Release notes

PL/Java 1.6.6

This is the sixth minor update in the PL/Java 1.6 series. It adds support for PostgreSQL 16 and confirms compatibility with Java 21, and fixes some bugs, with few other notable changes. Further information on the changes may be found below.

Version compatibility

PL/Java 1.6.6 can be built against recent PostgreSQL versions including 16, and older ones back to 9.5, using Java SE 9 or later. The Java version used at runtime does not have to be the same version used for building. PL/Java itself can run on any Java version 9 or later. PL/Java functions can be written for, and use features of, whatever Java version will be loaded at run time. See version compatibility for more detail.

Some builds of Java 20 are affected by a bug, JDK-8309515. PL/Java will report an error if detects it is affected by that bug, and the solution can be to use a Java version earlier than 20, or one recent enough to have the bug fixed. The bug has been fixed in Java 21.

PL/Java 1.6.6 will definitely no longer build on PostgreSQL versions older than 9.5. It has made no attempt to support them since 1.6.0, and lingering conditional code for older versions has now been removed.

Changes

Changes in XML support

Java 17’s standardized XML feature and property names added

Java 17 added standardized, easy-to-remember names for a number of features and properties the underlying XML implementations had formerly supported under implementation-specific names. PL/Java’s Adjusting.XML API already needed to know those various other names, to attempt using them to configure the desired features and properties. Now it tries the new standard names too.

Better control when a feature or property can’t be set as intended

The original documentation for the feature and property setters in the Adjusting.XML API said “the adjusting methods are best-effort and do not provide an indication of whether the requested adjustment was made”. (At the same time, failures could produce voluminous output to the log.)

The new lax(boolean) method offers more control. If not used, adjustment failures are logged (as before, but more compactly in the case of multiple failures in one sequence of adjustments). Or, lax(true) can be used to silently discard any failures up to that point in a sequence of adjustments, or lax(false) to have the exceptions chained together and thrown.

The addition of the new Java 17 standardized names can complicate version-agnostic configuration of other elements in the Java XML APIs, such as Transformer, that are not directly covered by PL/Java’s Adjusting.XML.Parsing methods. Client code may find the new Adjusting.XML method setFirstSupported convenient for that purpose; an example illustrates.

Packaging / testing

Support choice of PGJDBC or pgjdbc-ng in Node

The package jar produced as the last step of the build includes a test harness similar to the PostgresNode Perl module. It formerly worked only with the pgjdbc-ng driver. Now it works with either PGJDBC or pgjdbc-ng, and provides features for writing test scripts that do not depend on the driver chosen.

Source code

  • Minor changes to support PostgreSQL 16
  • Conditional code supporting PostgreSQL versions older than 9.5 removed
  • Old non-HTML5 elements (rejected by Javadoc 17 and later) removed from doc comments
  • The pureNonVirtualCalled method removed to quiet warnings from recent C compilers
  • Schema qualification in embedded SQL added to two operators that had been overlooked in the earlier round of adding such qualification

Bugs fixed

PL/Java 1.6.5

This is the fifth minor update in the PL/Java 1.6 series. It adds support for PostgreSQL 15 and fixes some bugs, with few other notable changes. Further information on the changes may be found below.

Version compatibility

PL/Java 1.6.5 can be built against recent PostgreSQL versions including 15, and older ones back to 9.5, using Java SE 9 or later. The Java version used at runtime does not have to be the same version used for building. PL/Java itself can run on any Java version 9 or later. PL/Java functions can be written for, and use features of, whatever Java version will be loaded at run time. See version compatibility for more detail.

Some builds of Java 20 are affected by a bug, JDK-8309515. PL/Java will report an error if detects it is affected by that bug, and the solution can be to use a Java version earlier than 20, or one recent enough to have the bug fixed.

Changes

Changes affecting administration
Bugs affecting install_jar from http/https URLs fixed

CI testing now makes sure that http URLs work and the appropriate java.net.URLPermission can be granted in pljava.policy where the comments indicate.

Improvements to the annotation-driven SQL generator
PL/Java functions can be declared on interfaces as well as classes

The SQL/JRT specification has always only said ‘class’, but it could be debated whether ‘class’ was intended strictly or inclusively. As there is no technical obstacle to using static methods declared on an interface, and PL/Java’s runtime already could do so, the SQL generator no longer disallows @Function annotations on them.

SQL generator reports compatibility with a more recent Java source version

Because PL/Java 1.6 retains compatibility for building on Java versions back to 9, the SQL generator previously reported 9 as the source version supported. This produced warnings building user code to target a later version of Java, which were only an issue for sites using a fail-on-warning policy.

The SQL generator now reports its supported source version as the earlier of: the Java version being used, or the latest Java version on which it has been successfully tested. In this release, that is Java 20.

Improvements to documentation
Use of --add-modules to access Java modules not read by default, explained

By default, PL/Java starts up with a fairly small set of Java modules readable. The documentation did not explain the use of --add-modules in pljava.vmoptions to expand that set when user code will refer to other modules. That is now documented.

Enhancement requests addressed

Bugs fixed

Credits

Thanks in release 1.6.5 to Francisco Miguel Biete Banon, Christoph Berg, Frank Blanning, Stephen Frost, Casey Lai, Krzysztof Nienartowicz, Yuril Rashkovskii, Tim Van Holder, aadrian, sincatter, tayalrun1.

Earlier releases

PL/Java 1.6.4

This is the fourth minor update in the PL/Java 1.6 series. It is a minor bug-fix release with few other notable changes. Further information on the changes may be found below.

Changes

Changes affecting administration
JEP 411 advisory message downgraded

PL/Java follows certain administrative actions with an advisory message about upcoming Java changes that will affect policy enforcement. In 1.6.3, that message was always at WARNING level. In 1.6.4, it is downgraded to NOTICE level when running on Java versions before 17.

As before, updates on how PL/Java will adapt to the future Java changes can be watched by bookmarking the JEP 411 topic on the PL/Java wiki.

Changes to runtime behavior
Java’s thread context class loader

Starting in 1.6.3, PL/Java has made sure the thread context class loader on entry to a function reflects the initiating loader for the implementing class, but in 1.6.3, the loader could be set too many times when executing a set-returning function. That could lead to the wrong loader being restored on function exit, likely only noticeable in an application with jars in multiple schemas and with nested Java function invocations.

That is fixed in this release.

Improvements to the annotation-driven SQL generator
Correct SQL now emitted for a function with one OUT parameter

PostgreSQL requires a different form of declaration for a function with one OUT parameter than for a function with two or more. PL/Java was emitting the wrong form for the one-out-parameter case.

The documentation has been clarified and new examples added to illustrate how Java methods should be structured and annotated for both cases.

The intended treatment can be ambiguous for some Java methods, without explicit annotations. For example, a Java method that returns boolean and has a trailing ResultSet parameter could be a single-row composite-returning function, or an ordinary function with a row-typed input parameter and boolean return. PL/Java has always silently chosen which interpretation to apply in such cases, and still does, but it now emits a comment into the generated deployment descriptor, explaining what annotation to use if a different meaning was intended.

BaseUDT annotation now has constants for known type categories

PostgreSQL user-defined types can be declared in categories, some of which are predefined. PL/Java’s BaseUDT annotation now includes constants for those, which can be used in place of the single-letter codes for readability.

Bugs fixed

PL/Java 1.6.3 (10 October 2021)

This is the third minor update in the PL/Java 1.6 series. It adds support for PostgreSQL 14, continues to improve the runtime behavior and the annotation-driven SQL generator, and fixes several bugs. Further information on the changes may be found below.

PL/Java with Java 17 and later: JEP 411

Current versions of PL/Java rely on Java security features that will be affected by JEP 411, beginning with Java 17. Java 17 itself will continue to provide the needed capabilities, with only deprecation marks and warnings added. Java 17 is also positioned as a long-term support release, so the option of continuing to run this PL/Java release will be available, with no loss of function, by continuing to run with Java versions up to and including 17.

For more on how PL/Java will adapt, please bookmark the JEP 411 topic on the PL/Java wiki.

For this release, PL/Java will suppress a JEP 411-related warning from the Java runtime itself that would otherwise be emitted for every PostgreSQL backend that starts Java, and instead will issue a more informative “migration advisory” warning only on certain administrative actions no more than once per session, with a goal of ensuring that responsible administrators are aware of the expected future developments. The advisory message includes the URL to the wiki topic above.

This release also adds code to detect if it is on a future, post-Java 17 runtime where the needed functionality is not available, and throw informative exceptions with suggested corrective actions.

Changes

Changes affecting administration
New function now created on update

The new SQLJ.ALIAS_JAVA_LANGUAGE function introduced with 1.6 will now be present after an ALTER EXTENSION UPDATE if it was not before.

Message if the Java runtime ends the backend process during startup

The Java runtime can behave antisocially for some startup issues, such as a misspelled jar in pljava.module_path, and silently terminate the backend process rather than reporting an error. (It writes a message, not to standard error, but to standard output, which from a PostgreSQL backend is not captured for logging, and may never be seen.) A message is now generated in that case, to provide at least some clue what has gone wrong.

Changes to runtime behavior
Java’s thread context class loader

PL/Java now supplies a known value for the current Java thread’s context class loader on entry to a PL/Java function. It is the class loader for the PostgreSQL schema where the function is declared. The context class loader is referred to by numerous Java libraries and by Java’s ServiceLoader class, which may expect to find services and resources along the PL/Java class path that has been configured for the function. Neglecting to set the context class loader opened the door to unexpected loading failures like issue #361 that can require awkward or slow contortions to work around in user Java code.

This change is documented in more detail. An opt-out setting is available.

Improvements to the annotation-driven SQL generator
Now able to declare function parameters that default to null

It has been possible to annotate a function parameter with @SQLType(defaultValue=...) to give it any non-null default value, but because Java disallows null annotation values, that notation wasn’t usable to declare a parameter that defaults to null. The new notation @SQLType(optional=true) means exactly that.

Fixes a bug in synthesis of COMMUTATOR/NEGATOR operators

It is now possible to code one method in Java and declare a full complement of operators based on it by combining commutation and negation. A new example is provided.

Bugs fixed

Updated PostgreSQL APIs tracked

  • Regularized spelling of pg_type OID symbols
  • Changed TOAST pointer format supporting selectable TOAST compression methods
  • Demise of ErrorData.show_funcname (it was a vestige of frontend/backend protocol v2, long obsolete)

Credits

Thanks to Krzysztof Nienartowicz, ricdhen, and JanaParthasarathy, who among them reported five of the issues fixed in this release.

PL/Java 1.6.2

This is the second minor update in the PL/Java 1.6 series, with two bugs fixed (including one that was likely to be a blocker for many applications). It adds a ‘trial’ security policy, useful when migrating code from PL/Java 1.5 to identify permission grants that may be needed, and some minor but useful example functionality.

Changes

Bug blocking use from an unprivileged PostgreSQL role fixed

In 1.6.0 and 1.6.1, PL/Java could fail to start in a backend process if the effective PostgreSQL role was not a superuser or a member of pg_read_all_settings. That is fixed in this release.

Trial security policy for migrating code from PL/Java pre-1.6.0

When migrating code from pre-1.6.0 PL/Java versions, it may be necessary to add permission grants to the PL/Java 1.6 security policy, which is distributed as a minimal starting point.

Configuring permissions in PL/Java covers the topic in general, and Migrating to policy-based permissions covers the new ‘trial’ policy introduced in 1.6.2 to simplify the process.

Example functions using XSLT 1.0 polished for simpler use

Although 1.0 is obsolete and very limited compared to current versions of XSLT, the implementation in Java has two attractive properties. First, it is provided in the Java runtime with no need of a separate download such as Saxon, and second, it allows use of any accessible Java methods or constructors from XPath expressions, which can often make it more useful than the limitations of strict XSLT 1.0 would suggest.

The example functions prepareXMLTransform, prepareXMLTransformWithJava, and transformXML, which demonstrate how such functions can be coded, have also been given streamlined parameter lists with defaults to make them simple to use for real work.

One small but useful result is an easy way to indent XML for readability, simply by passing a null transform name and indent => true to transformXML.

Bugs fixed

Credits

Thanks to Francisco Biete for the report of #331.

PL/Java 1.6.1 (16 November 2020)

Note: 1.6.1 was released with a bug likely to be a blocker for many applications. It was fixed in 1.6.2.

This is the first minor update in the PL/Java 1.6 series, with two bugs fixed. It also adds functionality in the SQL generator, allowing automated declaration of new PostgreSQL aggregates, casts, and operators, and functions with OUT parameters.

Changes

Limitations when built with Java 10 or 11 removed

PL/Java can now be built with any Java 9 or later (latest tested is 15 at time of writing), and the built artifact can use any Java 9 or later at run time (as selected by the pljava.libjvm_location configuration variable).

That was previously true when built with Java 9 or with Java 12 or later, but not when built with 10 (would not run on 9) or with 11 (would not run on 9 or 10). Those limits have been removed.

Functions with OUT parameters

PL/Java has long been able to declare a function that returns a composite type (or a set of such), by returning a named composite PostgreSQL type, or by being declared to return RECORD.

The former approach requires separately declaring a new composite type to PostgreSQL so it can be named as the function return. The RECORD approach does not require pre-declaring a type, but requires every caller of the function to supply a column definition list at the call site.

Declaring the function with OUT parameters offers a middle ground, where the function has a fixed composite return type with known member names and types, callers do not need to supply a column definition list, and no separate declaration of the type is needed.

There is no change to how such a function is coded at the Java source level; the new annotation element only changes the SQL generated to declare the function to PostgreSQL. Examples are provided.

Generation of aggregate, cast, and operator declarations

The SQL generator now recognizes @Aggregate, @Cast, and @Operator annotations, generating the corresponding SQL deploy/undeploy scripts. Some examples (for aggregates, casts, and operators) are provided. The reduction in boilerplate needed for a realistically-complete example can be seen in this comparison of Bear Giles’s pljava-udt-type-extension example; the two branches compared here are (1) using only the annotations supported in PL/Java 1.6.0 and (2) using also the new support in 1.6.1.

Bugs fixed

Credits

Thanks to Bear Giles for the pljava-udt-type-extension example, which not only illustrates the SQL generation improvements in this release, but also exposed both of the bugs fixed here.

PL/Java 1.6.0 (18 October 2020)

Note: 1.6.0 was released with a bug likely to be a blocker for many applications. It was fixed in 1.6.2.

This is the first release of a significantly refactored PL/Java 1.6 branch with a number of new features and changes. It requires Java 9 or later at build and run time, but retains the ability to run PL/Java application code built for earlier versions. It should be used with PostgreSQL 9.5 or later. For applications requiring an older Java or PostgreSQL version, the latest release in the PL/Java 1.5 line remains an option.

Note to package maintainers: these release notes should be reviewed before an installation moves to 1.6.0 from a 1.5 or earlier version, so it is best packaged in a way that requires an affirmative choice to upgrade.

Version compatibility

PL/Java 1.6.0 can be built against recent PostgreSQL versions including 13, and older ones back to 9.5, using Java SE 9 or later. The Java version used at runtime does not have to be the same version used for building. PL/Java itself can run on any Java version 9 or later if built with Java 9 or with 12 or later (bugs in the Java 10 and 11 compilers prevent running on 9 if built with 10, or on 9 or 10 if built with 11). PL/Java functions can be written for, and use features of, whatever Java version will be loaded at run time. See version compatibility for more detail.

When used with GraalVM as the runtime VM, PL/Java functions can use Graal’s “polyglot” capabilities to execute code in any other language available on GraalVM. In this release, it is not yet possible to directly declare a function in a language other than Java.

If building with GraalVM, please add -Dpolyglot.js.nashorn-compat=true on the mvn command line.

Changes

New configurable permissions may require configuration

Prior to 1.6.0, PL/Java hard-coded the permissions that were available to functions declared in the ‘trusted’ language java or the ‘untrusted’ language javaU. With 1.6.0, the exact permissions available for both cases can be configured in the pljava.policy file (found in the directory reported by pg_config --sysconfdir) as described in the new policy documentation.

Java’s policy language can conditionally grant permissions but not deny them if another clause grants them. Therefore, the default policy must be somewhat restrictive, so a desired policy can be built from it with grant clauses.

In the 1.6.0 default policy, ‘trusted’ (java) code has minimal permissions, suitable for general computation and interacting with the database, and ‘untrusted’ (javaU) code has only the additional permission to access the file system. Existing user functions that worked in PL/Java 1.5.x and performed other actions, such as making network connections, will need the appropriate permissions (such as java.net.URLPermission or java.net.SocketPermission) granted via the policy file.

The policy can grant permissions more selectively than just to java or javaU. The new documentation covers the details, and also how to log, for troubleshooting purposes, the permissions being requested.

Whatever the reason, all down the years, a favorite “is PL/Java working?” check found online has been to read a Java system property with System.getProperty. Not all of those examples pick properties that can be read under the default policy. So, even some familiar habits like that may need revision, at least to use a property like java.version that is readable by default.

The former hard-coded permissions were by turns too lax or too strict, depending on what was needed, and interfered in some cases with the operation of the Java runtime itself, breaking (at least) its XSLT implementation and the profiling functions of visualvm. This release fixes those issues.

Validation at CREATE FUNCTION time may force changes to deployment procedures

PL/Java can now detect problems with a function declaration, including missing dependencies, at the time of CREATE FUNCTION, rather than allowing the function to be created and reporting failure later when it is called.

This change may have an impact on some established procedures. For example, when installing a jar that contains deployment commands, deployment may fail if another required jar has not been installed and added to the class path first; in the past, the order did not matter. For details, see this section in the documentation for the supplied examples, and the description of check_function_bodies in the configuration variable reference.

Java 9 module system; pljava.classpath -> pljava.module_path

Because PL/Java itself is now modular code conforming to the module system introduced with Java 9, one configuration variable has changed: pljava.classpath is now pljava.module_path.

As before, its default value will be correct when PL/Java is installed to the usual locations. It should be rare for any installation to have needed to think about the old one, or to need to think about the new one. For a rare installation that does, the details are in the documentation.

In this release, user code is not treated as modular; the SQLJ.INSTALL_JAR routine still treats its jars as unnamed-module code on a class path, as before.

Improvements to the annotation-driven SQL generator
Infers additional implicit ordering dependencies

The SQL generator can now respect the implicit ordering constraints among user-defined types and functions that either use the types or are used in their definitions, which can eliminate many provides/requires annotation elements that had to be added by hand for PL/Java 1.5. The reduction in boilerplate needed for a realistic example can be seen by comparing the annotated version of Bear Giles’s pljava-udt-type-extension example at this commit (pre-1.6) and this one (1.6.0).

Generates variadic function declarations

PL/Java 1.6 can declare functions that can be called from SQL with varying numbers of arguments. Example code is provided.

Better support for PostgreSQL’s SQL_ASCII encoding

PostgreSQL’s legacy SQL_ASCII encoding is difficult to use in Java because 128 of its code points have no defined mapping to Unicode, which Java uses. The page on database character set encodings has a section suggesting workable approaches if PL/Java is used in a database with that encoding. A new addition among those options is a Java Charset supporting the encoding names X-PGSQL_ASCII or SQL_ASCII, which maps the ASCII characters as expected, and reversibly encodes the others using Unicode permanently-undefined codepoints.

Build system, continuous integration, quality assurance
  • The nar-maven-plugin formerly used in the build has been replaced with a newly-developed Maven plugin tailored to PostgreSQL extension building.

  • The new plugin respects the flags reported by pg_config when building the native library.

  • Building with the same flags used for PostgreSQL has eliminated the flood of uninformative warnings that, in prior versions, made troubleshooting actual build problems difficult.

  • Travis-CI and AppVeyor now regularly build and test PL/Java for Linux (x86_64 and ppc64le), Mac OS, and Windows (using MSVC and MinGW-w64), with results visible at GitHub.

  • PL/Java’s self-installer jar now includes utilities to simplify integration testing, similar to the PostgresNode Perl module provided with PostgreSQL. It is used in the Travis and AppVeyor builds to keep platform-specific code to a minimum, and may be useful for other purposes. Some documentation is included.

  • Having fixed the permission issues that were breaking the profiling functions of visualvm, it will be easier to incorporate profiling into future development.

Enhancement requests addressed

Bugs fixed

Updated PostgreSQL APIs tracked

  • 64-bit FuncCallContext.call_cntr (ResultSetProvider/ResultSetHandle can now return more than INT_MAX rows)

Credits

There is a PL/Java 1.6.0 thanks in part to Christoph Berg, Chapman Flack, Kartik Ohri, original creator Thomas Hallgren, and the many contributors to earlier versions.

The work of Kartik Ohri in summer 2020 on the build system renovation and continuous integration was supported by Google Summer of Code.

Releases prior to PL/Java 1.6.0