JNA 5.0.0 will break compatibility - is there interest in minimizing impact?

[Matthias Bläsing]

Hey all,

those of you following the development of JNA will have noticed, that
the master branch holds a version, that will released as 5.0.0.

I branched a bugfix branch 4.5.X, that will get (and already got) some
bugfixes, but feature work will be limited to master.

The upcoming 5.0 version will be an source and binary incompatible with
4.X, as for example the signature of the Native#loadLibrary method was
changed from:


public static <T> T loadLibrary(Class<T> interfaceClass);

to

public static <T extends Library> T loadLibrary(Class<T> interfaceClass);


The wildcard was bounded to require a subtype of Library as parameter,
turning the existing runtime check into an compiletime check.

This change as a drawback however: This change is binary incompatible,
as the return type of a method is part of the signature. After erasure,
the first method returns "Object", while the second returns "Library".


Any library building on JNA must be recompiled against 5.0 and you
can't design bindings, that work with both branches (ok you could, by
using reflection to invoke the correct version of loadLibrary).


There is an option though: It is possible to inject bridge methods into
the bytecode. The approach would use:

https://github.com/infradna/bridge-method-injector

Given the signature from above this:

@WithBridgeMethods(Object.class)
public static <T extends Library> T loadLibrary(Class<T> interfaceClass);

would generate the needed bridge method, if the class is postprocessed
with the method injector.

From the POV of java, the language, the old method is still gone, but
from the POV of the JVM, the method can be found.

A bit better visible is the consequence if you compare the attached
compatibility report. The one with the "_bridge_methods" suffix is the
postprocessed case, and the one without is the case without post
processing. The incompatible changes after applying the bridge methods
are reduced to removing deprecated methods and removing Pointer#SIZE
which was not deprecated.


I have mixed feelings about this:

* it reduces incompatibilities (with the bridge methods you can
trivially create bindings, that can be used with JNA 4.5.X and 5.X)
* it requires the developers of JNA to understand about bytecode
manipulation
* it introduces a new compile time dependency (asm)


What do you think?

Greetings

Matthias

[L Will Ahonen]

Hi!

Personally I prefer code changes to build pipeline changes, as it's much easier to revert the former - or build two different branches side-by-side.

Plus I've been burned by Mystery Bugs From The Seventh Dimension caused by bytecode manipulation, which makes me wary of them :)

Cheers,

Will

[Neil C Smith]

On Wed, 23 May 2018 at 20:47 Matthias Bläsing <mbla...@doppel-helix.eu> wrote:

I have mixed feelings about this:

 * it reduces incompatibilities (with the bridge methods you can
   trivially create bindings, that can be used with JNA 4.5.X and 5.X)
 * it requires the developers of JNA to understand about bytecode
   manipulation
 * it introduces a new compile time dependency (asm)

What do you think?

Agree with Will, really.  This seems an overly complex "solution" to the wrong problem.

How feasible do you think writing code compatible with JNA 4 and 5 will be?

If you want to encourage that, I think it would be better to deprecate the current methods and provide differently named alternatives (load?) with the generics change?

If you don't want to encourage it, then a clean break without these hacks sounds better, and people can use the reflection workaround if they really want to?

Best wishes,

Neil

--

Neil C Smith

Artist & Technologist

www.neilcsmith.net

Praxis LIVE - hybrid visual IDE for creative coding - www.praxislive.org

[Matthias Bläsing]

Hi,

Am Donnerstag, den 24.05.2018, 14:40 +0100 schrieb 'Neil C Smith' via Java Native Access:
> On Wed, 23 May 2018 at 20:47 Matthias Bläsing <mbla...@doppel-helix.eu> wrote:
> > I have mixed feelings about this:
> >
> > * it reduces incompatibilities (with the bridge methods you can
> > trivially create bindings, that can be used with JNA 4.5.X and 5.X)
> > * it requires the developers of JNA to understand about bytecode
> > manipulation
> > * it introduces a new compile time dependency (asm)
> >
> > What do you think?
>
> Agree with Will, really. This seems an overly complex "solution" to
> the wrong problem.
>
> How feasible do you think writing code compatible with JNA 4 and 5
> will be?

If you depend only on JNA, then it is mostly trivial to create a
library, that is backwards compatible.

I'll focus on a concrete sample: gst1-java-core

I choose this, because I had a look at the codebase before and also did
a prototype with it.

PR102 https://github.com/gstreamer-java/gst1-java-core/pull/102 holds
the necessary changes, so that the source code is compatible with JNA
4.X and 5.X.

These changes in the gst1-java-core code base mostly just switch to
newer constructs in the JNA codebase, that are present for more than 5
years.

The problem: You can't support both JNA versions currently(!), because
JNA 5.X breaks compatibility at the bytecode level and it can't be
fixed on the java level.

> If you want to encourage that, I think it would be better to
> deprecate the current methods and provide differently named
> alternatives (load?) with the generics change?

I'd like to go into 5.X without deprecated methods, but if bytecode
manipulation will be discouraged, I'll go down that route. For the
bikeshedding (name finding), "load" sounds good to me.

I'll escape there, because:

> If you don't want to encourage it, then a clean break without these
> hacks sounds better, and people can use the reflection workaround if
> they really want to?

They won't and the end-user will suffer. There are very slow moving
libraries out there and let me choose two I'm familiar with:

- gst1-java-core
- libpam4j

I can imagine applications, that depend on both libraries and you'll
need matching JNA versions as dependency.

At least libpam4j was releasing not too often.



I did a prototype and tested the build on linux and windows. This is
the changeset, that is required:

https://github.com/matthiasblaesing/jna/commit/5eb40ae7ae0669edde1effe3efc538a13b7cf798


So from my perspective there are two options:

- either add overloaded "load" variants to Native or
- use bytecode manipulation

I'd like to make it possible to create libraries, that are compatible
with JNA 4.X and JNA 5.X.

Greetings

Matthias

[Neil C Smith]

On Fri, 1 Jun 2018, 20:06 Matthias Bläsing, <mbla...@doppel-helix.eu> wrote:

I'd like to go into 5.X without deprecated methods,

IMO for a long running library like JNA it makes sense to remove methods that have been deprecated for some time, but not things that haven't. If you want to encourage people to support 4.x and 5.x for a period of time then deprecating now and removing in 6.x makes more sense. "Clean" APIs are nice, but there's a lot to be said for keeping it simple too! 

. There are very slow moving
libraries out there and let me choose two I'm familiar with:

- gst1-java-core

Yeah, yeah, it'll speed up this summer, promise! :-)

[Matthias Bläsing]

Hi Neil, hey Will,

Am Freitag, den 01.06.2018, 20:32 +0100 schrieb 'Neil C Smith' via Java Native Access:
>
> On Fri, 1 Jun 2018, 20:06 Matthias Bläsing, <mblaesing@doppel-helix.e

> u> wrote:
> > I'd like to go into 5.X without deprecated methods,
>
> IMO for a long running library like JNA it makes sense to remove
> methods that have been deprecated for some time, but not things that
> haven't. If you want to encourage people to support 4.x and 5.x for a
> period of time then deprecating now and removing in 6.x makes more
> sense. "Clean" APIs are nice, but there's a lot to be said for
> keeping it simple too!

ok - I took the time and implemented the alternative to bytecode
manipulation:

https://github.com/java-native-access/jna/pull/970

I would be ok with that change.

If no-one objects till end of this week, I'll merge that changeset.

Greetings

Matthias