Java 9

[shital patil]

Hello,

Any idea about plan to support GWT for java 9 which is to be released by july end. 

thanks.

[James Nelson]

As I've mentioned elsewhere, https://plus.google.com/+JamesNelsonX/posts/gBfpBDnwV9V , java 9 support is entirely possible.

When I last tried it out, it required a fair number of hacks, but it worked fine.

I expect the final version of java 9 modules to require slightly fewer hacks,

and for it to take a fair amount of time to properly refactor gwt jars enough to be usable in java 9 systems.

If you are in a rush to get java 9, you are going to have to roll up your sleeves, hold your nose, and dive into the early-access bug-zone.
I can get you jars that (can be made to) work, but are very likely to change when integrated properly into Gwt

(I did some unspeakable things to make it work the weekend I did it as my project).

If you are comfortable waiting until there is more official support, I can't really tell you what Google is planning...

I, however, will be releasing my fork w/ java 9 support when I am sure the underlying module system is completely done changing;

a few colleagues and I are considering opening a gwt-shop with support for the 2.X line, and would handle this kind of stuff directly,

however, it's still being done in my "spare" time, which is actually supposed to be for building a new ui templating language for gwt,

coming up with an javax.model replacement for gwt generators, and building a social network to fix democracy...  hehe.
"Sleep is the Cousin of Death" :D

[Colin Alworth]

Are there specific Java9 JRE classes that we should focus on to get into GWT?

Would making GWT support Java9 sources and running in a Java9 env cause backward incompatible changes that prevent GWT from running in Java8?

[James Nelson]

Are there specific Java9 JRE classes that we should focus on to get into GWT?

I didn't go through the full changelog, but I believe there have been more helper methods added to Optional and other classes; will need emul updates for sure.

 

Would making GWT support Java9 sources and running in a Java9 env cause backward incompatible changes that prevent GWT from running in Java8?

There is approximately zero likelihood of breaking changes there.
The only language changes are private interface methods, and relaxed try-with-resources.
So, the only risk would be using any updated emul code, and even then, that would only apply to any poor souls still using legacy dev mode, or people using java 8 importing a library using java 9.

In the case of using new language features, it would be possible to automate a transpile from 9 to 8, and in the case of emul, it's standard "don't use new things if you need to support old things". 

[James Nelson]

So, to be pedantic:

Language features -> no issues there at all, since JDT happily compiles java 8 code w/ a java 9-compatible compiler.

Emul updates -> standard "don't use java 9 methods if you need java 8 support"
Classpathery -> Made it use ugly reflection to not require java 9 classes on classpath; it does require more opening of modules and ugliness, but ensures that we don't break when using java 8.

I'm actually using the java 9 compatible version of gwt daily in java 8 code.

we could make the classpath nastiness a little better using sane feature detection and loading classes that don't need (as much) reflection to be able to extract the URLs from java 9 classloader...

...however, long term, I still think that we should reconsider how ResourceLoader works, and possibly consider something that accepts classpaths as arguments.  It's fairly trivial to have arbitrary java code use maven to cache/download dependencies, so we could, technically, get classpath into Gwt without relying on classloader scanning.

That would be a broader discussion to have though.

The one big caveat is code modularity.  We can't release two jars with code in the same packages (unless recent changes to module system now allowed "concealed package conflicts").  So, that means either one big, ugly uber-jar for compilation (my hack unzips gwt-user and gwt-dev locally to avoid these issues)... or we actually, finally modularize the codebase.

Given the intent to rip out useful bits and get them supported in the 3.X line, I'd angle towards modularization as that is necessary anyway to create smaller, independent libraries.

[Thomas Broyer]

On Thursday, June 8, 2017 at 12:13:35 AM UTC+2, James Nelson wrote:

So, to be pedantic:

Language features -> no issues there at all, since JDT happily compiles java 8 code w/ a java 9-compatible compiler.

Emul updates -> standard "don't use java 9 methods if you need java 8 support"
Classpathery -> Made it use ugly reflection to not require java 9 classes on classpath; it does require more opening of modules and ugliness, but ensures that we don't break when using java 8.

I'm actually using the java 9 compatible version of gwt daily in java 8 code.

we could make the classpath nastiness a little better using sane feature detection and loading classes that don't need (as much) reflection to be able to extract the URLs from java 9 classloader...

...however, long term, I still think that we should reconsider how ResourceLoader works, and possibly consider something that accepts classpaths as arguments.  It's fairly trivial to have arbitrary java code use maven to cache/download dependencies, so we could, technically, get classpath into Gwt without relying on classloader scanning.

Could that work if we complemented the "instanceof URLClassLoader" with some check for "is the system classloader" and then use System.getProperty("java.class.path") (or the equivalent ManagementFactory.getRuntimeMXBean().getClassPath()) to get the system classpath entries? (do we also need the bootclasspath?)

 

That would be a broader discussion to have though.

IIRC, when CodeServer was added, with its -src argument, I suggested passing classpaths as argument rather than using the system classpath.

At the time though, there was even more code that relied on the context classloader than today (rather than using the ResourceOracle). This was fixed later for the most part, but I believe there's still code that uses the context classloader (including looking up annotations from com.google.gwt.core.ext.typeinfo, and AFAIK also JDT loading bytecode that ends up being looked up in the context classloader; see 8b584f412d055ff39949d8b28a8e78d6b0bfe9b6 for example)

I suppose that could be fixed by using a child URLClassLoader from the classpath argument (that could be necessary anyway for generators and linkers)

 

The one big caveat is code modularity.  We can't release two jars with code in the same packages (unless recent changes to module system now allowed "concealed package conflicts").

Apparently not: http://openjdk.java.net/projects/jigsaw/spec/issues/#AvoidConcealedPackageConflicts

IIUC, they say this can be worked around using classloaders, or shading/shadowing (i.e. what you're doing)

 

So, that means either one big, ugly uber-jar for compilation (my hack unzips gwt-user and gwt-dev locally to avoid these issues)... or we actually, finally modularize the codebase.

Does that also apply to subpackage? Because it looks like the only conflicting packages are c.g.g.core.client and c.g.g.core.shared, so maybe moving classes around could be enough?

[James Nelson]

Could that work if we complemented the "instanceof URLClassLoader" with some check for "is the system classloader" and then use System.getProperty("java.class.path") (or the equivalent ManagementFactory.getRuntimeMXBean().getClassPath()) to get the system classpath entries? (do we also need the bootclasspath?)

Here is the hack in full.  Could be better using a more standard "get my classpath" mechanism (though, to be honest, I think it might be better to have an alternate means of specifying classpath for Gwt; for example, specifying maven-repo coordinates and resolving them via maven cli)

String java9Modules = System.getProperty("gwt.java9.modules");
for (ClassLoader candidate = wrapped; candidate != null; candidate = candidate.getParent()) {
  if (candidate instanceof URLClassLoader) {
    for (URL url : ((URLClassLoader) candidate).getURLs()) {
      result.put(url.toExternalForm(), url);
    }
  } else  if (java9Modules != null) {
  // If the user specified java 9 modules to load for classpath,
  // we'll want to do it all through reflection,
  // to avoid losing support for java 8
    try {
      final Class<?> modRef = candidate.loadClass("java.lang.module.ModuleReference");
      final Method getLocation = modRef.getMethod("location");
      // In the case of java 9, the only way to scrape classloaders is this ugly reflection
      // on an internal class (used to be a URLClassLoader).
      // you will need to add `--add-opens java.base/jdk.internal.loader=myJava9ModuleName`
      // to your gwt runtime as a VM argument
      final Class<?> loader = candidate.loadClass("jdk.internal.loader.BuiltinClassLoader");
      // TODO: Don't use classloader for resource loading
      final Method findMod = loader.getDeclaredMethod("findModule", String.class);
      // This is why we have to open the package; just being visible is not enough
      // to be allowed to act reflectively
      findMod.setAccessible(true);

      for (String source : java9Modules.split(",")) {
        System.out.println("Loading java 9 module " + source);
        Object mod = findMod.invoke(candidate, source);
        // Safe to cast; we must be in java 9 to get here,
        // so we know that this cast should be safe
        Optional<URI> location = (Optional<URI>) getLocation.invoke(mod);
        if (location.isPresent()) {
          final URL url = location.get().toURL();
          result.put(url.toExternalForm(), url);
        }
      }
    } catch (Exception ignored) {
      if (ignored instanceof InterruptedException) {
        Thread.currentThread().interrupt();
        throw new RuntimeException(ignored);
      }
      ignored.printStackTrace();
    }
  }
}

My method uses a system property to specify which java 9 modules to use as source, then rips those out of the BuiltinClassloader.

This is obviously too egregious to use in production, but worked enough to get a HelloWorld example to compile (w/ new language features for testing).

Due to java 9's encapsulation, we can't just load the whole classpath anyway, since the modules we want to read must be opened (thus the property, which could be a config prop instead).

Because we will already have to name the modules we want, it's not a far cry to specify source coordinates and resolve the jars as well (I have other tools that do this nicely).

It's not pretty, but at least it is possible. :-)

IIRC, when CodeServer was added, with its -src argument, I suggested passing classpaths as argument rather than using the system classpath.

At the time though, there was even more code that relied on the context classloader than today (rather than using the ResourceOracle). This was fixed later for the most part, but I believe there's still code that uses the context classloader (including looking up annotations from com.google.gwt.core.ext.typeinfo, and AFAIK also JDT loading bytecode that ends up being looked up in the context classloader; see 8b584f412d055ff39949d8b28a8e78d6b0bfe9b6 for example)

I suppose that could be fixed by using a child URLClassLoader from the classpath argument (that could be necessary anyway for generators and linkers)

 

Aye...  the way we load/compile annotations has always irked me.
Java 9 also comes with full-fledged support for dynamically compiling arbitrary source, and it is my opinion that we should use this for annotations (so we don't need to run javac before gwtc).
Because annotations contain classes and enums, those classes and enums must also be loaded to get annotation support, so it gets... sticky, especially once java 9 modules get involved.

By running the compiler on source, then the myriad of original modules won't really matter...

Another alternative is to go the way of apt and use something akin to AnnotationMirror... meaning you can't just do MyAnnoClass anno = member.getAnnotation(MyAnnoClass.class), but will instead be stuck operating on a generic mirror which can get members by string name.  Not exactly pretty, but if it's good enough for apt, it can clearly be made to work (plus, with the impetus to ditch generators, we'd be stuck with it anyway).

I (also) have another project which converts AnnotationMirror to a proxy of MyAnnoClass that can either load a dependent Class/Enum, or fail if that class is not on classpath (but only fail if you try to access that member).  Adding a dynamic runtime compiler (I didn't have support for that when I build the annotation proxy bit) would mean we could fallback to compiling the necessary classes, and still get nice typed annotations w/out requiring them to be precompiled (and thus remove all need for inspecting ClassLoader). 

 

The one big caveat is code modularity.  We can't release two jars with code in the same packages (unless recent changes to module system now allowed "concealed package conflicts").

Apparently not: http://openjdk.java.net/projects/jigsaw/spec/issues/#AvoidConcealedPackageConflicts

IIUC, they say this can be worked around using classloaders, or shading/shadowing (i.e. what you're doing)

 

Right, if you, yourself, load up a URLClassLoader and use it, it's business as usual.  But, that also means we need to populate said URLClassLoader correctly, which means either we manage classpath ourselves (then no need for URLClassLoader inspection, since we are supplying its URLs), or we force end users to figure this out themselves (ick).

The shading I am doing is too hideous to consider for anything serious.  It took like, four invocations of maven-dependency-plugin to unzip stuff in all the right locations, and resulted in an output artifact containing the entire user+dev gwt sdk.

So, the time has finally come that modularization is no longer nice-to-have, but now must-have. 

 

So, that means either one big, ugly uber-jar for compilation (my hack unzips gwt-user and gwt-dev locally to avoid these issues)... or we actually, finally modularize the codebase.

Does that also apply to subpackage? Because it looks like the only conflicting packages are c.g.g.core.client and c.g.g.core.shared, so maybe moving classes around could be enough?

 

It could be possible to get away with light refactoring, yourself and Colin have more experience in actually making an effort towards this.
I would be happy with a lighter weight solution for now, to make this workable, and a full solution later, as time permits.

[Thomas Broyer]

On Tuesday, June 13, 2017 at 5:55:33 PM UTC+2, James Nelson wrote:

Could that work if we complemented the "instanceof URLClassLoader" with some check for "is the system classloader" and then use System.getProperty("java.class.path") (or the equivalent ManagementFactory.getRuntimeMXBean().getClassPath()) to get the system classpath entries? (do we also need the bootclasspath?)

Fwiw, I think it would work, according to https://github.com/AdoptOpenJDK/openjdk-jdk9/blob/f9129bba032f3b75be9b67f45636f76940e029a6/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java#L70-L73

(for now, I'm not interested in supporting modules, just making GWT 2 work in a Java 9 VM)

Another alternative is to go the way of apt and use something akin to AnnotationMirror... meaning you can't just do MyAnnoClass anno = member.getAnnotation(MyAnnoClass.class), but will instead be stuck operating on a generic mirror which can get members by string name.  Not exactly pretty, but if it's good enough for apt, it can clearly be made to work (plus, with the impetus to ditch generators, we'd be stuck with it anyway).

But isn't the whole issue with annotations due to JDT?

 

I (also) have another project which converts AnnotationMirror to a proxy of MyAnnoClass that can either load a dependent Class/Enum, or fail if that class is not on classpath (but only fail if you try to access that member).

You mean like https://docs.oracle.com/javase/7/docs/api/javax/lang/model/element/Element.html#getAnnotation(java.lang.Class) ?

(ok, that one won't load the Class/enum from the classpath, but always throw when accessing members of those types)

[James Nelson]

Fwiw, I think it would work, according to https://github.com/AdoptOpenJDK/openjdk-jdk9/blob/f9129bba032f3b75be9b67f45636f76940e029a6/jdk/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java#L70-L73

(for now, I'm not interested in supporting modules, just making GWT 2 work in a Java 9 VM)

Hm.  One would assume that most people using Java 9 are going to be using modules, since, AFAIK, you can't access anything but the java.base module without requiring said modules (no java.logging, java.sql, java.xml, java.xml.bind, or jdk.management allowed).  Not sure how ReflectiveParser is going to create ModuleDefs w/out java.xml unless we repackage all of org.xml.sax... Though, I suppose we could try it and see if / how it blows up?

I mean, reading the java.class.path as a fallback would be ok, but we'd still want to prefer classloader scanning for cases where the user is launching their gwtc on a thread with a different classloader than the system classloader.

Though, I think any solution which requires "never run gwtc with modules involved at all" would be pretty poor, considering almost any dependency which also uses java 9 will almost assuredly choke and die.

Considering the ugly hack I have does technically work, it doesn't seem worth it to limit scope so we can use a less-bad hack.

 

Another alternative is to go the way of apt and use something akin to AnnotationMirror... meaning you can't just do MyAnnoClass anno = member.getAnnotation(MyAnnoClass.class), but will instead be stuck operating on a generic mirror which can get members by string name.  Not exactly pretty, but if it's good enough for apt, it can clearly be made to work (plus, with the impetus to ditch generators, we'd be stuck with it anyway).

But isn't the whole issue with annotations due to JDT?

It is my understanding that we use ASM to load the annotation attributes from source classes, and then create a Proxy to load member values / classes / enums off the classpath.  JDT is not involved at all (strange that it isn't...).  

Really, any solution which exposes actual instances of the annotation class are somewhat doomed to have to be able to load those types off the classpath.

One solution, not sure it's a good one, would be to generate a MyAnnotationMirror type, which still affords type safety and ease of use, but exposes classes and enums as objects containing only type / name information.  Obviously not going to work w/out a precompilation pass, but could work well in combination with other tools to help get generators out of gwtc (AKA, not a good solution for right now). 

 

 

I (also) have another project which converts AnnotationMirror to a proxy of MyAnnoClass that can either load a dependent Class/Enum, or fail if that class is not on classpath (but only fail if you try to access that member).

You mean like https://docs.oracle.com/javase/7/docs/api/javax/lang/model/element/Element.html#getAnnotation(java.lang.Class) ?

(ok, that one won't load the Class/enum from the classpath, but always throw when accessing members of those types)

Precisely that, actually.  Except I catch those exceptions (which contain the TypeMirror of the offending type), and then try to recover by loading the class (and could make that even better by looking for source and compiling on the fly, as needed... requires a dynamic classloader though, which can make things... complex.) 

[James Nelson]

It is my understanding that we use ASM to load the annotation attributes from source classes, and then create a Proxy to load member values / classes / enums off the classpath.  JDT is not involved at all (strange that it isn't...).  

The most likely reason we aren't using JDT to compile the annotaitons is we are really only using the parser, but not the linker (no code to emit classfiles).... 

[Roberto Lublinerman]

I have uploaded two patches for review to allow GWT to run under a Java 9 vm. 

https://gwt-review.googlesource.com/#/c/19000/

https://gwt-review.googlesource.com/#/c/19001/

The main issue is (as noted by James) that the class loading has been revamped and GWT can no longer assume that the class loaders are UrlClassLoaders.

The idea is first to be able to run GWT on a Java 9 vm compiling Java 8 sources.

On Wed, Jun 14, 2017 at 3:01 PM, James Nelson <Ja...@wetheinter.net> wrote:

It is my understanding that we use ASM to load the annotation attributes from source classes, and then create a Proxy to load member values / classes / enums off the classpath.  JDT is not involved at all (strange that it isn't...).  

The most likely reason we aren't using JDT to compile the annotaitons is we are really only using the parser, but not the linker (no code to emit classfiles).... 

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-contributors+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/d08d7ad2-4f67-4118-bf81-2924fedfddf3%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Roberto Lublinerman]

The correct link for the patches are 

https://gwt-review.googlesource.com/#/c/19000/

https://gwt-review.googlesource.com/#/c/19020/

On Wed, Jul 26, 2017 at 6:29 PM, Roberto Lublinerman <rlu...@google.com> wrote:

I have uploaded two patches for review to allow GWT to run under a Java 9 vm. 

https://gwt-review.googlesource.com/#/c/19000/

https://gwt-review.googlesource.com/#/c/19001/

The main issue is (as noted by James) that the class loading has been revamped and GWT can no longer assume that the class loaders are UrlClassLoaders.

The idea is first to be able to run GWT on a Java 9 vm compiling Java 8 sources.

On Wed, Jun 14, 2017 at 3:01 PM, James Nelson <Ja...@wetheinter.net> wrote:

It is my understanding that we use ASM to load the annotation attributes from source classes, and then create a Proxy to load member values / classes / enums off the classpath.  JDT is not involved at all (strange that it isn't...).  

The most likely reason we aren't using JDT to compile the annotaitons is we are really only using the parser, but not the linker (no code to emit classfiles).... 

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-contributors+unsu...@googlegroups.com.

[Goktug Gokdogan]

Colin, Thomas;

Since Google doesn't utilize all the code paths, I think it is important to manually test these in supported JVMs by our contributors. Any place that might have different classpath entries or places with potentially different class loaders probably good places to verify; like Eclipse plugins (superdev server, legacy dev server, junit web/dev mode).

To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-contributors+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/CAC7T7gnO-OFBnzEKyoF1mi3W4Tqn%2BjE_CDTrNpGU2i98yKPjhA%40mail.gmail.com.