Generator and Linker maintenance and changes

[Colin Alworth]

A few issues have come up (both in my own projects and in the issue tracker) where it seemed that continuing to maintain and update the GWT Generator and Linker types may be necessary. At least one was fairly low-hanging fruit (up for review at https://gwt-review.googlesource.com/#/c/14750/), but Daniel raised an important question that should be discussed: What is happening with GWT 2.8 Generators and Linkers.

In general the philosophy seems to be a mix of "if unmaintained, we'll phase it out", and "If you keep it up to date, we won't go out of our way to break it". Specifically in this case however, we'd like to begin the process of phasing out Generators (in favor of APT for your code generation needs) and Linkers (in favor of J2CL and the Closure Compiler ability to emit JS code and artifacts). 

---

My personal position is that GWT 2.8 is set for long-term maintenance - we expect point releases to keep it functional for teams not ready to make a big shift right away, for a variety of reasons (while I'll leave out to keep this short). GWT 2.8 is meant to be a bridge release for apps that wish to have modern Java support and JsInterop types at their disposal, and as they ready themselves for GWT3/J2CL, they can continue to function and stay up to date. The future will only be available beyond 2.8 and Generators, but in the meantime, they shouldn't be second class. 

The specific requests here are for static and default methods in AutoBeans or RequestFactory Proxies/Contexts. My goal (as stated in the bug reports) is not to add new features where getters/setters have their behavior altered, or get hooks before/after changes are made, but to let these methods be populated and called without breaking the compilation process in surprising ways. AutoBeans already support Categories, a sort of externally defined method that seems purpose-built to be eventually replaced by default methods - thats my goal in my own work here, outside the specific filed issues. I expect other features will be made easier to use too, such as lambda'd RPC callbacks, safehtmltemplates with logic and loops, and 

On the other hand, if we have published documents ready on what APIs will be available easily for linking, or for generating i18n/locale/device permutations through the use of external resources or alternate sets of constants in compiled code, my ground for complaints is limited to merely pointing out the existing stable code that could be better before it is ready to be ported. I don't think that this is going to be a hard point to sell.

---

So: in the spirit of always building better releases: Where does this argument go wrong? How much better off are we by shedding this weight now, and what great upgrade paths do we have? Is improving them but also marking them all as @Deprecated a good compromise? Please poke holes in this so that we can make GWT 2.8 an enduring release that is still able to be seen as a powerful tool for Java web developers!

Thanks,

Colin

[Jens]

Of course generators and linkers should be maintained, especially if it allows user code to be made more Java 8 friendly. I am pretty sure quite some GWT users will have large applications that will stay on 2.8 as long as possible because its too costly to rewrite these apps. However they will likely use Java 8 sooner or later and Java 8 friendly APIs should be added to GWT 2.8.x over the time.

I think its an error to assume that the current GWT compiler / generators / linkers should never be touched again just because Google is working on J2CL behind closed doors. J2CL is a different product and honestly GWT 3 will also be a different product if its based around J2CL.

So if Google engineers do not want to invest a huge amount of time in the GWT compiler anymore because they want to move all internal apps to J2CL in the long run then thats fine. However that should not prevent improvements to the current state of GWT. If I would be able to review the code of Colin I would do it but I do not have the knowledge of internal GWT compiler stuff. That actually brings me to my personal next point: It would be great to have an in-depth compiler documentation.

-- J.

[Thomas Broyer]

I think we should enable the use of Java 8-isms in interfaces (static and default methods) processed by existing generators; but we shouldn't enhance the generators to process those methods in any specific way to add new features.

For example, for AutoBeans, static methods should be allowed and not cause any compilation or runtime error; default methods should be prohibited, except possibly for non-accessor ones but even that is a non-negligible amount of work; but we shouldn't start processing static methods to create "categories", we already have an existing mechanism, default methods could possibly complement it.

For I18N for instance, static and default methods should be OK, the generator could simply ignore them (but what should it do if a default method is annotated with an I18N-related annotation such as @DefaultValue?)

Actually, default methods really need special treatment that need to be carefully thought out: either we ignore them, and the generated implementing class will just use the "inherited" default behavior, or we treat them like any other interface method and generate an implementation, or possibly some middle-ground depending on the signature of the method for instance.

So I think we should first disallow them (default methods), but put the small amount of work that make it fail with a proper error message.

Static methods, at first glance, don't have this problem, so could be "allowed by default".

What we should IMO refrain from doing is:

• add new generators / linkers
• add new features to existing generators / linkers

Every change to make them play better with Java 8-isms should be OK, particularly a) fixing the typemodel/typeinfo API to not lie about staticness/abstractness of the methods, and b) making generators fail the build with good error messages ("default methods can't be used in Xxx interfaces" rather than a cryptic error)

[Jens]

• add new features to existing generators / linkers

I think to some extend some new features should also be fine if they are related to Java 8. For example I could see GWT-RPC being improved to better support lambda/method references in the future.

-- J.

[Daniel Kurka]

Hi all,

thanks for Colin for writing this up.

To me this discussion is not so much about the particular feature (enhancing generators), but rather about us making smart choices.

We are already struggling to get 2.8 out the door (and making sure it will work with the next version of Guava). So I'd rather be lighten the work load and remove things that do not need to be done

I also feel that further enhancing generators dilutes the message we have been giving for the past year: Do not use generators anymore, use APTs.

I am also worried about possible problems that we will uncover once we allow these things to go into generators.

With the exception of GWT RPC all generators should be easily portable to an APT. I would love to see work being put into that direction rather than updating a system we are all not happy with.

-Daniel

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/1898a4b0-3eff-4e53-8aba-19e1851a646f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Paul Stockley]

I don't think migrating away from Generators to APT means you are forced to move off 2.8. In fact, I was hoping to move incrementally as each subsystem is migrated to become APT based. Then in the future, our code base would be close to moving to J2CL. In a couple of months or so I was planning on starting some prototyping, probably around replacing UiBinder and Resources.

On Tuesday, May 3, 2016 at 6:13:17 AM UTC-4, Daniel Kurka wrote:

Hi all,

thanks for Colin for writing this up.

To me this discussion is not so much about the particular feature (enhancing generators), but rather about us making smart choices.

We are already struggling to get 2.8 out the door (and making sure it will work with the next version of Guava). So I'd rather be lighten the work load and remove things that do not need to be done

I also feel that further enhancing generators dilutes the message we have been giving for the past year: Do not use generators anymore, use APTs.

I am also worried about possible problems that we will uncover once we allow these things to go into generators.

With the exception of GWT RPC all generators should be easily portable to an APT. I would love to see work being put into that direction rather than updating a system we are all not happy with.

-Daniel

On Tue, May 3, 2016 at 11:59 AM Jens <jens.ne...@gmail.com> wrote:

• add new features to existing generators / linkers

I think to some extend some new features should also be fine if they are related to Java 8. For example I could see GWT-RPC being improved to better support lambda/method references in the future.

-- J.

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-contributors+unsubscribe@googlegroups.com.

[Stephen Haberman]

I agree with Thomas and Daniel that it's good to avoid putting more "official" work into the generators, and encouraging people to move off.

But I think I agree more with Colin and Jens that, one way or another, people are using generators today, and will be using them for awhile (e.g. if they purposefully choose to stay on a GWT 2.8-based lineage), and it seems unfair to deny them improvements that they themselves want to make to the generators.

So, if Colin has a few patches waiting in the wings, or if people wander in with "hey, I made this incremental improvement to a generator", it seems "un-open source" to say "sorry, you should be doing something else".

I base my opinion mostly on the healthiest open source project I've ever seen: Apache Spark, where (at least in the early days), the committers were extremely wiling to accept patches about whatever itch you wanted to scratch (although still with tests/quality/etc.). (Granted, they were technically trying to win political points by "having as many contributors as Hadoop", but even if that was their core goal, the side affect of a healthy community was pretty nice.)

So, that's my only "post every 6 months" $0.02: if people have itches with the current generators, let them scratch it. That is what open source is about, IMO.

- Stephen

On Tue, May 3, 2016 at 3:40 PM Paul Stockley <pstoc...@gmail.com> wrote:

I don't think migrating away from Generators to APT means you are forced to move off 2.8. In fact, I was hoping to move incrementally as each subsystem is migrated to become APT based. Then in the future, our code base would be close to moving to J2CL. In a couple of months or so I was planning on starting some prototyping, probably around replacing UiBinder and Resources.

On Tuesday, May 3, 2016 at 6:13:17 AM UTC-4, Daniel Kurka wrote:

Hi all,

thanks for Colin for writing this up.

To me this discussion is not so much about the particular feature (enhancing generators), but rather about us making smart choices.

We are already struggling to get 2.8 out the door (and making sure it will work with the next version of Guava). So I'd rather be lighten the work load and remove things that do not need to be done

I also feel that further enhancing generators dilutes the message we have been giving for the past year: Do not use generators anymore, use APTs.

I am also worried about possible problems that we will uncover once we allow these things to go into generators.

With the exception of GWT RPC all generators should be easily portable to an APT. I would love to see work being put into that direction rather than updating a system we are all not happy with.

-Daniel

On Tue, May 3, 2016 at 11:59 AM Jens <jens.ne...@gmail.com> wrote:

• add new features to existing generators / linkers

I think to some extend some new features should also be fine if they are related to Java 8. For example I could see GWT-RPC being improved to better support lambda/method references in the future.

-- J.

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/1898a4b0-3eff-4e53-8aba-19e1851a646f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "GWT Contributors" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/ff572616-6851-464d-8b9c-4bfa8df031e7%40googlegroups.com.

[Colin Alworth]

Thanks everyone. Since we don't really have much of a consensus on this, perhaps we can take some middle ground here?

I certainly agree that we want people to move away from generators in general, though without finished solutions for some of the tricks generators can do, this will be tricky. Additionally, APT is not exactly a terribly user-friendly option - while I have managed to port a very small GWT generator, I wasn't enjoying myself, and am unsure how a larger project would be addressed and sanity retained. I have heard great things about JavaPoet, but have not yet had the time to learn enough of it to make another attempt.

Neither, apparently, has anyone else - AutoBeans/RequestFactionr, I18n, UiBinder, ClientBundle, and even the base UserAgent wiring are all generator-based - in fact, my SafeHtmlTemplates generator is the only one that I'm aware of that has been ported, at least publicly. If weren't not prepared to set an example on how to generally solve this, I'm unsure how strong of a position we can hold. 

Along the same lines, I agree that official work toward generators doesn't make any sense, and that instead work generally should be directed toward general solutions to our property/permutation and linker problems (features which will vanish with APT and J2CL as far as I can tell, with nothing yet set up to replace them). I'll generally agree that we should stop updating GWT-provided generators, though we certainly haven't yet stopped - over the past 12 months at a very brief glance at history, Thomas made changes in a year ago and in October, Julien made changes in June, October, December, I made changes in Feb. Some of these were purely bug-fixes, but about half of the changes actually appear to be introducing new features. It doesn't appear to be our position that further enhancing generators is contrary to our message.

My proposal is that first, the change I've submitted only removes unneeded (and arguably broken/meaningless) code - I don't see a position where it makes sense to keep it and refuse the patch.

Second, future work can be discussed, but actively forbidding contributions for fixing or improving existing code would require deprecating these classes and aiming to have them moved to their own jars, perhaps to be maintained externally. As opposed to my first point, I can see some wiggle room in this policy, but refusing changes merely because they improve a generator seems silly at best, especially when we have actively improved existing generators up until now. (Most of the examples I can come up with are pretty gratuitous and don't really make sense, but static or default methods in place of categories seems a natural fit for a hacked in feature that can be replaced with the real thing.)

Finally, if we are to be serious about this, as a GWT user, contributor, and steering committee member, I have to expect to see usable solutions in the near future to move away from these tools. This has to include tutorials on setting up APT to play nicely with SDM in reasonable development environments, a pathway for non-GWT generators to move in, and clear evidence that we are not simply abandoning all existing generators that are part of GWT itself. More than any other thing we are doing around this topic, _this_ is our messaging problem, and efforts to solve it there will be a lot clearer for users than closing bugs as WONTFIX or refusing patches.

The first is to allow the community the freedom to continue to improve before new tools are available, the second recognizes that contributor time is not a zero sum game whereby refusing commits will make other work magically work faster, and the third brings these new tools into the limelight as soon as possible. Ten years of constantly improving generators aren't going to be counteracted by ignoring fixes - we need a better approach. 

[Predrag Pesic]

Great post Colin. You are "the voice of the people" here. I couldn't agree more with the statement: "Finally, if we are to be serious about this, as a GWT user, contributor, and steering committee member, I have to expect to see usable solutions in the near future to move away from these tools."

[Thomas Broyer]

On Monday, May 9, 2016 at 5:39:36 AM UTC+2, Colin Alworth wrote:

Thanks everyone. Since we don't really have much of a consensus on this, perhaps we can take some middle ground here?

I certainly agree that we want people to move away from generators in general, though without finished solutions for some of the tricks generators can do, this will be tricky. Additionally, APT is not exactly a terribly user-friendly option - while I have managed to port a very small GWT generator, I wasn't enjoying myself, and am unsure how a larger project would be addressed and sanity retained. I have heard great things about JavaPoet, but have not yet had the time to learn enough of it to make another attempt.

JavaPoet is orthogonal to that; it's a tool to help generate Java code; it can be used in GWT Generators today, or in annotation processors, or in any other tool that needs to generate Java code (e.g. generate Java classes from protobuf definitions, from the command-line or a Maven plugin: https://github.com/square/wire )

If you want tools to make it easier to work with/in annotation processors, I'd recommend Auto-Common from Google: https://github.com/google/auto

 

Neither, apparently, has anyone else - AutoBeans/RequestFactionr,

I've been thinking about this for years (literally), but don't have the bandwidth and motivation (part of the work would also be to replace dynamic java.lang.reflect.Proxy in the JVM with generated implementations; reusing client-side code as much as possible; that was my main motivation initially actually, to make it easier to debug things in the JVM, not to replace the GWT generator)

 

I18n, UiBinder, ClientBundle,

Those have dependencies on non-Java files, so annotation processing is not really a good fit; it can (and would) work, but incremental build tools would need to re-trigger Java compilation when non-Java files change. This is the tricky part (easy with Gradle –a one-liner–, trickier with Maven for instance)

 

and even the base UserAgent wiring are all generator-based

That one would likely move to JSNI/JsInterop and system properties actually; but it's currently too linked to the concept of "permutations" to be efficiently refactored I believe.

 

- in fact, my SafeHtmlTemplates generator is the only one that I'm aware of that has been ported, at least publicly.

Also https://github.com/tbroyer/gwt-places (announced here: https://groups.google.com/d/topic/google-web-toolkit/jbOWVAYGwqU/discussion)

 

If weren't not prepared to set an example on how to generally solve this, I'm unsure how strong of a position we can hold. 

Along the same lines, I agree that official work toward generators doesn't make any sense, and that instead work generally should be directed toward general solutions to our property/permutation and linker problems (features which will vanish with APT and J2CL as far as I can tell, with nothing yet set up to replace them). I'll generally agree that we should stop updating GWT-provided generators, though we certainly haven't yet stopped - over the past 12 months at a very brief glance at history, Thomas made changes in a year ago and in October, Julien made changes in June, October, December, I made changes in Feb. Some of these were purely bug-fixes, but about half of the changes actually appear to be introducing new features. It doesn't appear to be our position that further enhancing generators is contrary to our message.

My proposal is that first, the change I've submitted only removes unneeded (and arguably broken/meaningless) code - I don't see a position where it makes sense to keep it and refuse the patch.

+1