"GWT RPC" and its role in the future of GWT
616 views
Skip to first unread message
Colin Alworth
Sep 24, 2018, 11:54:57 AM9/24/18
to GWT Contributors
As has been discussed previously in various venues, I have a mostly-working port of GWT-RPC that drops the use of legacy GWT APIs like Generators and JSNI, future-proofing both beyond GWT 2.x, and making it functional in other runtimes (JVM, Android, TeaVM, etc). It isn't perfect, and isn't quite complete, but before I work on softening those rough edges, I wanted to have a conversation here about whether it should remain a part of GWT going forward, and be migrated to the org.gwtproject package/groupId, and hosted at github.com/gwtproject.
There are two main reasons I see that would lead us to avoid doing this - RPC doesn't enjoy the popularity it once did, and the use of my current draft implementation could be considered to be far enough a way from the old Generator implementation that it might seem to be a different tool entirely.
* RPC-based mechanisms in general seem to be out of favor (though with gRPC gaining popularity, the idea might be coming back), with REST (whether JSON, Protobuf, or whatever else you like) leading in one direction and query mechanisms like GraphQL in another. I don't want to debate the pros and cons of these, or even to break down how "GWT-RPC" can be treated like a RPC or query mechanism, only to note that if RPC is considered by itself to be "too old-school", that might be are a reason to do nothing more than provide upgrade paths *within GWT itself*.
* In order to support being run by annotation processors in a performant way, my particular implementation all but mandates that the RPC interfaces and beans exist in their own "shared" project, apart from either client or server. Arguably this is a best practice to begin with, but it hasn't actually been required. Additionally, to support external dependencies (either other projects within your own product, or outside artifacts like the JRE or Guava that might have serializable types you'd be interested in) a manifest of some kind is required to point at the classes that should be considered while compiling. Presently this is supported through annotation processor arguments, but other mechanisms could be supported as well, but nothing as simple and obvious as "just assume that all possible subclasses will work" that GWT2's RPC supports. Reading private members isn't supported either - it is assumed that either those members will be non-private, or that getters and setters will be provided.
These two points probably need to be weighed relative to each other as well: we could decide that as long as the migration path for 90% of projects out there is simple enough that despite being out of favor, at least part of this tool should end up in org.gwtproject for now. Or, perhaps despite its continued popularity, the project is different enough that it shouldn't be included in org.gwtproject merely due to having its roots in "classic" GWT-RPC.
--
A quick technical overview - I don't want to get bogged down in this discussion just yet, just providing a summary to inform the topic at hand (though I'll be happy to provide more details as it helps to make this decision).
The project is composed of two annotation processors, and is intended to support a variety of use cases out of the box:
* old pipe-delimited string format, or TypedArray/ByteBuffer based encoding (including not writing Strings to the typedarray at all but leaving them in a plain JS array for cheaper copying between the window and workers)
* transport tools for page/worker, websockets, and fetch/xhr calls
* interface-based approach to handling streams allowing other implementations to be added or further specialized, and transport mechanisms to be built per-platform so interfaces, beans, and generated serializers can be reused between platforms, easily enabling adding new platforms, or devising your own stream formats or transport mechanisms.
At runtime, the serialization stream interface is nearly identical to the old one, with the addition of one additional method when reading from the stream, to generalize mitigation on a class of attack that GWT-RPC historically only handled in the servlet. No reflection is used in the server implementation for de/serializing fields in objects, but it is instead assumed that either a field serializer has been created by the code generation mechanism, or a custom field serializer has been provided.
Instead of a .gwt.rpc file on the server to enumerate types that are serializable and a JSNI map on the client, a single class is generated to describe this for each platform - server gets one, and client gets another for example, and these are almost certainly compliments of each other. Avoiding the JSNI map may pose a small risk at increasing generated code size, but I've tried to mitigate this by generating specific serializer types which describe exactly which operations are supported, so that any pruning step when compiling code can delete as much code as possible (and support weird cases like only reading a type sent from the server, but both reading and writing that same type when communicate with a web worker).
The SerializableTypeOracleBuilder remains at the heart of the code generation mechanism, updated to use TypeMirrors and Elements instead of GWT's JType hierarchy. Instead of being invoked by the RemoteService code generator, this is instead triggered by its own annotation, expected to be found on an interface with helper methods to add various types to an existing stream, or decode a stream and read expected data. This interface in turn is expected to be generated by another, higher-level annotation processor, based on the specific use case (XMLHttpRequest, WebSocket, etc). The built-in "high level" processor implementation takes an interface full of asynchronous interface methods and generates boilerplate to serialize each method's arguments as necessary. Presently, there is one such high level processor, which relies on each interface it sees to either offer specifics on a base class to extend, or extend another interface which in turn provides those specifics - I've planned to add a second ("high-er" level?) processor, just for the purpose of allowing a synchronous interface to be provided and implemented on the server, and the async instance generated automatically.
Potential future work:
* "Opaque serialization" for server-originating objects to sign them and return them to the server unchanged, for use in eventually implementing the JPA features (which otherwise have been left out so far due to the easy attack vector it presents).
* More options for automatic de/serialization, possibly including some kind of "violator pattern" tool, support for builder/factories, @ConstructorProperties for final fields, etc.
* Allow disabling polymorphism in whole or in part to save bytes over the wire (and more easily support non-Java remote endpoints).
* "Allocation-free" (or at least "reduced allocation") serialization/deserialization, for memory/allocation constrained situations, allowing bean-like data to be read or written, but handled on at least one endpoint on a field-by-field basis.
--
So, as I continue to polish this and prepare to push builds for others to begin trying, I would like to start making packages and groupIds consistent. If the direction seems to be that it doesn't belong in gwtproject, that's fine by me, plenty of other popular GWT libraries live outside the official project, but if we think it might belong here, then to avoid later churn in downstream projects, I'd like to start adopting the org.gwtproject namespace right away.
There are two main reasons I see that would lead us to avoid doing this - RPC doesn't enjoy the popularity it once did, and the use of my current draft implementation could be considered to be far enough a way from the old Generator implementation that it might seem to be a different tool entirely.
* RPC-based mechanisms in general seem to be out of favor (though with gRPC gaining popularity, the idea might be coming back), with REST (whether JSON, Protobuf, or whatever else you like) leading in one direction and query mechanisms like GraphQL in another. I don't want to debate the pros and cons of these, or even to break down how "GWT-RPC" can be treated like a RPC or query mechanism, only to note that if RPC is considered by itself to be "too old-school", that might be are a reason to do nothing more than provide upgrade paths *within GWT itself*.
* In order to support being run by annotation processors in a performant way, my particular implementation all but mandates that the RPC interfaces and beans exist in their own "shared" project, apart from either client or server. Arguably this is a best practice to begin with, but it hasn't actually been required. Additionally, to support external dependencies (either other projects within your own product, or outside artifacts like the JRE or Guava that might have serializable types you'd be interested in) a manifest of some kind is required to point at the classes that should be considered while compiling. Presently this is supported through annotation processor arguments, but other mechanisms could be supported as well, but nothing as simple and obvious as "just assume that all possible subclasses will work" that GWT2's RPC supports. Reading private members isn't supported either - it is assumed that either those members will be non-private, or that getters and setters will be provided.
These two points probably need to be weighed relative to each other as well: we could decide that as long as the migration path for 90% of projects out there is simple enough that despite being out of favor, at least part of this tool should end up in org.gwtproject for now. Or, perhaps despite its continued popularity, the project is different enough that it shouldn't be included in org.gwtproject merely due to having its roots in "classic" GWT-RPC.
--
A quick technical overview - I don't want to get bogged down in this discussion just yet, just providing a summary to inform the topic at hand (though I'll be happy to provide more details as it helps to make this decision).
The project is composed of two annotation processors, and is intended to support a variety of use cases out of the box:
* old pipe-delimited string format, or TypedArray/ByteBuffer based encoding (including not writing Strings to the typedarray at all but leaving them in a plain JS array for cheaper copying between the window and workers)
* transport tools for page/worker, websockets, and fetch/xhr calls
* interface-based approach to handling streams allowing other implementations to be added or further specialized, and transport mechanisms to be built per-platform so interfaces, beans, and generated serializers can be reused between platforms, easily enabling adding new platforms, or devising your own stream formats or transport mechanisms.
At runtime, the serialization stream interface is nearly identical to the old one, with the addition of one additional method when reading from the stream, to generalize mitigation on a class of attack that GWT-RPC historically only handled in the servlet. No reflection is used in the server implementation for de/serializing fields in objects, but it is instead assumed that either a field serializer has been created by the code generation mechanism, or a custom field serializer has been provided.
Instead of a .gwt.rpc file on the server to enumerate types that are serializable and a JSNI map on the client, a single class is generated to describe this for each platform - server gets one, and client gets another for example, and these are almost certainly compliments of each other. Avoiding the JSNI map may pose a small risk at increasing generated code size, but I've tried to mitigate this by generating specific serializer types which describe exactly which operations are supported, so that any pruning step when compiling code can delete as much code as possible (and support weird cases like only reading a type sent from the server, but both reading and writing that same type when communicate with a web worker).
The SerializableTypeOracleBuilder remains at the heart of the code generation mechanism, updated to use TypeMirrors and Elements instead of GWT's JType hierarchy. Instead of being invoked by the RemoteService code generator, this is instead triggered by its own annotation, expected to be found on an interface with helper methods to add various types to an existing stream, or decode a stream and read expected data. This interface in turn is expected to be generated by another, higher-level annotation processor, based on the specific use case (XMLHttpRequest, WebSocket, etc). The built-in "high level" processor implementation takes an interface full of asynchronous interface methods and generates boilerplate to serialize each method's arguments as necessary. Presently, there is one such high level processor, which relies on each interface it sees to either offer specifics on a base class to extend, or extend another interface which in turn provides those specifics - I've planned to add a second ("high-er" level?) processor, just for the purpose of allowing a synchronous interface to be provided and implemented on the server, and the async instance generated automatically.
Potential future work:
* "Opaque serialization" for server-originating objects to sign them and return them to the server unchanged, for use in eventually implementing the JPA features (which otherwise have been left out so far due to the easy attack vector it presents).
* More options for automatic de/serialization, possibly including some kind of "violator pattern" tool, support for builder/factories, @ConstructorProperties for final fields, etc.
* Allow disabling polymorphism in whole or in part to save bytes over the wire (and more easily support non-Java remote endpoints).
* "Allocation-free" (or at least "reduced allocation") serialization/deserialization, for memory/allocation constrained situations, allowing bean-like data to be read or written, but handled on at least one endpoint on a field-by-field basis.
--
So, as I continue to polish this and prepare to push builds for others to begin trying, I would like to start making packages and groupIds consistent. If the direction seems to be that it doesn't belong in gwtproject, that's fine by me, plenty of other popular GWT libraries live outside the official project, but if we think it might belong here, then to avoid later churn in downstream projects, I'd like to start adopting the org.gwtproject namespace right away.
Stefano Ciccarelli
Oct 17, 2018, 8:51:40 AM10/17/18
to google-web-tool...@googlegroups.com
I hope to have GWT RPC in the official repository, I think GWT RPC is far most the best choice for a consistent Java development.
--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/ba0b000c-2228-4897-94a0-cc9faa42e60f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Stefano Ciccarelli
YAP Division
/ Director
stefano.c...@mmbsoftware.it
M.M.B. s.r.l.
via Granarolo, 177/7 - 48018 Faenza (RA) - Italy
tel. +39.0546.637711 - fax +39.0546.46077
www.mmbsoftware.it - in...@mmbsoftware.it
Le informazioni contenute in questa comunicazione sono riservate e destinate esclusivamente alla/e persona/e o all'ente sopra indicati. E' vietato ai soggetti diversi dai destinatari qualsiasi uso, copia, diffusione di quanto in esso contenuto sia ai sensi dell'art. 616 c.p., sia ai sensi del DL n. 196/03. Se questa comunicazione Vi e' pervenuta per errore, Vi preghiamo di rispondere a questa e-mail e successivamente cancellarla dal Vostro sistema.
Learner Evermore
Oct 17, 2018, 9:46:42 AM10/17/18
to GWT Contributors
As I noted multiple times before, such as here (didn't search more):
RPC is intergral to the value of GWT (or any similar solution). This should not be confused with public APIs - it isn't good for that and should not be targeting that use case. What is it best for (as in - there is nothing better) is linking the parts of otherwise the same application that are separated by the network. Original GWT had/has a lot of problems. From what I've seen, those who hate it focused on either these or misused it for a public API - which it isn't for. The problems themselves aren't unsolvable and the benefits are great.
I know this is largely preaching to the choir, but here it is again - try to find another approach that ticks *all* of the following boxes (there are quite a few that don't tick all):
- Can use the same language (and code) on both sides of the network divide.
- Do not need to have separate implementations of "DTOs" on each side. Common objects can be communicated in either direction and use the same code. DTOs are an antipattern in at least this case (and I would argue most others). Typical arguments revolve around controlling what is exposed (this may be a public API and security concern but could be addressed with custom/filtering serializers and, if enhanced, annotations; the caller could also specify how much is needed using something like projection specifications, JPA entity graphs, etc) and about "forcing one to think hard about what is communicated, for performance reasons" (thinking is always required, if you don't use it in one approach, then don't in another either). DTOs create a huge issue with code duplication, having to code, recode, code again and then some copying of object graphs to/from DTOs where other approaches (such as mentioned) would actually do better. DTOs also cause a loss of logic - methods in main classes that could and should work everywhere - or cause duplication of these.
- Natural support for preserving instance cardinality - referencing a single object multiple times will not yield multiple instances of it "on the other side". Cyclic references supported.
- Does not require "decorating" the code with anything for it to work. This translates to the ability to use 3rd party code/libraries and communicate their objects that are already marked for (Java) serialization. Step away from this and you enter the world where you cannot easily use two libraries that depend on different (non-Java) serialization. In other words, the only reliable standard is "Java serialization".
- Compact payloads.
- Protocol details hidden away, abstracted, so that it can evolve and be enhanced over time without having to redo any of the client or server code.
I don't care what is "in" vs not in terms of hype. We use things that have merit and get the job done. If we get it, use will follow. Maybe "RPC" as a term isn't popular any more. I personally don't care about it, but one easy way around this is to break away from that name and use a different one that better aligns with intended purpose. Brainstorming seed terms: application/network bridge, (Java) Object Graph transporter/Replicator (OGRe? :) ), Split Application Linking Transporter (SALT?) - you may not think much of salt but try going without it at all...
Relja Pcela
Oct 17, 2018, 10:36:23 AM10/17/18
to GWT Contributors
I have never seen any other solution (for GWT) that is even close to RPC. It doesn't matter for me if it's part of GWT or it will be a separate library. You can change the name if it's not popular any more why not ? Not only that, I really think it's better to change the name of future GWT (v3) to something else, as "GWT "is not popular any more. In contrary it's very unpopular last few years and almost everyone is trying to escape from it.
Colin Alworth
Oct 17, 2018, 11:09:05 AM10/17/18
to GWT Contributors
At the risk of sounding like I'm telling off the people who agree with me that GWT-RPC has value, the purpose of this thread isn't to establish that RPC will live on in some form, but to discuss how it will be maintained. It _will_ still exist in some form, either as part of the "official GWT project", or as an external "spiritual successor", and will largely provide the same functionality we all either love or hate. If you'd like to support that goal directly, please contact me off-list.
I'm looking for some consensus among maintainers and contributors if something that looks like the GWT-RPC of old belongs under the general umbrella of GWT, even if it has slightly different setup and configuration, and grows to be general enough to function on non-GWT clients, servers. Of the discussed modules being migrated, this one seems to have the most "diverse" opinions around it, so I started this thread to take our collective temperature on keeping the "GWT" name on it and supporting it. Barring any strong disagreement, I'm going to take a day and renamespace everything in the next few days and open the repository up for review and comments, before starting to actually migrate it to the gwtproject github.
Learner Evermore
Oct 17, 2018, 12:48:58 PM10/17/18
to GWT Contributors
With respect to location/name I personally do not care. However, you may want to consider whether people could confuse the "new" RPC with the old one and expect the same things to work. Can they work together? If they can, maybe they should not be called the same and placed in the same location. Will it (ever) do what GWT 2.x RPC did? If one thinks about it, GWT 3 really isn't anything GWT 2 is and the name is only adding to, not helping with confusion... Additionally, your new stuff may be better and only provide "migration and some compatibility" with GWT 2.x RPC.
Will this evolve with "GWT" or will it evolve separately? Does it require hooks or it it standalone? Does it even require GWT or can it be, technically, used with something else similar to GWT? If not today, it may be reasonable to assume that it is possible it could evolve into something that isn't GWT specific.
Finally, a lot of people, myself included, have developed an allergy for things coming from Google due to the rock solid record of breaking compatibility with everything (GWT? Angular? ...). While there are some valid reasons for this, there are many more against it and ways to deal with it that Google never cared about (I am not saying it should in any way, I am just merely stating the fact). I am actually in a position to select and start developing a new UI using a technology of choice in a company that already uses (used?) GWT but cannot, with a straight face, tell anyone to use anything that begins with "G" because we'd be in it for the long run, however much I'd like to. As such, we will probably have to select something else right now :(
My completely (admittedly, subconscious) irrational hope is to be able to use something like your new RPC with something other than GWT... But that would still mean Java on the client side and, even though I actually despise JavaScript for numerous reasons, what Oracle is doing is not helping Java's future either. Don't get me wrong - things of that size can't die quickly, but the trends and new development may shift quickly to something else.
Peter Donald
Oct 17, 2018, 6:51:47 PM10/17/18
to google-web-tool...@googlegroups.com
Hi,
IMNSHO I think that the best idea would be to host the new "gwt-rpc"
project outside the project have a completely separate moniker -
something that offers easy iconography (Mmmm ... TigerRPC?). Once it
is stabilised and offers a credible alternative to current gwt-rpc, at
that point I think it would be good to start documenting it as the
spiritual successor of gwt-rpc.
The advantage of this approach is that you get to iterate much faster
with a lot less overhead. You also potentially get developers involved
that have the time/interest to work on that specific project.
--
Cheers,
Peter Donald
IMNSHO I think that the best idea would be to host the new "gwt-rpc"
project outside the project have a completely separate moniker -
something that offers easy iconography (Mmmm ... TigerRPC?). Once it
is stabilised and offers a credible alternative to current gwt-rpc, at
that point I think it would be good to start documenting it as the
spiritual successor of gwt-rpc.
The advantage of this approach is that you get to iterate much faster
with a lot less overhead. You also potentially get developers involved
that have the time/interest to work on that specific project.
--
Cheers,
Peter Donald
Thomas Broyer
Oct 18, 2018, 3:17:53 AM10/18/18
to GWT Contributors
Everyone knows I don't like GWT-RPC (specifically related to some of the points "Learner" listed), but I cannot deny that it's been a big selling point of GWT over the years.
For that reason alone, I won't oppose to it being in the org.gwtproject namespace. So let's do that (but IMO make it more a "compatibility lib" for those coming from GWT 2, replacing its use in tutorials and samples with something else -- RESTish JSON APIs are easy nowadays with JsInterop, so probably use that instead, as that makes it easier to understand that GWT could be used with any backend)
For that reason alone, I won't oppose to it being in the org.gwtproject namespace. So let's do that (but IMO make it more a "compatibility lib" for those coming from GWT 2, replacing its use in tutorials and samples with something else -- RESTish JSON APIs are easy nowadays with JsInterop, so probably use that instead, as that makes it easier to understand that GWT could be used with any backend)
Learner Evermore
Oct 18, 2018, 7:16:36 AM10/18/18
to GWT Contributors
I strongly disagree that this should be just a compatibility library. Thomas, will all due respect, if you don't find it useful for your work it does not mean others don't and REST APIs are seeing their own sunset as well and JavaScript approaches come nowhere near where they need to be to cover the points I listed. Even if I could expose APIs of any other kind (REST, GraphQL, JavaScript/JSON or not) the "GWT RPC (NG)" still has a great place, perhaps parallel (in addition to them) for all the reasons listed. RPC was what completed the GWT value proposition - without it it isn't much better (if at all better or even close to) than a number of other solutions out there and doesn't fulfill its own promise. Instead of performant abstraction we'd have a thin translator requiring the developers to continue to know and think about both Java and JavaScript world of things.
JavaScript is an *awful* language and system in any of its incarnation for anything complex. Arguments that it is the best because of performance, size and library availability are the same as those used for assembly and Fortran a while back and are pointless, especially due to the prospect of WebAssembly. In my opinion, GWT should completely abstract that platform and be able to, eventually, leverage whatever is the best for it without impacting the apps built using it. As such, abstracting JS away as much as possible is a benefit just like it is using the paradigms of the core language of GWT - Java.
Learner Evermore
Original Message
From: t.br...@gmail.com
Sent: October 18, 2018 3:17 AM
To: google-web-tool...@googlegroups.com
Reply to: google-web-tool...@googlegroups.com
Subject: [gwt-contrib] Re: "GWT RPC" and its role in the future of GWT
Everyone knows I don't like GWT-RPC (specifically related to some of the points "Learner" listed), but I cannot deny that it's been a big selling point of GWT over the years.
For that reason alone, I won't oppose to it being in the org.gwtproject namespace. So let's do that (but IMO make it more a "compatibility lib" for those coming from GWT 2, replacing its use in tutorials and samples with something else -- RESTish JSON APIs are easy nowadays with JsInterop, so probably use that instead, as that makes it easier to understand that GWT could be used with any backend)
--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/7785d7c8-4f2e-4d95-9ea9-b287c0b65544%40googlegroups.com.
Goktug Gokdogan
Oct 18, 2018, 5:39:13 PM10/18/18
to google-web-toolkit-contributors
I understand GWT-RPC would be interesting at least for some people but that still doesn't make it a trustworthy, decent RPC system and has fundamental flaws which we have enumerated couple of times already.
I see it could be potentially replaced with something more sane and useable but like some others have already suggested it seems like the replacement is better to evolve and mature independently and later evaluated to see if it is part of compatibility lib or something we can recommend for future development.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/2shv5gsgoe900hgcebeo5p86.1539861391103%40gmail.com.
Learner Evermore
Oct 18, 2018, 6:28:49 PM10/18/18
to google-web-toolkit-contributors
I don't see this as a compatibility thing. That may be impossible to achieve 100% due to generators and custom serializers among other reasons. I see this as retaining the goal/spirit but doing it properly. I can facilitate easy migration.
Learner Evermore
To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/CAN%3DyUA0WfeX%3DpJ_BDNf5MBwD%2Bm2-107o5DsbkWqkgx4bevaErw%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages