Protocol Buffers for GWT: Issue 2649

[V.B.]

For those of you still waiting for Issue 2649 to be completed, I opened a parallel feature request on the Protocol Buffers issue tracker (Issue 431). Be sure to star both issues. Hopefully, the missing feature will be completed soon - it has been 4+ years since the original request.

[Andrea Boscolo]

It is indeed a nice feature to have out of the box.

Lately I've been using a really nice library to use protobuf-like messages with GWT. It can generate jso implementation of server-side proto-generated classes, or you can even easily define your own class generator using StringTemplate (e.g., to generate RequestFactory Proxy interfaces). And much more. Take a look http://code.google.com/p/protostuff/ Of course it is not entirely Protobuf, see http://code.google.com/p/protostuff/wiki/ThingsYouNeedToKnow

There is also http://code.google.com/p/protobuf-gwt/ by Broyer although I do not know its current status (but I guess, watching the changes, it is suspended since a couple of years).

[V.B.]

Hi Andrea,
    Thanks for the interest, and for your thoughtful post!
Given our particular requirements, the reason we ruled out protostuff and protobuf-gwt is that we need something officially supported by Google. I saw from the GWT Wishlist that we are not the only ones waiting for baked-in Protocol Buffer support in GWT. My hope is that the Protobuf Team would pick up where the GWT Team left off. Or, perhaps Vaadin and Sencha can make this happen now that they are part of the steering committee.
    Our project spans both mobile and web. We have native Android clients, native iOS clients, and GWT web clients, all serviced by AppEngine. Protocol Buffers fit our needs perfectly!... except for the glaring lack of official support in GWT.

[Darren Robinson]

I just starred both issues. I really hope to see support for this issue also. I would much prefer to use Protocol Buffers over JSON or XML for web applications. Protocol Buffers are so much more elegant.

[Goktug Gokdogan]

GWT's built-in library space is already bloated and getting harder to maintain. Inside GWT, we need to do less but do better.

And good thing is, a lot of stuff doesn't need to be baked in; like the protocol buffers; it is a valuable feature but I personally don't see GWT having it as a first class citizen; at least in the near term.

On the other hand we can definitely help out on removing any blockers and support anybody who would like to implement it as a 3rd party library.

It looks like there is enough demand, why don't you guys start a project for that?

On Fri, Dec 21, 2012 at 2:28 PM, Darren Robinson <agen...@gmail.com> wrote:

I just starred both issues. I really hope to see support for this issue also. I would much prefer to use Protocol Buffers over JSON or XML for web applications. Protocol Buffers are so much more elegant.

--
You received this message because you are subscribed to the Google Groups "Google Web Toolkit" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-web-toolkit/-/S28zwRl68GcJ.

To post to this group, send email to google-we...@googlegroups.com.
To unsubscribe from this group, send email to google-web-tool...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.

[Thomas Broyer]

On Saturday, December 22, 2012 2:50:00 AM UTC+1, Goktug Gokdogan wrote:

GWT's built-in library space is already bloated and getting harder to maintain. Inside GWT, we need to do less but do better.

And good thing is, a lot of stuff doesn't need to be baked in; like the protocol buffers; it is a valuable feature but I personally don't see GWT having it as a first class citizen; at least in the near term.

On the other hand we can definitely help out on removing any blockers and support anybody who would like to implement it as a 3rd party library.

+1

Wondering what Google is doing though ;-)

Are you somehow generating, say, RequestFactory ValueProxy-s from .proto files? It looks like what BobV was planning.

 

It looks like there is enough demand, why don't you guys start a project for that?

Feel free to continue protobuf-gwt where I left it, I can add committers/owners if anyone's interested (let's discuss it in https://groups.google.com/d/forum/protobuf-gwt then)

I wonder whether it would make sense to support the binary serialization using TypedArrays in browsers that support them.

As for generating client code, the patch I made was never merged into protobuf, so wrt codegen, I think I'd rather use the approach taken by Wave: use protoc to generate "normal" Java classes, then load the classes and use the protobuf "reflection" API to generate GWT-specific implementations.

[V.B.]

Hi Goktug and Thomas,
   Thank you for your responses. Thomas, I recognize your name immediately. Goktug, I see that you are also a member of the GWT team. Obviously, you are both in a position to know how this feature would best be implemented. If you say that it is better for Protocol Buffer support to be external to GWT (rather than directly baked into GWT), then I can accept that at face value. In either case, we need a solution that we know will be actively maintained by Google.

   Currently, we are developing an educational platform (Android client, iOS client, and GWT web client, hosted on AppEngine) using grants from the National Science Foundation, and from Google itself. We were contacted by Sprint and may partner with them for a large scale (mobile client) deployment of our platform to classrooms here in the United States. As new features and bugfixes are added to the protobuf project, we need to know that Protocol Buffer support for GWT will stay current, for the sake of the teachers and parents that will depend on our web client. We use Protocol Buffers directly in Android, iOS, and AppEngine. But currently, we are stuck maintaining a parallel set of GWT-compatible classes that are translated from Protocol Buffers in AppEngine before being sent to the GWT web client, and vice versa. This is quite cumbersome and will only get worse as we continue development. Naturally, the ideal situation is that the GWT web client can handle the exact same Protocol Buffers that the mobile clients already send and receive. No manual translation in the middle.

    Thomas, I really like your suggestion about using TypedArrays (in browsers that support them) for binary serialization/deserialization, which makes perfect sense since we are dealing with Protocol Buffers. If requiring our users to have only up-to-date modern browsers is the tradeoff, then that is a small price to pay. We'll take it.

Thank you again for your responses.

[Goktug Gokdogan]

On Fri, Dec 21, 2012 at 8:20 PM, Thomas Broyer <t.br...@gmail.com> wrote:

On Saturday, December 22, 2012 2:50:00 AM UTC+1, Goktug Gokdogan wrote:

GWT's built-in library space is already bloated and getting harder to maintain. Inside GWT, we need to do less but do better.

And good thing is, a lot of stuff doesn't need to be baked in; like the protocol buffers; it is a valuable feature but I personally don't see GWT having it as a first class citizen; at least in the near term.

On the other hand we can definitely help out on removing any blockers and support anybody who would like to implement it as a 3rd party library.

+1

Wondering what Google is doing though ;-)

Are you somehow generating, say, RequestFactory ValueProxy-s from .proto files? It looks like what BobV was planning.

 

There is compiler plugin that generates lightweight GWT emulations of the messages based on JavaScript overlays.

On top of that, there is a custom RPC mechanism that mimics GWT RPC. Wire format is like JSON-RPC where 'params' is js array encoded protobuf.

I think there has been multiple solutions but I guess this one got wider acceptance. Unfortunately, nobody actively working on it and not ready to open-source.

 

It looks like there is enough demand, why don't you guys start a project for that?

Feel free to continue protobuf-gwt where I left it, I can add committers/owners if anyone's interested (let's discuss it in https://groups.google.com/d/forum/protobuf-gwt then)

I wonder whether it would make sense to support the binary serialization using TypedArrays in browsers that support them.

As for generating client code, the patch I made was never merged into protobuf, so wrt codegen, I think I'd rather use the approach taken by Wave: use protoc to generate "normal" Java classes, then load the classes and use the protobuf "reflection" API to generate GWT-specific implementations.

--

You received this message because you are subscribed to the Google Groups "Google Web Toolkit" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-web-toolkit/-/vj4wP3pdv3MJ.

[Thomas Broyer]

On Friday, December 28, 2012 12:33:19 AM UTC+1, Goktug Gokdogan wrote:

On Fri, Dec 21, 2012 at 8:20 PM, Thomas Broyer <t.br...@gmail.com> wrote:

On Saturday, December 22, 2012 2:50:00 AM UTC+1, Goktug Gokdogan wrote:

GWT's built-in library space is already bloated and getting harder to maintain. Inside GWT, we need to do less but do better.

And good thing is, a lot of stuff doesn't need to be baked in; like the protocol buffers; it is a valuable feature but I personally don't see GWT having it as a first class citizen; at least in the near term.

On the other hand we can definitely help out on removing any blockers and support anybody who would like to implement it as a 3rd party library.

+1

Wondering what Google is doing though ;-)

Are you somehow generating, say, RequestFactory ValueProxy-s from .proto files? It looks like what BobV was planning.

 

There is compiler plugin that generates lightweight GWT emulations of the messages based on JavaScript overlays.

On top of that, there is a custom RPC mechanism that mimics GWT RPC. Wire format is like JSON-RPC where 'params' is js array encoded protobuf.

I think there has been multiple solutions but I guess this one got wider acceptance. Unfortunately, nobody actively working on it and not ready to open-source.

Thanks.

Looks a bit like what I did years ago to call "REST-alike web services", except I coded everything by hand.

Also looks a bit like gwt-json-rpc used by Gerrit, but interfaces and JSOs are hand-coded too.

[V.B.]

Our team is revisiting this issue again in hopes that some lateral (i.e. non-reflection) workarounds could be added to GWT 2.6.1 or GWT 3.0.

   In particular, the original problem with protobuf-generated Java classes is that Reflection isn't emulated in GWT. I admit that emulating reflection is of dubious value, at best. However, it turns out this can be side-stepped by compiling with the LITE_RUNTIME flag, which excludes reflection code, solving 80% of the problem.

   The next problem has to do with very common classes/methods that are not emulated by GWT, but reasonably could be. These include parts of java.io.* (Issue 604), String.format (Issue 3945), and a few others (try compiling the attached .proto file into Java). Adding emulation for these would be very beneficial far beyond just protocol buffers.

What would it take to get these side-issues included in 2.6.1/3.0 ?

[Jens]

https://github.com/dcodeIO/ProtoBuf.js

Seems like a very good JS implementation of protobuf that uses typed arrays. Before trying to (probably badly) emulate java.io stuff in GWT I would try to build a GWT wrapper for that project.

-- J.