GWT-RPC broken in GAE/J

[Robert Hanson]

I didn't see a thread on this yet, but the GWT-RPC doesn't work well
in GAE with JPA. It seems that the JPA entities are "enhanced" at
compile time, and this is not compatible with the GWT serialization on
the client side. The GWT app complains that the serialization is not
compatible.

Ray has a workaround, but it is less than ideal
http://timepedia.blogspot.com/2009/04/google-appengine-and-gwt-now-marriage.html

And you can always use a DTO, but I recall that Serializable was added
to GWT just so that you didn't need to do that.

So... I'm not really asking for a fix... I am more wondering what the
"official response" is. Is this something that you want to fix,
something that isn't safe to fix, or something that you don't want to
fix?

Thanks.

Rob

[Ray Cromwell]

The only 'real' way to make detached entities to work with GWT would be to make the 'enhancer' generate Java source like a GWT generator OR make an interface like 'Detachable' and have a GWT generator do it, because not only do you need the extra state fields (jdoDetachedState), but the method calls of the class have to be intercepted to manipulate the detached state. Actually, a generator probably can't do it because IIRC JDO enhancement involves intercepting even field references, and Generators don't get an AST.

So it's either transient instances, or some kind of Ant task that generates GWT-friendly DTO helper classes for you along with the enhancement process.

-Ray

[Arthur Kalmenson]

I haven't tried GAE/J just yet, but I'm guessing the problem with JPA
is the same one you would have with any database. If your domain
objects aren't very large, you can use Dozer to map the object onto
itself which removes all the enhancements and makes them regular
objects.

--
Arthur Kalmenson

[Robert Hanson]

> you can use Dozer to map the object onto itself

This doesn't with GAE/J.

I copied persisted entities to non-persisted copies, and again, it
fails. I needed to create a separate class, with the same fields as
the model object, and then copy the data to that. It seems that any
class with the @Entity annotation is unable to be passed to the
client.

Rob

[John Tamplin]

On Fri, Apr 10, 2009 at 6:48 PM, Robert Hanson <iamrobe...@gmail.com> wrote:

I copied persisted entities to non-persisted copies, and again, it
fails.  I needed to create a separate class, with the same fields as
the model object, and then copy the data to that.  It seems that any
class with the @Entity annotation is unable to be passed to the
client.

Conceivably the RPC generator could recognize the JDO/JPA annotations and take into account the bytecode rewriting that will happen in the server version, but I don't know what all that would entail.

--
John A. Tamplin
Software Engineer (GWT), Google

[Ray Cromwell]

Don't use JPA annotations, use JDO, because you can specify
non-detachable objects. Even if you hack the generators and it passes
the CRC checks you'll still end up with possibly broken code, because
on the server, the methods of the class are rewritten to modify these
hidden fields, and on the client, you'll be getting the original code.
Basically, trying to ship detached objects over RPC and maintain all
of the detached 'dirty' state, while being wire-efficient I think is
impractical and error-prone, so the style of programming where you
ship objects straight out of the ORM to the client should be
discouraged.

I'm of the opinion of some kind of ProtocolBuffer-like system, with
built-in capability of efficiently copying state to-and-from ORM and
PB classes would be better and more deterministic (and also support
non-GWT clients like Android!)

-Ray

[Sami Jaber]

Too bad that Google has to launch AppEngine to discover that ORM is definitively a problem for GWT. I send a mail to Bruce in early summer 2007 about that issue. His answer (that I perfectly understand) was to use DTO.

The fact is that DTO is no more a recommended pattern when your domain model is pure Java/POJO and client and server are tightly coupled.  According to Gavin Kind and Rod Johnson, “DTO are evil” (http://forum.springsource.org/archive/index.php/t-15538.html). There are many situations where DTO are useful, but in general they could be avoided.

There is no “clean” solution for this problem unfortunately. All the ORM tools provide the “dirty state checking” mechanism by enhancing the entities object (Hibernate uses  javassist during the attach process, jdo pre-enhanced during the compilation step). The issue comes from the fact that most of enhancers rely on java specific classes that are not serializable (SessionImplementor for Hibernate, …).  Sending any Lazy initialized POJO over the wire to a non Java Client (Soap, XML, Json, …) leads to the same problem.

The idea behind hibernate4gwt is to replace on the fly (before InvokeAndEncodeRequest()) the proxies/specific classes by their JavaScript counter-part (a hashmap containing all the lazy initialized fields). This mechanism is implemented either with a generator (the generator generates subclasses that holds the hashmap proxy fields/information) or with a “magic” super class (LightEntity) which holds the same map.  

Whatever we do, the process is necessarily specific to the ORM implementation because the way you get back the JavaScriptObject and you convert it back in Java depends on the Proxy implementation.  The only way to get rid of this problem is to provide a mechanism based on “adaptors” that would be plugged during the Serialization process for converting attached entities to detached javascript compliant entities. Those Hibernate Adaptors would be provided by the ORM providers (Hibernate, DataNucleus, OpenJPA, etc …) and would handle for us all the nasty under-the-cover conversions stuff.

This problem requires a big implication of all the actors (ORM providers and Google/Community).  Without this big bang, all the solutions based on Dozer, BeanLib, protocolbuffer generator (as stated by Ray) and DTO conversion tools will be a pain (many of them do not handle circular reference, deep graphs, etc..) and will lead to poor performance.

I’m working with Bruno (hibernate4gwt) to try to move the adaptor mechanism in the custom serialization (ServerSerializationWriter/Reader) process. I know that those classes are not meant to be extended as they are internal implementation and will probably be removed in the next version. Is there any work on a kind of (real) pluggable serialization mechanism for GWT 2 ?

 Sami

[noon]

Hi all,

As Gilead (and Hibernate4GWT) author, I completely agree with Sami :
GWT serialization has to support extension points to allow third
parties adapters.
A good inspiration is probably Adobe BlazeDS (RPC communication for
Flex), which was *designed* to seamlessly support adapters and
factories.

As Sami mentioned, this kind of issues are known since first releases
of GWT, and the DTO answer is not a good one (it is error prone, non
productive and breaks Domain Driven Design that GWT Java support
promises). In my opinion, GWT team must not ignore this problem
anymore and should start to work on this really fast ("we will think
about it for RPC 2.0" was already the answer 2 years ago !).

Regards
Bruno

> On Sat, Apr 11, 2009 at 3:05 AM, Ray Cromwell <cromwell...@gmail.com> wrote:
>
> > Don't use JPA annotations, use JDO, because you can specify
> > non-detachable objects. Even if you hack the generators and it passes
> > the CRC checks you'll still end up with possibly broken code, because
> > on the server, the methods of the class are rewritten to modify these
> > hidden fields, and on the client, you'll be getting the original code.
> > Basically, trying to ship detached objects over RPC and maintain all
> > of the detached 'dirty' state,  while being wire-efficient I think is
> > impractical and error-prone, so the style of programming where you
> > ship objects straight out of the ORM to the client should be
> > discouraged.
>
> > I'm of the opinion of some kind of ProtocolBuffer-like system, with
> > built-in capability of efficiently copying state to-and-from ORM and
> > PB classes would be better and more deterministic (and also support
> > non-GWT clients like Android!)
>
> > -Ray
>
> > On Fri, Apr 10, 2009 at 4:21 PM, John Tamplin <j...@google.com> wrote:
> > > On Fri, Apr 10, 2009 at 6:48 PM, Robert Hanson <

> > iamroberthan...@gmail.com>

[datanucleus]

> The issue comes from the fact that most of enhancers rely on java specific classes
> that are not serializable (SessionImplementor for Hibernate, …).

JDO enhancers don't, since all contents of "jdoDetachedState" (Object
[]) are serializable, although in our case GWT apparently has problems
with BitSet (due to long[] being used inside I'm told).

> Those HibernateAdaptors would be provided by the ORM providers (Hibernate, DataNucleus,

> OpenJPA, etc …) and would handle for us all the nasty under-the-cover conversions stuff.
> This problem requires a big implication of all the actors (ORM providers and Google/Community).

> I’m working with Bruno (hibernate4gwt) to try to move the adaptor mechanism in the custom serialization

If anything is required from DataNucleus just let us know.

--Andy (DataNucleus)

[datanucleus]

Hi,
didn't see this thread since I tried on the main GWT group. See
http://groups.google.com/group/Google-Web-Toolkit/browse_thread/thread/162beffc8e33e041

Having your RPC generator recognise JDO/JPA annotations isn't a way to
go since the user may have defined persistence in XML. The one thing
is, the field is always called jdoDetachedState so can't this be
recognised ? (non-GWT person here so excuse stupid questions ;-) ).

Ray's workaround (of using transient) obviously will work, and we
(DataNucleus) could provide complete helper methods to allow this
copying back in of a "transient GWTed" object to aid the process, but
something more complete would be desirable.

[datanucleus]

From a DataNucleus perspective, everything included in enhanced
classes is Serializable (deliberately). We are willing to contribute
to the process of aiding the fix of GWT where you require any specific
handler for DataNucleus usage. Just let us know.

[I've tried to post to this group twice so far and it always says Post
successful, but never appears, so who knows if this will]

--Andy (DataNucleus)

[Ray Cromwell]

One possibility might be to add features to the JDO enhancer to spit
out Java source for IsSerializable tagged classes instead of just
generating enhanced byte code. This code could then be seen by the GWT
compiler. In which case, all of the extra jdo methods, as well as
jdoDetachedState would be on the object. You could even emit custom
serializers.

However, I am nervous about the idea of sending actual detached
objects over the wire. The philosophy of GWT is to minimize latency,
by reducing HTTP requests and wire size. People might have a tendency
to blindly ship detached instances over the wire, which could result
in either too much serialized data being sent, or too much lazy
fetching. It is true that JDO allows fetch groups to control this, but
it seems like it would more consideration to the client than is usual
for JDO clients, as it seems many people using ORMs are trying to
optimize RDBMS performance, not serialization performance.

That's why I like the PB approach, because with PB's, it's very clear
that cycles can't occur, that optional nullable fields take up no
space, and that relations require explicit fetching on the client. The
downside is, it violates DRY, and I agree, I hate the DTO pattern.

I am not really an expert in the ORM domain, so the datanucleus guys
probably have alot better recommendations for patterns on using JDO to
minimize fetching and wire-size.

I do think that with Datanucleus's help, there can probably be a real
solution, and if done right, perhaps one that addresses common
mistakes people make with wire-size and latency.

-Ray

[Miroslav Pokorny]

Andy

The major stumbling block remains that the enhanced classes are
unknown and unavailsble in your client/GWT application. If the server
did allow (and it could - it's no big deal to recursively serialize
all fields) the puoblem remains on the client. Failing on the server
is actually good practice e - and csn be considered to be failing fast.

The gwt serielization architecture does not use reflection but rather
generates a new serializer for each and every type it feels can occur
in a serialized stream response from the server. This can be easily
witnessed by setting logging to debut or using the option that writes
generated classes to a specific directory "-gen <directory>".

Back to the enhanced classes - they are unknown and do not exist at
compile time when the generator creates all tbe required serializer
types. The client code rpc proxy stuff contains a big switch statement
mapping serializable classes to generated/custom serializers. Problem
is the names of the enhanced classes are not present in the switch
thus deserialuzation fails .

This is the same reason onecannot serialize anon classes. Javac
generates names for anon classes and these of course don't match this
on the client. Icould be wrong ok the anon class thing but the
statement is true if older gwt versions.

Ultimately to solveyour problem and avoid copying Instances of
enganced classes onto regular classes the easiest thing is to probably
hack the server code which does the serislizing. Rather than sending
down enhanced type instances write the state of the class/instance
underneath and skip the fields holding the detached state. Naturally
one loses the ability to reattach but it could be argued passing dtos
thru all those layers all the way to client is exposi g way too much
of your persistance impl detail.

Hth

On 12/04/2009, at 12:07 AM, datanucleus <andy_je...@yahoo.com>
wrote:

[Sami Jaber]

Absolutely Ray, PB violates DRY principles and could be easily handled as a specific adaptor in the whole extensibility model.

This idea is to agree on an extensibility pattern similar to the way one extend RPC processCall(). CFS (Custom Field Serializer) may be an answer but they are supposed to be removed in further versions.

Customizing the JDO enhancer certainly does the trick but responds to a global problem by a specific solution. The main ORM used today is Hibernate (thru JPA with annotations or XML files), DataNucleus will probably gain momentum with AppEngine but people are waiting responses in priority for their ORM implementation.

To finish on the Latency, this problem is not specific to GWT/ORM but to ORM in general. Anyone who uses ORM should take care of FetchGroup, EagerFetching, N+1 select and minimizing roundtrip.

The way Hibernate4GWT solves this problem should be a starting point of thinking IMHO. This Framework looks like complex because of the lack of extensibility model in the serialization process, it is up to us (or GWT team) to simplify this mechanism.

Hope (you would) help

Sami

[Miguel Ping]

I discovered the hard way that GWT and ORM needs some custom code.
IMHO, the real problem is that class enhancers do not provide a way to
unenhance an instance according to some rules. I ended up writing my
custom bean copier based on dozer and beanmapper, so I can specify a
shallow/deep clone, though most of the times I'll just be getting rid
of proxies, since I fetch ahead what I know I'll need.

To my understanding, GWT-like applications (read: the service layer)
don't need lazy load for most of the time; lazy load works most for
open-session-in-view. I think almost all devs code to fetch the needed
objets up front anyway.

The real problem for me is the following:
- I want to send an enhanced object over the wire.
- The object may or may not be serializable due to class enhancing.
- My object class declaration is fully serializable (but the
underlying type may not be due to proxies).

I think we have two choices:
- copy the object to a fully serializable class using some bean mapper
with some rules
- describe a way to convert the proxied/enhanced class to an
unenhanced one, again with some rules

Both these ways benefit from an extensible serialization mechanism in
GWT. In the end, I'd like to see a plugin mechanism from GWT to
provide custom (de)serializers for RPC mechanisms, and a simple way of
"dehydrating" a persistent/enhanced class through some rules.

On Apr 12, 11:41 am, Sami Jaber <sami.ja...@gmail.com> wrote:
> Absolutely Ray, PB violates DRY principles and could be easily handled as a
> specific adaptor in the whole extensibility model.
>
> This idea is to agree on an extensibility pattern similar to the way one
> extend RPC processCall(). CFS (Custom Field Serializer) may be an answer but
> they are supposed to be removed in further versions.
>
> Customizing the JDO enhancer certainly does the trick but responds to a
> global problem by a specific solution. The main ORM used today is Hibernate
> (thru JPA with annotations or XML files), DataNucleus will probably gain
> momentum with AppEngine but people are waiting responses in priority for
> their ORM implementation.
>
> To finish on the Latency, this problem is not specific to GWT/ORM but to ORM
> in general. Anyone who uses ORM should take care of FetchGroup,
> EagerFetching, N+1 select and minimizing roundtrip.
>
> The way Hibernate4GWT solves this problem should be a starting point of
> thinking IMHO. This Framework looks like complex because of the lack of
> extensibility model in the serialization process, it is up to us (or GWT
> team) to simplify this mechanism.
>
> Hope (you would) help
>
> Sami
>

> > On Sat, Apr 11, 2009 at 2:34 AM, datanucleus <andy_jeffer...@yahoo.com>

> > wrote:
>
> > > Hi,
> > > didn't see this thread since I tried on the main GWT group. See
>

> >http://groups.google.com/group/Google-Web-Toolkit/browse_thread/threa...

[noon]

Hi,

I fully agree with Miguel analysis and solution proposal.
In my mind, I wrote Gilead as just a serialization adapter for
Hibernate objects, and similar work should have to be done for
DataNucleus, OpenJPA and al (by the supplier of such libraries, or
community...)

Can we have the "official GWT team" point of view about it ?

Regards
Bruno

[philip]

Hi,

So I like GWT and I like Google App Engine, all I want is a simple
recipe for getting the two to work without hitting too many big
problems.
I'm willing to use DTO, I'm willing to suffer whatever I have to do -
but I need to know what's the best current way to do it.

Thanks, Philip

On Apr 10, 10:19 pm, Robert Hanson <iamroberthan...@gmail.com> wrote:
> I didn't see a thread on this yet, but the GWT-RPC doesn't work well
> in GAE with JPA.  It seems that the JPA entities are "enhanced" at
> compile time, and this is not compatible with the GWT serialization on
> the client side.  The GWT app complains that the serialization is not
> compatible.
>

> Ray has a workaround, but it is less than idealhttp://timepedia.blogspot.com/2009/04/google-appengine-and-gwt-now-ma...

[John Tamplin]

On Tue, Apr 14, 2009 at 10:26 AM, philip <phili...@gmail.com> wrote:

So I like GWT and I like Google App Engine, all I want is a simple
recipe for getting the two to work without hitting too many big
problems.
I'm willing to use DTO, I'm willing to suffer whatever I have to do -
but I need to know what's the best current way to do it.

GWT RPC does not preserve identity across the wire, and the GWT compiler needs the source of the classes at compile time.  Since JDO/JPA rewrite the bytecode on the server, and may do so based on information not in the source, there won't be any short-term workaround.  Even if the enchancer produced source code instead, trying to serialize Object[] (what the enhancer adds) means every serializable type in the program has to have deserializers present in the client, which would be a massive size increase for programs of any significant size.

The only way I see it working would be to add a flag requiring preserving object identity across RPC, and the protocol would include an object ID with each object if that flag is set.  The server would then save/restore additional hidden fields on its end for any persistent classes which were rewritten, and use the object ID to know what values should be recreated for these hidden fields when a persistent object comes back from the client.

A few issues with this approach:

• what happens for a persistent object created on the client that the server has never seen before?
• how are field references handled?  Since a class can be made persistent without the GWT compiler's knowledge, it can't do anything about this (and even if it could you wouldn't want to for client code-size reasons).  I assume this is only needed to know which fields are "dirty" and need to be updated -- maybe you can just assume anything coming from the client needs to be replaced entirely.  Since the GAE data store works that way anyway that shouldn't be a big inefficiency, but could affect using this approach with non-GAE persistence engines.
• Since you don't have clone in JS, any copy of an object would have to be treated as a new instance, which could be the wrong answer in some situations.  Is making it mostly work as expected acceptable, or will the odd cases confuse developers more than just requiring them to do things explicitly?
  

This would require significant work in the GWT RPC subsystems and could mean a significant cost in the wire-protocol size.

IMHO, blindly transferring your stored data across the wire is convenient but produces big, slow client code.  That may be useful for development, and it may be worth providing it for that reason and for intranet apps where the client size is less important, but I think having to write your own DTOs so you know exactly what is going over the wire and why forces you to think about why you need it in the client and produce a more efficient app.  It is too easy to just send a Customer record to the client because that is what you have, even when all you need is just the name/address portions for your shipping form, etc.

[Toby Reyelts]

I think this is a false dichotomy. JDO has fetch-groups for just that kind of scenario. You send down a slice of an object graph that meets exactly your needs. You don't have to re-invent an entire DTO layer just to send a subset of your data. You can re-use the same domain objects in several different RPC calls with several different fetch-groups each tailored for the specific use case.

I think we need to tread carefully here if we're to make progress and produce something everyone can be happy with. We need people who really understand this domain to get together and think this through offline, coming back with at least semi-baked proposals we can discuss, evaluate, and adjust. We'd end up with much more productive discussion instead of lots of idle speculation.

[Ray Cromwell]

On Tue, Apr 14, 2009 at 9:03 AM, John Tamplin <j...@google.com> wrote:

> GWT RPC does not preserve identity across the wire, and the GWT compiler
> needs the source of the classes at compile time.
>

> The only way I see it working would be to add a flag requiring preserving
> object identity across RPC, and the protocol would include an object ID with
> each object if that flag is set.

Actually, JDO has a concept called application identity where object
identity is handled by the application. This typically boils down to
the PrimaryKey being used as object identity.

> what happens for a persistent object created on the client that the server
> has never seen before?

Calling persist() on it would insert it rather than merge it.

> IMHO, blindly transferring your stored data across the wire is convenient
> but produces big, slow client code.

This I agree with, I've been drum beating on this for awhile. ORM +
RPC is the surest way to produce 10,000 messages in the newsgroup
about "why my RPC calls are so slow and I get slow script warnings"
For me, the issue is to find a middle ground, a way of using DTOs, a
pattern, a DSL, a utility, that gives you DTO semantics without
excessive boilerplate. First of all, in almost all cases of an ORM
object sent over RPC, lazy association fetching would be wrong
(resulting in excess roundtrips), but pre-emptive fetches would also
be wrong much of the time. JDO/JPA work on the paradigm that the
server controls fetch semantics, but often times, it is the client
that knows what it wants to pull down, not the server, and this is
especially true if the client and server are written by different
people. Explicit finder methods with explicit DTOs are rather self
documenting and obvious on their face, what it is they do and what's
being transferred, the downside is a multiplication of boilerplate and
house keeping code.

This is certainly an area where DSLs + generators, libraries, and
compilers can help. Consider how easy Guice + Warp-Persist makes ORM
finders.

public interface CustomerDAO {
@Finder("select c from Customer c where c.zipCode = :zipcode")
Customer findCustomerByZipCode(@Named("zipcode") String zip);
}

You inject this and everything is handled automatically, getting
persistent context, opening/closing transactions, setting up the
query. It makes things painless.

Now we just need to find an equivalent for DTOs.

-Ray

[Ray Cromwell]

On Tue, Apr 14, 2009 at 10:27 AM, Toby Reyelts <to...@google.com> wrote:
> I think this is a false dichotomy. JDO has fetch-groups for just that kind
> of scenario. You send down a slice of an object graph that meets exactly
> your needs. You don't have to re-invent an entire DTO layer just to send a
> subset of your data. You can re-use the same domain objects in several
> different RPC calls with several different fetch-groups each tailored for
> the specific use case.

That's true, but what bothers me about this is that it seems brittle.
If I pull down a domain object, one that comes with fetch group A, and
pull down another domain object from fetch group B, and stick them in
a Collection together, and then someone else maps over this
collection, it can result in serious breakage. The assumption being
made is that people working with domain objects are usually the ones
calling the RPC method, and so have read the documentation that says
"This returns a Customer object, but does not fetch the
AccountRepresentative. Do not call
Customer.getAccountRepresentative()!" It true, from the context of an
AsyncCallback, you will usually know what restrictions are in effect
on the domain object returned, but this will be lost as it goes down
the stack chain.

Of course, you could document Customer.getAccountRepresentative() as
potentially returning null, but that's sorta brittle too, because if
null represents the non-existence of a representative, then other code
might just to assign one!

And given the asynchronous nature of RPC, you can't really fetch these
associations on demand, unless you make the getter method take an
AsyncCallback, another pollution of the interface.

I know you can make this work if you're careful, but it seems brittle
and subject to subtle breakage as the number of people on a project
increase.

-Ray

[Toby Reyelts]

I agree that it's not a panacea. I am not aware of any panacea in existence.

My opinion is that GWT users would be best-served if they had both DTO and DTO-less solutions. I believe that the solution that best fits your needs depends upon your team and your project requirements. Rather than philosophising, I'd like to see "expert" teams working out possible approaches with working prototypes.

-Ray

[John Tamplin]

On Tue, Apr 14, 2009 at 1:28 PM, Ray Cromwell <cromw...@gmail.com> wrote:

On Tue, Apr 14, 2009 at 9:03 AM, John Tamplin <j...@google.com> wrote:

> The only way I see it working would be to add a flag requiring preserving
> object identity across RPC, and the protocol would include an object ID with
> each object if that flag is set.

Actually, JDO has a concept called application identity where object
identity is handled by the application. This typically boils down to
the PrimaryKey being used as object identity.

Is there an equivalent for JPA?

> what happens for a persistent object created on the client that the server
> has never seen before?

Calling persist() on it would insert it rather than merge it.

Well, what I was getting at was more what does the server-side RPC code populate the hidden fields with if it gets a new object of that type from the client?

If the hidden fields can be null and the @PrimaryKey field will be used to determine if it is a new object or a replacement of an existing one, then it seems pretty easy -- the server-side RPC code simply strips out the effect of the bytecode rewriting for serialization.  I don't know enough about what rewriting happens to know how to detect/reverse it, but it seems like it should be doable.

[Robert Hanson]

I hope I am not being naive (or reiterating an existing post), but I
am not concerned with identity, nor concerned with expecting my ORM
solution to deal with entities created on the client.

I think all I need is a way to use an entity as a DTO without needing
a second set of model objects.

What about something as simple as keeping the existing semantics in
tact, with the ability to flag properties that are send-able to the
client (as opposed to using transient to flag the reverse behavior).

public class Data implements Serializable {
@GwtDto
private Long id;
@GwtDto
private String partA;
private String partB;
}

In this case the GWT compiler would create a serializer for the client
that only handled the id and partA properties, and ignore partB. The
partB property would simply be null on the client side.

On the server reflection would be used to only serialize the marked
properties, ignoring the rest. Besides ignoring partB it would also
ignore any fields added by enhancers.

Besides fixing GWT-RPC with GAE, it also allows me to trim data that I
don't really need o

[Robert Hanson]

[Continued - Mail got away from me]

...Besides fixing GWT-RPC with GAE, it also allows me to trim data
that I don't really need on the client side.

The rest, like knowing when to persist vs merge an incoming object
from the client, is my problem to figure out.

Rob

[Ray Cromwell]

On Tue, Apr 14, 2009 at 8:39 PM, Robert Hanson
<iamrobe...@gmail.com> wrote:
>
> I hope I am not being naive (or reiterating an existing post), but I
> am not concerned with identity, nor concerned with expecting my ORM
> solution to deal with entities created on the client.
>
> I think all I need is a way to use an entity as a DTO without needing
> a second set of model objects.

If this is all you're interested in, I described a way to make GAE and
GWT-RPC work together "out of the box". Just declare your entities as:

@PersistenceCapable(identityType = IdentityType.APPLICATION,
detachable = "false")
public class MyPojo implements Serializable {

}

and everything will work, but you'll have to manually deal with
re-attachment when sending objects from the client back to the server.

-Ray

[John Tamplin]

On Tue, Apr 14, 2009 at 11:39 PM, Robert Hanson <iamrobe...@gmail.com> wrote:

I hope I am not being naive (or reiterating an existing post), but I
am not concerned with identity, nor concerned with expecting my ORM
solution to deal with entities created on the client.

If you don't preserve identity, I don't see how you can modify the objects in the client.  For example (client-side code):

Customer cust = service.getCustomer(custId);
cust.setCustomerId(newId);
service.updateCustomer(cust);

If the server doesn't know that the argument passed to updateCustomer is the same as the one it returned from getCustomer, how can it update the database?  As I understand it, these hidden fields are used to record a unique identifier so it can tell a modified one from a different one (ie, insert vs update) as well as dirty flags.  Even if you can go by the primary key (presuably customerId here), when you persist the Customer object on the server it will add a new one rather than reflect the change of the id.

Thus, I think the issue does boil down to tracking identity across RPC.
 

I think all I need is a way to use an entity as a DTO without needing
a second set of model objects.

What about something as simple as keeping the existing semantics in
tact, with the ability to flag properties that are send-able to the
client (as opposed to using transient to flag the reverse behavior).

public class Data implements Serializable {
 @GwtDto
 private Long id;
 @GwtDto
 private String partA;
 private String partB;
}

In this case the GWT compiler would create a serializer for the client
that only handled the id and partA properties, and ignore partB.  The
partB property would simply be null on the client side.

On the server reflection would be used to only serialize the marked
properties, ignoring the rest.  Besides ignoring partB it would also
ignore any fields added by enhancers.

So when you create a new Data instance on the client, what gets stored in the database?  What do you do when the class is something you can't or don't want to modify to add these extra annotations?  If we require that the source has the JDO/JPA annotations, it is at least feasible for the compiler to generate client-side code to deal with it.

If you start adding lots of unusual restrictions like you can't modify the primary key in the client or create new instances there, it isn't clear what value you are gaining over just creating DTOs (perhaps with a generator, though that makes the developer's life more awkward in the IDE).

[philip]

Hi John,

Thats fine your debating how best to do it for the longer term, and
thats great for the longer term. How about now today? how best can I
do it now?

For the shorter term I am sure there is a bunch of people who look at
this and think, I've got GWT and Google App Engine, I want to put them
together and make an nice app, then they come across these
discussions. Alternatively, they try to serialize the JDO classes and
find they have problems. Either way, they hit the ugly wall.
They probably wonder why GWT and App Engine are not such good friends
such it would make a great programming platform and boost GWT.

This class of people who just came to these technologies for the first
or second time, then find a serious problem between the two of them
and drop the idea. Its just like the people who go to Ruby on rails to
do their application development quickly, they don't care so much at
first about the performance, or what is going down the wire. They care
about getting the application done quickly at the start. The people
who did adopt Ruby on Rails near the start didn't care about
scalability or anything complex, just to get the app done and deal
with those problems later. Ha, they wouldn't have choosen ROR if they
cared about performance. Not only that, I don't really want to
optimize my application early, I want to optimize the code later, so I
don't care about the data transfer. All my customers have broadband.

In my case I'm a programmer with 15 years experience and I've played
with GWT for some time but want to write a more serious application, I
want to know how best to send my data from App Engine to the GWT
client, how best to send it back and update the record. For something
like SEAM framework, there are best practices for this. These are
fairly simple things, so I expected there would be a simple way to do
it in GWT/GAE.
I can write an application and try some of the methods discussed here
such as Dozer, BeanLib, but it feels wrong.

Seam framework has good solutions for bring the data to the page via
JPA, maybe you could look at that for some ideas as well. I use that
for a lot of work and it avoids the lazyloading exception and you
don't need to pre-fetch all data. Although, its not really technically
applicable for this problem, the general ideas inside it are good.

Thanks, Philip


On Apr 15, 12:03 am, John Tamplin <j...@google.com> wrote:

> On Tue, Apr 14, 2009 at 10:26 AM, philip <philip14...@gmail.com> wrote:
> > So I like GWT and I like Google App Engine, all I want is a simple
> > recipe for getting the two to work without hitting too many big
> > problems.
> > I'm willing to use DTO, I'm willing to suffer whatever I have to do -
> > but I need to know what's the best current way to do it.
>
> GWT RPC does not preserve identity across the wire, and the GWT compiler
> needs the source of the classes at compile time.  Since JDO/JPA rewrite the
> bytecode on the server, and may do so based on information not in the
> source, there won't be any short-term workaround.  Even if the enchancer
> produced source code instead, trying to serialize Object[] (what the
> enhancer adds) means every serializable type in the program has to have
> deserializers present in the client, which would be a massive size increase
> for programs of any significant size.
>
> The only way I see it working would be to add a flag requiring preserving
> object identity across RPC, and the protocol would include an object ID with
> each object if that flag is set.  The server would then save/restore
> additional hidden fields on its end for any persistent classes which were
> rewritten, and use the object ID to know what values should be recreated for
> these hidden fields when a persistent object comes back from the client.
>
> A few issues with this approach:
>

>    - what happens for a persistent object created on the client that the

>    server has never seen before?

>    - how are field references handled?  Since a class can be made persistent

>    without the GWT compiler's knowledge, it can't do anything about this (and
>    even if it could you wouldn't want to for client code-size reasons).  I
>    assume this is only needed to know which fields are "dirty" and need to be
>    updated -- maybe you can just assume anything coming from the client needs
>    to be replaced entirely.  Since the GAE data store works that way anyway
>    that shouldn't be a big inefficiency, but could affect using this approach
>    with non-GAE persistence engines.

>    - Since you don't have clone in JS, any copy of an object would have to

[noon]

Hi Robert,

For information, Gilead provides the @ServerOnly annotation to prevent
sending sensible fields over the wire.

Regards
Bruno

On 15 avr, 05:39, Robert Hanson <iamroberthan...@gmail.com> wrote:
> I hope I am not being naive (or reiterating an existing post), but I
> am not concerned with identity, nor concerned with expecting my ORM
> solution to deal with entities created on the client.
>
> I think all I need is a way to use an entity as a DTO without needing
> a second set of model objects.
>
> What about something as simple as keeping the existing semantics in
> tact, with the ability to flag properties that are send-able to the
> client (as opposed to using transient to flag the reverse behavior).
>
> public class Data implements Serializable {
>   @GwtDto
>   private Long id;
>   @GwtDto
>   private String partA;
>   private String partB;
>
> }
>
> In this case the GWT compiler would create a serializer for the client
> that only handled the id and partA properties, and ignore partB.  The
> partB property would simply be null on the client side.
>
> On the server reflection would be used to only serialize the marked
> properties, ignoring the rest.  Besides ignoring partB it would also
> ignore any fields added by enhancers.
>
> Besides fixing GWT-RPC with GAE, it also allows me to trim data that I
> don't really need o
>
> On Tue, Apr 14, 2009 at 2:18 PM, John Tamplin <j...@google.com> wrote:

[Miguel Ping]

John, I'm no JPA expert but the way I've been doing it is exactly how
you describe it. I think dirty flags and hidden fields are meant to be
working within an attach/detach context. In fact, I really don't give
a damn about advanced persistence context capabilites like this one
(Cited from Java Persistence with Hibernate book - I hope I'm not
violating some copyright by pasting it here):

■A primitive persistence layer with no identity scope makes no
guarantees
that if a row is accessed twice the same Java object instance will be
returned to the application. This becomes problematic if the
application
modifies two different instances that both represent the same row in a
single
unit of work. (How should we decide which state should be propagated
to the database?)
■ A persistence layer using persistence context-scoped identity
guarantees that, in
the scope of a single persistence context, only one object instance
represents
a particular database row. This avoids the previous problem and also
allows for some caching at the context level.
■ Process-scoped identity goes one step further and guarantees that
only one
object instance represents the row in the whole process (JVM)"

I really just want to generate queries based on my domain objects, and
I think this is the majority (80/20 rule maybe) of GWT users.
Application vs. Db identity is a complicated issue, I'll past another
example from the hib book:

Set allObjects = new HashSet();
allObjects.add(a);
allObjects.add(b);
allObjects.add(c);

If a and b are the same DB object loaded from the same session and c
is a detached object loaded from another session, how many entries are
there in the set?
My answer is I don't care, since I'll avoid this scenarios (two
sessions for loading the same db object). This is a hard problem.
Besides, I think it falls out of GWT scope. GWT will never be able to
provide referential identity (== identity), but programmers should be
aware of this. In the end, I think the GWT team should only be
concerned with the GWT scope, ie, serialization, and the community
should focus on providing sensible answers the the problems around JDO/
JPA/enhanced classes.


All of my GWT RPC <> DB interactions are based on 2 things
- Relationship responsability: who is responsible for managing eg: a
one-to-many relation. This is tricky because if I am sending an object
A that manages a Set<B> from the client, hibernate can overwrite the
set of B's in the DB with the set that comes from the client if A is
managing the relation. I had to write custom code and change my
relations mappings to handle this
- Database identity: to distinguish between an insert and an update

For me, GWT at the RPC level is just really a transport layer, which
(de)serializes what it is asked. My program has to provide the rules
to define what is to be serialized when responding an RPC request.

On Apr 15, 5:09 am, John Tamplin <j...@google.com> wrote:
> On Tue, Apr 14, 2009 at 11:39 PM, Robert Hanson

> <iamroberthan...@gmail.com>wrote:

[Isaac Truett]

> The rest, like knowing when to persist vs merge an incoming object
> from the client, is my problem to figure out.
>
> Rob

I'm definitely in agreement with Robert on this. I just started trying
to write a simple app for GWT + GAEJ and over half my server code
seems to be jumping through hoops for JDO and converting between my
persistent data object and my GWT-serializable data object. I'd like
to see that problem area addressed. Problems of identity and
client-origin data? I've been dealing with those for years, and I'm
okay with it.

- Isaac


On Tue, Apr 14, 2009 at 11:42 PM, Robert Hanson

[Ray Cromwell]

It seems like you guys don't want the transient solution I proposed,
where you reuse the domain objects, but must implement special copying
logic on the server to merge changes from the client. It seems like
what you're really asking for is detached object support. This should
be possible to do. One possibility for example, is to override the
default RPC generator with one that generates RPC subs and subclasses
of serialized types that have detachment support.

For example, let's say you have

public interface CustomerService extends RemoteService {
Customer getCustomer(String name);
}

public interface CustomerServiceAsync {
void getCustomer(String name, AsyncCallback<Customer> callback);
}

The RPC generator could implement this in a way, such that the
received object from the RPC call is not a Customer, but a
DetachableCustomerImpl (which extends Customer). This would be a
client side 'enhanced' version which does dirty bit tracking, by
overriding all of the Persistent setter fields of the class, along
with the stipulation that mutation of persistent fields only occurs
through these setters. (Generators do not have AST access and thus
cannot enhance method bodies to track field mutations)

Meanwhile, on the server, create a JdoRemoteServiceServlet which
treats the serialization of Detachable objects in a special way, by
serializing the detached state fields in a way that is consistent with
the client side encoding of those fields. The logic for tracking class
type CRCs for verification would probably have to be override. The
object would be for the servlet to map a native JDO-encoding of
detached state into a GWT-friendly client-side encoding, and
vice-versa on the return trip.

According to the JDO 2.2 spec, here is what is tracked in enhanced classes:

The detached state is stored as a field in each instance of
Detachable. The field is serialized so
as to maintain the state of the instance while detached. While
detached, only the BitSet of mod-
ified fields will be modified. The structure is as follows.
Object[] jdoDetachedState;
jdoDetachedState[0]: the Object Id of the instance
jdoDetachedState[1]: the Version of the instance
jdoDetachedState[2]: a BitSet of loaded fields
jdoDetachedState[3]: a BitSet of modified fields

Only the BitSet of modified fields is potentially modified by the
client domain object, so the rest of the state could potentially be
encoded by the server as an opaque token to carry around on the client
object until it returns to the server over RPC.

The JDO spec has a lot of complexity, so I'm sure there's other
issues, and probably having the Datanucleus guys help would be
fruitful, but I think there are enough hooks available in RPC right
now to make it work. I was able to practically replace the entire RPC
machinery for my Google Gadget-RPC stuff by just overriding the RPC
generator in my module.


-Ray

[Robert Hanson]

> If you don't preserve identity, I don't see how you can modify the objects
> in the client. For example (client-side code):
>
> Customer cust = service.getCustomer(custId);
> cust.setCustomerId(newId);
> service.updateCustomer(cust);

I typically don't include a setter for the ID value. If it is null,
then it is an insert, if not null it is an update. Either way, it is
really my problem. In some cases I might not even need to send the ID
to the client, because it is read-only.

> As I understand it, these hidden fields are used to record a
> unique identifier so it can tell a modified one from a different one

I am willing to take the responsibility of handling this myself. Like
I said, if the ID is read only, and it is non-null, then it is an
update. Easy enough.

> Thus, I think the issue does boil down to tracking identity across RPC.

I am already handling identifying updates vs inserts myself because I
am using DTOs. I am not asking for a cure-all, all I am asking for is
to be able to use the same objects because it is easier.

The use of Serializable instead of IsSerializable was for the same
reason... just to make it easier. GWT's serialization never meant to
abide by the Seralizable contract.

> If you start adding lots of unusual restrictions like you can't modify the
> primary key in the client or create new instances there, it isn't clear what
> value you are gaining over just creating DTOs

The point is that I don't need a second set of classes for the same
data. I don't want to have to duplicate code, and I don't want to
have to write routines to copy fields between model objects and DTOs.
Less code is always a good thing.

Rob

On Apr 15, 12:09 am, John Tamplin <j...@google.com> wrote:
> On Tue, Apr 14, 2009 at 11:39 PM, Robert Hanson

> <iamroberthan...@gmail.com>wrote:

[Robert Hanson]

Thanks Ray, I will need to follow what you did a little more closely.

Rob

On Apr 15, 12:02 am, Ray Cromwell <cromwell...@gmail.com> wrote:
> On Tue, Apr 14, 2009 at 8:39 PM, Robert Hanson
>

[Robert Hanson]

RAY> @PersistenceCapable(identityType = IdentityType.APPLICATION,
RAY> detachable = "false")

Ray, I reread your post, and I think that perhaps I had thought that
this only applied to JDO, but seems to also apply to JPA as well,
correct?

I guess this works, but that leaves one outstanding issue for me. I
want to use detachable instances on the server-side, yet still have
unenhanced objects to send to the GWT client.

BRUNO> For information, Gilead provides the @ServerOnly annotation to prevent
BRUNO> sending sensible fields over the wire.

Thanks Bruno, I will need to check that out.

RAY> The RPC generator could implement this in a way, such that the
RAY> received object from the RPC call is not a Customer, but a
RAY> DetachableCustomerImpl (which extends Customer). This would be a
RAY> client side 'enhanced' version which does dirty bit tracking

I like this idea. It seems to solve a most(?) of the issues.

Rob

[Miguel Ping]

Can someone show me the benefits of implementing detached instances
for GWT? AFAIK, the advantage is detecting that an entity bean is
dirty/not dirty, thus preventing some unecessary db queries. If
someone could post some code showing some scenarios, that would be
great. Like Rob, I can handle inserts vs. updates fairly easy (my
persistence manager can tell based on the @Id property if its an
insert or update), and with a little bit of conscious effort I can
avoid unecessary db calls.

Besides, adding extra fields to entities in order to provide for
client-side detached bookeeping would increase the size of my payload
in a non-transparent way (it would depend heavily on the underlying
persistence impl). Maybe I'm missing something here, so please
enlighten me :)

Best Regards,
Miguel

[Ray Cromwell]

On Wed, Apr 15, 2009 at 8:45 PM, Robert Hanson
<iamrobe...@gmail.com> wrote:

> Ray, I reread your post, and I think that perhaps I had thought that
> this only applied to JDO, but seems to also apply to JPA as well,
> correct?

AFAIK, JDO is a superset of JPA now, and DataNucleus implements JPA as
a facade around their JDO implementation. As the GAE ORM sample that
comes in the distribution shows, you can persist a JDO class in either
an EntityManager or PersistenceManager, and JPA/JDO annotations seem
rather interchangeable.

> I guess this works, but that leaves one outstanding issue for me.  I
> want to use detachable instances on the server-side, yet still have
> unenhanced objects to send to the GWT client.

The best way this should be done is to modify the DataNucleus/JDO
enhancer to add @GwtTransient annotation to the jdoDetachedState
field. This will tell GWT RPC to ignore it, but Java RMI/Serialization
won't. Optionally, RemoteServiceServlet should have some kind of
callback like isPersistent(Field) and let the app decide if it wants,
teaching SerializabilityUtil about this delegation.

Keep in mind, if you design your application to "open transaction in
view/open session in view" you can keep the domain objects attached
all throughout the request. Detach only seems relevant if you're
sending objects through RPC, or sticking them in a JCache or something
that lives across requests.

[Ray Cromwell]

If the goal is to handle inserts vs updates yourself and simply reuse
ORM objects as DTOs over RPC, you can achieve this today, even on GWT
1.5.3, with zero modifications to either GWT or GAE, you simply tell
the enhancer to turn off detachability.

-Ray

[Miroslav Pokorny]

Another useful RemoteServiceServlet method would be soMething that
allows the developer to coerse types into another compatible type.
This would make it easy to send foe example the value object instead
of the enhanced type instance. It might also potentially allow
anonymous Lists to be sent as ArrayLists and do on.

Naturally most of the time the method mentioned above would simply
return the type given.


If Hibernate.isEnhancwdType( type) then
return type.gwtSuperclass();
else
return type.

Since hibernate enhances by subclassing the above should work.

Another visitor method that sees each and every field before being
serialized field with the option of returning a differrnt but
hopefully compatible ibstance might also be useful.

Both are naturally powerful but there ate cases where developers need
a mechanism where they can tailor the serialization process. Maybe
another answer is to open up the factory method that returns the
ServerSerisluxationOutputStream and let the user then have the
opportunity to decorate etc as they wish...

[Gary Miller]

I haven't yet got my hands dirty and tried GAE/J with GWT+JDO but I
came across some blogs that seem to get it working kind-of out-of-the-
box without using detachable = "false".

http://fredsa.allen-sauer.com/2009/04/1st-look-at-app-engine-using-jdo.html
and
http://blog.sriramnarayan.com/2009/04/gwt-and-jdo-on-java-appengine.html

Both have issues working round the GWT compiler not being able to find
source code, but don't mention the RPC system having any issues.

I'm really interest if anyone can shed some light on why they don't
mention running into GWT-RPC issues.

[brendan]

It's sounds like Philip is describing me. I've said "GWT and GAE/J
looks impressive, I want to put them together and make an nice app".

In my test application i'm creating, I want to have a grid of
customers that has new, edit and delete buttons.

A customer has a name and an email.

I've created a customer class on the server that has name and email as
strings, with getters/setters.

I've created an RPC service that has the following methods:
List<Customer> list()
Customer create(Customer customer)
void update(Customer customer)
void delete(Customer customer)

Initially list was returning a hard coded list, and I put some
breakpoints in the other methods to see that objects were successfully
being passed over RPC.

Everything seemed to work and I was thinking this GWT+GAE/J stuff is
pretty awesome.

Then I tried to add persistence, and things stopped working.
I added the @PersistenceCapable annotation to my customer class, along
with an extra id attribute that had an @PrimaryKey annotation.

When I try to return the results of a query.execute() (casted to
List<Customer>) I got the following error.

SEVERE: [1240842247754000] javax.servlet.ServletContext log: Exception
while dispatching incoming RPC call
com.google.gwt.user.client.rpc.SerializationException: Type
'org.datanucleus.store.appengine.query.StreamingQueryResult' was not
included in the set of types which can be serialized by this
SerializationPolicy or its Class object could not be loaded. For
security purposes, this type will not be serialized.

I fiddled around a bit more and found that if I copied Customers out
of the list returned by query.execute to a new list (using the
detachCopy method) and returned that, I could populate my grid.

However when it comes time to saving/deleting Customers, I tried
"reattaching" by calling makePersistant, it wouldn't work I got more
errors.

Pretty much, I'm a javascript/c++/c#.net programmer coming to GWT and
GAE/J, I want to put them together and make an nice app.

If I want things to work right now, what is best practice for having
persistent objects on the server, and sending the data back and forth
from the server to the client.

Brendan

On Apr 15, 6:40 pm, philip <philip14...@gmail.com> wrote:
> Thats fine your debating how best to do it for the longer term, and
> thats great for the longer term. How about now today? how best can I
> do it now?
>
> For the shorter term I am sure there is a bunch of people who look at
> this and think, I've got GWT and Google App Engine, I want to put them
> together and make an nice app, then they come across these
> discussions. Alternatively, they try to serialize the JDO classes and
> find they have problems. Either way, they hit the ugly wall.
> They probably wonder why GWT and App Engine are not such good friends
> such it would make a great programming platform and boost GWT.
>

> ...
>
> Thanks, Philip

[Julio Faerman]

I am having the same problem as brendan. I thing that everyone trying
to get beyond hello-world gwt + gae/j also hit this wall.
Is there a "best workaround" for this? must i use gilead?
I really would like to see this on google tutorials, like the other
frameworks mentioned by Philip, so that learning gwt+gae/j gets less
fustrating.

[noon]

Hello,

For information, I just finished a first version of
"adapter4appengine" module for Gilead.
It is just a 1st milestone, and it is only available in the project
SVN trunk, but it seems to work pretty well with my demo application
(also available in 'samples' subproject).

The main issues I am facing are about GAE DataNucleus module, which
seems to have a lot of bugs (DataNucleus does not seem to be the cause
of this) : even simple "detached state" fails with JPA encapsulation.
JDO works better, but still have some annoying issues.

I will probably make an official announce very soon, and will open a
new topic on Contributor Group, because I had to modify the GWT
Serialization code to make it work (I added support for custom
transformers and filters in it, for clean third party integration).

Stay tuned !
Bruno

[KaffeineComa]

Hi,

Just jumping into an old thread to see if there is any news on this
problem. I'm just getting started with App Engie and immediately ran
into the same issue. For now, I'm planning to duplicate my model
objects, which, as others have mentioned, is a serious violation of
DRY. But for now it seems the simplest solution. What is everyone
else doing?

Regards,
K

On Apr 14, 11:42 pm, Robert Hanson <iamroberthan...@gmail.com> wrote:
> [Continued - Mail got away from me]
>
> ...Besides fixing GWT-RPC with GAE, it also allows me to trim data
> that I don't really need on the client side.
>
> The rest, like knowing when to persist vs merge an incoming object
> from the client, is my problem to figure out.
>
> Rob
>
> On Tue, Apr 14, 2009 at 11:39 PM, Robert Hanson
>

[Ray Cromwell]

I think there's going to be a mini-summit at Google I/O next week. The
problem goes beyond attachment/detachment and how to handle that with
GWT, there's also the fact that Datanucleus inserts custom versions of
java.util.Collections, so RemoteServiceServlet has to be alot smarter
about serializing these fields on the way out, as well as reattaching
transient instances on the way in, plus there's the issue of maybe
tracking dirty fields on the client.

I've pretty much decided to go the DTO approach at this point as it is
known to work 100%. I don't feel the current hacks are production
worthy.

-Ray

[philip andrew]

Hi Brendan,

 

Just to follow up - I use DTO (Data Transfer Objects) to transfer the data back and forward.

I use BeanUtils.copyProperties(obj, dto); as the data goes to the client and comes back to the server.

 

Also use this key strategy in the JDO class:

    @PrimaryKey
    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
    @Extension(vendorName="datanucleus", key="gae.encoded-pk", value="true")
    private String encodedKey;

    @Extension(vendorName="datanucleus", key="gae.pk-name", value="true")
    private String keyName;

I'm using it but it doesn't mean I'm happy about it - don't see what else I can do.

 

I've used JBoss SEAM which has a nice way to have "conversations" and binding to the UI. Also between FLEX and JBoss SEAM I've used GraniteDS, these have very advanced bindings between the database and the user interface.

Someone should consider these when thinking about binding GWT with Google App Engine, as they have much more advanced and easier binding strategies.

 

Thanks, Philip

[noon]

Hello all,

For information, I published last week a first milestone of
"adapter4appengine", available in Gilead site, which handles the
"jdoDetachedState", backed collections and other enhancement of
DataNucleus entities. I hope to provide a seamless integration between
GAE datastore and GWT in the same way that Gilead does for Hibernate.

To achieve it, I had to "open" the GWT RPC serialization process, to
support clean custom filters and transformers, and hope to propose it
for contribution to GWT very soon.

Finally, I have to say that I disagree the idea of DTO could be a
*productive* and *efficient* solution to this problem. Yes, it is
probably the one that currently works, but it is really a pain to
maintain two different class hierarchy.
We have to use it because it is a GWT lack, and the good solution
would probably be that GWT allow third party libraries using class
enhancement to perform a clean detachement inside RPC process.

My 2 cents
Bruno

On 26 mai, 04:12, philip andrew <philip14...@gmail.com> wrote:
> Hi Brendan,
>
> Just to follow up - I use DTO (Data Transfer Objects) to transfer the data
> back and forward.
> I use BeanUtils.copyProperties(obj, dto); as the data goes to the client and
> comes back to the server.
>
> Also use this key strategy in the JDO class:
>     @PrimaryKey
>     @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
>     @Extension(vendorName="datanucleus", key="gae.encoded-pk", value="true")
>     private String encodedKey;
>     @Extension(vendorName="datanucleus", key="gae.pk-name", value="true")
>     private String keyName;
>
> I'm using it but it doesn't mean I'm happy about it - don't see what else I
> can do.
>
> I've used JBoss SEAM which has a nice way to have "conversations" and
> binding to the UI. Also between FLEX and JBoss SEAM I've used GraniteDS,
> these have very advanced bindings between the database and the user
> interface.
> Someone should consider these when thinking about binding GWT with Google
> App Engine, as they have much more advanced and easier binding strategies.
>
> Thanks, Philip
>

[Piotr Jaroszyński]

Hey Ray,

> I've pretty much decided to go the DTO approach at this point as it is
> known to work 100%. I don't feel the current hacks are production
> worthy.

You have mentioned on your blog that you use protobufs for DTOs. Could
you please elaborate on that?

P.S. Can't wait for gwt sessions to be uploaded on youtube!

--
Best Regards,
Piotr Jaroszyński

[Mark Renouf]

Glad to see progress on this. Ray, did anything come out of
discussions at Google I/O?

There seems to be two schools of thought here (both equally valid
approaches mind you):

1. Help the client participate in the persistence, maintain detached
state, dirty field lists, etc. This seems lofty and makes me more
concerned about tight coupling, but is probably exactly what is needed
for some applications.

2. Make a clean cut and keep the client free from persistence
concerns. This could be RPC with seperate DTOs or marshalled XML/JSON.
The client is only able to save by sending the modified object which
is then fully updated in the database (all fields, since we lost track
of what changed). This would typically happen by having the server
reconstruct the object (unmarshal), then merge this over an existing
entity.

Our app actually functions as #2. Currently we use GWT-generated XML
conversion code on the client. It's inefficient if your entity has
many fields and you change just one. We've employed "sparse xml" to
help, where we send back only the elements we want to modify (all of
them are optional). When this is applied to the entity, only one field
is triggered in the update. In theory this is great but I haven't
implemented to dirty field tracking in the client, so in practice we
serialize the whole object and we don't get any benefit from it.

So I guess I'm sort of sitting on fence on this one. I beleive #1 can
work, but I need to see proof of it in a large-ish app before we go
and touch anything.

[Ray Ryan]

I'm really skeptical of #1. The new AdWords UI uses a tool supported variant of #2, and I think that's the direction any general GWT support needs to go. 

ORM objects just don't make good decoupled wire objects. I think what we need to do is provide tooling to make the inevitable DRY violations as mechanical and painless as possible. 

rjrjr

2009/6/1 Mark Renouf <mark....@gmail.com>

[tonjac]

Hi all,
I think we need something that is in between #1 and #2. A "deep cut"
as proposed in #2 but without the need to maintain dual models when
they are exactly the same. We don't need to save the detached state as
in #1 it works just fine with JPA to merge a entity when turned back
to the server.

I've used GWT with openJPA for some time now and it works just fine. I
was glad to see the Java support at GAE but was put down when the RPC/
persistence/JPA issue slapped me in the face and my app couldn't be
ported to GAE without a lot of work.

When looking in to what differs between openJPA and Datanucleus JPA I
found that openJPA stores it "detachedstate" in a transient fields:

// Open JPA enhanced entity
private transient Object pcDetachedState;

// Datanucleus enhanced entity
protected Object jdoDetachedState[];

As I see it this would do the trick, can Datanucleus enhance be
altered to work in this way?

Regards
Toni Thomsson

On 2 Juni, 22:07, Ray Ryan <rj...@google.com> wrote:
> I'm really skeptical of #1. The new AdWords UI uses a tool supported variant
> of #2, and I think that's the direction any general GWT support needs to
> go.
> ORM objects just don't make good decoupled wire objects. I think what we
> need to do is provide tooling to make the inevitable DRY violations as
> mechanical and painless as possible.
> rjrjr
>

> 2009/6/1 Mark Renouf <mark.ren...@gmail.com>

[Mark Renouf]

To toss another idea out there, has anyone considered actually trying
protocol buffers in the browser? Shouldn't it be possible to write a
code generator that uses protocol buffers, and ships them over as
binary data? Protobuf support is already there for Java, C++,
Python...