I didn't see a thread on this yet, but the GWT-RPC doesn't work well in GAE with JPA. It seems that the JPA entities are "enhanced" at compile time, and this is not compatible with the GWT serialization on the client side. The GWT app complains that the serialization is not compatible.
And you can always use a DTO, but I recall that Serializable was added to GWT just so that you didn't need to do that.
So... I'm not really asking for a fix... I am more wondering what the "official response" is. Is this something that you want to fix, something that isn't safe to fix, or something that you don't want to fix?
The only 'real' way to make detached entities to work with GWT would be to
make the 'enhancer' generate Java source like a GWT generator OR make an
interface like 'Detachable' and have a GWT generator do it, because not only
do you need the extra state fields (jdoDetachedState), but the method calls
of the class have to be intercepted to manipulate the detached state.
Actually, a generator probably can't do it because IIRC JDO enhancement
involves intercepting even field references, and Generators don't get an
AST.
So it's either transient instances, or some kind of Ant task that generates
GWT-friendly DTO helper classes for you along with the enhancement process.
-Ray
On Fri, Apr 10, 2009 at 7:19 AM, Robert Hanson <iamroberthan...@gmail.com>wrote:
> I didn't see a thread on this yet, but the GWT-RPC doesn't work well
> in GAE with JPA. It seems that the JPA entities are "enhanced" at
> compile time, and this is not compatible with the GWT serialization on
> the client side. The GWT app complains that the serialization is not
> compatible.
> And you can always use a DTO, but I recall that Serializable was added
> to GWT just so that you didn't need to do that.
> So... I'm not really asking for a fix... I am more wondering what the
> "official response" is. Is this something that you want to fix,
> something that isn't safe to fix, or something that you don't want to
> fix?
I haven't tried GAE/J just yet, but I'm guessing the problem with JPA
is the same one you would have with any database. If your domain
objects aren't very large, you can use Dozer to map the object onto
itself which removes all the enhancements and makes them regular
objects.
On Fri, Apr 10, 2009 at 12:30 PM, Ray Cromwell <cromwell...@gmail.com> wrote:
> The only 'real' way to make detached entities to work with GWT would be to
> make the 'enhancer' generate Java source like a GWT generator OR make an
> interface like 'Detachable' and have a GWT generator do it, because not only
> do you need the extra state fields (jdoDetachedState), but the method calls
> of the class have to be intercepted to manipulate the detached state.
> Actually, a generator probably can't do it because IIRC JDO enhancement
> involves intercepting even field references, and Generators don't get an
> AST.
> So it's either transient instances, or some kind of Ant task that generates
> GWT-friendly DTO helper classes for you along with the enhancement process.
> -Ray
> On Fri, Apr 10, 2009 at 7:19 AM, Robert Hanson <iamroberthan...@gmail.com>
> wrote:
>> I didn't see a thread on this yet, but the GWT-RPC doesn't work well
>> in GAE with JPA. It seems that the JPA entities are "enhanced" at
>> compile time, and this is not compatible with the GWT serialization on
>> the client side. The GWT app complains that the serialization is not
>> compatible.
>> Ray has a workaround, but it is less than ideal
>> And you can always use a DTO, but I recall that Serializable was added
>> to GWT just so that you didn't need to do that.
>> So... I'm not really asking for a fix... I am more wondering what the
>> "official response" is. Is this something that you want to fix,
>> something that isn't safe to fix, or something that you don't want to
>> fix?
I copied persisted entities to non-persisted copies, and again, it
fails. I needed to create a separate class, with the same fields as
the model object, and then copy the data to that. It seems that any
class with the @Entity annotation is unable to be passed to the
client.
On Fri, Apr 10, 2009 at 3:59 PM, Arthur Kalmenson <arthur.k...@gmail.com> wrote:
> I haven't tried GAE/J just yet, but I'm guessing the problem with JPA
> is the same one you would have with any database. If your domain
> objects aren't very large, you can use Dozer to map the object onto
> itself which removes all the enhancements and makes them regular
> objects.
> --
> Arthur Kalmenson
> On Fri, Apr 10, 2009 at 12:30 PM, Ray Cromwell <cromwell...@gmail.com> wrote:
>> The only 'real' way to make detached entities to work with GWT would be to
>> make the 'enhancer' generate Java source like a GWT generator OR make an
>> interface like 'Detachable' and have a GWT generator do it, because not only
>> do you need the extra state fields (jdoDetachedState), but the method calls
>> of the class have to be intercepted to manipulate the detached state.
>> Actually, a generator probably can't do it because IIRC JDO enhancement
>> involves intercepting even field references, and Generators don't get an
>> AST.
>> So it's either transient instances, or some kind of Ant task that generates
>> GWT-friendly DTO helper classes for you along with the enhancement process.
>> -Ray
>> On Fri, Apr 10, 2009 at 7:19 AM, Robert Hanson <iamroberthan...@gmail.com>
>> wrote:
>>> I didn't see a thread on this yet, but the GWT-RPC doesn't work well
>>> in GAE with JPA. It seems that the JPA entities are "enhanced" at
>>> compile time, and this is not compatible with the GWT serialization on
>>> the client side. The GWT app complains that the serialization is not
>>> compatible.
>>> Ray has a workaround, but it is less than ideal
>>> And you can always use a DTO, but I recall that Serializable was added
>>> to GWT just so that you didn't need to do that.
>>> So... I'm not really asking for a fix... I am more wondering what the
>>> "official response" is. Is this something that you want to fix,
>>> something that isn't safe to fix, or something that you don't want to
>>> fix?
On Fri, Apr 10, 2009 at 6:48 PM, Robert Hanson <iamroberthan...@gmail.com>wrote:
> I copied persisted entities to non-persisted copies, and again, it > fails. I needed to create a separate class, with the same fields as > the model object, and then copy the data to that. It seems that any > class with the @Entity annotation is unable to be passed to the > client.
Conceivably the RPC generator could recognize the JDO/JPA annotations and take into account the bytecode rewriting that will happen in the server version, but I don't know what all that would entail.
-- John A. Tamplin Software Engineer (GWT), Google
Don't use JPA annotations, use JDO, because you can specify
non-detachable objects. Even if you hack the generators and it passes
the CRC checks you'll still end up with possibly broken code, because
on the server, the methods of the class are rewritten to modify these
hidden fields, and on the client, you'll be getting the original code.
Basically, trying to ship detached objects over RPC and maintain all
of the detached 'dirty' state, while being wire-efficient I think is
impractical and error-prone, so the style of programming where you
ship objects straight out of the ORM to the client should be
discouraged.
I'm of the opinion of some kind of ProtocolBuffer-like system, with
built-in capability of efficiently copying state to-and-from ORM and
PB classes would be better and more deterministic (and also support
non-GWT clients like Android!)
On Fri, Apr 10, 2009 at 4:21 PM, John Tamplin <j...@google.com> wrote:
> On Fri, Apr 10, 2009 at 6:48 PM, Robert Hanson <iamroberthan...@gmail.com>
> wrote:
>> I copied persisted entities to non-persisted copies, and again, it
>> fails. I needed to create a separate class, with the same fields as
>> the model object, and then copy the data to that. It seems that any
>> class with the @Entity annotation is unable to be passed to the
>> client.
> Conceivably the RPC generator could recognize the JDO/JPA annotations and
> take into account the bytecode rewriting that will happen in the server
> version, but I don't know what all that would entail.
> --
> John A. Tamplin
> Software Engineer (GWT), Google
Too bad that Google has to launch AppEngine to discover that ORM is
definitively a problem for GWT. I send a mail to Bruce in early summer 2007
about that issue. His answer (that I perfectly understand) was to use DTO.
The fact is that DTO is no more a recommended pattern when your domain model
is pure Java/POJO and client and server are tightly coupled. According to
Gavin Kind and Rod Johnson, “DTO are evil” (
http://forum.springsource.org/archive/index.php/t-15538.html). There are
many situations where DTO are useful, but in general they could be avoided.
There is no “clean” solution for this problem unfortunately. All the ORM
tools provide the “dirty state checking” mechanism by enhancing the entities
object (Hibernate uses javassist during the attach process, jdo
pre-enhanced during the compilation step). The issue comes from the fact
that most of enhancers rely on java specific classes that are not
serializable (SessionImplementor for Hibernate, …). Sending any Lazy
initialized POJO over the wire to a non Java Client (Soap, XML, Json, …)
leads to the same problem.
The idea behind hibernate4gwt is to replace on the fly (before
InvokeAndEncodeRequest()) the proxies/specific classes by their JavaScript
counter-part (a hashmap containing all the lazy initialized fields). This
mechanism is implemented either with a generator (the generator generates
subclasses that holds the hashmap proxy fields/information) or with a
“magic” super class (LightEntity) which holds the same map.
Whatever we do, the process is necessarily specific to the ORM
implementation because the way you get back the JavaScriptObject and you
convert it back in Java depends on the Proxy implementation. The only way
to get rid of this problem is to provide a mechanism based on “adaptors”
that would be plugged during the Serialization process for converting
attached entities to detached javascript compliant entities. Those Hibernate
Adaptors would be provided by the ORM providers (Hibernate, DataNucleus,
OpenJPA, etc …) and would handle for us all the nasty under-the-cover
conversions stuff.
This problem requires a big implication of all the actors (ORM providers and
Google/Community). Without this big bang, all the solutions based on Dozer,
BeanLib, protocolbuffer generator (as stated by Ray) and DTO conversion
tools will be a pain (many of them do not handle circular reference, deep
graphs, etc..) and will lead to poor performance.
I’m working with Bruno (hibernate4gwt) to try to move the adaptor mechanism
in the custom serialization (ServerSerializationWriter/Reader) process. I
know that those classes are not meant to be extended as they are internal
implementation and will probably be removed in the next version. Is there
any work on a kind of (real) pluggable serialization mechanism for GWT 2 ?
On Sat, Apr 11, 2009 at 3:05 AM, Ray Cromwell <cromwell...@gmail.com> wrote:
> Don't use JPA annotations, use JDO, because you can specify
> non-detachable objects. Even if you hack the generators and it passes
> the CRC checks you'll still end up with possibly broken code, because
> on the server, the methods of the class are rewritten to modify these
> hidden fields, and on the client, you'll be getting the original code.
> Basically, trying to ship detached objects over RPC and maintain all
> of the detached 'dirty' state, while being wire-efficient I think is
> impractical and error-prone, so the style of programming where you
> ship objects straight out of the ORM to the client should be
> discouraged.
> I'm of the opinion of some kind of ProtocolBuffer-like system, with
> built-in capability of efficiently copying state to-and-from ORM and
> PB classes would be better and more deterministic (and also support
> non-GWT clients like Android!)
> -Ray
> On Fri, Apr 10, 2009 at 4:21 PM, John Tamplin <j...@google.com> wrote:
> > On Fri, Apr 10, 2009 at 6:48 PM, Robert Hanson <
> iamroberthan...@gmail.com>
> > wrote:
> >> I copied persisted entities to non-persisted copies, and again, it
> >> fails. I needed to create a separate class, with the same fields as
> >> the model object, and then copy the data to that. It seems that any
> >> class with the @Entity annotation is unable to be passed to the
> >> client.
> > Conceivably the RPC generator could recognize the JDO/JPA annotations and
> > take into account the bytecode rewriting that will happen in the server
> > version, but I don't know what all that would entail.
> > --
> > John A. Tamplin
> > Software Engineer (GWT), Google
As Gilead (and Hibernate4GWT) author, I completely agree with Sami :
GWT serialization has to support extension points to allow third
parties adapters.
A good inspiration is probably Adobe BlazeDS (RPC communication for
Flex), which was *designed* to seamlessly support adapters and
factories.
As Sami mentioned, this kind of issues are known since first releases
of GWT, and the DTO answer is not a good one (it is error prone, non
productive and breaks Domain Driven Design that GWT Java support
promises). In my opinion, GWT team must not ignore this problem
anymore and should start to work on this really fast ("we will think
about it for RPC 2.0" was already the answer 2 years ago !).
Regards
Bruno
On 11 avr, 12:00, Sami Jaber <sami.ja...@gmail.com> wrote:
> Too bad that Google has to launch AppEngine to discover that ORM is
> definitively a problem for GWT. I send a mail to Bruce in early summer 2007
> about that issue. His answer (that I perfectly understand) was to use DTO.
> The fact is that DTO is no more a recommended pattern when your domain model
> is pure Java/POJO and client and server are tightly coupled. According to
> Gavin Kind and Rod Johnson, “DTO are evil” (http://forum.springsource.org/archive/index.php/t-15538.html). There are
> many situations where DTO are useful, but in general they could be avoided.
> There is no “clean” solution for this problem unfortunately. All the ORM
> tools provide the “dirty state checking” mechanism by enhancing the entities
> object (Hibernate uses javassist during the attach process, jdo
> pre-enhanced during the compilation step). The issue comes from the fact
> that most of enhancers rely on java specific classes that are not
> serializable (SessionImplementor for Hibernate, …). Sending any Lazy
> initialized POJO over the wire to a non Java Client (Soap, XML, Json, …)
> leads to the same problem.
> The idea behind hibernate4gwt is to replace on the fly (before
> InvokeAndEncodeRequest()) the proxies/specific classes by their JavaScript
> counter-part (a hashmap containing all the lazy initialized fields). This
> mechanism is implemented either with a generator (the generator generates
> subclasses that holds the hashmap proxy fields/information) or with a
> “magic” super class (LightEntity) which holds the same map.
> Whatever we do, the process is necessarily specific to the ORM
> implementation because the way you get back the JavaScriptObject and you
> convert it back in Java depends on the Proxy implementation. The only way
> to get rid of this problem is to provide a mechanism based on “adaptors”
> that would be plugged during the Serialization process for converting
> attached entities to detached javascript compliant entities. Those Hibernate
> Adaptors would be provided by the ORM providers (Hibernate, DataNucleus,
> OpenJPA, etc …) and would handle for us all the nasty under-the-cover
> conversions stuff.
> This problem requires a big implication of all the actors (ORM providers and
> Google/Community). Without this big bang, all the solutions based on Dozer,
> BeanLib, protocolbuffer generator (as stated by Ray) and DTO conversion
> tools will be a pain (many of them do not handle circular reference, deep
> graphs, etc..) and will lead to poor performance.
> I’m working with Bruno (hibernate4gwt) to try to move the adaptor mechanism
> in the custom serialization (ServerSerializationWriter/Reader) process. I
> know that those classes are not meant to be extended as they are internal
> implementation and will probably be removed in the next version. Is there
> any work on a kind of (real) pluggable serialization mechanism for GWT 2 ?
> Sami
> On Sat, Apr 11, 2009 at 3:05 AM, Ray Cromwell <cromwell...@gmail.com> wrote:
> > Don't use JPA annotations, use JDO, because you can specify
> > non-detachable objects. Even if you hack the generators and it passes
> > the CRC checks you'll still end up with possibly broken code, because
> > on the server, the methods of the class are rewritten to modify these
> > hidden fields, and on the client, you'll be getting the original code.
> > Basically, trying to ship detached objects over RPC and maintain all
> > of the detached 'dirty' state, while being wire-efficient I think is
> > impractical and error-prone, so the style of programming where you
> > ship objects straight out of the ORM to the client should be
> > discouraged.
> > I'm of the opinion of some kind of ProtocolBuffer-like system, with
> > built-in capability of efficiently copying state to-and-from ORM and
> > PB classes would be better and more deterministic (and also support
> > non-GWT clients like Android!)
> > -Ray
> > On Fri, Apr 10, 2009 at 4:21 PM, John Tamplin <j...@google.com> wrote:
> > > On Fri, Apr 10, 2009 at 6:48 PM, Robert Hanson <
> > iamroberthan...@gmail.com>
> > > wrote:
> > >> I copied persisted entities to non-persisted copies, and again, it
> > >> fails. I needed to create a separate class, with the same fields as
> > >> the model object, and then copy the data to that. It seems that any
> > >> class with the @Entity annotation is unable to be passed to the
> > >> client.
> > > Conceivably the RPC generator could recognize the JDO/JPA annotations and
> > > take into account the bytecode rewriting that will happen in the server
> > > version, but I don't know what all that would entail.
> > > --
> > > John A. Tamplin
> > > Software Engineer (GWT), Google
> The issue comes from the fact that most of enhancers rely on java specific classes
> that are not serializable (SessionImplementor for Hibernate, …).
JDO enhancers don't, since all contents of "jdoDetachedState" (Object
[]) are serializable, although in our case GWT apparently has problems
with BitSet (due to long[] being used inside I'm told).
> Those HibernateAdaptors would be provided by the ORM providers (Hibernate, DataNucleus,
> OpenJPA, etc …) and would handle for us all the nasty under-the-cover conversions stuff.
> This problem requires a big implication of all the actors (ORM providers and Google/Community).
> I’m working with Bruno (hibernate4gwt) to try to move the adaptor mechanism in the custom serialization
If anything is required from DataNucleus just let us know.
Having your RPC generator recognise JDO/JPA annotations isn't a way to
go since the user may have defined persistence in XML. The one thing
is, the field is always called jdoDetachedState so can't this be
recognised ? (non-GWT person here so excuse stupid questions ;-) ).
Ray's workaround (of using transient) obviously will work, and we
(DataNucleus) could provide complete helper methods to allow this
copying back in of a "transient GWTed" object to aid the process, but
something more complete would be desirable.
From a DataNucleus perspective, everything included in enhanced
classes is Serializable (deliberately). We are willing to contribute
to the process of aiding the fix of GWT where you require any specific
handler for DataNucleus usage. Just let us know.
[I've tried to post to this group twice so far and it always says Post
successful, but never appears, so who knows if this will]
One possibility might be to add features to the JDO enhancer to spit
out Java source for IsSerializable tagged classes instead of just
generating enhanced byte code. This code could then be seen by the GWT
compiler. In which case, all of the extra jdo methods, as well as
jdoDetachedState would be on the object. You could even emit custom
serializers.
However, I am nervous about the idea of sending actual detached
objects over the wire. The philosophy of GWT is to minimize latency,
by reducing HTTP requests and wire size. People might have a tendency
to blindly ship detached instances over the wire, which could result
in either too much serialized data being sent, or too much lazy
fetching. It is true that JDO allows fetch groups to control this, but
it seems like it would more consideration to the client than is usual
for JDO clients, as it seems many people using ORMs are trying to
optimize RDBMS performance, not serialization performance.
That's why I like the PB approach, because with PB's, it's very clear
that cycles can't occur, that optional nullable fields take up no
space, and that relations require explicit fetching on the client. The
downside is, it violates DRY, and I agree, I hate the DTO pattern.
I am not really an expert in the ORM domain, so the datanucleus guys
probably have alot better recommendations for patterns on using JDO to
minimize fetching and wire-size.
I do think that with Datanucleus's help, there can probably be a real
solution, and if done right, perhaps one that addresses common
mistakes people make with wire-size and latency.
> Having your RPC generator recognise JDO/JPA annotations isn't a way to
> go since the user may have defined persistence in XML. The one thing
> is, the field is always called jdoDetachedState so can't this be
> recognised ? (non-GWT person here so excuse stupid questions ;-) ).
> Ray's workaround (of using transient) obviously will work, and we
> (DataNucleus) could provide complete helper methods to allow this
> copying back in of a "transient GWTed" object to aid the process, but
> something more complete would be desirable.
The major stumbling block remains that the enhanced classes are
unknown and unavailsble in your client/GWT application. If the server
did allow (and it could - it's no big deal to recursively serialize
all fields) the puoblem remains on the client. Failing on the server
is actually good practice e - and csn be considered to be failing fast.
The gwt serielization architecture does not use reflection but rather
generates a new serializer for each and every type it feels can occur
in a serialized stream response from the server. This can be easily
witnessed by setting logging to debut or using the option that writes
generated classes to a specific directory "-gen <directory>".
Back to the enhanced classes - they are unknown and do not exist at
compile time when the generator creates all tbe required serializer
types. The client code rpc proxy stuff contains a big switch statement
mapping serializable classes to generated/custom serializers. Problem
is the names of the enhanced classes are not present in the switch
thus deserialuzation fails .
This is the same reason onecannot serialize anon classes. Javac
generates names for anon classes and these of course don't match this
on the client. Icould be wrong ok the anon class thing but the
statement is true if older gwt versions.
Ultimately to solveyour problem and avoid copying Instances of
enganced classes onto regular classes the easiest thing is to probably
hack the server code which does the serislizing. Rather than sending
down enhanced type instances write the state of the class/instance
underneath and skip the fields holding the detached state. Naturally
one loses the ability to reattach but it could be argued passing dtos
thru all those layers all the way to client is exposi g way too much
of your persistance impl detail.
Hth
On 12/04/2009, at 12:07 AM, datanucleus <andy_jeffer...@yahoo.com>
wrote:
> From a DataNucleus perspective, everything included in enhanced
> classes is Serializable (deliberately). We are willing to contribute
> to the process of aiding the fix of GWT where you require any specific
> handler for DataNucleus usage. Just let us know.
> [I've tried to post to this group twice so far and it always says Post
> successful, but never appears, so who knows if this will]
Absolutely Ray, PB violates DRY principles and could be easily handled as a
specific adaptor in the whole extensibility model.
This idea is to agree on an extensibility pattern similar to the way one
extend RPC processCall(). CFS (Custom Field Serializer) may be an answer but
they are supposed to be removed in further versions.
Customizing the JDO enhancer certainly does the trick but responds to a
global problem by a specific solution. The main ORM used today is Hibernate
(thru JPA with annotations or XML files), DataNucleus will probably gain
momentum with AppEngine but people are waiting responses in priority for
their ORM implementation.
To finish on the Latency, this problem is not specific to GWT/ORM but to ORM
in general. Anyone who uses ORM should take care of FetchGroup,
EagerFetching, N+1 select and minimizing roundtrip.
The way Hibernate4GWT solves this problem should be a starting point of
thinking IMHO. This Framework looks like complex because of the lack of
extensibility model in the serialization process, it is up to us (or GWT
team) to simplify this mechanism.
On Sat, Apr 11, 2009 at 7:30 PM, Ray Cromwell <cromwell...@gmail.com> wrote:
> One possibility might be to add features to the JDO enhancer to spit
> out Java source for IsSerializable tagged classes instead of just
> generating enhanced byte code. This code could then be seen by the GWT
> compiler. In which case, all of the extra jdo methods, as well as
> jdoDetachedState would be on the object. You could even emit custom
> serializers.
> However, I am nervous about the idea of sending actual detached
> objects over the wire. The philosophy of GWT is to minimize latency,
> by reducing HTTP requests and wire size. People might have a tendency
> to blindly ship detached instances over the wire, which could result
> in either too much serialized data being sent, or too much lazy
> fetching. It is true that JDO allows fetch groups to control this, but
> it seems like it would more consideration to the client than is usual
> for JDO clients, as it seems many people using ORMs are trying to
> optimize RDBMS performance, not serialization performance.
> That's why I like the PB approach, because with PB's, it's very clear
> that cycles can't occur, that optional nullable fields take up no
> space, and that relations require explicit fetching on the client. The
> downside is, it violates DRY, and I agree, I hate the DTO pattern.
> I am not really an expert in the ORM domain, so the datanucleus guys
> probably have alot better recommendations for patterns on using JDO to
> minimize fetching and wire-size.
> I do think that with Datanucleus's help, there can probably be a real
> solution, and if done right, perhaps one that addresses common
> mistakes people make with wire-size and latency.
> -Ray
> On Sat, Apr 11, 2009 at 2:34 AM, datanucleus <andy_jeffer...@yahoo.com>
> wrote:
> > Hi,
> > didn't see this thread since I tried on the main GWT group. See
> > Having your RPC generator recognise JDO/JPA annotations isn't a way to
> > go since the user may have defined persistence in XML. The one thing
> > is, the field is always called jdoDetachedState so can't this be
> > recognised ? (non-GWT person here so excuse stupid questions ;-) ).
> > Ray's workaround (of using transient) obviously will work, and we
> > (DataNucleus) could provide complete helper methods to allow this
> > copying back in of a "transient GWTed" object to aid the process, but
> > something more complete would be desirable.
I discovered the hard way that GWT and ORM needs some custom code.
IMHO, the real problem is that class enhancers do not provide a way to
unenhance an instance according to some rules. I ended up writing my
custom bean copier based on dozer and beanmapper, so I can specify a
shallow/deep clone, though most of the times I'll just be getting rid
of proxies, since I fetch ahead what I know I'll need.
To my understanding, GWT-like applications (read: the service layer)
don't need lazy load for most of the time; lazy load works most for
open-session-in-view. I think almost all devs code to fetch the needed
objets up front anyway.
The real problem for me is the following:
- I want to send an enhanced object over the wire.
- The object may or may not be serializable due to class enhancing.
- My object class declaration is fully serializable (but the
underlying type may not be due to proxies).
I think we have two choices:
- copy the object to a fully serializable class using some bean mapper
with some rules
- describe a way to convert the proxied/enhanced class to an
unenhanced one, again with some rules
Both these ways benefit from an extensible serialization mechanism in
GWT. In the end, I'd like to see a plugin mechanism from GWT to
provide custom (de)serializers for RPC mechanisms, and a simple way of
"dehydrating" a persistent/enhanced class through some rules.
On Apr 12, 11:41 am, Sami Jaber <sami.ja...@gmail.com> wrote:
> Absolutely Ray, PB violates DRY principles and could be easily handled as a
> specific adaptor in the whole extensibility model.
> This idea is to agree on an extensibility pattern similar to the way one
> extend RPC processCall(). CFS (Custom Field Serializer) may be an answer but
> they are supposed to be removed in further versions.
> Customizing the JDO enhancer certainly does the trick but responds to a
> global problem by a specific solution. The main ORM used today is Hibernate
> (thru JPA with annotations or XML files), DataNucleus will probably gain
> momentum with AppEngine but people are waiting responses in priority for
> their ORM implementation.
> To finish on the Latency, this problem is not specific to GWT/ORM but to ORM
> in general. Anyone who uses ORM should take care of FetchGroup,
> EagerFetching, N+1 select and minimizing roundtrip.
> The way Hibernate4GWT solves this problem should be a starting point of
> thinking IMHO. This Framework looks like complex because of the lack of
> extensibility model in the serialization process, it is up to us (or GWT
> team) to simplify this mechanism.
> Hope (you would) help
> Sami
> On Sat, Apr 11, 2009 at 7:30 PM, Ray Cromwell <cromwell...@gmail.com> wrote:
> > One possibility might be to add features to the JDO enhancer to spit
> > out Java source for IsSerializable tagged classes instead of just
> > generating enhanced byte code. This code could then be seen by the GWT
> > compiler. In which case, all of the extra jdo methods, as well as
> > jdoDetachedState would be on the object. You could even emit custom
> > serializers.
> > However, I am nervous about the idea of sending actual detached
> > objects over the wire. The philosophy of GWT is to minimize latency,
> > by reducing HTTP requests and wire size. People might have a tendency
> > to blindly ship detached instances over the wire, which could result
> > in either too much serialized data being sent, or too much lazy
> > fetching. It is true that JDO allows fetch groups to control this, but
> > it seems like it would more consideration to the client than is usual
> > for JDO clients, as it seems many people using ORMs are trying to
> > optimize RDBMS performance, not serialization performance.
> > That's why I like the PB approach, because with PB's, it's very clear
> > that cycles can't occur, that optional nullable fields take up no
> > space, and that relations require explicit fetching on the client. The
> > downside is, it violates DRY, and I agree, I hate the DTO pattern.
> > I am not really an expert in the ORM domain, so the datanucleus guys
> > probably have alot better recommendations for patterns on using JDO to
> > minimize fetching and wire-size.
> > I do think that with Datanucleus's help, there can probably be a real
> > solution, and if done right, perhaps one that addresses common
> > mistakes people make with wire-size and latency.
> > -Ray
> > On Sat, Apr 11, 2009 at 2:34 AM, datanucleus <andy_jeffer...@yahoo.com>
> > wrote:
> > > Hi,
> > > didn't see this thread since I tried on the main GWT group. See
> > > Having your RPC generator recognise JDO/JPA annotations isn't a way to
> > > go since the user may have defined persistence in XML. The one thing
> > > is, the field is always called jdoDetachedState so can't this be
> > > recognised ? (non-GWT person here so excuse stupid questions ;-) ).
> > > Ray's workaround (of using transient) obviously will work, and we
> > > (DataNucleus) could provide complete helper methods to allow this
> > > copying back in of a "transient GWTed" object to aid the process, but
> > > something more complete would be desirable.
I fully agree with Miguel analysis and solution proposal.
In my mind, I wrote Gilead as just a serialization adapter for
Hibernate objects, and similar work should have to be done for
DataNucleus, OpenJPA and al (by the supplier of such libraries, or
community...)
Can we have the "official GWT team" point of view about it ?
Regards
Bruno
On 13 avr, 19:04, Miguel Ping <miguel.p...@gmail.com> wrote:
> I discovered the hard way that GWT and ORM needs some custom code.
> IMHO, the real problem is that class enhancers do not provide a way to
> unenhance an instance according to some rules. I ended up writing my
> custom bean copier based on dozer and beanmapper, so I can specify a
> shallow/deep clone, though most of the times I'll just be getting rid
> of proxies, since I fetch ahead what I know I'll need.
> To my understanding, GWT-like applications (read: the service layer)
> don't need lazy load for most of the time; lazy load works most for
> open-session-in-view. I think almost all devs code to fetch the needed
> objets up front anyway.
> The real problem for me is the following:
> - I want to send an enhanced object over the wire.
> - The object may or may not be serializable due to class enhancing.
> - My object class declaration is fully serializable (but the
> underlying type may not be due to proxies).
> I think we have two choices:
> - copy the object to a fully serializable class using some bean mapper
> with some rules
> - describe a way to convert the proxied/enhanced class to an
> unenhanced one, again with some rules
> Both these ways benefit from an extensible serialization mechanism in
> GWT. In the end, I'd like to see a plugin mechanism from GWT to
> provide custom (de)serializers for RPC mechanisms, and a simple way of
> "dehydrating" a persistent/enhanced class through some rules.
> On Apr 12, 11:41 am, Sami Jaber <sami.ja...@gmail.com> wrote:
> > Absolutely Ray, PB violates DRY principles and could be easily handled as a
> > specific adaptor in the whole extensibility model.
> > This idea is to agree on an extensibility pattern similar to the way one
> > extend RPC processCall(). CFS (Custom Field Serializer) may be an answer but
> > they are supposed to be removed in further versions.
> > Customizing the JDO enhancer certainly does the trick but responds to a
> > global problem by a specific solution. The main ORM used today is Hibernate
> > (thru JPA with annotations or XML files), DataNucleus will probably gain
> > momentum with AppEngine but people are waiting responses in priority for
> > their ORM implementation.
> > To finish on the Latency, this problem is not specific to GWT/ORM but to ORM
> > in general. Anyone who uses ORM should take care of FetchGroup,
> > EagerFetching, N+1 select and minimizing roundtrip.
> > The way Hibernate4GWT solves this problem should be a starting point of
> > thinking IMHO. This Framework looks like complex because of the lack of
> > extensibility model in the serialization process, it is up to us (or GWT
> > team) to simplify this mechanism.
> > Hope (you would) help
> > Sami
> > On Sat, Apr 11, 2009 at 7:30 PM, Ray Cromwell <cromwell...@gmail.com> wrote:
> > > One possibility might be to add features to the JDO enhancer to spit
> > > out Java source for IsSerializable tagged classes instead of just
> > > generating enhanced byte code. This code could then be seen by the GWT
> > > compiler. In which case, all of the extra jdo methods, as well as
> > > jdoDetachedState would be on the object. You could even emit custom
> > > serializers.
> > > However, I am nervous about the idea of sending actual detached
> > > objects over the wire. The philosophy of GWT is to minimize latency,
> > > by reducing HTTP requests and wire size. People might have a tendency
> > > to blindly ship detached instances over the wire, which could result
> > > in either too much serialized data being sent, or too much lazy
> > > fetching. It is true that JDO allows fetch groups to control this, but
> > > it seems like it would more consideration to the client than is usual
> > > for JDO clients, as it seems many people using ORMs are trying to
> > > optimize RDBMS performance, not serialization performance.
> > > That's why I like the PB approach, because with PB's, it's very clear
> > > that cycles can't occur, that optional nullable fields take up no
> > > space, and that relations require explicit fetching on the client. The
> > > downside is, it violates DRY, and I agree, I hate the DTO pattern.
> > > I am not really an expert in the ORM domain, so the datanucleus guys
> > > probably have alot better recommendations for patterns on using JDO to
> > > minimize fetching and wire-size.
> > > I do think that with Datanucleus's help, there can probably be a real
> > > solution, and if done right, perhaps one that addresses common
> > > mistakes people make with wire-size and latency.
> > > -Ray
> > > On Sat, Apr 11, 2009 at 2:34 AM, datanucleus <andy_jeffer...@yahoo.com>
> > > wrote:
> > > > Hi,
> > > > didn't see this thread since I tried on the main GWT group. See
> > > > Having your RPC generator recognise JDO/JPA annotations isn't a way to
> > > > go since the user may have defined persistence in XML. The one thing
> > > > is, the field is always called jdoDetachedState so can't this be
> > > > recognised ? (non-GWT person here so excuse stupid questions ;-) ).
> > > > Ray's workaround (of using transient) obviously will work, and we
> > > > (DataNucleus) could provide complete helper methods to allow this
> > > > copying back in of a "transient GWTed" object to aid the process, but
> > > > something more complete would be desirable.
So I like GWT and I like Google App Engine, all I want is a simple
recipe for getting the two to work without hitting too many big
problems.
I'm willing to use DTO, I'm willing to suffer whatever I have to do -
but I need to know what's the best current way to do it.
Thanks, Philip
On Apr 10, 10:19 pm, Robert Hanson <iamroberthan...@gmail.com> wrote:
> I didn't see a thread on this yet, but the GWT-RPC doesn't work well
> in GAE with JPA. It seems that the JPA entities are "enhanced" at
> compile time, and this is not compatible with the GWT serialization on
> the client side. The GWT app complains that the serialization is not
> compatible.
> And you can always use a DTO, but I recall that Serializable was added
> to GWT just so that you didn't need to do that.
> So... I'm not really asking for a fix... I am more wondering what the
> "official response" is. Is this something that you want to fix,
> something that isn't safe to fix, or something that you don't want to
> fix?
On Tue, Apr 14, 2009 at 10:26 AM, philip <philip14...@gmail.com> wrote: > So I like GWT and I like Google App Engine, all I want is a simple > recipe for getting the two to work without hitting too many big > problems. > I'm willing to use DTO, I'm willing to suffer whatever I have to do - > but I need to know what's the best current way to do it.
GWT RPC does not preserve identity across the wire, and the GWT compiler needs the source of the classes at compile time. Since JDO/JPA rewrite the bytecode on the server, and may do so based on information not in the source, there won't be any short-term workaround. Even if the enchancer produced source code instead, trying to serialize Object[] (what the enhancer adds) means every serializable type in the program has to have deserializers present in the client, which would be a massive size increase for programs of any significant size.
The only way I see it working would be to add a flag requiring preserving object identity across RPC, and the protocol would include an object ID with each object if that flag is set. The server would then save/restore additional hidden fields on its end for any persistent classes which were rewritten, and use the object ID to know what values should be recreated for these hidden fields when a persistent object comes back from the client.
A few issues with this approach:
- what happens for a persistent object created on the client that the server has never seen before? - how are field references handled? Since a class can be made persistent without the GWT compiler's knowledge, it can't do anything about this (and even if it could you wouldn't want to for client code-size reasons). I assume this is only needed to know which fields are "dirty" and need to be updated -- maybe you can just assume anything coming from the client needs to be replaced entirely. Since the GAE data store works that way anyway that shouldn't be a big inefficiency, but could affect using this approach with non-GAE persistence engines. - Since you don't have clone in JS, any copy of an object would have to be treated as a new instance, which could be the wrong answer in some situations. Is making it mostly work as expected acceptable, or will the odd cases confuse developers more than just requiring them to do things explicitly?
This would require significant work in the GWT RPC subsystems and could mean a significant cost in the wire-protocol size.
IMHO, blindly transferring your stored data across the wire is convenient but produces big, slow client code. That may be useful for development, and it may be worth providing it for that reason and for intranet apps where the client size is less important, but I think having to write your own DTOs so you know exactly what is going over the wire and why forces you to think about why you need it in the client and produce a more efficient app. It is too easy to just send a Customer record to the client because that is what you have, even when all you need is just the name/address portions for your shipping form, etc.
-- John A. Tamplin Software Engineer (GWT), Google
On Tue, Apr 14, 2009 at 12:03 PM, John Tamplin <j...@google.com> wrote:
> On Tue, Apr 14, 2009 at 10:26 AM, philip <philip14...@gmail.com> wrote:
>> So I like GWT and I like Google App Engine, all I want is a simple
>> recipe for getting the two to work without hitting too many big
>> problems.
>> I'm willing to use DTO, I'm willing to suffer whatever I have to do -
>> but I need to know what's the best current way to do it.
> GWT RPC does not preserve identity across the wire, and the GWT compiler
> needs the source of the classes at compile time. Since JDO/JPA rewrite the
> bytecode on the server, and may do so based on information not in the
> source, there won't be any short-term workaround. Even if the enchancer
> produced source code instead, trying to serialize Object[] (what the
> enhancer adds) means every serializable type in the program has to have
> deserializers present in the client, which would be a massive size increase
> for programs of any significant size.
> The only way I see it working would be to add a flag requiring preserving
> object identity across RPC, and the protocol would include an object ID with
> each object if that flag is set. The server would then save/restore
> additional hidden fields on its end for any persistent classes which were
> rewritten, and use the object ID to know what values should be recreated for
> these hidden fields when a persistent object comes back from the client.
> A few issues with this approach:
> - what happens for a persistent object created on the client that the
> server has never seen before?
> - how are field references handled? Since a class can be made
> persistent without the GWT compiler's knowledge, it can't do anything about
> this (and even if it could you wouldn't want to for client code-size
> reasons). I assume this is only needed to know which fields are "dirty" and
> need to be updated -- maybe you can just assume anything coming from the
> client needs to be replaced entirely. Since the GAE data store works that
> way anyway that shouldn't be a big inefficiency, but could affect using this
> approach with non-GAE persistence engines.
> - Since you don't have clone in JS, any copy of an object would have to
> be treated as a new instance, which could be the wrong answer in some
> situations. Is making it mostly work as expected acceptable, or will the
> odd cases confuse developers more than just requiring them to do things
> explicitly?
> This would require significant work in the GWT RPC subsystems and could
> mean a significant cost in the wire-protocol size.
> IMHO, blindly transferring your stored data across the wire is convenient
> but produces big, slow client code. That may be useful for development, and
> it may be worth providing it for that reason and for intranet apps where the
> client size is less important, but I think having to write your own DTOs so
> you know exactly what is going over the wire and why forces you to think
> about why you need it in the client and produce a more efficient app. It is
> too easy to just send a Customer record to the client because that is what
> you have, even when all you need is just the name/address portions for your
> shipping form, etc.
I think this is a false dichotomy. JDO has fetch-groups for just that kind
of scenario. You send down a slice of an object graph that meets exactly
your needs. You don't have to re-invent an entire DTO layer just to send a
subset of your data. You can re-use the same domain objects in several
different RPC calls with several different fetch-groups each tailored for
the specific use case.
I think we need to tread carefully here if we're to make progress and
produce something everyone can be happy with. We need people who really
understand this domain to get together and think this through offline,
coming back with at least semi-baked proposals we can discuss, evaluate, and
adjust. We'd end up with much more productive discussion instead of lots of
idle speculation.
On Tue, Apr 14, 2009 at 9:03 AM, John Tamplin <j...@google.com> wrote: > GWT RPC does not preserve identity across the wire, and the GWT compiler > needs the source of the classes at compile time.
> The only way I see it working would be to add a flag requiring preserving > object identity across RPC, and the protocol would include an object ID with > each object if that flag is set.
Actually, JDO has a concept called application identity where object identity is handled by the application. This typically boils down to the PrimaryKey being used as object identity.
> what happens for a persistent object created on the client that the server > has never seen before?
Calling persist() on it would insert it rather than merge it.
> IMHO, blindly transferring your stored data across the wire is convenient > but produces big, slow client code.
This I agree with, I've been drum beating on this for awhile. ORM + RPC is the surest way to produce 10,000 messages in the newsgroup about "why my RPC calls are so slow and I get slow script warnings" For me, the issue is to find a middle ground, a way of using DTOs, a pattern, a DSL, a utility, that gives you DTO semantics without excessive boilerplate. First of all, in almost all cases of an ORM object sent over RPC, lazy association fetching would be wrong (resulting in excess roundtrips), but pre-emptive fetches would also be wrong much of the time. JDO/JPA work on the paradigm that the server controls fetch semantics, but often times, it is the client that knows what it wants to pull down, not the server, and this is especially true if the client and server are written by different people. Explicit finder methods with explicit DTOs are rather self documenting and obvious on their face, what it is they do and what's being transferred, the downside is a multiplication of boilerplate and house keeping code.
This is certainly an area where DSLs + generators, libraries, and compilers can help. Consider how easy Guice + Warp-Persist makes ORM finders.
public interface CustomerDAO { @Finder("select c from Customer c where c.zipCode = :zipcode") Customer findCustomerByZipCode(@Named("zipcode") String zip);
}
You inject this and everything is handled automatically, getting persistent context, opening/closing transactions, setting up the query. It makes things painless.
On Tue, Apr 14, 2009 at 10:27 AM, Toby Reyelts <to...@google.com> wrote: > I think this is a false dichotomy. JDO has fetch-groups for just that kind > of scenario. You send down a slice of an object graph that meets exactly > your needs. You don't have to re-invent an entire DTO layer just to send a > subset of your data. You can re-use the same domain objects in several > different RPC calls with several different fetch-groups each tailored for > the specific use case.
That's true, but what bothers me about this is that it seems brittle. If I pull down a domain object, one that comes with fetch group A, and pull down another domain object from fetch group B, and stick them in a Collection together, and then someone else maps over this collection, it can result in serious breakage. The assumption being made is that people working with domain objects are usually the ones calling the RPC method, and so have read the documentation that says "This returns a Customer object, but does not fetch the AccountRepresentative. Do not call Customer.getAccountRepresentative()!" It true, from the context of an AsyncCallback, you will usually know what restrictions are in effect on the domain object returned, but this will be lost as it goes down the stack chain.
Of course, you could document Customer.getAccountRepresentative() as potentially returning null, but that's sorta brittle too, because if null represents the non-existence of a representative, then other code might just to assign one!
And given the asynchronous nature of RPC, you can't really fetch these associations on demand, unless you make the getter method take an AsyncCallback, another pollution of the interface.
I know you can make this work if you're careful, but it seems brittle and subject to subtle breakage as the number of people on a project increase.
On Tue, Apr 14, 2009 at 1:38 PM, Ray Cromwell <cromwell...@gmail.com> wrote:
> On Tue, Apr 14, 2009 at 10:27 AM, Toby Reyelts <to...@google.com> wrote:
> > I think this is a false dichotomy. JDO has fetch-groups for just that
> kind
> > of scenario. You send down a slice of an object graph that meets exactly
> > your needs. You don't have to re-invent an entire DTO layer just to send
> a
> > subset of your data. You can re-use the same domain objects in several
> > different RPC calls with several different fetch-groups each tailored for
> > the specific use case.
> That's true, but what bothers me about this is that it seems brittle.
> If I pull down a domain object, one that comes with fetch group A, and
> pull down another domain object from fetch group B, and stick them in
> a Collection together, and then someone else maps over this
> collection, it can result in serious breakage. The assumption being
> made is that people working with domain objects are usually the ones
> calling the RPC method, and so have read the documentation that says
> "This returns a Customer object, but does not fetch the
> AccountRepresentative. Do not call
> Customer.getAccountRepresentative()!" It true, from the context of an
> AsyncCallback, you will usually know what restrictions are in effect
> on the domain object returned, but this will be lost as it goes down
> the stack chain.
> Of course, you could document Customer.getAccountRepresentative() as
> potentially returning null, but that's sorta brittle too, because if
> null represents the non-existence of a representative, then other code
> might just to assign one!
> And given the asynchronous nature of RPC, you can't really fetch these
> associations on demand, unless you make the getter method take an
> AsyncCallback, another pollution of the interface.
> I know you can make this work if you're careful, but it seems brittle
> and subject to subtle breakage as the number of people on a project
> increase.
I agree that it's not a panacea. I am not aware of any panacea in existence.
My opinion is that GWT users would be best-served if they had both DTO and
DTO-less solutions. I believe that the solution that best fits your needs
depends upon your team and your project requirements. Rather than
philosophising, I'd like to see "expert" teams working out possible
approaches with working prototypes.
On Tue, Apr 14, 2009 at 1:28 PM, Ray Cromwell <cromwell...@gmail.com> wrote: > On Tue, Apr 14, 2009 at 9:03 AM, John Tamplin <j...@google.com> wrote: > > The only way I see it working would be to add a flag requiring preserving > > object identity across RPC, and the protocol would include an object ID > with > > each object if that flag is set.
> Actually, JDO has a concept called application identity where object > identity is handled by the application. This typically boils down to > the PrimaryKey being used as object identity.
Is there an equivalent for JPA?
> what happens for a persistent object created on the client that the server > > has never seen before?
> Calling persist() on it would insert it rather than merge it.
Well, what I was getting at was more what does the server-side RPC code populate the hidden fields with if it gets a new object of that type from the client?
If the hidden fields can be null and the @PrimaryKey field will be used to determine if it is a new object or a replacement of an existing one, then it seems pretty easy -- the server-side RPC code simply strips out the effect of the bytecode rewriting for serialization. I don't know enough about what rewriting happens to know how to detect/reverse it, but it seems like it should be doable.
-- John A. Tamplin Software Engineer (GWT), Google
I hope I am not being naive (or reiterating an existing post), but I
am not concerned with identity, nor concerned with expecting my ORM
solution to deal with entities created on the client.
I think all I need is a way to use an entity as a DTO without needing
a second set of model objects.
What about something as simple as keeping the existing semantics in
tact, with the ability to flag properties that are send-able to the
client (as opposed to using transient to flag the reverse behavior).
public class Data implements Serializable {
@GwtDto
private Long id;
@GwtDto
private String partA;
private String partB;
}
In this case the GWT compiler would create a serializer for the client
that only handled the id and partA properties, and ignore partB. The
partB property would simply be null on the client side.
On the server reflection would be used to only serialize the marked
properties, ignoring the rest. Besides ignoring partB it would also
ignore any fields added by enhancers.
Besides fixing GWT-RPC with GAE, it also allows me to trim data that I
don't really need o
On Tue, Apr 14, 2009 at 2:18 PM, John Tamplin <j...@google.com> wrote:
> On Tue, Apr 14, 2009 at 1:28 PM, Ray Cromwell <cromwell...@gmail.com> wrote:
>> On Tue, Apr 14, 2009 at 9:03 AM, John Tamplin <j...@google.com> wrote:
>> > The only way I see it working would be to add a flag requiring
>> > preserving
>> > object identity across RPC, and the protocol would include an object ID
>> > with
>> > each object if that flag is set.
>> Actually, JDO has a concept called application identity where object
>> identity is handled by the application. This typically boils down to
>> the PrimaryKey being used as object identity.
> Is there an equivalent for JPA?
>> > what happens for a persistent object created on the client that the
>> > server
>> > has never seen before?
>> Calling persist() on it would insert it rather than merge it.
> Well, what I was getting at was more what does the server-side RPC code
> populate the hidden fields with if it gets a new object of that type from
> the client?
> If the hidden fields can be null and the @PrimaryKey field will be used to
> determine if it is a new object or a replacement of an existing one, then it
> seems pretty easy -- the server-side RPC code simply strips out the effect
> of the bytecode rewriting for serialization. I don't know enough about what
> rewriting happens to know how to detect/reverse it, but it seems like it
> should be doable.
> --
> John A. Tamplin
> Software Engineer (GWT), Google
<iamroberthan...@gmail.com> wrote: > I hope I am not being naive (or reiterating an existing post), but I > am not concerned with identity, nor concerned with expecting my ORM > solution to deal with entities created on the client.
> I think all I need is a way to use an entity as a DTO without needing > a second set of model objects.
> What about something as simple as keeping the existing semantics in > tact, with the ability to flag properties that are send-able to the > client (as opposed to using transient to flag the reverse behavior).
> public class Data implements Serializable { > @GwtDto > private Long id; > @GwtDto > private String partA; > private String partB; > }
> In this case the GWT compiler would create a serializer for the client > that only handled the id and partA properties, and ignore partB. The > partB property would simply be null on the client side.
> On the server reflection would be used to only serialize the marked > properties, ignoring the rest. Besides ignoring partB it would also > ignore any fields added by enhancers.
> Besides fixing GWT-RPC with GAE, it also allows me to trim data that I > don't really need o
|
