Proposal for improving JDO/JPA RPC support

[Daniel Rice (דניאל רייס)‎]

Hi all -

  I've been working on a patch to improve support for RPC of persistence-enhanced objects that would replace the one I recently submitted as trunk revision 5672.  I'd like to give those of you who are interested in the interaction between RPC and persistence a chance to help me validate the design before moving forward.

 The idea is to be able to deal with different persistence mechanisms in a way that does not depend too much on the details of their implementations. We assume that the persistence mechanism provides a way for instances to be detached from the object store, serialized, deserialized, and reattached in a well-defined way. Our strategy leverages this capability as follows:

1) User code on the server detaches the object and places it into a state such that serialization is valid

2) GWT detects whether any instance fields are present on the object that were not known to client code

3) GWT performs hybrid serialization:

  a) GWT uses its regular RPC mechanism for the client-visible fields

  b) GWT uses Java serialization for any additional server-only instance fields

  c) GWT encodes the results of step (b) as a String

  d) GWT prepends the encoded String to the regular RPC data and transmits to the client

4) Client code treats the object in the normal way and does not interact with the data from step (3b)

5) When the object is sent back to the server, the normal RPC mechanism is altered as follows

  a) the encoded server data is decoded and deserialized into the new object instance

  b) the client data is populated using setXXX() method invocations rather than by directly setting field values

All this is done only for classes that can be send bidirectionally, and which are determined to be (potentially) enhanced.

The potential for enhancement is signaled in one of three ways:

o The user adds the fully-qualified class name to a 'gwt.enhancedClasses' configuration property

o The class is determined to have the JDO @PersistenceCapable annotation with detachable=true

o The class is determined to have the JPA @Entity annotation

If a class is determined to be potentially enhanced, the list of fields known to the client is added to the '.gwt.rpc' file that contains the RPC whitelist.  This list is used by step 2 of the RPC process which compares the set of client- and server-visible fields.

  Step (5b) is necessary to accommodate mechanisms like that of JDO, where the setter methods are enhanced to provide object state tracking (dirty bits).  If we were to set the fields directly, the object detached state would not be updated properly.

  My hope is that this mechanism would be general enough to support a variety of persistence mechanisms without the need to add a lot of special-case code.  I'm looking forward to any comments that you have as to whether this will work with your favorite persistence API, or any other thoughts that you have.  Thanks,

Dan

[Isaac Truett]

> 5) When the object is sent back to the server, the normal RPC mechanism is
> altered as follows
> a) the encoded server data is decoded and deserialized into the new object
> instance
> b) the client data is populated using setXXX() method invocations rather
> than by directly setting field values

Is this introducing a requirement that all serializable fields have
setters? Or will direct field access be used where a setter can't be
found?

[Daniel Rice (דניאל רייס)‎]

  The latter.  The first time the class is encountered, GWT would search for setters (field '<T> theField' would cause a search for a setter 'void setTheField(<T> arg)') and create a map from field names to setters.  If there's no setter for a given field, the field would be set directly.

Dan

[Sami Jaber]

Hi Dan,

That sounds to be a really good step in the right direction.
This mecanisms seems to cover all the nominal use cases that we would face regarding the different o/r mapping impl.
I would just make a few remarks :

1) When you call all the setXX() methods when the object is sent back to the server, the o/r mapper will assume that those fields have been modified and the dirty state tracking (which is activated before the setXX() call if I well understand) will raise an "update" operation for all the fields. I think you have to compare bean n-1 field values and bean n field values to call only the setXX() for the modified values. What about collection ? Hibernate provides a PersistentSet that ensure the object state tracking, will you call setXX() for all items of client data ?

2) If you analyze an enhanced entity in the Hibernate land you will see that it contains many coarse-grained information, like a SessionImplementor (it depends on the dynamic proxy factory and impl you have used) -> https://www.hibernate.org/hib_docs/v3/api/org/hibernate/proxy/ProxyFactory.html. Serializing those data in a String based representation could be potentially harmful in term of bandwith and payload. Why not providing a way to store this client data info in the HttpSession ?

Sami

[Daniel Rice (דניאל רייס)‎]

Hi Sami -

1) To track the previous state would seem to require doubling the amount of data sent from server to client, which doesn't seem like a good tradeoff in general.  My proposal avoids the need to send extra data by simply allowing everything to be marked as dirty.

  What are your specific questions about collections and PersistentSet?

2) The only places I can see to store the extra field data are a) on the client or b) in the persistent store.  The HTTP session won't work, I think.  For example, the server could be behind a load-balancer, so the object would be materialized on server A, sent to the client, and sent back to server B.  If the extra data is not sent along with the object, it has to be somewhere that both A and B can access.  It can't just live in A's VM heap.  My thought is that it would be counterintuitive for GWT RPC to attempt to modify the persistent store, and probably would create problems.

Dan

[Sami Jaber]

Hi Dan,

Please correct me if I'm wrong. As I understand your proposal, when entity A get back to the server after being modified by the client, you assume that all fields are dirty if extra data are not present ? If extra data exists, you don't compare extra-data state representation (which would be in fact n-1 fields state) with the actual state of your entity ? In other terms, are extra data only a raw base64 serialization of the previous bean state ?

My concern about PersistentCollection is the following -> Hibernate ensure that dirty state tracking is handled by a kind of client side collection wrapper. If you remove the PersistentCollection type and its behavior, you lose the ability to track on the client side the elements inserted/removed/modified. When you go back to the server, you can only assume that all the elements of your collection are dirty, which would raise (with Hibernate) a [delete all/insert all] operation instead of a simple update. Transitive persistence  and cascading merging are handled by information stored in PersistentCollection (that other would call an Observable Collection).

Regarding the second point, I agree with you that storing the extra-data on the client is better to ensure that you are stateless architecture compliant. You will probably get complains coming from people that have a pretty complex object graph to serialize.

Sami

[Daniel Rice (דניאל רייס)‎]

  At least in JDO, the extra data is essentially a couple of bit sets.  It doesn't contain any additional copies of field values.  I'm not sure what would happen with collections -- if one were used as a field value, we would call a setter and hand it entire collection, which I suppose Hibernate would have to treat as an entirely new object.  Trying to track the individual changes on the client seems like a much harder problem than what I am proposing at the moment.

Dan