Serialisation of Object/BitSet

[datanucleus]

Hi,

I'm not a GWT user, instead I write DataNucleus (used by AppEngine).
There's an issue around GWT and RPC serialisation when using a
bytecode enhanced class. The class (when enhanced) has a field

Object[] jdoDetachedState;

which basically contains
Object[0] is an object of the primary key type of the class
(serialisable)
Object[1] is an object representing the version of the object (Long or
Timestamp, so serialisable too)
Object[2] is a BitSet
Object[3] is a BitSet

See http://groups.google.com/group/google-appengine-java/browse_thread/thread/9b6983753a843b99/37a87031b5ccd768
for what GWT gives with this. My understanding is that the problem is
GWT's problem serialising the BitSet. Is this correct ? And if so is
there any chance of a fix to GWT for this? Or is there more to it ...
e.g the "Object[]" and you'd need changes at our side?
Would be nice to understand the scope of the problem at the very least
so we can plan how to go forward.

Regards
--Andy

[Dean S. Jones]

If you look at the Java source for BitSet, you'll see it uses an array
of long[].... JavaScript has no concept of long, so GWT has an
"simulation" of long, but it's not perfect.

One could easily write an emulation class and serializers, but it's
not likely the bit math would be correct without totally rewriting the
classes using int.

[datanucleus]

Thx Dean. So if this "Object[]" field had Object[2] stored as boolean
[] and Object[3] stored as boolean[] then this would work ?

[Vitali Lovich]

Are you sure the GWT long acts differently than a true Java long?  My understanding was that as of 1.5 the long on the client correctly emulates a real long primitive.

No?

http://groups.google.com/group/Google-Web-Toolkit-Contributors/browse_thread/thread/3c768d8d33bfb1dc seems to be the thread that discusses the issue in your thread.

I think the conclusion is (I only skimmed the thread) that JPA injects other code & that's where the problem comes in, because GWT doesn't know how to serialize that.

[Dean S. Jones]

At this point, I'm not going to bet my life on GWT simulation of long
and bit operations.

boolean[] is less efficient, but easier to trust.

[Vitali Lovich]

On Sat, Apr 11, 2009 at 5:02 AM, Dean S. Jones <deans...@gmail.com> wrote:

At this point, I'm not going to bet my life on GWT simulation of long
and bit operations.

I would because first of all you can check the code - this kind of stuff has been written hundreds of times for all sorts of different bit sizes.  I'm sure there's textbooks on the stuff.  Secondly, just run in hosted mode & then you'll have the real native Java long.

boolean[] is less efficient, but easier to trust..

Does this actually solve the problem though?  If it doesn't (and I suspect it doesn't) you're focusing on the wrong problem.
 

[datanucleus]

Well not being a GWT user I can't comment on the details of what is
going on in the GWT side, but from the enhanced class side here you
can see an enhanced class
http://db.apache.org/jdo/enhancement.html
with the before and after. The field "jdoDetachedState" is the one
referred to, and this relies on any updates to the object coming in
via setters which in turn set the flags (in jdoDetachedState) for
modified (e.g look at setB() method, relaying to jdoSetb). If that
notion doesn't hold in GWT space for some reason then people can
simply be referred to Rays workaround for now.

Storing as boolean[] would not be a problem but as Vitali said may not
be the real issue.

[datanucleus]

I'll move discussion to the other thread that Vitali linked to avoid
duping. Thx for your time.

[Dean S. Jones]

Every time I run into this kind of issue, BitSet, BigInteger,
BigDouble.... I dig into the source code and find one common element:
long, long, long.

3 GWT projects I thought I was smarter than the GWT team and
implemented the emul classes and serializers. Worked most of the time,
but failed on the edge cases. When you work at a Bank and your numbers
go wrong, and the skew is in the Billions, well, you fall back to
strings and do the math on the server.

[Vitali Lovich]

I have no experience with the JDO stuff, so I can't comment on it.

On Sat, Apr 11, 2009 at 5:33 AM, Dean S. Jones <deans...@gmail.com> wrote:

Every time I run into this kind of issue, BitSet, BigInteger,
BigDouble.... I dig into the source code and find one common element:
long, long, long.

How big exactly are the numbers that you are dealing with in your data entry fields that they don't fit into a regular integer (+/- about 2 billion).  And the long use in BigInteger & BigDouble would be strictly for performance reasons.  You should be able to (carefully) replace them with ints & it'll still work, albeit more slowly for bigger numbers.

3 GWT projects I thought I was smarter than the GWT team and
implemented the emul classes and serializers. Worked most of the time,
but failed on the edge cases.

There's plenty of stuff out there about how to emulate big integers (it really isn't that difficult to do - but difficult to get right with good performance).
 

When you work at a Bank and your numbers
go wrong, and the skew is in the Billions, well, you fall back to
strings and do the math on the server.

Yes - that is the correct approach that should be taken when dealing with important data - let the server deal with parsing it.