1/2 off topic: Could graph comparision be efficiently implemented on any of the serializers?

[Sebastien Diot]

I have this StackOverflow question with a +50 bonus running out in a few hours:

http://stackoverflow.com/questions/10044476/is-there-a-object-change-tracking-versioning-java-api-out-there#comment13634053_10044476

It's basically about the ability to repeatedly serialize an object graph, but with the purpose of finding the difference and storing them instead of the whole graph. The main business purpose being to keep an history, and be able to reverse individual changes, under certain conditions.

As I am expecting that some of you here might have a detailed knowledge of many different Java serializers, it might be that you could answer it, at least better than what I got so far. I should have posted this here earlier, but I hadn't realized about that there was a forum where the results were discussed, and also it's not related in any way to benchmarks.

[Tatu Saloranta]

Interesting question! My guess is that this is something that needs to
live at higher level; efficient serializers are hard enough to do,
without worrying about 'advanced' concepts like diffing. So in some
sense it is orthogonal concept; although some aspects are overlapping
(esp. if one has to handle typing, cyclic references).

-+ Tatu +-

> --
> You received this message because you are subscribed to the Google Groups
> "java-serialization-benchmarking" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/java-serialization-benchmarking/-/ixW5pHOZCRMJ.
> To post to this group, send email to
> java-serializat...@googlegroups.com.
> To unsubscribe from this group, send email to
> java-serialization-be...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/java-serialization-benchmarking?hl=en.

[Nate]

Kryo v1 had a serializer that knows about the last data that was serialized and only emits a delta. When reading, it knows about the last data received and applies the delta. The delta is done on at the byte level. Here is the serializer:
https://code.google.com/p/kryo/source/browse/tags/kryo-1.05/src/com/esotericsoftware/kryo/compress/DeltaCompressor.java
Most of the work is done by this class:
https://code.google.com/p/kryo/source/browse/tags/kryo-1.05/src/com/esotericsoftware/kryo/compress/Delta.java
This could be used in a few useful ways, eg networking similar to Quake 3.

This was omitted in Kryo v2 because AFAIK it had never been put to use. Also, it did not have an extensive set of tests. It could be ported though and may do what you need, or serve as the basis for what you need.

-Nate

On Thu, May 10, 2012 at 11:07 AM, Sebastien Diot <skunki...@googlemail.com> wrote:

--

[Tatu Saloranta]

On Thu, May 10, 2012 at 2:09 PM, Nate <nathan...@gmail.com> wrote:
> Kryo v1 had a serializer that knows about the last data that was serialized
> and only emits a delta. When reading, it knows about the last data received
> and applies the delta. The delta is done on at the byte level. Here is the
> serializer:
> https://code.google.com/p/kryo/source/browse/tags/kryo-1.05/src/com/esotericsoftware/kryo/compress/DeltaCompressor.java
> Most of the work is done by this class:
> https://code.google.com/p/kryo/source/browse/tags/kryo-1.05/src/com/esotericsoftware/kryo/compress/Delta.java
> This could be used in a few useful ways, eg networking similar to Quake 3.
>
> This was omitted in Kryo v2 because AFAIK it had never been put to use.
> Also, it did not have an extensive set of tests. It could be ported though
> and may do what you need, or serve as the basis for what you need.

Very interesting. Thank you for sharing this.

Come to think of it, perhaps I could play with one follow-up idea.
Jackson has "updateValue" functionality, in which one can do partial
'merge' of data, giving existing POJO: although this is currently just
a shallow update, it could (and should given time) be possible to
allow deep merges too.
Now: diff can obviously be viewed as the reverse of this... and might
not be super hard, using some of existing machinery.

-+ Tatu +-