Issue 400 in google-gson: Dynamic serialization and deserialization via class interface
googl...@googlecode.com
Owner: ----
Labels: Type-Defect Priority-Medium
New issue 400 by mint...@easyesi.com: Dynamic serialization and
deserialization via class interface
http://code.google.com/p/google-gson/issues/detail?id=400
I have created a patch against gson-2.1 that adds code for allowing classes
to define serialization and deserialization by implementing a simple
interface. The benefit of this approach is that there is no longer a need
to register special handlers for each class that requires custom
serialization. It can be used as follows:
public class Foo implements JsonSerialization,
JsonDeserializedBy<FooDeserializer> {
public String name;
public JsonElement serialize (JsonSerializationContext jsc) {
return new JsonPrimitive(name);
}
}
public class FooDeserializer implements JsonDeserializer<Foo> {
public Foo deserialize(JsonElement je, Type type,
JsonDeserializationContext jdc) {
Foo f = new Foo();
f.name = je.getAsString();
return f;
}
}
...
Foo foo = new Foo();
foo.name = "foo";
gson.toJson(foo); // "foo"
gson.fromJson(gson.toJson(foo), Foo.class).name; // "foo"
With my patch, a Gson object now checks the argument sent to toJson to
determine if it implements JsonSerialization. If so, it calls its
serialize(jsc) method to retrieve a JsonElement for that class.
When fromJson is called, the given class is inspected to determine if it
implements the JsonDeserializer interface. If so, the class specified in
the template parameter (in this case FooDeserializer) is used for
deserialization by creating a new instance of that class (using the
no-argument constructor), and then deserialize is called as with any
JsonDeserializer object.
This patch adds the two new interfaces (JsonSerialization and
JsonDeserialization) and adds a few lines of code to Gson in order to
provide the aforementioned functionality. I'm not sure I chose the
appropriate location to insert the functionality, and the code is pretty
ugly, but it seems to work pretty well in my limited test cases.
I'll license the patch under the same license as gson, in case anyone is
concerned about that.
Attachments:
gson-2.1-dynamic_serialization.patch 4.0 KB
googl...@googlecode.com
Comment #1 on issue 400 by mint...@easyesi.com: Dynamic serialization and
http://code.google.com/p/google-gson/issues/detail?id=400
Sorry, the example code has a small mistake, as I renamed one of the
interfaces before submitting the patch. Where it says "JsonDeserializedBy",
it should say "JsonDeserialization". The patch and the rest of my post are
accurate.
googl...@googlecode.com
Comment #2 on issue 400 by jessewil...@google.com: Dynamic serialization
http://code.google.com/p/google-gson/issues/detail?id=400
That's extremely clever!
FYI, if you're willing to make a single call to
GsonBuilder.registerTypeAdapterFactory(), I don't think you need to make
any changes to Gson 2.1 to make this work.
googl...@googlecode.com
Comment #3 on issue 400 by mint...@easyesi.com: Dynamic serialization and
http://code.google.com/p/google-gson/issues/detail?id=400
Will that actually work, though? From what I could tell looking at the
code, a TypeAdaptor is registered against a specific type that is later
retrieved with a map lookup. Since my method relies on any object simply
implementing an interface, doesn't that require a change to Gson.
The alternative would be for each object to register itself with some
globally-used Gson object. I preferred the interface approach.
googl...@googlecode.com
Comment #4 on issue 400 by mint...@easyesi.com: Dynamic serialization and
http://code.google.com/p/google-gson/issues/detail?id=400
Will that actually work, though? From what I could tell looking at the
code, a TypeAdaptor is registered against a specific type that is later
retrieved with a map lookup. Since my method relies on any object simply
implementing an interface, doesn't that require a change to Gson?
googl...@googlecode.com
Comment #5 on issue 400 by jessewil...@google.com: Dynamic serialization
http://code.google.com/p/google-gson/issues/detail?id=400
It'll work, but you need to register a TypeAdapterFactory, not a
TypeAdapter. The factory lets you support any type.
googl...@googlecode.com
Comment #6 on issue 400 by mint...@easyesi.com: Dynamic serialization and
http://code.google.com/p/google-gson/issues/detail?id=400
Thank you for the continued guidance, Jesse. A TypeAdapterFactory is
definitely the right way to do this, but I've run into a few issues with
accessibility in implementing it. I've attached my TypeAdapterFactory and
the interfaces associated with it, and I would appreciate any suggestions
you can give me on a better approach. I am aware that using
ReflectiveTypeAdapter in the way that I have is something of a hack (I
should be following an approach like TreeTypeAdapter's delegate()), but I
don't think it changes the idea much.
There are a few calls to my own custom class called "Reflection". I am not
going to include it in the attachment, but I'll specify the methods instead:
Class classOfType(Type t)
Return the upper bound on t. If (t instanceof Class), it's simply ((Class)
t). For a ParameterizedType, e.g., List<String> would be List.class. For
something like "? extends Comparable", it's Comparable.class.
Class[] getTypeParameters(Class implClass, Class genClass)
Ascends and descends the class hierarchy between implClass and genClass to
return the array indicating genClass's instantiated type parameters as
specifically as possible. For any parameter that doesn't resolve
completely, the behavior follows that of classOfType(). (In its usage here,
it's used to find the actual class DESERIALIZER when a class implements
JsonDeserialization<DESERIALIZER>.)
T newInstance(Class<T> c)
Just like c.newInstance() except that exceptions are rethrown as a
RuntimeException and the accessibility is set to true before invoking the
constructor (so that non-public constructors can be invoked).
Field getAccesibleField(Class c, String fieldName)
Calls c.getDeclaredField(fieldName), rethrowing exceptions as a
RuntimeException and setting the resulting Field's accessibility to true
and before returning it.
Attachments:
factory.zip 2.2 KB
googl...@googlecode.com
Comment #7 on issue 400 by jessewil...@google.com: Dynamic serialization
http://code.google.com/p/google-gson/issues/detail?id=400
Yeah, you probably shouldn't have to do that much work. See the
TypeAdapterFactory documentation for an example that includes delegation:
For serialization, you should delegate to the concrete class of the type
being serialized. For deserialization, you should delegate to the adapter
of the concrete class that implements your interface. You can get both type
adapters using the Gson instance passed in to create().
googl...@googlecode.com
Comment #8 on issue 400 by mint...@easyesi.com: Dynamic serialization and
http://code.google.com/p/google-gson/issues/detail?id=400
I'm sorry, but I'm having trouble following your suggestion. My mechanism
allows a class to implement one or both of JsonSerialization and
JsonDeserialization<DESERIALIZER>. If Obj obj implements the former, calls
to gson.toJson(obj) execute a callback to
obj.serialize(gson.serializationContext). If Obj implements the latter,
calls to gson.fromJson(Obj.class) effectively execute
DESERIALIZER.newInstance().deserialize(JsonElement je, Obj.class,
gson.deserializationContext).
I don't see how I can handle that with a delegate. I understand that I can
avoid my usage of ReflectiveTypeAdapter, but that's a more minor issue. The
issue I'm running up against is that in implementing my own
TypeAdapterFactory, I can't technically access gson.de/serializationContext
without hacking around the Java security system. This leads me to believe
I'm doing something wrong.
In the case where I wish to override the default de/serialization, I don't
believe that I can perform the delegation you suggest because I'm not
actually registering a type adapter for every type implementing
JsonSerialization and JsonDeserialization (my two custom interfaces). Is
there some hook to perform that registration without resorting to hacks
that use the Reflections package to find every subclass of the
aforementioned interfaces and register a TypeAdapter for each?
The point of my factory is to allow objects to simply implement an
interface instead of having to register themselves with a canonical Gson
instance. That means that I can't "delegate to the concrete class of the
type being serialized" for classes that implement JsonSerialization because
I need to ensure that the object's serialize(jsc) method is called instead.
I also don't think I can delegate for deserialization because the
deserializer is not registered with any Gson instance.
Am I missing something simple?
googl...@googlecode.com
Comment #9 on issue 400 by jessewil...@google.com: Dynamic serialization
http://code.google.com/p/google-gson/issues/detail?id=400
Got it. The JsonSerializationContext/JsonDeserializationContext APIs aren't
present nor necessary for streaming type adapters implementing the
TypeAdapter interface. Instead that interface uses 'Gson' which provides a
superset of the functionality of JsonSerializationContext and
JsonDeserializationContext.
If you want, change your interface to take a Gson instance instead. I
posted another big TypeAdapterFactory example on issue 43; you may want to
read it through.
googl...@googlecode.com
Comment #10 on issue 400 by mint...@easyesi.com: Dynamic serialization and
http://code.google.com/p/google-gson/issues/detail?id=400
Perfect! Thank you very much. I'll reply here soon with a cleaner version
of my InterfaceTypeAdapterFactory mechanism. Perhaps it will turn out to be
something worth including in trunk after a few iterations.
googl...@googlecode.com
Comment #11 on issue 400 by mint...@easyesi.com: Dynamic serialization and
http://code.google.com/p/google-gson/issues/detail?id=400
Thanks a lot for working through this issue with me. Attached is an
implementation of the mechanism I discussed. It uses the Drink example you
provided in issue 43. My goal was to make it as simple as possible for an
implementor to perform the conversion to and from Json.
It's also available on github: https://github.com/BMintern/gson-interface
In order for it to work, it must be registered with the Gson instance:
Gson gson = new GsonBuilder()
.registerTypeAdapterFactory(new InterfaceAdapterFactory())
.create();
Note that I changed the deserializer class to implement JsonDeserializes
instead of JsonDeserializer. I felt that the method signature defined there
was more consistent with JsonSerialization (and slightly easier to use).
Note also that this provides a dead-simple way for a class to recursively
serialize itself.
googl...@googlecode.com
Comment #12 on issue 400 by mint...@easyesi.com: Dynamic serialization and
http://code.google.com/p/google-gson/issues/detail?id=400
Oops, forgot attachment.
Attachments:
gson-interface.zip 15.9 KB
googl...@googlecode.com
Comment #13 on issue 400 by mint...@easyesi.com: Dynamic serialization and
http://code.google.com/p/google-gson/issues/detail?id=400
Thanks a lot for working through this issue with me. Attached is an
implementation of the mechanism I discussed. It uses the Drink example you
provided in issue 43 . My goal was to make it as simple as possible for an
implementor to perform the conversion to and from Json.
It's also available on github: https://github.com/BMintern/gson-interface
In order for it to work, it must be registered with the Gson instance:
Gson gson = new GsonBuilder()
.registerTypeAdapterFactory(new InterfaceAdapterFactory())
.create();
Note that I changed the deserializer class to implement JsonDeserializes
instead of JsonDeserializer. I felt that the method signature defined there
was more consistent with JsonSerialization (and slightly easier to use).
Note also that this provides a dead-simple way for a class to recursively
serialize itself.
Attachments:
gson-interface.zip 16.0 KB
googl...@googlecode.com
Owner: limpbiz...@gmail.com
Cc: joel....@gmail.com inder123
Comment #14 on issue 400 by limpbiz...@gmail.com: Dynamic serialization and
deserialization via class interface
http://code.google.com/p/google-gson/issues/detail?id=400
Nice work!
FYI Gson team, mintern's library is a use case for a public
getNextAdapter() method.
googl...@googlecode.com
Comment #15 on issue 400 by mint...@easyesi.com: Dynamic serialization and
http://code.google.com/p/google-gson/issues/detail?id=400
Thanks, I'm glad you like it! Unfortunately, it also uses TypeToken(Type)
in a gross way. I'd be interested in hearing an alternative. See
GsonContext.nextAdapter(Type).
googl...@googlecode.com
Comment #16 on issue 400 by inder123: Dynamic serialization and
http://code.google.com/p/google-gson/issues/detail?id=400
Looks interesting. mintern/Jesse, do you have any take on the performance
impact of this? I remember that in the past, annotations caused us quite a
bit of hit on performance. Does this reflection have any such issues?
googl...@googlecode.com
Comment #17 on issue 400 by mint...@easyesi.com: Dynamic serialization and
http://code.google.com/p/google-gson/issues/detail?id=400
I did not perform any performance tests on my mechanism. I can say that in
the case of large class hierarchies where none actually implements
JsonDeserialization, the Reflection.getTypeParameters(...) call will visit
every ancestor of the type. I expect that this could be trivially improved
by first checking JsonDeserialization.class.isAssignableFrom(...) before
calling getTypeParameters.
Based on my (somewhat limited) understanding of Gson, the most expensive
part will be called just once per (de/serialized) type per Gson instance.
That is, the first time a user calls a Gson method on a type, the
reflection introspection will slow it down somewhat. Subsequent operations,
however, should be as fast as they currently are.
googl...@googlecode.com
Comment #18 on issue 400 by limpbiz...@gmail.com: Dynamic serialization and
http://code.google.com/p/google-gson/issues/detail?id=400
We're going to publish the getNextAdapter API in Gson 2.2, though I believe
we may rename it to getDelegateAdapter().
mintern, any further action you'd like us to take here?
googl...@googlecode.com
Comment #19 on issue 400 by mint...@easyesi.com: Dynamic serialization and
http://code.google.com/p/google-gson/issues/detail?id=400
That's a good start, but I found in the implementation of my interface that
I sometimes needed to call getNextAdapter(..., Type). Currently,
getNextAdapter only accepts a TypeToken as an argument. The solution, then,
is to either implement getNextAdapter(..., Type) or to make the
TypeToken(Type) constructor public. A justification follows.
I provide a pair of methods (called thisToJson and thisFromJson) that allow
one to use Gson to perform the de/serialization while avoiding infinite
recursion. As you're aware, this makes use of getNextAdapter.
In some cases, however, I have to call getNextAdapter with a different
type--for example, when using thisFromJson to construct a subclass. In this
case, I need to call getNextAdapter with a TypeToken other than the one
with which my TypeAdapter was constructed.
For an example of what I'm talking about, see:
https://github.com/BMintern/gson-interface/blob/master/InterfaceAdapterFactory.java#L137
called by:
https://github.com/BMintern/gson-interface/blob/master/GsonContext.java#L85
called by:
https://github.com/BMintern/gson-interface/blob/master/InterfaceExample.java#L81
googl...@googlecode.com
Comment #20 on issue 400 by limpbiz...@gmail.com: Dynamic serialization and
http://code.google.com/p/google-gson/issues/detail?id=400
Can you use TypeToken.get(type) ?
googl...@googlecode.com
Comment #21 on issue 400 by mint...@easyesi.com: Dynamic serialization and
http://code.google.com/p/google-gson/issues/detail?id=400
Yes, I can. I don't know how I missed that. Thanks! This bug can be closed.
googl...@googlecode.com
Comment #22 on issue 400 by a...@shapeways.com: Dynamic serialization and
http://code.google.com/p/google-gson/issues/detail?id=400
This is a very useful addition. Worked fine inside our own codebase. I
think it would make a good addition to the core library.
googl...@googlecode.com
Comment #23 on issue 400 by mint...@easyesi.com: Dynamic serialization and
http://code.google.com/p/google-gson/issues/detail?id=400
Thanks! I'm glad it was useful.
Some notes based on our usage:
1. If a class has a non-static inner class that extends Runnable, it
results in an infinite loop during serialization. This is a problem with
Gson in general, as far as I can tell. Implementing JsonSerialization and
avoiding the use of thisToJson allows one to avoid this problem.
2. As Gson appears to be moving toward a streaming JSON interface, I'm not
sure how much sense my interface makes. Streaming is clearly superior for
large datasets, but it adds a lot of complexity when you just want to
serialize a simple object.
3. Make sure you have the latest version. I made a change nearly a month
ago that improved efficiency and fixed the problem outlined in (1) above.
Just today I committed the change suggested by comment #20 above.
4. My package introduces several new classes that might be confusing in the
general case (JsonSerialization, JsonDeserialization, JsonDeserializes,
GsonContext, InterfaceTypeAdapter, InterfaceAdapterFactory). Some of my
GsonContext functionality could potentially be rolled into the Gson object
itself, but otherwise the departure from standard Gson means that I can't
simplify the interfaces all that much.
5. Requiring an object to be deserialized by a separate class that
implements JsonDeserialization is a bit cumbersome, but there's not a good
way around it as far as I can tell. One pattern that eases the indirection
is to make the deserializer be a static inner class of the class it is
deserializing. In order to do that, in YourClass.java you'll have to import
YourClass.YourClassDeserializer, where YourClass implements
JsonDeserialization<YourClassDeserializer>, and YourClassDeserializer
implements JsonDeserializes<YourClass>.
I hope that helps. Definitely report any issues you have on my github...
I'm generally pretty responsive.
googl...@googlecode.com
Status: Done
Comment #24 on issue 400 by limpbizkit: Dynamic serialization and
(No comment was entered for this change.)