deserialize into an instances instead specifying target class/type

[Pawel]

Hi.

I had a thought that may be there should be a set of fromJson()
methods that will deserialize JSON into an existing root object,
instead of always having Gson create the instance of the root object
first.

The reasoning behind this is really the specifics of Java, and to
reduce the boilerplate code that would be necessary for self-
serializing and self-deserializing objects.

public abstract JsonObject {

protected JsonObject(String source) {
// can't use fromJson(), it's probably too overloaded
getMyBuilder().create().fromJsonToObject(source, this);
}

protected void writeOut(Appendable whereTo) {
getMyBuilder().create().toJson(whereTo, this);
}

protected GsonBuilder createMyBuilder() {
// creates / returns builder specific for this instance
// for specific customization, call abstract tweaking methods, or
// methods that are expected to be overwritten from default
behavior
}

protected abstract GsonBuilder tweakSomething1(GsonBuilder b);
protected GsonBuilder tweakSomething2(GsonBuilder b) {
// default behavior here, may be the concrete will override.
}

}

Right now, only static methods of a specific implementation can be
used, which stands in the way of inheritance, and object specific code
has to be part of static methods to instantiate it; most of that code
is expected to be the the same however.

[Brandon Mintern]

I have a personal project on github that might help some, especially
with the self-serializing part:

https://github.com/BMintern/gson-interface

It also includes an interface-based deserialization mechanism, but I'm
not in love with that approach due to the inheritance problems you
mention and because the deserializer has to be aware of the entire
object hierarchy.

I would love to hear more about your ideas around deserializing into a
root object. Specifically:

1. What would the Gson-using code look like (i.e., describe a call to fromJson)?
2. When deserializing a JsonObject into an arbitrary class, how would
a deserialized-into-root-object field be created?
3. How would you select a specific subclass to construct and
deserialize into based on the JSON (see
https://github.com/BMintern/gson-interface/blob/master/InterfaceExample.java
for an example of what I mean)?
4. How would you deserialize into an instance that is immutable (e.g.,
a Boolean), that has no notion of a base root object (i.e., no
sensible default constructor), or no runtime-constructed instances at
all (e.g., an enum)?

Looking forward to your input,
Brandon

> --
> You received this message because you are subscribed to the Google Groups "google-gson" group.
> To post to this group, send email to googl...@googlegroups.com.
> To unsubscribe from this group, send email to google-gson...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/google-gson?hl=en.
>

[Pawel]

Hi Brandon.

On May 23, 1:24 pm, Brandon Mintern <mint...@easyesi.com> wrote:
> I have a personal project on github that might help some, especially
> with the self-serializing part:
>
> https://github.com/BMintern/gson-interface
>
> It also includes an interface-based deserialization mechanism, but I'm
> not in love with that approach due to the inheritance problems you
> mention and because the deserializer has to be aware of the entire
> object hierarchy.
>
> I would love to hear more about your ideas around deserializing into a
> root object. Specifically:
>
> 1. What would the Gson-using code look like (i.e., describe a call to fromJson)?

fromJsonToInstance(JsonElement, Object);
fromJsonToInstnace(JsonReader, Object);
fromJsonToInstance(Reader, Object);
fromJsonToInstance(String, Object);

> 2. When deserializing a JsonObject into an arbitrary class, how would
> a deserialized-into-root-object field be created?

It will not be created. My thinking is to use exactly the same logic
as it is used now, except for instantiation. Superficially, the code
for fromJson(jsonish, Class cls) calls cls.newInstance() and then
attempts to fit fields from json at this level into the instantiated
object. This proposal is to allow the instantiation step to be done by
the caller, instead of having it being done solely by Gson.

> 3. How would you select a specific subclass to construct and
> deserialize into based on the JSON (see https://github.com/BMintern/gson-interface/blob/master/InterfaceExample.java
> for an example of what I mean)?

The idea is that a specific subclass is already instantiated. This RFE
does not cover use cases for selecting concrete implementation based
on particular JSON stream.

> 4. How would you deserialize into an instance that is immutable (e.g.,
> a Boolean), that has no notion of a base root object (i.e., no
> sensible default constructor), or no runtime-constructed instances at
> all (e.g., an enum)?

To keep it simple, the same code that is used to populate an instance
created by Gson, should be applied to the instance passed to the
caller, as if json has created it. I understand it that Gson will
still manipulate immutable classes all the same, purely because of how
it's implemented. This won't work for enums, but enums are static
classes, and are not instantiable.

If I look through implementation of TypeAdapter, I see these things
instantiate classes somewhere in their read() methods (and I can see
changing this will have quite an avalanche effect); now suppose each
type adapter could also have an instance that was passed to it
separately, and it will attempt, to it's best, use that instance,
instead of creating one of it's own.

> Looking forward to your input,

Hope that helped :)

> Brandon
>

[Brandon Mintern]

Thanks for the quick reply. I mainly asked my questions because they
are problems I ran into while considering the deserialization problem.
(Just to clarify, I'm not actually a Gson developer; we just use it
heavily at my workplace.)

On Wed, May 23, 2012 at 4:22 PM, Pawel <pawel....@gmail.com> wrote:
>> 1. What would the Gson-using code look like (i.e., describe a call to fromJson)?
>
> fromJsonToInstance(JsonElement, Object);
> fromJsonToInstnace(JsonReader, Object);
> fromJsonToInstance(Reader, Object);
> fromJsonToInstance(String, Object);

That seems pretty straightforward, except...

>> 2. When deserializing a JsonObject into an arbitrary class, how would
>> a deserialized-into-root-object field be created?
>
> It will not be created. My thinking is to use exactly the same logic
> as it is used now, except for instantiation. Superficially, the code
> for fromJson(jsonish, Class cls) calls cls.newInstance() and then
> attempts to fit fields from json at this level into the instantiated
> object. This proposal is to allow the instantiation step to be done by
> the caller, instead of having it being done solely by Gson.

...that this seems like a pretty big departure from the way Gson works
right now. An illustrative example of what I'm concerned about is:

public interface JsonObject {
void setFromJson(JsonElement json, Gson gson);
}

class Foo implements JsonObject {
String foo;
Bar bar;

@Override
public void setFromJson(JsonElement json, Gson gson) {
foo = json.getAsJsonObject().get("foo").getAsString();
bar = new Bar(); // These two lines seem troublesome to me
gson.fromJsonToInstance(json.getAsJsonObject().get("bar"), gson);
}
}

class Bar implements JsonObject { ... }

(I've changed some method names and approaches, but the basic
mechanism seems to be the same.)

In this model, every class that includes a JsonObject field has to
manually construct and call fromJsonToInstance on it. As soon as an
object includes an embedded JsonObject field, it also must implement
JsonObject and construct the nested object manually. This just doesn't
seem like a win to me versus the current TypeAdapter-based (or my
JsonDeserialization<FooDeserializer> interface-based) deserialization
paradigm.

>> 4. How would you deserialize into an instance that is immutable (e.g.,
>> a Boolean), that has no notion of a base root object (i.e., no
>> sensible default constructor), or no runtime-constructed instances at
>> all (e.g., an enum)?
>
> To keep it simple, the same code that is used to populate an instance
> created by Gson, should be applied to the instance passed to the
> caller, as if json has created it. I understand it that Gson will
> still manipulate immutable classes all the same, purely because of how
> it's implemented. This won't work for enums, but enums are static
> classes, and are not instantiable.
>
> If I look through implementation of TypeAdapter, I see these things
> instantiate classes somewhere in their read() methods (and I can see
> changing this will have quite an avalanche effect); now suppose each
> type adapter could also have an instance that was passed to it
> separately, and it will attempt, to it's best, use that instance,
> instead of creating one of it's own.

Note that a TypeAdapter doesn't always construct an instance. Therein
lies its power: it doesn't have to create a new instance. It can use a
static valueOf method to fetch an instance. It can call a getInstance
method that returns cached objects which are constructed only when
necessary. It can call a non-default constructor. The default
implementation that magically builds class instances does call
newInstance. But for your own types, you can create a JsonDeserializer
class that builds your class from a JsonElement in whatever way you
wish. That seems to enable some of the same ideas that you're looking
for with your approach.

Don't get me wrong. Your approach seems like it can simplify things
for a simple, flat class. It can also afford the interesting
possibility of updating an existing instance with new JSON data.
Unfortunately, it seems to break down--becoming at least as cumbersome
as the existing solution--when a JsonObject instance is a field in
another class. Do you disagree?

Thanks for sharing your ideas. You've given me some interesting things
to think about. In an ideal world, deserialization would be performed
by instances so that the methods can be overridden. What if a certain
interface indicated that Gson should construct a default instance and
then call its fromJson method? That seems to get at the most
interesting part of your proposal while still allowing reasonable
behavior when the class is nested.

Let me know if you run into any difficulties,
Brandon

p.s. If you think your idea as it is now will simplify your codebase,
I definitely recommend taking a shot at implementing it. Our codebase
includes a Json utility class that provides convenience methods which
delegate to a static internal Gson instance. You could definitely do
something similar yourself, adding your fromJsonToInstance methods to
do what you like and delegating to a Gson instance when necessary.

[Pawel]

Hi Brandon.

My request is to support this as an exception, rather than to have a
full-fledged serialization that always work on constructed instances
internally.

I don't think it's an entirely bad idea to have support for
deserializing into existing instances. The only way to override this
right now - is to write your own deserializer, but if you create your
own instances, you have to have full code to populate each instance
field yourself, you can no longer invoke default behavior.

You said a lot of good things about how Gson deals with selecting
instances and all that, and I agree, it's doing a great job, but it
has a semi-monopoly on it, if you choose to intervene in element
deserialization process, you are left completely on your own, and you
can't just create an instance, and ask GSon to pick it up from there.
It would be nice if I could have control on just that aspect piece of
each element deconstruction.

To repeat myself, the original problem that prompted my email is this:

I am high level code, I'm expecting some data and meta-data in a
package. All my data is some kind of animal.
I received a package, I read the meta-data, I know that I expect to
create a ferret. I have data for a ferret.
Now, each of my animals have a special way of processing their input
json data. The problem that I'm facing here, is pure in the fact that
I can't create an instance of animal, and have them control exactly
how they are going to deserialize, before I deserialize (at which
point it's too late).

So, I have to have an Animal interface, and an AnimalHelper interface,
or use static methods in concrete classes. Either way it is increasing
boiler-plate code, and/or relying on internal (un-enforceable at
compile time) conventions of having special (static) methods.

Thank you,
Pawel.

On May 23, 5:58 pm, Brandon Mintern <mint...@easyesi.com> wrote:
> Thanks for the quick reply. I mainly asked my questions because they
> are problems I ran into while considering the deserialization problem.
> (Just to clarify, I'm not actually a Gson developer; we just use it
> heavily at my workplace.)
>

[Brandon Mintern]

Hi Pawel,

I've been thinking about your proposal, and I think it's a good idea
with some slight modifications. This is what I've come up with:

Deserializing from JSON involves three conceptually separate steps:

1. Deciding what type to deserialize to. This is handled by the
argument passed to gson.fromJson or by the fields of an object being
deserialized.
2. Constructing an object of that type.
3. Populating the fields of the constructed object with the JSON data.

The problem with Gson in its current form is that the three steps are
rather intertwined. There is a mechanism for overriding #2 only:
InstanceCreator. Unfortunately, if you want to change the behavior of
#3, it can only be done by a separate class which does all 3 steps.
Likewise for #1: if you want to dispatch to different types, you have
to peek at the JsonElement and then recursively call the read method
of a delegate TypeAdapter. I think this situation is suboptimal.

I've begun work on a Gson fork (hopefully some the ideas make it back
to trunk) that:

1. Separates deserialization into the three steps outlined above,
providing interfaces for type selection, instance creation, and field
population

2. Allows objects to implement an interface so that Gson will
automatically call their member methods to perform serialization and
deserialization

3. Provides both streaming- and JsonElement-based interfaces. It is
generally easier to implement a JsonElement-based interface, but for
large objects and certain other situations, streaming is often
preferable

4. Provides an annotation so that an object can specify its InstanceCreator

5. Simplifies the interfaces as much as possible. For example, I found
no code that uses the typeOfT argument that is passed to
JsonSerializer.serialize

6. Adds recursion protection so that, for example, calling gson.toJson
from a JsonSerializer's write method automatically delegates to the
next adapter in order to avoid infinite recursion

7. Provides configuration options to GsonBuilder to disable many of
these features to optimize for performance-critical contexts

8. Provides methods to get GsonBuilder instances that are conveniently
initialized with both feature-rich and performance-minded
configurations

9. Maintains complete backward compatibility with Gson 2.2.1;
unfortunately, this may involve a confusing explosion of interface
names

That's a long-winded way of saying that I liked your idea and I think
it can fit well into a different approach to deserialization. In
Gson's current state, I think object-based deserialization is going to
be cumbersome to implement.

Would some of the changes I've mentioned above help you in your
use-case? In your case, a class would simply implement the following:

class Foo implements FromJson {
private Bar bar;
private boolean deserializedFromJson;

@Override
void fromJson(JsonElement json, Gson gson) {
bar = gson.fromJson(json.getAsJsonObject().get("bar"), Bar.class);
deserializedFromJson = true;
}
}

My changes would also add to Gson the fromJson methods that take an
instance as an argument.

Thanks,
Brandon