Number formatting

[Ping]

Hello,

My project is using Gson to format JSON for HTTP requests and
replies. What I'd like to do is to format certain number fields with
a limited number of significant digits. I have just gone through a
grueling couple of days fighting with Gson to try to do this, but have
been defeated at every turn. Having tried everything I can think of,
I now write to ask for your help. Here is my sad tale...

1. Suppose I create a JsonDouble class to represent a number with
limited precision, and use JsonDoubles in my structures instead of
doubles. But implementing the "toString()" method on JsonDouble has
no effect, because Gson doesn't respect that method and there's no way
to tell Gson to use "toString()". (Why not? Would you consider
adding this?)

2. There is no method that I can implement on JsonDouble to achieve my
goal, because Gson doesn't let classes serialize themselves — there's
nothing like a "toJson()" method that Gson calls to get the JSON
string serialization for an object. (Why not? Would you consider
supporting this?)

3. I can't extend JsonElement to achieve my goal, because the default
serializer doesn't pass JsonElement through. (Why not?) Instead it
tries to introspect the fields of JsonElement and turn it into a JSON
dictionary like any other class, and fails on deserialization.

4. I can't extend any other class to achieve my goal, because TypeInfo
is hardcoded to accept only a specific list of types as primitives;
other subclasses of Number are not accepted, and the list of types is
not extensible. (Why not?)

5. Instead I have to create a separate JsonSerializer class, and then
register that class with a GsonBuilder, and then use only Gson objects
constructed by that GsonBuilder. Fine — I did that. But that still
doesn't give any formatting control, because the serialize() method is
required to return a JsonElement, not a String. (Why not? Would you
consider adding support for this?)

6. However, the JsonElement does have a toString() method that I can
override. I tried deriving JsonDouble from JsonElement, and
overriding the toString() method. But this still doesn't work,
because JsonTreeNavigator tries to cast the element to a JsonPrimitive
with a call to getAsJsonPrimitive. The only reason it does this cast
is to help the Java compiler select the correct version of the
overloaded "visitObjectMember" method on JsonElementVisitor — but none
of this would be necessary if the JsonElementVisitor simply used an
appropriately named method (such as "visitPrimitive") instead of a
type-overloaded method. (Why not? Would you consider fixing this
interface?)

7. I could satisfy getAsJsonPrimitive if I could derive JsonDouble
from JsonPrimitive. But I can't do that either, because JsonPrimitive
is final. (Why? Would you consider removing the "final" keyword?)

8. Alternatively, I could pass a JsonDouble as the argument to the
JsonPrimitive constructor — if JsonDouble extends Number. Okay — I
did that. But that doesn't work either, because ObjectNavigator's
accept() method doesn't respect custom handlers for primitives on
deserialization. (Why not?) During serialization, the
ObjectNavigator sees a JsonDouble, so the custom serializer is used.
But during deserialization, the ObjectNavigator never looks for a
custom deserializer because the incoming JSON looks like a number.
And I can't avoid having to provide a custom deserializer, because
Gson has no other way to know how to turn a JavaScript number into a
JsonDouble object.

I see a recent revision (http://code.google.com/p/google-gson/source/
detail?r=296) that claims to allow customization of the serialization
and deserialization of Primitives. But I don't see how this revision
helps at all; even if it did cause ObjectNavigator to call a custom
serializer or deserializer (which I think it doesn't, as described in
item 8 above), I still would only be able to substitute one primitive
for another — I wouldn't be able to change the actual serialization of
a primitive, would I?

I would really appreciate if you could take a moment to enlighten me.
Have I made incorrect assumptions about how Gson works? Are there
possibilities I've missed? Most of all, is there a simple way to
achieve this goal that I have just failed to see?

Many thanks for your time and assistance.


— ?!ng

[inde...@gmail.com]

On Nov 17, 10:30 am, Ping <p...@zesty.ca> wrote:
> Hello,
>
> My project is using Gson to format JSON for HTTP requests and
> replies.  What I'd like to do is to format certain number fields with
> a limited number of significant digits.  I have just gone through a
> grueling couple of days fighting with Gson to try to do this, but have
> been defeated at every turn.  Having tried everything I can think of,
> I now write to ask for your help.  Here is my sad tale...
>
> 1. Suppose I create a JsonDouble class to represent a number with
> limited precision, and use JsonDoubles in my structures instead of
> doubles.  But implementing the "toString()" method on JsonDouble has
> no effect, because Gson doesn't respect that method and there's no way
> to tell Gson to use "toString()".  (Why not?  Would you consider
> adding this?)

Would you want toString() on all methods to be used for their JSON
representation? How is this different from forcing the class writer to
write all the JSON themselves and not use Gson at all.

> 2. There is no method that I can implement on JsonDouble to achieve my
> goal, because Gson doesn't let classes serialize themselves — there's
> nothing like a "toJson()" method that Gson calls to get the JSON
> string serialization for an object.  (Why not?  Would you consider
> supporting this?)

This seems like a reasonable suggestion. Instead of a magic method
name, we can use annotations that essentially mark a method as
serializer, deserializer and instance creator for the class. We will
consider this for our next major release (due in Dec).

> 3. I can't extend JsonElement to achieve my goal, because the default
> serializer doesn't pass JsonElement through.  (Why not?)  Instead it
> tries to introspect the fields of JsonElement and turn it into a JSON
> dictionary like any other class, and fails on deserialization.

Yes, JsonElement and its subclasses re used as the nodes for the parse
tree. We didn't design them to be subclassed by users, so it is
probably not a good idea.

> 4. I can't extend any other class to achieve my goal, because TypeInfo
> is hardcoded to accept only a specific list of types as primitives;
> other subclasses of Number are not accepted, and the list of types is
> not extensible.  (Why not?)

I am not sure what you mean here.

> 5. Instead I have to create a separate JsonSerializer class, and then
> register that class with a GsonBuilder, and then use only Gson objects
> constructed by that GsonBuilder.  Fine — I did that.  But that still
> doesn't give any formatting control, because the serialize() method is
> required to return a JsonElement, not a String.  (Why not?  Would you
> consider adding support for this?)

I see. I presume that you want to write it out as double, not double
in a string.
Because you can always use it like this:
String str = // your formatted string here
new JsonPrimitive(str);

You are right, Gson gives you no control over formatting of floating
points.

> 6. However, the JsonElement does have a toString() method that I can
> override.  I tried deriving JsonDouble from JsonElement, and
> overriding the toString() method.  But this still doesn't work,
> because JsonTreeNavigator tries to cast the element to a JsonPrimitive
> with a call to getAsJsonPrimitive.  The only reason it does this cast
> is to help the Java compiler select the correct version of the
> overloaded "visitObjectMember" method on JsonElementVisitor — but none
> of this would be necessary if the JsonElementVisitor simply used an
> appropriately named method (such as "visitPrimitive") instead of a
> type-overloaded method.  (Why not?  Would you consider fixing this
> interface?)

We did not design Gson to be extensible in the manner you are
expecting. Even though you can extend JsonElement and return that in
your custom serializer (Indeed a smart trick that I didnt think
someone would do), our code is written to support only the four
subtypes of JsonElement: JsonPrimitive, JsonNull, JsonObject and
JsonArray.

We do have flexibility in changing JsonTreeNavigator, but we wouldn't
want you to rely on the internal workings of this class.

> 7. I could satisfy getAsJsonPrimitive if I could derive JsonDouble
> from JsonPrimitive.  But I can't do that either, because JsonPrimitive
> is final.  (Why?  Would you consider removing the "final" keyword?)

We chose final for classes just as a precaution. We can always relax
that requirement in a future release if we are convinced of the
merits.

> 8. Alternatively, I could pass a JsonDouble as the argument to the
> JsonPrimitive constructor — if JsonDouble extends Number.  Okay — I
> did that.  But that doesn't work either, because ObjectNavigator's
> accept() method doesn't respect custom handlers for primitives on
> deserialization.  (Why not?)  

This specific issue should be fixed in the release 1.2.3.

> During serialization, the
> ObjectNavigator sees a JsonDouble, so the custom serializer is used.
> But during deserialization, the ObjectNavigator never looks for a
> custom deserializer because the incoming JSON looks like a number.
> And I can't avoid having to provide a custom deserializer, because
> Gson has no other way to know how to turn a JavaScript number into a
> JsonDouble object.
>
> I see a recent revision (http://code.google.com/p/google-gson/source/
> detail?r=296) that claims to allow customization of the serialization
> and deserialization of Primitives.  But I don't see how this revision
> helps at all; even if it did cause ObjectNavigator to call a custom
> serializer or deserializer (which I think it doesn't, as described in
> item 8 above), I still would only be able to substitute one primitive
> for another — I wouldn't be able to change the actual serialization of
> a primitive, would I?

Well, the serialization of a JsonPrimitive calls the toString() of the
primitive object. So, you extend your double class from Number, and
have the correct toString() method, it should work as you expect it
to.

> I would really appreciate if you could take a moment to enlighten me.
> Have I made incorrect assumptions about how Gson works?  Are there
> possibilities I've missed?  Most of all, is there a simple way to
> achieve this goal that I have just failed to see?

You certainly explored extension use-cases of Gson that we didn't plan
for. So, our mechanisms to prevent those came in your way.

There is no simple way yet to achieve what you want (except what you
describe in the para after 8). I would recommend that you rename
JsonDouble to something else (so as not to confuse it with a subclass
of JsonElement) and do not extend JsonElement.

However, this seems like a reasonable feature that we should build in
Gson itself.

How about if we provide a method in GsonBuilder that controls
formatting of floating point numbers?
Will that work for you? How would you like the method to look like?

Sorry for all the troubles you had to go through in using Gson, and we
appreciate your patience in sticking with it.

Thanks
Inder