Can we have Date object support as JsonDate in the API?

[Atul Dambalkar]

Hi,

Can we have a support for JsonDate class which presumably can extend com.google.gson.JsonElement?

I have created a project Mongo2gson (googlecode) which basically converts Mongo Java objects to Json objects. As MongoDB supports java.util.Date as one of the value objects, it will be great if I am able to convert the Date object (that I get from Mongo API) to com.google.gson.JsonDate representation.

Please note: Current abstract com.google.gson.JsonElement class can't be extended out of it's own package as one can't over-ride API - "protected void toString(Appendable appendable, Escaper escaper) throws IOException" as the Escaper class has package-private visibility (, which I perfectly understand as an implementation choice).

It will be good to have JsonDate implemented. I can also help implement it, if needed.

Thanks and regards.

-Atul

[Inderjeet Singh]

Hi Atul, 

How standard is this concept of Json Date? I don't see it in http://www.ietf.org/rfc/rfc4627.txt or the spec at http://www.json.org/  Is it used widely? This is the first time I have heard of it.

I am not sure if defining a top-level JsonDate class that extends JsonElement is the right approach.

One option would be to revise the built-in type adapters for java.util.Date (and other variants of that class) to support deserializing from a Json Date String. There could also be a GsonBuilder setting that outputs all date objects into JsonDate format.  How does that sound?

You can help us by creating a new issue on Gson project for this, and possibly writing such a TypeAdapter and posting it on the issue.

Thanks

Inder

http://www.singhinderjeet.com/

[Atul Dambalkar]

Hi Inder,

 

Thanks for your response. And also good points. I agree there is no standard surrounding Json Date and that can create issue such as the allowed input and output formats to the JsonDate object, which will have to be defined/standardized for any such implementation.

 

But I think it will be nice to have some Date support in Gson API, just because it’s a Java-based API and Java has a good Date support. And anybody using Gson will encounter the need and will end-up implementing the adapter for that. I like the type-adapter based approach you have suggested. Can you pass me some more information about the built-in type adapters? How can I get handle on that? The user-guide does mention about built-in adapter for Java.net.URL/URI, but I didn’t see any Javadoc for that.

 

Thanks and regards,

 

-Atul

[Inderjeet Singh]

On Thursday, 23 August 2012 00:50:57 UTC-7, Atul Dambalkar wrote:

Thanks for your response. And also good points. I agree there is no standard surrounding Json Date and that can create issue such as the allowed input and output formats to the JsonDate object, which will have to be defined/standardized for any such implementation. 

But I think it will be nice to have some Date support in Gson API, just because it’s a Java-based API and Java has a good Date support. And anybody using Gson will encounter the need and will end-up implementing the adapter for that. I like the type-adapter based approach you have suggested. Can you pass me some more information about the built-in type adapters? How can I get handle on that? The user-guide does mention about built-in adapter for Java.net.URL/URI, but I didn’t see any Javadoc for that.

What kind of formatting does JsonDate uses? Is it mappable to any of the standard Java date formatting? If so, then all you need to do is call GsonBuilder.setDateFormat(). 

If you are do decided to write a new adapter, you first need to decide the class which it will serialize to/deserialize from. For example, you can write a type adapter for java.util.Date and have it write to/from JsonDate format.

Regarding the built-in adapters for java.net.URL/URI, there is no need for Javadocs. Anything that is part of standard libraries is automatically serialized/deserialized in sensible formats. For example, a URL/URI is serialized using toExternalFormat() and deserialized from the equivalent String.

HTH

Inder

[Atul Dambalkar]

Thanks Inder, that’s helpful.

 

Subsequent question is - if we have GsonBuilder.setDateFormat() then I can simply use that. Then in terms of deserializing it’s fine, but while serializing, what Gson API object it’s going to return? Will it be JsonPrimitive or JsonObject with the date value as string with the specified formatting?

 

I see that, JsonObject doesn’t have addProperty(property, value) API that takes Date as a value object. What do you think on adding Date as one of the supported value types? And the serialization/deserialization can still use the date format that user or client application has set, so we don’t get into the Date format standardization problem.

[Inderjeet Singh]

On Friday, August 24, 2012 1:01:00 AM UTC-7, Atul Dambalkar wrote:

Thanks Inder, that’s helpful.

 

Subsequent question is - if we have GsonBuilder.setDateFormat() then I can simply use that. Then in terms of deserializing it’s fine, but while serializing, what Gson API object it’s going to return? Will it be JsonPrimitive or JsonObject with the date value as string with the specified formatting?

You can try it out. I think it is going to return a string with the specified formatting.

 

I see that, JsonObject doesn’t have addProperty(property, value) API that takes Date as a value object. What do you think on adding Date as one of the supported value types? And the serialization/deserialization can still use the date format that user or client application has set, so we don’t get into the Date format standardization problem.

We do have a gson.toJsonElement() method that does what you want (create a DOM tree of JsonElement nodes given any object). 

I wouldn't want to add a special behavior for Date in JsonObject. addProperty is meant only for primitives and String. Why should date be treated any special than any other type? Date is anyway a poorly designed type in Java.

Inder

[Atul Dambalkar]

 

 

From: Inderjeet Singh [mailto:inde...@gmail.com]
Sent: Sunday, August 26, 2012 11:54 AM
To: googl...@googlegroups.com
Cc: 'Inderjeet Singh'; at...@entrib.com
Subject: Re: Can we have Date object support as JsonDate in the API?

 

On Friday, August 24, 2012 1:01:00 AM UTC-7, Atul Dambalkar wrote:

Thanks Inder, that’s helpful.

 

Subsequent question is - if we have GsonBuilder.setDateFormat() then I can simply use that. Then in terms of deserializing it’s fine, but while serializing, what Gson API object it’s going to return? Will it be JsonPrimitive or JsonObject with the date value as string with the specified formatting?

You can try it out. I think it is going to return a string with the specified formatting.

 

[atul@entrib] Yea, I understand.

I see that, JsonObject doesn’t have addProperty(property, value) API that takes Date as a value object. What do you think on adding Date as one of the supported value types? And the serialization/deserialization can still use the date format that user or client application has set, so we don’t get into the Date format standardization problem.

We do have a gson.toJsonElement() method that does what you want (create a DOM tree of JsonElement nodes given any object). 

 

[atul@entrib] Yea, but that JsonElement is going to be String, right?

 

I wouldn't want to add a special behavior for Date in JsonObject. addProperty is meant only for primitives and String. Why should date be treated any special than any other type? Date is anyway a poorly designed type in Java.

 

[atul@entrib] I would say yes and no. May be you should consider giving a special treatment to Date just because ECMAScript (ECMA-262) has Date support, even Java and JDBC treat Date as special data-type.  SQL and NO-SQL (e.g. MongoDB) databases support Date. XML schema also gives special treatment to Date. But I see your point, as JSON specs is not giving any special treatment to Date type. So, I will leave it to you to decide and how to take this forward, as then it can be JSON specs issue. But IMO, more and more developers will face (or they already have) the similar issue as JSON is already so much in use. The issue will be seen mostly in cases like non-browser apps that are using JSON for content formatting over some transport or simply for data inter-change (effectively replacing XML).

 

Regards,

 

-Atul

[Inderjeet Singh]

On Sunday, August 26, 2012 11:34:42 PM UTC-7, Atul Dambalkar wrote:

[atul@entrib] Yea, but that JsonElement is going to be String, right?

I had a typo earlier. The method to use is gson.toJsonTree() that returns a DOM tree. 

 

 

I wouldn't want to add a special behavior for Date in JsonObject. addProperty is meant only for primitives and String. Why should date be treated any special than any other type? Date is anyway a poorly designed type in Java.

 

[atul@entrib] I would say yes and no. May be you should consider giving a special treatment to Date just because ECMAScript (ECMA-262) has Date support, even Java and JDBC treat Date as special data-type.  SQL and NO-SQL (e.g. MongoDB) databases support Date. XML schema also gives special treatment to Date. But I see your point, as JSON specs is not giving any special treatment to Date type. So, I will leave it to you to decide and how to take this forward, as then it can be JSON specs issue. But IMO, more and more developers will face (or they already have) the similar issue as JSON is already so much in use. The issue will be seen mostly in cases like non-browser apps that are using JSON for content formatting over some transport or simply for data inter-change (effectively replacing XML).

We like to drive new features by real use-cases. What problem are you trying to solve that will be solved much more elegantly if Date was an intrinsic type in Gson?

Regards

Inder

 

[Atul Dambalkar]

 

 

From: Inderjeet Singh [mailto:inde...@gmail.com]
Sent: Tuesday, August 28, 2012 10:29 PM
To: googl...@googlegroups.com
Cc: 'Inderjeet Singh'; at...@entrib.com
Subject: Re: Can we have Date object support as JsonDate in the API?

 

On Sunday, August 26, 2012 11:34:42 PM UTC-7, Atul Dambalkar wrote:

 

[atul@entrib] Yea, but that JsonElement is going to be String, right?

I had a typo earlier. The method to use is gson.toJsonTree() that returns a DOM tree. 

 

 

[atul@entrib] Yea, thought so.

 

 

I wouldn't want to add a special behavior for Date in JsonObject. addProperty is meant only for primitives and String. Why should date be treated any special than any other type? Date is anyway a poorly designed type in Java.

 

[atul@entrib] I would say yes and no. May be you should consider giving a special treatment to Date just because ECMAScript (ECMA-262) has Date support, even Java and JDBC treat Date as special data-type.  SQL and NO-SQL (e.g. MongoDB) databases support Date. XML schema also gives special treatment to Date. But I see your point, as JSON specs is not giving any special treatment to Date type. So, I will leave it to you to decide and how to take this forward, as then it can be JSON specs issue. But IMO, more and more developers will face (or they already have) the similar issue as JSON is already so much in use. The issue will be seen mostly in cases like non-browser apps that are using JSON for content formatting over some transport or simply for data inter-change (effectively replacing XML).

 

We like to drive new features by real use-cases. What problem are you trying to solve that will be solved much more elegantly if Date was an intrinsic type in Gson?

 

[atul@entrib] It is a real use-case actually. In one of our tools, I am converting Mongo objects (http://api.mongodb.org/java/current/) to Gson objects (http://code.google.com/p/mongo2gson/). Now, since I do get a Date object from Mongo objects just like other data types such as primitives and String, I need to convert Date to some meaningful object so that I can use it in type-safe manner. But since right now there is no Date support in Gson (as per JSON standards), I end-up converting it to a JsonPrimitive with String (with a certain date-format) as underlying data type. Also, as I said earlier, JSON is already popular and is also claimed as a fat-free alternative to XML. So if someone is using it as transport medium, they are most likely going to end-up into similar issue. I am sure people at 10gen (MongoDB) have faced this issue and ended up adding new type to represent date.

 

Thanks and regards,

 

-Atul

 

Regards

Inder

 

[Inderjeet Singh]

On Tuesday, August 28, 2012 10:30:56 AM UTC-7, Atul Dambalkar wrote:

 [atul@entrib] It is a real use-case actually. In one of our tools, I am converting Mongo objects (http://api.mongodb.org/java/current/) to Gson objects (http://code.google.com/p/mongo2gson/). Now, since I do get a Date object from Mongo objects just like other data types such as primitives and String, I need to convert Date to some meaningful object so that I can use it in type-safe manner. But since right now there is no Date support in Gson (as per JSON standards), I end-up converting it to a JsonPrimitive with String (with a certain date-format) as underlying data type. Also, as I said earlier, JSON is already popular and is also claimed as a fat-free alternative to XML. So if someone is using it as transport medium, they are most likely going to end-up into similar issue. I am sure people at 10gen (MongoDB) have faced this issue and ended up adding new type to represent date.

I think there are two ways to represent it in JSON. one is the long value representing epoch time, the other is String in the ISO format.

In my view, epoch time is the best in-memory representation.

However, while sending it on the wire, it may make sense to use a String format: How does Mongo represent it as on-the-wire format? How do they make it work with other systems?

Inder

[Atul Dambalkar]

 

 

From: Inderjeet Singh [mailto:inde...@gmail.com]
Sent: Wednesday, August 29, 2012 5:06 AM
To: googl...@googlegroups.com
Cc: 'Inderjeet Singh'; at...@entrib.com
Subject: Re: Can we have Date object support as JsonDate in the API?

 

On Tuesday, August 28, 2012 10:30:56 AM UTC-7, Atul Dambalkar wrote:

 [atul@entrib] It is a real use-case actually. In one of our tools, I am converting Mongo objects (http://api.mongodb.org/java/current/) to Gson objects (http://code.google.com/p/mongo2gson/). Now, since I do get a Date object from Mongo objects just like other data types such as primitives and String, I need to convert Date to some meaningful object so that I can use it in type-safe manner. But since right now there is no Date support in Gson (as per JSON standards), I end-up converting it to a JsonPrimitive with String (with a certain date-format) as underlying data type. Also, as I said earlier, JSON is already popular and is also claimed as a fat-free alternative to XML. So if someone is using it as transport medium, they are most likely going to end-up into similar issue. I am sure people at 10gen (MongoDB) have faced this issue and ended up adding new type to represent date.

 

I think there are two ways to represent it in JSON. one is the long value representing epoch time, the other is String in the ISO format.

In my view, epoch time is the best in-memory representation.

[atul@entrib] Yes, you are right it can be a long representing epoch time or just a string in ISO date format. And in-memory epoch time will be more convenient representation.

 

However, while sending it on the wire, it may make sense to use a String format: How does Mongo represent it as on-the-wire format? How do they make it work with other systems?

[atul@entrib] Please take a look http://www.mongodb.org/display/DOCS/Dates This link talks about how Mongo takes care of date on the Mongo shell. Mongo stores data in BSON.BSON defines the data type for Date and also all JSON data types.  In order to interface with other systems, Mongo has language specific drivers that handle the serialization and deserialization for all the data types including Date. For a Java driver it simply converts the BSON Date  to Java date. I looked at the BSON specs (http://bsonspec.org/#/specification)for  on-the-wire format for Date, they do use UTC epoch time (int64) as the representation, they propagate the type information before the actual value (\x09). So when I query Mongo using Java driver, I do get Java Date object back if the underlying value is of type Date.

 

-Atul

 

Inder

[Inderjeet Singh]

Hi Atul,

Thanks for sharing the use-case. Now I understand better about your need for a JsonElement subclass representing date: You are doing operations exclusively with the DOM tree generated by Gson.

How would you use JsonDate differently that just a primitive? What operations are you doing on this DOM tree that require you to actually know that you are dealing with a date? Also, how is the current workaround of using a primitive working out?

I do know that a lot of storage technologies (especially databases) provide Date as a native type. However, most languages don't. It's not clear to me that that the storage technologies made the right choice, but my knowledge is limited in the area.

Regards

Inder

[Atul Dambalkar]

Hi Inder,

 

I appreciate you spending time on this.

 

From: Inderjeet Singh [mailto:inde...@gmail.com]
Sent: Thursday, August 30, 2012 10:49 PM
To: googl...@googlegroups.com
Cc: 'Inderjeet Singh'; at...@entrib.com
Subject: Re: Can we have Date object support as JsonDate in the API?

 

Hi Atul,

 

Thanks for sharing the use-case. Now I understand better about your need for a JsonElement subclass representing date: You are doing operations exclusively with the DOM tree generated by Gson.

 

[atul@entrib] Yes, you could say that, but I am still one step before that and  I am simply converting the Mongo objects to Gson objects. These Gson objects then are being used by some other client code.

 

How would you use JsonDate differently that just a primitive? What operations are you doing on this DOM tree that require you to actually know that you are dealing with a date? Also, how is the current workaround of using a primitive working out?

 

[atul@entrib] This is where the problem is. In order to programmatically use JsonPrimitive object for Date, I have to use it as long or String, which effectively means loss of type information. So, now I have no way to know that underlying long or String is actually a Date. If I were to know that the underlying JsonPrimitive is Date, things will be simplified.

 

I do know that a lot of storage technologies (especially databases) provide Date as a native type. However, most languages don't. It's not clear to me that that the storage technologies made the right choice, but my knowledge is limited in the area.

 

[atul@entrib] Isn’t JSON  just a declarative language,  like XML (and not really an imperative one)? IMHO, what I feel is, you could still add JsonDate class (although JSON spec doesn’t talk about it) which can still be used by the custom Type Adapters to convert the given string (for user defined/supported format). It will solve my problem greatly as then I will base my code on the underlying type. Just to elaborate my use-case further, I use Velocity Template Engine to generate formatted content based on Gson objects. So if I get a JsonDate, I can base my Velocity code to  use underlying Date object to implement the logic.

 

Thanks,

 

-Atul

[Brandon Mintern]

In my own implementations, I've run into several cases where a
package-visible method or a final class forced me to do unnecessary
gymnastics in order to accomplish what I wanted to. I understand that
declaring certain things as final or package-visible more accurately
reflects the library authors' intent and is a sort of compatibility
protection, but can't that be accomplished with documentation instead,
including warnings where overriding certain methods is likely to break
in future implementations?

I don't think it will be possible to anticipate every user's JSON
needs. Making Gson and JsonElement able to be extended would be a
great start. Obviously this would include changing many methods that
have default visibility to be protected instead.

> --
> You received this message because you are subscribed to the Google Groups
> "google-gson" group.
> To post to this group, send email to googl...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-gson...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-gson?hl=en.

[Inderjeet Singh]

Good points, Brandon. It was a deliberate choice to prevent extension of Gson by others: the library was evolving and we didn't want to end up supporting use-cases that we didn't intend. Even then I was amazed at some of the stuff people used Gson for: One small example, I never thought that deserializing incoming JSON into JsonElement.class would make sense, but people were using it. We actually now support it better in the library itself.

Your point about warnings in Javadoc is probably good enough. I am still worried that people will complain if we break backward compatibility in any form. But worth a discussion anyway: we will explore this further. 

Happy Friday
Inder

-----

Checkout my new Android multiplayer word game that uses Gson: 

https://play.google.com/store/apps/details?id=com.applimobile.unscramble

[Inderjeet Singh]

On Thursday, August 30, 2012 10:37:02 PM UTC-7, Atul Dambalkar wrote:

Hi Inder,

How would you use JsonDate differently that just a primitive? What operations are you doing on this DOM tree that require you to actually know that you are dealing with a date? Also, how is the current workaround of using a primitive working out?

 

[atul@entrib] This is where the problem is. In order to programmatically use JsonPrimitive object for Date, I have to use it as long or String, which effectively means loss of type information. So, now I have no way to know that underlying long or String is actually a Date. If I were to know that the underlying JsonPrimitive is Date, things will be simplified.

Yes, there is no easy way to add metadata like this. Though if you really want to hack around it, you can keep a separate map of (JsonElement, Type) to keep track of which object has which type. You can store that map in ThreadLocal or a field.

Hacky, I know. 

 

I do know that a lot of storage technologies (especially databases) provide Date as a native type. However, most languages don't. It's not clear to me that that the storage technologies made the right choice, but my knowledge is limited in the area.

 

[atul@entrib] Isn’t JSON  just a declarative language,  like XML (and not really an imperative one)? IMHO, what I feel is, you could still add JsonDate class (although JSON spec doesn’t talk about it) which can still be used by the custom Type Adapters to convert the given string (for user defined/supported format). It will solve my problem greatly as then I will base my code on the underlying type. Just to elaborate my use-case further, I use Velocity Template Engine to generate formatted content based on Gson objects. So if I get a JsonDate, I can base my Velocity code to  use underlying Date object to implement the logic.

Adding part is easy. My concern would be whether it is a compelling enough use-case to add another JsonElement sub-class. It adds some complexity to Gson API. In Gson, we focused a lot on automatic Data binding. We haven't spent proportionately as much on DOM model. With the latest versions (thanks to Jesse Wilson), we even skip creation of DOM entirely if the user hasn't explicitly asked for it.

My interest in continuing this discussion is to see what improvements we can make to our DOM model.

Cheers

Inder

[Atul Dambalkar]

 

 

 

From: Inderjeet Singh [mailto:inde...@gmail.com]
Sent: Friday, August 31, 2012 10:25 PM
To: googl...@googlegroups.com
Cc: 'Inderjeet Singh'; at...@entrib.com
Subject: Re: Can we have Date object support as JsonDate in the API?

 

On Thursday, August 30, 2012 10:37:02 PM UTC-7, Atul Dambalkar wrote:

Hi Inder,

How would you use JsonDate differently that just a primitive? What operations are you doing on this DOM tree that require you to actually know that you are dealing with a date? Also, how is the current workaround of using a primitive working out?

 

[atul@entrib] This is where the problem is. In order to programmatically use JsonPrimitive object for Date, I have to use it as long or String, which effectively means loss of type information. So, now I have no way to know that underlying long or String is actually a Date. If I were to know that the underlying JsonPrimitive is Date, things will be simplified.

Yes, there is no easy way to add metadata like this. Though if you really want to hack around it, you can keep a separate map of (JsonElement, Type) to keep track of which object has which type. You can store that map in ThreadLocal or a field.

Hacky, I know. 

 

 

[atul@entrib] Yea, not really the best way to get it done.

 

I do know that a lot of storage technologies (especially databases) provide Date as a native type. However, most languages don't. It's not clear to me that that the storage technologies made the right choice, but my knowledge is limited in the area.

 

[atul@entrib] Isn’t JSON  just a declarative language,  like XML (and not really an imperative one)? IMHO, what I feel is, you could still add JsonDate class (although JSON spec doesn’t talk about it) which can still be used by the custom Type Adapters to convert the given string (for user defined/supported format). It will solve my problem greatly as then I will base my code on the underlying type. Just to elaborate my use-case further, I use Velocity Template Engine to generate formatted content based on Gson objects. So if I get a JsonDate, I can base my Velocity code to  use underlying Date object to implement the logic.

Adding part is easy. My concern would be whether it is a compelling enough use-case to add another JsonElement sub-class. It adds some complexity to Gson API. In Gson, we focused a lot on automatic Data binding. We haven't spent proportionately as much on DOM model. With the latest versions (thanks to Jesse Wilson), we even skip creation of DOM entirely if the user hasn't explicitly asked for it.

My interest in continuing this discussion is to see what improvements we can make to our DOM model.

 

[atul@entrib] Sure, I understand, you guys are  the ones who can decide the right  way going forward. Let’s see if there is interest on this in the community.

 

Thanks and regards,

 

-Atul

 

Cheers

Inder