[2.0] JSON in Play

[James Roper]

Hey all,

I'd like to address the topic of JSON parsing in Play again.  Just to put some hard numbers on this, yesterday, unrelated to play framework, I was looking at some code that someone told me was going much slower than expected.  It was taking about 120 milliseconds to do something that they expected to only take a few milliseconds.  We quickly determined that the major part of the issue was that they weren't reusing the Jackson ObjectMapper.  Once they reused the Jackson ObjectMapper, the time of this thing (including the IO involved in this operation) went down to 5 milliseconds.  So you can see that creating an ObjectMapper each time you need to use it can have a massive performance impact.

I submitted this pull request, but it was rejected because it made json support a "plugin":

https://github.com/playframework/Play20/pull/225

Actually, I wasn't interested in making JSON support a plugin, all I wanted was the ability to initialise an ObjectMapper on application start, and reinitialise it each time the application is updated (in development, this is needed, because Jackson might not pick up changes in annotations).  The only way I can see for an internal component in play to hook into play to do this is to implement the Plugin interface, from where it gets notified of application start events.  This is the same way that other core non plugin components of Play seem to do it.  In fact, I put the JSON plugin into Plays own play.plugins file, which means it's impossible to disable it... which means it's not really a plugin at all, it's just using the plugin mechanism to hook into application start events.  I would have thought this is the sensible way to go about doing that, am I wrong?  If I am wrong, what is the correct way for a core play component to hook into application start events?

I also wanted to provide a way to configure the ObjectMapper play uses (for example, to register custom serialisers/deserialisers), which also requires doing things on application start... and using the Plugin interface seemed to be the right way to do this too.

Also, Scala JSON support is already using a singleton instance, though perhaps we want to provide a mechanism for configuring the ObjectMapper in Scala, but I think this is a separate issue, because the code bases in both the Java and the Scala support are completely different.

So, should I implement it in some different way, or can my original pull request be accepted?

Cheers,

James

[Guillaume Bort]

> So, should I implement it in some different way, or can my original pull
> request be accepted?

Sorry, you must definitely implement this in a different way. I agree
that it would be an interesting optimization but it shouldn't be tied
to any Plugin concept (and neither to the application lifecycle).
Please think about the Json support as a separate library.

> --
> You received this message because you are subscribed to the Google Groups
> "play-framework" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/play-framework/-/zsJdd-rJ-sMJ.
> To post to this group, send email to play-fr...@googlegroups.com.
> To unsubscribe from this group, send email to
> play-framewor...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/play-framework?hl=en.



--
Guillaume Bort, http://guillaume.bort.fr

[James Roper]

But lot's of separate libraries need to do initialisation as part of the application lifecycle.  I could consider caching a separate library too, but the ehcache support for Play is tied to the application lifecycle and implemented as a plugin.  I'm also not sure that Json support can be considered a separate library, it is used by some of the most fundmental parts in Play, for example, by the request and response classes to serialise/deserialise json responses/requests.

It's not uncommon, particularly for performance reasons, for a library to require doing some application lifecycle bound initialisation.  Surely play must support this?

> To post to this group, send email to play-framework@googlegroups.com.

> To unsubscribe from this group, send email to

> play-framework+unsubscribe@googlegroups.com.

> For more options, visit this group at
> http://groups.google.com/group/play-framework?hl=en.



--
Guillaume Bort, http://guillaume.bort.fr

> To post to this group, send email to play-framework@googlegroups.com.

> To unsubscribe from this group, send email to

> play-framework+unsubscribe@googlegroups.com.

[Guillaume Bort]

That's because it is required by the core that it can't be defined as
a Plugin (unlike the cache plugin that is fully optional and can be
swapped by any other cache plugin). But because we will probably
release the Play JSON lib as a separate project (like the template
engine or Anorm), it can't depends itself of the application
lifecycle.

I suppose that you are doing it this way because the ObjectMapper
keeps a lot of cache and can't be kept between reload right?

>> > To post to this group, send email to play-fr...@googlegroups.com.

>> > To unsubscribe from this group, send email to

>> > play-framewor...@googlegroups.com.

>> > To post to this group, send email to play-fr...@googlegroups.com.

>> > To unsubscribe from this group, send email to

>> > play-framewor...@googlegroups.com.

>> > For more options, visit this group at
>> > http://groups.google.com/group/play-framework?hl=en.
>>
>>
>>
>> --
>> Guillaume Bort, http://guillaume.bort.fr
>

> --
> You received this message because you are subscribed to the Google Groups
> "play-framework" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/play-framework/-/Z8g-Hlhc2L8J.
>
> To post to this group, send email to play-fr...@googlegroups.com.

> To unsubscribe from this group, send email to

> play-framewor...@googlegroups.com.

[James Roper]

I suppose that you are doing it this way because the ObjectMapper
keeps a lot of cache and can't be kept between reload right?

That's correct.  Also, it's my assumption that it can't be kept between reloads, though I haven't tested this.  I'm not sure how play classloading works in development, that is, if it does any hotswapping of classes, or just drops classloaders and reloads them.  If it's working at the classloader level, then it technically doesn't need to be reloaded, though a memory leak would be present (one that would keep old classloaders live).

The other reason that initially led me to implement it as a plugin was so that a user could define a custom class that supplied an ObjectMapper, so that they could reconfigure the ObjectMapper at their will (the list of reasons for wanting to do this is huge, but a likely common reason would be to configure whether dates are formatted as numbers or strings, and if string, what date format should be used).  This would also require hooking into the lifecycle, in case the user had changed the code that supplied the ObjectMapper.

So, assuming the play JSON library is a separate project, would an acceptable solution be for it to provide an interface that could be used to provide an object mapper, and lazily initialise this on first use to be one that just used a simple singleton ObjectMapper?  Then play could implement this interface, and set it to be the provider for the library, and use some mechanism for binding the object mapper to the lifecycle?  As for that mechanism, is the plugin concept appropriate there? Or would something else be better?

Another solution might be for the json library to simply have a var reference to an ObjectMapper, and let play set it whenever it wants to.

 

>> > To post to this group, send email to play-framework@googlegroups.com.

>> > To unsubscribe from this group, send email to

>> > play-framework+unsubscribe@googlegroups.com.

>> > To post to this group, send email to play-framework@googlegroups.com.

>> > To unsubscribe from this group, send email to

>> > play-framework+unsubscribe@googlegroups.com.

>> > For more options, visit this group at
>> > http://groups.google.com/group/play-framework?hl=en.
>>
>>
>>
>> --
>> Guillaume Bort, http://guillaume.bort.fr
>
> --
> You received this message because you are subscribed to the Google Groups
> "play-framework" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/play-framework/-/Z8g-Hlhc2L8J.
>

> To post to this group, send email to play-framework@googlegroups.com.

> To unsubscribe from this group, send email to

> play-framework+unsubscribe@googlegroups.com.

[Ben McCann]

I like the idea of providing an interface accepts an object mapper and having Play create and pass in a singleton ObjectMapper.  Continually making new ObjectMappers like we are now is insane.  What do you think Guillaume?

[Guillaume Bort]

Could we first check that we have really a problem if we reuse the
same ObjectMapper between reloading? It depends how the caching stuff
is done in Jerkson. If it's done properly it shouldn't cause any
problem.

>>> >> > play-fr...@googlegroups.com.

>>> >> > To unsubscribe from this group, send email to

>>> >> > play-framewor...@googlegroups.com.

>>> >> > play-fr...@googlegroups.com.

>>> >> > To unsubscribe from this group, send email to

>>> >> > play-framewor...@googlegroups.com.

>>> >> > For more options, visit this group at
>>> >> > http://groups.google.com/group/play-framework?hl=en.
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Guillaume Bort, http://guillaume.bort.fr
>>> >
>>> > --
>>> > You received this message because you are subscribed to the Google
>>> > Groups
>>> > "play-framework" group.
>>> > To view this discussion on the web visit
>>> > https://groups.google.com/d/msg/play-framework/-/Z8g-Hlhc2L8J.
>>> >

>>> > To post to this group, send email to play-fr...@googlegroups.com.

>>> > To unsubscribe from this group, send email to

>>> > play-framewor...@googlegroups.com.

>>> > For more options, visit this group at
>>> > http://groups.google.com/group/play-framework?hl=en.
>>>
>>>
>>>
>>> --
>>> Guillaume Bort, http://guillaume.bort.fr
>
> --
> You received this message because you are subscribed to the Google Groups
> "play-framework" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/play-framework/-/CFNcdKqQch8J.
>
> To post to this group, send email to play-fr...@googlegroups.com.

> To unsubscribe from this group, send email to

> play-framewor...@googlegroups.com.

[Alex Jarvis]

Is it possible to access the ObjectMapper now?

I'm trying to find a way to parse ISO8601 Date/Time strings and have found the joda module but it requires access to the ObjectMapper.

From what I can see, the ObjectMapper object is in a private val inside a private object.. https://github.com/playframework/Play20/blob/master/framework/src/play/src/main/scala/play/api/libs/json/JsValue.scala

I guess an alternative would be to use the jackson-module-scala ?

Another option might be to write a Formatter like the following https://gist.github.com/2833989 but for Joda Time, except this means that I'm stuck with having to write Reads/Writes or Format for every Json case class rather than being able to parse/generate dynamically.

Any help, or insight would be appreciated!

[Pascal Voitot Dev]

On Wed, Oct 31, 2012 at 12:16 AM, Alex Jarvis <alex.j...@gmail.com> wrote:

Is it possible to access the ObjectMapper now?

I'm trying to find a way to parse ISO8601 Date/Time strings and have found the joda module but it requires access to the ObjectMapper.

From what I can see, the ObjectMapper object is in a private val inside a private object.. https://github.com/playframework/Play20/blob/master/framework/src/play/src/main/scala/play/api/libs/json/JsValue.scala

I guess an alternative would be to use the jackson-module-scala ?

Another option might be to write a Formatter like the following https://gist.github.com/2833989 but for Joda Time, except this means that I'm stuck with having to write Reads/Writes or Format for every Json case class rather than being able to parse/generate dynamically.

JodaTime Reads/Writes have been added in Play2.1-SNAPSHOT master
They need a few more tests but the code is there.

Regards
Pascal
 

To view this discussion on the web visit https://groups.google.com/d/msg/play-framework/-/-u6jiaa1raIJ.

[Alex Jarvis]

Thank you Pascal,

Do you have an example of it being used? I see that the default joda pattern is "yyyy-MM-dd", so what (and where) would be the best way to set my own implicit val with a different pattern.

Also, I would love it if you can explain where the implicit val for a Reads[T] is used. I've had a brief look at the code but it's above my understanding for now.

Cheers!

Alex

[Pascal Voitot Dev]

Before going further, I thihk you should learn a few basics about Scala implicits and then Json in Play which uses them intensively.

Just to give you a glimpse:
Json.fromJson[T](js: JsValue)(implicit val r: Reads[T]): T
Json.toJson[T](t: T)(implicit val w: Writes[T]): JsValue

You see the implicit Reads[T]/Writes[T] ?
It's called implicit typeclasses and is one of the most powerful pattern of Scala.
The compiler will find the right Reads[T] depending on the T you gave and if it can't find any, it generates an error.
Json.fromJson[Int](JsNumber(5)) gives (5: Int) because the compiler found an implicit Reads[Int] that is provided by Play...

You can find lots of info on the web about implicit in Scala.
I let you discover by yourself, it would be too long to explain ;)

Are you using Play2.1? (with ReactiveMongo at least ;) )
If yes, maybe you could read my recent articles presenting new JSON API for Play2.1:
http://mandubian.com/2012/09/08/unveiling-play-2-dot-1-json-api-part1-jspath-reads-combinators/
http://mandubian.com/2012/10/01/unveiling-play-2-dot-1-json-api-part2-writes-format-combinators/

and I should publish part3 today...

Pascal

To view this discussion on the web visit https://groups.google.com/d/msg/play-framework/-/pwEPn9AnwWUJ.

[Alex Jarvis]

Ah yes, I didn't realise that it was an implicit on the to/from Json functions :)

I've read through your articles previously (thanks for writing them) and just even did a search for 'Date'. I guess what wanted to understand is how can I use Reads[T] for a case class which has a joda date time.

A simplified example:

case class UserDTO(
  id: Option[String] = None,
  email: String,
  updated: Option[DateTime]
)

object UserDTO {

  implicit val userReads: Reads[UserSignUpDTO] = (

    (__ \ "id").readOpt[String] and

    (__ \ "email").read(email provided minLength[String](5)) and

    (__ \ "updated").read[???] and

  )(UserSignUpDTO.apply _)

}

Can I just place DateTime instead of ??? and it will use the right Reads[T] ?

If so, how can I:

1. Specify my own pattern.

2. Provide an alternative Reads[DateTime] which uses ISODateTimeFormat (if necessary, or maybe I can just provide an ISO8601 pattern).

[Pascal Voitot Dev]

On Wed, Oct 31, 2012 at 11:31 AM, Alex Jarvis <alex.j...@gmail.com> wrote:

Ah yes, I didn't realise that it was an implicit on the to/from Json functions :)

I've read through your articles previously (thanks for writing them) and just even did a search for 'Date'. I guess what wanted to understand is how can I use Reads[T] for a case class which has a joda date time.

A simplified example:

case class UserDTO(
  id: Option[String] = None,
  email: String,
  updated: Option[DateTime]
)

object UserDTO {

  implicit val userReads: Reads[UserSignUpDTO] = (

    (__ \ "id").readOpt[String] and

    (__ \ "email").read(email provided minLength[String](5)) and

    (__ \ "updated").read[???] and

  )(UserSignUpDTO.apply _)

}

Can I just place DateTime instead of ??? and it will use the right Reads[T] ?

If so, how can I:

1. Specify my own pattern.

2. Provide an alternative Reads[DateTime] which uses ISODateTimeFormat (if necessary, or maybe I can just provide an ISO8601 pattern).

in Play 2.1, you can use Reads[DateTime] directly (minus little potential bugs because this is not tested totally) but it uses a default pattern

  implicit val DefaultJodaDateReads = jodaDateReads("yyyy-MM-dd")

So if you want a custom pattern, create your own Reads[DateTime] using this function

val yourJodaDateReads = jodaDateReads("YOUR_PATTERN")

and then use it directly, not as an implicit

Json.fromJson[DateTime](js)(yourJodaDateReads)


Pascal

[Martin Grotzke]

Picking up this rather old thread. I'm also interested in customizable, reused ObjectMapper support in play2 java, as described by James. Is there anything new regarding this topic?

The current state (referring to play2 java) is rather disappointing, especially for a framework like play that claims to be modern / on the edge. The docs (http://www.playframework.org/documentation/2.0/JavaJsonRequests) show examples with maps containing 2 entries or so, but what about complex object graphs (e.g. a request with an order,,  orderlines, customer, address, payment etc., or a product search response with products, facets)? How would such examples look like?

Cheers,

Martin

>> > To post to this group, send email to play-fr...@googlegroups.com.

>> > To unsubscribe from this group, send email to

>> > play-framewor...@googlegroups.com.

>> > To post to this group, send email to play-fr...@googlegroups.com.

>> > To unsubscribe from this group, send email to

>> > play-framewor...@googlegroups.com.

>> > For more options, visit this group at
>> > http://groups.google.com/group/play-framework?hl=en.
>>
>>
>>
>> --
>> Guillaume Bort, http://guillaume.bort.fr
>
> --
> You received this message because you are subscribed to the Google Groups
> "play-framework" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/play-framework/-/Z8g-Hlhc2L8J.
>

> To post to this group, send email to play-fr...@googlegroups.com.

> To unsubscribe from this group, send email to

> play-framewor...@googlegroups.com.

[Michael Janiak]

It's an old thread but remains valid for me at least. Any news?