Why sjson for json de/serialization
Manuel Bernhardt
I have a question to the developers: why choose sjson for JSON de/serialization?
I so far have been through:
- gson + custom code for handling scala collections (until that got
too big to maintain)
- a tentative of sjson, which didn't work out for reasons I'll explain below
- lift-json, until I ran into a bug that would make deserialization
impossible if not all of the members were provided
- jerkson, a scala wrapper for the jackson library, which works nicely.
What I'm looking for in a JSON library is the possibility to
de/serialize case classes very easily, like:
val serialized = generate[MyCaseClass](someInstanceOfMyCaseClass)
val deserialized = parse[MyCaseClass](someInstanceOfMyCaseClass)
(the above is how it looks like with Jerkson, without having to
register an additional format; it's also similar to how it looks like
with lift-json once the format is provided)
Additionally I need to have support for custom de/serializers, mostly
because of the use of BSON ObjectIds (our system has mongo in the
backend).
So far I found no easy way to register custom serializers for Java
beans with sjson. It seems to only support scala case classes unless
writing an own Protocol in full length. Or has this evolved since the
last time I checked?
Besides of that, I must say that the syntax of registering formats
with sjson looks rather bad.
Taking as example a previous mail on the list:
implicit val UserFormat: Format[User] = asProduct2("id",
"name")(User)(User.unapply(_).get)
Does look rather strange, and if I understood correctly needs to be
repeated everywhere, which does seem to be quite an overhead.
Any comments would be welcome,
thanks,
Manuel
Erwan Loisant
Gson is pretty much disqualified because it's a Java library that not
only doesn't support Scala collections, but also doesn't make use of
the Scala concepts. Typically, you can't prevent it to do reflection,
and when you use custom serializers you have to specify them (while
scala-ish libraries use implicit parameters). I don't know Jerkson,
but being based on a Java library I would suspect it has the same
limitations.
So the real candidates are sjson and lift-json. We studied both, and
figured out sjson provided a better type safety. I don't think the
problem you had is enough to through away sjson; it's actually more a
lack in the documentation than a bug.
You've been using the type-safe way to serialize classes: that's
indeed the recommended way. The advantage is that if you try to
serialize a type that it doesn't know how to serialize, you get a
compilation error. This way, you don't have to wait to get the error
at runtime. Other advantage, you only expose what you want to expose
and separate your model from your API.
The consequence is obviously that you have a bit more work to do:
serializer have to be written. If you don't like that, you can still
use sjson's reflection-based serialization.
https://github.com/debasishg/sjson/wiki/Reflection-based-JSON-Serialization
Of course, the integration of sjson in Play is pretty basic; it's
mostly provided to get a full stack. You can use any library you like
by including the necessary jars in your project.
> --
> You received this message because you are subscribed to the Google Groups "play-framework" group.
> To post to this group, send email to play-fr...@googlegroups.com.
> To unsubscribe from this group, send email to play-framewor...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/play-framework?hl=en.
>
>
--
Erwan Loisant
Julien Richard-Foy
> only doesn't support Scala collections, but also doesn't make use of
> the Scala concepts. Typically, you can't prevent it to do reflection,
> and when you use custom serializers you have to specify them (while
> scala-ish libraries use implicit parameters). I don't know Jerkson,
> but being based on a Java library I would suspect it has the same
> limitations.
Is it not possible to write things like that:
def generate[A : JsonSerializer](a: A) = JavaishJsonApi.generate(a,
implicitly[JsonSerializer])
??
Manuel Bernhardt
> The consequence is obviously that you have a bit more work to do:
> serializer have to be written. If you don't like that, you can still
> use sjson's reflection-based serialization.
> https://github.com/debasishg/sjson/wiki/Reflection-based-JSON-Serialization
Yes but that one is pretty much broken too in regards to custom
serializers for Java beans. Read: you can't serialize Java Beans in a
custom way without writing out the _whole_ serializer. At least it was
like this 2-3 weeks ago when I tried it out.
> Gson is pretty much disqualified because it's a Java library that not
> only doesn't support Scala collections, but also doesn't make use of
> the Scala concepts. Typically, you can't prevent it to do reflection,
> and when you use custom serializers you have to specify them (while
> scala-ish libraries use implicit parameters). I don't know Jerkson,
> but being based on a Java library I would suspect it has the same
> limitations.
What do you mean with "you can't prevent it to do reflection, and when
you use custom serializers you have to specify them"? I think that's
the whole point of a custom serializer; you want it to be custom
because you're not happy with the default serialization that takes
place. Or do you refer to sjson's way of specifying what fields get
serialized?
> Of course, the integration of sjson in Play is pretty basic; it's
> mostly provided to get a full stack. You can use any library you like
> by including the necessary jars in your project.
And this is what we do :)
However, the reason I sent this mail in the first place is that I
think that JSON support is a pretty darn important thing to have for a
full-stack framework. I really like the fact that Play does not impose
a specific javascript framework upon the user, though in compensation
I'd say that data exchange needs to have a good support
out-of-the-box. Play-Java's integration with Gson works nicely, and
Gson is powerful enough in the Java space to do whatever you need
without a whole lot of overhead.
The current - even though basic - integration of sjson for Play-Scala
doesn't feel very "Play-ish" to me. From a user perspective, so far
when working with Play we're used to clean-cut APIs that do things
with the minimum amount of overhead, so when seeing things like
implicit val UserFormat: Format[User] =
asProduct2("id","name")(User)(User.unapply(_).get)
it looks just plain ugly to me. "User" appears 4 times in the line
above, and it hasn't even serialized anything yet.
----
Besides of the non-Play-ish API design, I would like to re-iterate one
bigger concern for the library: it does not, to my knowledge, support
custom serializers for Java classes in combination with "easy"
de/serialization. This is a big issue, whatever you do, please take it
into account. I'd dare to say that I'm not the only one who'll want at
some point to serialize beasts like the BSON ObjectId.
----
About type-safety: you mention that there is an advantage of being
warned at compile-time if the library doesn't know how to serialize
something. I'd argue at this point that:
- during development, getting a RuntimeException because of a
non-serializable thing is a very acceptable behavior. And I don't
think you can develop a front-end view without correctly serialized
data (which is to say: I don't think that type-safe checks on JSON
serialization would provide more safety against issues in production,
since you can't realistically get a view to production without having
proper JSON data exchange anyway - you'd notice that during
development)
- at least from what I've seen so far Jerkson serializes everything.
For Java Beans it just takes the whole bean and decomposes it, the
same goes for case classes. And its support for Scala types is very
good, see what it offers in the specs at at
https://github.com/codahale/jerkson/tree/development/src/test/scala/com/codahale/jerkson/tests)
I don't know what kind of mechanism is in place with Jerkson
(reflection or not), I'd have to ask.
Manuel
Erwan Loisant
<bernhard...@gmail.com> wrote:
> Hi,
>
>> The consequence is obviously that you have a bit more work to do:
>> serializer have to be written. If you don't like that, you can still
>> use sjson's reflection-based serialization.
>> https://github.com/debasishg/sjson/wiki/Reflection-based-JSON-Serialization
>
> Yes but that one is pretty much broken too in regards to custom
> serializers for Java beans. Read: you can't serialize Java Beans in a
> custom way without writing out the _whole_ serializer. At least it was
> like this 2-3 weeks ago when I tried it out.
Yes, that's the whole point of avoiding reflection.
>> Gson is pretty much disqualified because it's a Java library that not
>> only doesn't support Scala collections, but also doesn't make use of
>> the Scala concepts. Typically, you can't prevent it to do reflection,
>> and when you use custom serializers you have to specify them (while
>> scala-ish libraries use implicit parameters). I don't know Jerkson,
>> but being based on a Java library I would suspect it has the same
>> limitations.
>
> What do you mean with "you can't prevent it to do reflection, and when
> you use custom serializers you have to specify them"? I think that's
> the whole point of a custom serializer; you want it to be custom
> because you're not happy with the default serialization that takes
> place. Or do you refer to sjson's way of specifying what fields get
> serialized?
I mean that if it doesn't find a serializer, it will fall back to
reflection. But I don't want to do reflection because if it fails, it
fails at runtime. I want it to fail at compile time so I know I have
to fix it.
Here is a simple example: you have a class, and you serialize it with
reflection. It works fine, until you add a field that correspond to a
type impossible to serialize (because of a loop). With sjson, it won't
compile, saying "I don't know how to serialize X". With reflection,
you won't know it breaks until you run that specific line. If it's in
your API for example, you may not test it and you may be releasing a
broken version of your web app.
>> Of course, the integration of sjson in Play is pretty basic; it's
>> mostly provided to get a full stack. You can use any library you like
>> by including the necessary jars in your project.
>
> And this is what we do :)
>
> However, the reason I sent this mail in the first place is that I
> think that JSON support is a pretty darn important thing to have for a
> full-stack framework. I really like the fact that Play does not impose
> a specific javascript framework upon the user, though in compensation
> I'd say that data exchange needs to have a good support
> out-of-the-box. Play-Java's integration with Gson works nicely, and
> Gson is powerful enough in the Java space to do whatever you need
> without a whole lot of overhead.
>
> The current - even though basic - integration of sjson for Play-Scala
> doesn't feel very "Play-ish" to me. From a user perspective, so far
> when working with Play we're used to clean-cut APIs that do things
> with the minimum amount of overhead, so when seeing things like
>
> implicit val UserFormat: Format[User] =
> asProduct2("id","name")(User)(User.unapply(_).get)
>
> it looks just plain ugly to me. "User" appears 4 times in the line
> above, and it hasn't even serialized anything yet.
Yes, that's unfortunate but the problem with case classes is that
they're look similar but they're not. So if you want to keep type
safety, that's the best we can do for now.
Or, you can simply rewrite the serializer using by building the JSON
with the DSL. It's fairly simple.
> Besides of the non-Play-ish API design, I would like to re-iterate one
> bigger concern for the library: it does not, to my knowledge, support
> custom serializers for Java classes in combination with "easy"
> de/serialization. This is a big issue, whatever you do, please take it
> into account. I'd dare to say that I'm not the only one who'll want at
> some point to serialize beasts like the BSON ObjectId.
Point taken, I'll see what we can do for that.
> ----
> About type-safety: you mention that there is an advantage of being
> warned at compile-time if the library doesn't know how to serialize
> something. I'd argue at this point that:
>
> - during development, getting a RuntimeException because of a
> non-serializable thing is a very acceptable behavior. And I don't
> think you can develop a front-end view without correctly serialized
> data (which is to say: I don't think that type-safe checks on JSON
> serialization would provide more safety against issues in production,
> since you can't realistically get a view to production without having
> proper JSON data exchange anyway - you'd notice that during
> development)
Typically you can have an API only used by third party. Your
application works because it's not based on the broken JSON, but your
API is broken.
Type safety is one of the great feature of Scala, so we try to
preserve it as much as possible.
> - at least from what I've seen so far Jerkson serializes everything.
> For Java Beans it just takes the whole bean and decomposes it, the
> same goes for case classes. And its support for Scala types is very
> good, see what it offers in the specs at at
> https://github.com/codahale/jerkson/tree/development/src/test/scala/com/codahale/jerkson/tests)
>
> I don't know what kind of mechanism is in place with Jerkson
> (reflection or not), I'd have to ask.
No matter how good it is it will never be able to serialize loops.
(For example user references group, and group contains list of users).
There is an other issue with reflection (because that's really what
we're talking about): it usually works to just dump all fields of your
models to your json, but it's seldom the best approach. Usually you
want to pick the fields, maybe reformat them... Not doing it is
introducing a dependency on your model to your API, while your API
should be designed independently from the model.
--
Erwan Loisant