ANNOUNCE: spray-json 1.3.0
1,236 views
Skip to first unread message
Mathias Doenitz
Sep 22, 2014, 5:39:33 AM9/22/14
to spray...@googlegroups.com
Dear sprayers,
we have just released version 1.3.0 of spray-json, which is mostly a performance-related upgrade.
The key improvement over the previous release (1.2.6) is a completely rewritten JSON parser, which delivers a speedup factor of ca. 15 over the old implementation!
Here are the results of the JSON parsing benchmark from parboiled2* running on my Core i7 machine:
https://gist.github.com/sirthias/3179a1c47d6ee7030eef
Apart front the parser upgrade we have made some smaller changes to the JSON AST, which might break your code in certain cases.
However, these things should be really easy to fix.
The full CHANGELOG:
- Upgraded to Scala 2.11.2, dropped support for Scala 2.9
- Switched to fast, hand-written parser (#86, #108)
- Removed dependency on parboiled 1.x
- Changed parser to produce JsObject(HashMap) rather than JsObject(ListMap)
- Switched JsArray(List) to JsArray(Vector)
- Improved JsonPrinter to support printing to custom StringBuilder
- Added support for parameter-less case classes (#41)
The release has been uploaded to Sonatype and should appear on Maven Central shortly.
Please let us know if you have any unexpected issues with upgrading!
Cheers,
Mathias
*https://github.com/sirthias/parboiled2/blob/master/jsonBenchmark/src/main/scala/org/parboiled/examples/JsonParserBenchmark.scala
---
mat...@spray.io
http://spray.io
we have just released version 1.3.0 of spray-json, which is mostly a performance-related upgrade.
The key improvement over the previous release (1.2.6) is a completely rewritten JSON parser, which delivers a speedup factor of ca. 15 over the old implementation!
Here are the results of the JSON parsing benchmark from parboiled2* running on my Core i7 machine:
https://gist.github.com/sirthias/3179a1c47d6ee7030eef
Apart front the parser upgrade we have made some smaller changes to the JSON AST, which might break your code in certain cases.
However, these things should be really easy to fix.
The full CHANGELOG:
- Upgraded to Scala 2.11.2, dropped support for Scala 2.9
- Switched to fast, hand-written parser (#86, #108)
- Removed dependency on parboiled 1.x
- Changed parser to produce JsObject(HashMap) rather than JsObject(ListMap)
- Switched JsArray(List) to JsArray(Vector)
- Improved JsonPrinter to support printing to custom StringBuilder
- Added support for parameter-less case classes (#41)
The release has been uploaded to Sonatype and should appear on Maven Central shortly.
Please let us know if you have any unexpected issues with upgrading!
Cheers,
Mathias
*https://github.com/sirthias/parboiled2/blob/master/jsonBenchmark/src/main/scala/org/parboiled/examples/JsonParserBenchmark.scala
---
mat...@spray.io
http://spray.io
Age Mooij
Sep 22, 2014, 8:42:12 AM9/22/14
to spray...@googlegroups.com
Great work Mathias, thanks!
Good to see your benchmark shows spray-json now within about 20% of Jackson. Perhaps you could post these results (and maybe some more micro benchmarks for different document sizes) to the project README?
I would also be interested to see how the new version compares with Jawn (https://github.com/non/jawn), which seems to claim to be faster than Jackson.
Age
> --
> You received this message because you are subscribed to the Google Groups "spray.io User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to spray-user+...@googlegroups.com.
> Visit this group at http://groups.google.com/group/spray-user.
> To view this discussion on the web visit https://groups.google.com/d/msgid/spray-user/8045096A-957E-4580-8E9F-B86146F4CDBB%40spray.io.
> For more options, visit https://groups.google.com/d/optout.
Good to see your benchmark shows spray-json now within about 20% of Jackson. Perhaps you could post these results (and maybe some more micro benchmarks for different document sizes) to the project README?
I would also be interested to see how the new version compares with Jawn (https://github.com/non/jawn), which seems to claim to be faster than Jackson.
Age
> You received this message because you are subscribed to the Google Groups "spray.io User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to spray-user+...@googlegroups.com.
> Visit this group at http://groups.google.com/group/spray-user.
> To view this discussion on the web visit https://groups.google.com/d/msgid/spray-user/8045096A-957E-4580-8E9F-B86146F4CDBB%40spray.io.
> For more options, visit https://groups.google.com/d/optout.
Mathias Doenitz
Sep 22, 2014, 8:55:41 AM9/22/14
to spray...@googlegroups.com
Age,
with spray-json 1.3.0 the bottleneck in parsing JSON has been moved away from the parser into AST construction.
The parser itself is much faster that the benchmark numbers suggest. However, the current spray-json AST has elements like
case class JsNumber(value: BigDecimal) extends JsValue
which means that we need to instantiate a `BigDecimal` for every JSON number, even if would perfectly fit into a 32bit Integer.
If we want to further increase JSON processing performance we need a better AST, which is not something that we’d like to do in the context of the spray-json project.
The important thing with spray-json 1.3 is that parsing is not a perf issue anymore, which should be all that is required for most users.
If you have special performance requirements you’ll probably want to use another library and an entirely different JSON handling approach anyway.
Cheers,
Mathias
---
mat...@spray.io
http://spray.io
> To view this discussion on the web visit https://groups.google.com/d/msgid/spray-user/FDBAE350-9195-4EB6-844F-14E71A8F5786%40gmail.com.
with spray-json 1.3.0 the bottleneck in parsing JSON has been moved away from the parser into AST construction.
The parser itself is much faster that the benchmark numbers suggest. However, the current spray-json AST has elements like
case class JsNumber(value: BigDecimal) extends JsValue
which means that we need to instantiate a `BigDecimal` for every JSON number, even if would perfectly fit into a 32bit Integer.
If we want to further increase JSON processing performance we need a better AST, which is not something that we’d like to do in the context of the spray-json project.
The important thing with spray-json 1.3 is that parsing is not a perf issue anymore, which should be all that is required for most users.
If you have special performance requirements you’ll probably want to use another library and an entirely different JSON handling approach anyway.
Cheers,
Mathias
---
mat...@spray.io
http://spray.io
Age Mooij
Sep 22, 2014, 9:02:29 AM9/22/14
to spray...@googlegroups.com
I totally agree. I just thought it would be nice for new users if the README would contain a few basic performance numbers so they can make an informed decision.
This performance boost is more than enough for our needs so my interests in seeing further performance improvements (and comparisons with other libraries) are purely academic ;)
Thanks again1
Age
> To view this discussion on the web visit https://groups.google.com/d/msgid/spray-user/C620271B-3D31-4495-BE8B-0EA46D7EC9D2%40spray.io.
This performance boost is more than enough for our needs so my interests in seeing further performance improvements (and comparisons with other libraries) are purely academic ;)
Thanks again1
Age
Mathias Doenitz
Sep 22, 2014, 9:35:55 AM9/22/14
to spray...@googlegroups.com
Yes, of course we’d also be interested in officially grabbing the JSON parsing crown on the JVM (purely for academic reasons, of course! :),
however, with the current AST this is simply not possible.
Once the main spray-to-akka port is done we might have some time for a proper JSON AST/parser re-design, which will certainly come with more features (like being fully reactive-stream enabled).
> To view this discussion on the web visit https://groups.google.com/d/msgid/spray-user/650E6A4A-D74A-4AE0-BD11-F3913E5C6E61%40gmail.com.
however, with the current AST this is simply not possible.
Once the main spray-to-akka port is done we might have some time for a proper JSON AST/parser re-design, which will certainly come with more features (like being fully reactive-stream enabled).
Koert Kuipers
Sep 22, 2014, 10:40:11 AM9/22/14
to spray...@googlegroups.com
i tried to use it by simply bumping the version of spray-json in my projects and recompiling and re-running tests, trusting the compiler will tell me what changed.
it compiles without errors, tests also compile, but all my spray-routing (1.2.1) tests fail.
java.lang.NoSuchMethodError: spray.json.JsonParser$.apply(Ljava/lang/String;)Lspray/json/JsValue;
[info] at spray.httpx.SprayJsonSupport$$anonfun$sprayJsonUnmarshaller$1.applyOrElse(SprayJsonSupport.scala:36)
[info] at spray.httpx.SprayJsonSupport$$anonfun$sprayJsonUnmarshaller$1.applyOrElse(SprayJsonSupport.scala:34)
[info] at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:33)
[info] at spray.httpx.unmarshalling.Unmarshaller$$anon$1$$anonfun$unmarshal$1.apply(Unmarshaller.scala:29)
[info] at spray.httpx.unmarshalling.SimpleUnmarshaller.protect(SimpleUnmarshaller.scala:40)
[info] at spray.httpx.unmarshalling.Unmarshaller$$anon$1.unmarshal(Unmarshaller.scala:29)
[info] at spray.httpx.unmarshalling.SimpleUnmarshaller.apply(SimpleUnmarshaller.scala:29)
[info] at spray.httpx.unmarshalling.SimpleUnmarshaller.apply(SimpleUnmarshaller.scala:23)
[info] at spray.httpx.unmarshalling.UnmarshallerLifting$$anon$3.apply(UnmarshallerLifting.scala:35)
[info] at spray.httpx.unmarshalling.UnmarshallerLifting$$anon$3.apply(UnmarshallerLifting.scala:34)
it compiles without errors, tests also compile, but all my spray-routing (1.2.1) tests fail.
java.lang.NoSuchMethodError: spray.json.JsonParser$.apply(Ljava/lang/String;)Lspray/json/JsValue;
[info] at spray.httpx.SprayJsonSupport$$anonfun$sprayJsonUnmarshaller$1.applyOrElse(SprayJsonSupport.scala:36)
[info] at spray.httpx.SprayJsonSupport$$anonfun$sprayJsonUnmarshaller$1.applyOrElse(SprayJsonSupport.scala:34)
[info] at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:33)
[info] at spray.httpx.unmarshalling.Unmarshaller$$anon$1$$anonfun$unmarshal$1.apply(Unmarshaller.scala:29)
[info] at spray.httpx.unmarshalling.SimpleUnmarshaller.protect(SimpleUnmarshaller.scala:40)
[info] at spray.httpx.unmarshalling.Unmarshaller$$anon$1.unmarshal(Unmarshaller.scala:29)
[info] at spray.httpx.unmarshalling.SimpleUnmarshaller.apply(SimpleUnmarshaller.scala:29)
[info] at spray.httpx.unmarshalling.SimpleUnmarshaller.apply(SimpleUnmarshaller.scala:23)
[info] at spray.httpx.unmarshalling.UnmarshallerLifting$$anon$3.apply(UnmarshallerLifting.scala:35)
[info] at spray.httpx.unmarshalling.UnmarshallerLifting$$anon$3.apply(UnmarshallerLifting.scala:34)
Koert Kuipers
Sep 22, 2014, 11:05:39 AM9/22/14
to spray...@googlegroups.com
oh silly of me. thats not my code. thats spray-httpx. guess spray-httpx also needs to be rebuild against this new version of spray-json?
Mathias Doenitz
Sep 22, 2014, 3:13:28 PM9/22/14
to spray...@googlegroups.com
Koert,
yes, spray-json 1.3.0 does introduce some breaking changes, which cause it to be incompatible with the current spray releases.
Here is the ticket to follow: https://github.com/spray/spray/issues/932
However, you can already use spray-json 1.3.0 with the existing spray releases simply by not using the pre-defined `SprayJsonSupport` but a verbatim copy thereof.
So this does it:
trait SprayJsonSupport {
implicit def sprayJsonUnmarshallerConverter[T](reader: RootJsonReader[T]) =
sprayJsonUnmarshaller(reader)
implicit def sprayJsonUnmarshaller[T: RootJsonReader] =
Unmarshaller[T](MediaTypes.`application/json`) {
case x: HttpEntity.NonEmpty ⇒
val json = JsonParser(x.asString(defaultCharset = HttpCharsets.`UTF-8`))
jsonReader[T].read(json)
}
implicit def sprayJsonMarshallerConverter[T](writer: RootJsonWriter[T])(implicit printer: JsonPrinter = PrettyPrinter) =
sprayJsonMarshaller[T](writer, printer)
implicit def sprayJsonMarshaller[T](implicit writer: RootJsonWriter[T], printer: JsonPrinter = PrettyPrinter) =
Marshaller.delegate[T, String](ContentTypes.`application/json`) { value ⇒
val json = writer.write(value)
printer(json)
}
}
object SprayJsonSupport extends SprayJsonSupport
In case you wonder why the exact same code works when you compile it but fails as a part of spray 1.x.1:
The breaking change here is source compatible but not binary compatible. So simply recompiling against spray-json 1.3.0 is all that is required.
> To view this discussion on the web visit https://groups.google.com/d/msgid/spray-user/CANx3uAjQ02bOUQ06Ou0jr%3DXAMf52za_k5bUGsu-wP7e6BSp20A%40mail.gmail.com.
yes, spray-json 1.3.0 does introduce some breaking changes, which cause it to be incompatible with the current spray releases.
Here is the ticket to follow: https://github.com/spray/spray/issues/932
However, you can already use spray-json 1.3.0 with the existing spray releases simply by not using the pre-defined `SprayJsonSupport` but a verbatim copy thereof.
So this does it:
trait SprayJsonSupport {
implicit def sprayJsonUnmarshallerConverter[T](reader: RootJsonReader[T]) =
sprayJsonUnmarshaller(reader)
implicit def sprayJsonUnmarshaller[T: RootJsonReader] =
Unmarshaller[T](MediaTypes.`application/json`) {
case x: HttpEntity.NonEmpty ⇒
val json = JsonParser(x.asString(defaultCharset = HttpCharsets.`UTF-8`))
jsonReader[T].read(json)
}
implicit def sprayJsonMarshallerConverter[T](writer: RootJsonWriter[T])(implicit printer: JsonPrinter = PrettyPrinter) =
sprayJsonMarshaller[T](writer, printer)
implicit def sprayJsonMarshaller[T](implicit writer: RootJsonWriter[T], printer: JsonPrinter = PrettyPrinter) =
Marshaller.delegate[T, String](ContentTypes.`application/json`) { value ⇒
val json = writer.write(value)
printer(json)
}
}
object SprayJsonSupport extends SprayJsonSupport
In case you wonder why the exact same code works when you compile it but fails as a part of spray 1.x.1:
The breaking change here is source compatible but not binary compatible. So simply recompiling against spray-json 1.3.0 is all that is required.
Eric Weise
Sep 23, 2014, 12:50:05 PM9/23/14
to spray...@googlegroups.com
Hi Mathias,
Will this version be used for the next round of Techempower benchmarks? Looks like the current version used is 1.2.4. Not sure how much impact it would have.
Eric
Will this version be used for the next round of Techempower benchmarks? Looks like the current version used is 1.2.4. Not sure how much impact it would have.
Eric
Reply all
Reply to author
Forward
0 new messages