Request for opinions on a rewrite of the lift-json parser
45 views
Skip to first unread message
Matt Farmer
Mar 18, 2017, 2:18:29 PM3/18/17
to Lift
Hey all,
As some of you know I’ve been working in PR 1768 on improving the lift-json parser to support
better handling of tuples. Unfortunately, I’ve hit a very annoying wall that wherein the Scala
primitive types appear as java.lang.Object to the JVM. This isn’t the first time that the lift-json
parser has run into this situation, and in the past we’ve special cased our handling of places
where java.lang.Object comes up in these type signatures by using scalap to pull the full
Scala type signature for the parameterized type of a class.
Attempting to continue down the road of adding more and more special cases is an option. Yet
I think that it’s probably time to consider other options. Lift-json’s parser has served me and
many others well. However, I’m starting to become convinced that it’s become only more
cumbersome to maintain over time.
We’ve held off using scala.reflect as it was experimental in 2.10, which was the last time we
considered implementing it. However, I can’t find any markings of experimental-ness in 2.12.
How would folks feel at this point about potentially rewriting the lift-json parser to use
scala.reflect?
I think that the first iteration of this would probably involve starting with the implementation of
a parallel interface for extraction. As we iterate through Lift 3 minor releases, we would move
more and more in the direction of using it everywhere with an eye toward making it the default
for Lift 4.
You may now submit your opinions. =)
Antonio Salazar Cardozo
Mar 19, 2017, 10:51:05 AM3/19/17
to Lift
Quick note here: the scala reflection guide (http://docs.scala-lang.org/overviews/reflection/overview.html) still indicates scala.reflect is experimental. In the API docs I think it's the macros package that reflects this (http://www.scala-lang.org/api/2.12.1/scala-reflect/scala/reflect/macros/index.html). Additionally, I believe the intent is to in some future release replace it with the outcome of the scala.meta project (http://scalameta.org). From a reading of the current status (https://github.com/scalameta/scalameta/issues/604) it looks like maybe they'll be leaning on some of the facilities in scala.reflect for some of the functionality in scala.meta... but it's somewhat unclear.
I think there's some investigation needed to figure out which parts of scala.reflect will be of use here (do we want to use macros? Manifest is technically in scala.reflect, but does it provide everything we need? Probably not) and whether these are marked experimental and what the forthcoming strategy in scala proper is for those bits. That'll probably require asking in scala-lang or somesuch once we know what we need.
In the meantime, we can always start developing this extractor as a module outside of lift/framework, depended on separately, and moved into lift-json when we feel it's ready. This decouples it completely from the lift release cycle while it's iterating fast early on. Not unlike what scala.meta is doing, really :)
If we do pursue that latter approach, though, we'll want to be clear about the expected support lifetime of a given version of that module. It presumably will not be the same as that of a lift version, nor will the API stability :)
Would be super exciting to have an improved extractor!
Thanks,
Antonio
Matt Farmer
Mar 19, 2017, 2:04:24 PM3/19/17
to Lift
Of course Antonio finds the one URL that I couldn’t highlighting it as Experimental. I kind of hate
the Scala website.
To start with, from reading the docs that I did find my thinking is that the new TypeTag
stuff might provide us richer access to type information and we’d be able to lean less on Java
reflection and guesswork in the form of “if this is a java.lang.Object call out to the compiler
to figure out what it actually is!”
I could, however, be wrong about what TypeTag can actually deliver. That API, however, isn’t
marked as experimental in the API:
The problem, of course, and the one place Macros would come in handy, is that I would like
for us to continue using any available constructors as data comes in. Not being able to do
that was a very annoying limitation of play-json during my interactions with it.
I hear you on introducing this as a separate module to start with. That’s probably a good
idea. I’ll probably spin something up and start playing with it soon. Speaking of which, should
this be considered a lift module? Our story for discovering other things that can be bolted
onto Lift right now isn’t super great. It would probably be ideal to work toward unifying
where we store those things as committers, yeah?
(Also I don’t have access to the liftmodules org which would also be cool to have. Hah.)
--
--
Lift, the simply functional web framework: http://liftweb.net
Code: http://github.com/lift
Discussion: http://groups.google.com/group/liftweb
Stuck? Help us help you: https://www.assembla.com/wiki/show/liftweb/Posting_example_code
---
You received this message because you are subscribed to the Google Groups "Lift" group.
To unsubscribe from this group and stop receiving emails from it, send an email to liftweb+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Richard Dallaway
Mar 19, 2017, 4:19:17 PM3/19/17
to lif...@googlegroups.com
On Sun, 19 Mar 2017 at 18:04 Matt Farmer <ma...@frmr.me> wrote:
(Also I don’t have access to the liftmodules org which would also be cool to have. Hah.)
You do now :-)
ti com
Mar 19, 2017, 9:38:53 PM3/19/17
to lif...@googlegroups.com
Do you mean switching runtime reflection to use the scala.reflect api or do you mean using macros?
The latter is being replaced by the inline and meta feature. For the former, you might want to ask about plans at contributors.scala-lang.org
Matt Farmer
Mar 21, 2017, 12:04:28 AM3/21/17
to Lift
I think I’m leaning toward switching runtime reflection to scala.reflect since that would be the smallest
change in our API, I think.
I actually started playing with it in a side project and it is doing what I originally hoped which is avoiding
having to edge case things…. so far, at least. You can see what I’ve got so far here:
The big difference so far is that primitive types are just there for me to analyze in the constructor signatures
so I don’t have to call out to the compiler to parse those out. Scala’s reflector already has the information.
More to come, I expect. :)
Arseny Tolmachev
Mar 21, 2017, 9:08:31 PM3/21/17
to Lift
I have a working prototype of typeclass and macro-based API, if anyone interested.
Basically, this:
and this
https://github.com/kotonoha/server/blob/master/lift-macros/src/main/scala/ws/kotonoha/lift/json/SerializationMacros.scala
https://github.com/kotonoha/server/blob/master/lift-macros/src/main/scala/ws/kotonoha/lift/json/SerializationMacros.scala
However, it is incompatible with current API.
I am using quasiquotes only, so port to scala.meta will not be difficult at all.
Reply all
Reply to author
Forward
0 new messages