Fwd: JSON library prototype

[Marc Esquerrà]

Hi all,

I would like to share with you a project I've been working on: a Scala library to work with JSON documents. I know, the idea may be not the most original but I'm sure that you would find some innovative ideas on how it's implemented and on how to use it. I call it KissJson.

Feel free to download it and try it. And please... any comment or collaboration would be really, really appreciated :)

https://github.com/marcesquerra/KissJson

Since you are almost the first people I share this project with and since I'm writing this email from what I get fresh out of my mind, please excuse me if It's not as clear as It should be... or if I write too much.

Well, let's go.

First of all, if you take a look at the name I've given the library you can see that I'm huge fan of the Kiss principle (Keep it Simple) and that is exactly what I've tried to achieve implementing this.

Also, the full concept and the main design principles of the library revolve around the following four concepts:

• A Scala representation of a JSON document (JsonValue, JsonString, ...)
• A DSL to write JSON "literals" directly in the scala source code
• A set of bidirectional JsonValue convertors
• from string representation
• from scala types (base types, case classes, arrays, collections, options, ...)
• Tools to work directly with the JsonValues

JsonValue

The following types conform the core of the library:

JsonValue:  The base type of all JSON values

JsonBoolean, JsonString, JsonNumber, JsonInteger, JsonReal

JsonArray

JsonObject

JsonNull

This collection of types have been designed to replicate what Option, Some and None do in standard Scala.

Option  →  JsonValue

None  →  JsonNull

Some  →  All the others

But, with a really important difference, contrarily to what happens with Option this is perfectly valid in KissJson:

val str: JsonString = JsonNull

This allows to keep the concept of "null" in the JSON sense of it, combined with the clean versatility and safety of working with an "Option" like pattern:

val result = str.getOrElse("defaultString")

This way, JsonArray wraps a Vector of JsonValues (if asked for an invalid index it returns JsonNull) and JsonObject wraps a Map of String → JsonValue (which also, if asked for an invalid key will return JsonNull).

Another cool feature of KissJson is the way you can ask for an specific key of a JsonObject. I wanted it to be idiomatic, like with the usage of dynamic types (being able to make queries of the like of obj.company.name) and also to keep the safety provided by the compiler (detecting the error if I write "obj.tostring" instead of "obj.toString"). To achieve that, I've created the ".?." accessor so you can do this:

val name: JsonValue = obj.?.company.?.name

Literals

This is the bit I'm probably most proud of. An example is the best explanation in this case. And that the example explains by itself probably 's why I'm so proud ;)

val json = J(

    `type`     = "Sample Json Object",

     id        = 12,

     rating    = 3.7,

     content   = "characters",

     enabled   = true,

     items     = J("Spock", "Luck")

)

Conversors

A JSON library wouldn't make much sense if you can't get data from a String representation or render your data into a String, ready for sending. With KissJson the parsing works like this:

val jsonObjectString =

"""

{

"name": "John",

"surname": "Whatson"

}

"""

val jsonObject:Try[JsonValue] = jsonObjectString.asJson

And rendering back to string is done with the render method:

val msg: String = jsonValue.render

Similarly, it's really handy to be able to convert jsonValues to standar scala clases:

case class Person(name: String, age: Int)

val p = Person("John", 32)

val r: Try[JsonValue] = p.asJson

and vice-versa:

val p2 = J(name = "Jhon", age = 32)

val r2: Try[Person] = p2.as[Person]

Ok, I think this is enough for a presentation. As you can see the library is really easy to use and I think it's also quite powerful. There are several things to polish, but It has a lot of potential. I wish you find both this email and KissJson useful :)

Kind regards,

Marc Esquerrà i Bayo

http://www.bryghts.com

@marcesquerra

[Kevin Wright]

Google groups always block first posts until moderation. For some reason though, your email was also flagged as spam (perhaps because of the "fwd: "?), so I missed it.

You may now consider yourself approved for future posts :)

--
You received this message because you are subscribed to the Google Groups "London Scala User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-london...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Kevin Wright
mail: kevin....@scalatechnology.com
gtalk / msn : kev.lee...@gmail.com

quora: http://www.quora.com/Kevin-Wright

google+: http://gplus.to/thecoda

twitter: @thecoda

vibe / skype: kev.lee.wright

steam: kev_lee_wright

"My point today is that, if we wish to count lines of code, we should not regard them as "lines produced" but as "lines spent": the current conventional wisdom is so foolish as to book that count on the wrong side of the ledger" ~ Dijkstra

[Marc Esquerrà]

Thanks Kevin for your explanation,

It's always good to learn something new, even if it is that is better to know how something works before clicking "send" :) I'm sorry for all of you that might have received the email twice, it has been an accidental spamming.

While I'm at it, and thinking that my explanatory eMail may be a little bit too long, I would like to know your opinion about two ideas I'm trying in the library:

Do you think it's a good idea to mix the concepts of "null" and "None"?

Also, what do you think about the option of making a safer version of the "Dynamic" trait?

Kind regards,

Marc Esquerrà i Bayo

http://www.bryghts.com

@marcesquerra

[bryan hunt]

Nice idea, with the literals, presumably that's implemented using
dynamic classes?

[Kevin Wright]

It's always good to learn something new, even if it is that is better to know how something works before clicking "send" :) I'm sorry for all of you that might have received the email twice, it has been an accidental spamming.

Don't worry, it only went to moderators.

 

While I'm at it, and thinking that my explanatory eMail may be a little bit too long, I would like to know your opinion about two ideas I'm trying in the library:

Do you think it's a good idea to mix the concepts of "null" and "None"?

Usually not, but the rules are a bit different for DSLs.

I'll often have a method taking an argument with null as the default value then immediately wrap the thing in an Option for subsequent use.

 

Also, what do you think about the option of making a safer version of the "Dynamic" trait?

"Dynamic" doesn't have to mean "unsafe".  It can be combined with macros to allow logic past the initial typecheck before macro expansion takes place, but you still don't end up with reflection or anything else risky in the resulting bytecode.

rillit gives a nice example of this in action: https://github.com/akisaarinen/rillit/blob/master/core/src/main/scala/rillit/Lenser.scala

Also worth noting that macros are rapidly evolving (especially macro-paradise and scala 2.11) so that such tricks won't be necessary in the future.

[Marc Esquerrà i Bayo]

Thanks Bryan,

Yes, I'm using the option that allows to dynamically specify the parameter names. It's intended to be used with the "apply" method name but you can use whatever method name you want (although I plan to us a macro to allow only "apply"). The current implementation is really easy:

object J extends Dynamic

{

//...

def applyDynamicNamed(method: String)(in: (String, JsonValue)*) = JsonObject(fixNames(in :_*):_*)

//...

}

The only tricky bit is dealing with unicode. Scala allows to use anything as an identfier if you put it between back-quotes, like in:

val `My Identifier` = 3

which allows characters otherwise prohivited in identefiers (e.g. whitespaces). This is essential in KissJson to allow all possible values in the keys, but the scala compiler when it does the rewriting to create the invocation to the "applyDynamicName" method, converts this special characters to special unicode notation.  If I run this example:

object J extends Dynamic

{

def applyDynamicNamed(method: String)(in: (String, _)*) = 

in foreach {case (s, _) => println(s)}

}

J(`My Identifier` = 0)

I will get:

My$u0020Identifier

Which means that I need to replace all this $u[Unicode Point] for the correct character.

Cheers,

Marc Esquerrà i Bayo

http://www.bryghts.com

@marcesquerra

[Marc Esquerrà i Bayo]

Do you think it's a good idea to mix the concepts of "null" and "None"?

Usually not, but the rules are a bit different for DSLs.

I'll often have a method taking an argument with null as the default value then immediately wrap the thing in an Option for subsequent use.

I'm talking about mixing the ideas behaind 'null' and 'None' in a single entity, in my case 'JsonNull' that have some of the typical capabilities of null:

val b: JsonBoolean = true

val n: JsonBoolean = JsonNull

and some of the Option/None:

val r = b.getOrElse(false)

Also, what do you think about the option of making a safer version of the "Dynamic" trait?

"Dynamic" doesn't have to mean "unsafe".  It can be combined with macros to allow logic past the initial typecheck before macro expansion takes place, but you still don't end up with reflection or anything else risky in the resulting bytecode.

I'm not sure if I understand your explanation. What I mean by unsafe is the following:

One of the greatest advantages of using a non-dynamic type system is the compiler detects a lot of possible errors for you in compile time instead of the project crashing in runtime (i.e. obj.tostrin will not compile). But, if you add the dynamic trait to a class, all the non dynamic methods of the class lose that protection (the compiler can't tell if you where planning to call the "toString" method or some random dynamic method named "tostrin").

What I've thought is in creating a special accessor for the dynamic methods ( obj.?.tostrin ), leaving the standard ones as they are ( obj.toString )

Cheers,

[Jon Pretty]

Hi Marc,

I've been working on providing better support for JSON for Scala in Rapture I/O.

I've documented the JSON features of the library at

http://rapture.io/jsonSupport

It looks like we've taken similar (but not identical) approaches for
some of the features. I'd be interested in discussing this further. Is
there any chance you'll be around for a chat at either the Scala
eXchange in London on Scala I/O in Paris?

Cheers,
Jon

> --
> You received this message because you are subscribed to the Google Groups
> "London Scala User Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to scala-london...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.



--

Jon Pretty

[Marc Esquerrà i Bayo]

Hi Jon,

Thanks for your interest. It's true, it's very tempting to use Dynamic typing to access this Map like data structures :)

And yes, I would be really pleased to have a chat with you, but unfortunately I won't be able to attend neither of the two conferences. Since I live in London we can manage to meet during the eXchange anyway. But better go private for that, I'll send you an email.

Cheers,

Marc Esquerrà i Bayo

http://www.bryghts.com

@marcesquerra

[Olivier Chafik]

Hi guys,

I'm late to the party, but here's my modest contribution to the JSON macros landscape:

https://github.com/ochafik/Scalaxy/tree/master/JSON

It provides similar features as Jon's cool Rapture lib, except it goes a bit further for the string interpolation: it actually assembles and parses the JSON string fragments during compilation and macro-expands to a reified version of the objects (which allows translating JSON errors to compilation errors, and provides renormalization for free).

I also mixed Dynamic and macros to provide a json(a = 10, b = 12) syntax.

I didn't want to add a new JSON lib so I built it on top of https://github.com/json4s/json4s, but it should be easy to decouple and adapt to any sensible JSON library.

Maybe we'll meet at scala.io :-)

Cheers

--
Olivier
http://ochafik.com
http://twitter.com/ochafik

2013/9/28 Marc Esquerrà i Bayo <esqu...@gmail.com>

[Marc Esquerrà]

Hi Olivier,

Your json extractor through interpolation looks great. It's really neat.

I'm planning to enter the Macros world to improve performance and safety but I'm working on the usage first of all.

Cheers,

Marc Esquerrà i Bayo

http://www.bryghts.com

@marcesquerra

Atentament

Marc Esquerrà i Bayo

http://jawos.w3d.cc