Nested JsonObjects for static typing of contained objects

[Christopher Piggott]

I've been playing around with nested JsonObjects just to see how far I could get with it.  To summarize the specific item I was trying to fix, here is a snippet from the key documentation on the subject:

abstract class Language {
  String language;
  List targets;
  Map website;
}

In the spirit of providing a static interface between my library and end users wishing to use it, I really would like to have an object in there, rather than a Map.  The end goal is more like:

abstract class Language {
  String language;
  List targets;
  Website website;
}

I've been toying around with a bunch of different ideas about how to do this, and here is what I came up with:

class LanguageImpl extends JsonObject implements Language {
  LanguageImpl();

  /* I'm not sure if final is what I really want here */
  final Website _website;

  /* If someone external asks for the website, give him the internal one */
  Website get website => _website;

  /* When JsonObject.fromJsonString tries to set the website it will
   * examine the JSON and see it as a map.  This setter picks that
   * up and creates a version that conforms to the Website interface
   */
  Website set website(Map map) => _website = new WebsiteImpl.fromMap(map);
  
  factory LanguageImpl.fromJsonString(string) {
    return new JsonObject.fromJsonString(string, new LanguageImpl());
  }
}

Website looks something like::

abstract class Website {
  String name;
  String url;
}

class WebsiteImpl extends JsonObject implements Website {
  WebsiteImpl(); 

  factory WebsiteImpl.fromMap(map) {
    return new JsonObject.fromMap(map);
  }
}

The code above is notional (not ready to share the real thing yet) so excuse any obvious typos.  The concept works, though.  In a nutshell, I use JsonObject.fromJsonString() only for the top level object; the others are built from maps.  JsonObject does most of the heavy lifting, and I end up with a strongly-typed object that I can access with normal dot notation.

All this said, I'm pretty new to dart so I'm curious if others think this is a good or terrible idea.  One thing that bothers me is that I think it would be better if it didn't map Website unless a user asked for it.  I couldn't figure out a way to be lazy about actually transforming the object.  I was hoping to hold the map, but in the JsonObject the map would be called 'website' which precludes me from having a transformed version also called Website.  Possible, but adds a lot of code (relative to what it is now) so I decided to let someone else figure that out.

Feedback welcome, I'm an experienced developer but new to dart.

--Chris

[Jim Trainor]

I have a generic json class builder implementation.  It takes a json strings parsed to a map (using the dart sdk json support), or a map instance such as that that comes from index_db, and turns that into a strongly typed Dart object hierarchy.

Classes that I want to convert from a map (e.g. from a json.parse(..) result) must register a class name, property name and type information, and a class factory with generic builder that recursively traverses a map converting embedded map instances into class instances.  That registration could be done generically someday using a dart mirror but I don't want to use mirrors today so it is done "manually" by a static method in each class called once at program init time.

I started with the same JsonObject implementation you reference as a base class. It provided excellent inspiration, however, I've since moved away from it to my own very much slimmed down and streamlined equivalent  (because I didn't need such a generic solution as my requirements became clearer, e.g. I don't need/want dot notation access to my objects).  That will probably be streamlined one step further by eliminating it totally since the requirement to extend from a common base simply to support serialization is too heavy IMO.  What happens at that point depends on what happens in Dart.  For now, things are okay.

I don't hand code most of my serializable classes. They are generated from jaxb annotated java class that are defined on my server - another reason why I don't mind the manual registration because it's not actually manual - it's automated. I have the odd wrapper around the generated classes that is only used in Dart and is hand coded.  I have some rather deep and complex objects going back and for to/from an http server and to/from an index_db cache without problems.

[Alan Knight]

It sounds like you might be interested in the serialization package, which aims to do precisely these sorts of things. By which I mean convert object graphs into some form that's easily transmittable (JSON or otherwise) and reconstruct them as objects. Somewhat similarly to what Jim describes you register a class by creating a SerializationRule, which can be implemented in various ways. The simplest is one that you just tell it the class and it constructs a rule using mirrors, but you can also provide your own hand-coded or generated rules. There are still quite a bit of work that could be done on it, including documentation, but it does work and I'm happy to provide pointers/help debug problems.

--
Consider asking HOWTO questions at Stack Overflow: http://stackoverflow.com/tags/dart
 
 

[Alan Knight]

And, replying to myself, I should add that there are pluggable formats and it sounds like what you'd want is the SimpleJsonFormat, which produces something that a client side that expects JSON maps would understand, doesn't have cycles, etc.

[Christopher Piggott]

Thanks, Alan and Jim.  I have a ways to go ... I don't understand how to make this work yet:

    Serialization s = new Serialization();

    s.addDefaultRules();

    Reader r = s.newReader(new SimpleJsonFormat());   

    MyObject obj = r.read(data);

At least for now I only need to go in the one direction (deserialize) ... but I've found the documentation and will noodle it for a while.

Thanks again for the pointer.

[Alex Tatumizer]

@Alan: any data on performance of serializer? 

E.g. take some JSON, deserialize it into object, serialize it back (so output=input), compute output in MB/sec.

[Iván Zaera Avellón]

In the json library (not JsonObject) there's an optional argument you can give to the parse() function, called reviver:

<<<

 * The optional [revivier] function, if provided, is called once for each

 * object or list property parsed. The arguments are the property name

 * ([String]) or list index ([int]), and the value is the parsed value.

 * The return value of the revivier will be used as the value of that property

 * instead the parsed value.

external parse(String json, [reviver(var key, var value)]);

>>>

I haven't tried it, but sounds like it could help you to convert your Map to a Website during the translation without having to hack getters and setters.

Unfortunately json_object doesn't allow you to specify any reviver:

<<<

factory JsonObject.fromJsonString(String _jsonString, [JsonObject t]) {

...

t._objectData = JSON.parse(t._jsonString);

...

}

>>>

I don't know if json_object could be changed (if it makes sense) in some way to take advantage of all this.

Just another thing more to investigate ;-). If you do it I'll be interested in knowing what you finally got. I'm also working with JSON objects for REST interfaces but I haven't yet arrived to the implementation of any 1-N or N-N relations, I'm just working with plain  objects with only "primitive" properties as of now.

2013/5/10 Alex Tatumizer <tatu...@gmail.com>

@Alan: any data on performance of serializer? 

E.g. take some JSON, deserialize it into object, serialize it back (so output=input), compute output in MB/sec.

[jim.trainor.kanata]

Interesting, I had not noticed the "reviver" argument on json.parse(...).  I'll may end up using that in my own solution. A  simple json deserializer solution then turns into a registry of class names and factories and property name/type info, combined with a json.parse(...) reviver impl.  The one thing that may keep me away is that the deserializer, for me, is not really just a json deserializer, ultimately it is a generic map converter hence is used in other situations where maps of maps is the return type - such as objects that come back from indexed_db (there is no "reviver" in that interface).

I wasn't aware of the serialization package when I did my impl.  My work mostly when into the Java/JAXB parser that outputs Dart classes anyways, which would not have changed.  The dart json deserialize bit is quite small if you're not looking for a generalized solution.

[Alan Knight]

None whatsoever :-) It's on my to-do list to do some measurement and tuning.

Back of the envelope it will vary a great deal depending on exactly what you're doing. If you use the default configuration without adding explicit rules, then it will use mirrors and serialization to json. I don't think mirrors have had any serious optimization work, and the json format is not very space-efficient. Also, when the serialization output is "json" it really means reduced to maps, lists, strings, numbers, etc.  You would still have to run it through stringify and parse, and I don't think that library is particularly fast yet. And SimpleJsonFormat is making an extra pass through looking for cycles. The flat format would be significantly more compact because it doesn't repeat the field names. CustomRule subclasses or other hard-coded rules that access the data more directly will probably be faster than mirrors. 

However, I'm suspicious that right now the dominant performance factor for a large serialization would be the use of IdentityMap (in pkg/serialization/lib/serialization_helpers.dart) to keep track of the objects we've already seen. It's implemented via linear search, as a performance optimization. Seriously. Object.hashCode is implemented via expandos, which in the VM are doing a linear search. That's dartbug.com/5746 , which fortunately looks like it should be addressed soon.

On Thu, May 9, 2013 at 6:54 PM, Alex Tatumizer <tatu...@gmail.com> wrote:

@Alan: any data on performance of serializer? 

E.g. take some JSON, deserialize it into object, serialize it back (so output=input), compute output in MB/sec.

[Alan Knight]

Ah. So the problem in your case is that if the JSON data comes from an external source there probably isn't any information about what classes to create from the data. Or if there is, it isn't in the form that serialization understands. Also, rules can store their data either as lists (more efficient) or maps (more readable, more likely to be what another program expects) and the default is lists. So here's a code fragment that writes and reads back, but it uses the "storeRoundTripData" flag on the format which adds an extra parameter that's meaningless to other programs but enough for this one to read it back. You can take that out, but then it can't read it back again.

It would also be possible to make a subclass of the format or a different format that knew how to interpret some other piece of information to determine the class to create. You wouldn't have identity, but you typically don't have that with straight JSON data anyway. On the other hand, you're dragging in a fair bit of code in the serialization library and not using most of it, so a simpler mechanism might work just as well and be more lightweight.

import "package:serialization/serialization.dart";

import "package:unittest/unittest.dart";

import "dart:json" as json;


class Person {

  var name, address;

}


class Address {

  var street, city;

}


main() {

  var p = new Person()..name="Alan"..address = (

      new Address()..street="123 456th St"..city="Seattle");

  var serialization = new Serialization()

      ..addRuleFor(p).configureForMaps()

      ..addRuleFor(p.address).configureForMaps();

  var format = new SimpleJsonFormat(storeRoundTripInfo: true);

  var output = serialization.write(p, format);

  print(json.stringify(output));

  var input = serialization.newReader(format).read(output);

  print(input.address.street);

}