Re-opening the Deriving Json can of worms

[Joey Eremondi]

With the release of 0.14, my Haskell->Elm translation library Haskelm will need major rewrites. One of the main features it provided was automatically driving Elm functions to convert JSON to and from Elm. This will be much easier given the new Json.Decode library, but I wanted to run a few things by the community first.

1. Is automatically deriving to/fromJSON instances something in the works for Elm? If I put a bunch of effort into writing the Haskelm derivation for these, am I going to be duplicating effort?

2. Can we develop an official specification for how Elm values can be represented as JSON? It would make sense to use the underlying format, since then converting them at runtime would be pretty trivial. Having a specification like this would also mean we could have conversion functions across multiple backends, so Scala or Clojure could use the same JSON format as Haskell. (Also, such a specification could be used to interact with Ports).

The idea is that I'd use this as the standard for writing toElmJSON and fromElmJSON typeclasses in Haskell.

3. Is Haskelm/some preprocessor the preferred way to implement deriving JSON? i.e. is this preferable to having some sort of "deriving" keyword in the language?

Either way, I'm just looking for opinions and trying to start a discussion on these issues before I do much development. There's no rush on any of this.

Thanks!

Joey

[Evan Czaplicki]

Here's the latest thinking on deriving JSON conversions. A more generalized notion is described here. There's a link in there about the canonical form of union types in JSON. I have recently been thinking about how this stuff might look in practice. A rough idea from a few days ago was sort of like this:

type alias macro

  Json name args tipe =

    { encode = ...

    , decode = ...

    }

type alias Point = { ... } deriving Json

Something like that may be plausible so that people can define type-directed macros in a reasonable way for whatever they need to do. This needs way more work, particularly around making a strong case that this would make things better overall. Macros are dangerous territory, and though I like this direction, I'd be curious to hear from web-y-ier folks how this would change their workflows.

My personal priority right now is around making a Promises API that'll take the place of the existing Http and WebSockets library, perhaps even becoming the general way to do any effectful stuff that cannot fit into Signals easily. This will start with APIs, but I'm also revisiting the command syntax though I think we don't need the concept of "commands" given how things have played out. Just call anything that has andThen "chainable" and things are pretty clear.

I should do a more concrete status report soon! I'm spending the holidays with my family in Texas, and I already feel more relaxed. Lots of Bela Fleck and the Flecktones today :P I'll try to iron out some more details in the next few days and do a more complete update!

--
You received this message because you are subscribed to the Google Groups "Elm Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elm-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[John Mayer]

Some context: there's two families of macros.

C family (and more recently Rust) are more of the templating type; they generate source code via basic  symbol substitution. C is straight up format strings, while Rust will at least check that the template is valid and sorta types the macro parameters, though still at its heart is just substitution.

Haskell (TH) is full blown code generating AST values, which are interpolated at the call site. At compile time, TH will evaluate imported code - it can even do IO.

Ok, so pretty diverse prior art. Cool.

A separate feature of Haskell is genetics, which allow you to write code that operates on a representation of the type - a value encoding the shape of a type (like the fields and types of a record, or the constructors and parameters of union types). In modern Ghc, virtually all Haskell types can do this generic stuff and get a "Typerep" for free.

===

Type directed macros (invoked with deriving syntax) could be basically the "evaluation at compile time" of a pure Elm function from a "type representation value" to a "value of Elm AST". Notice the three or four big challenges in there!

[Richard Feldman]

Some comments relevant to this discussion:

1. I haven't done anything with Http yet (in part because I'm waiting for Promises, but also because my current main Elm project has no back end by design...although if things work out I'll soon have a different main Elm project with a very, very large back end), but my intuition is that this would cut down on a ton of boilerplate. At work we currently have dynamically typed languages on both the front end and back end, so when we need to communicate via a JSON API, we will basically write out a sample JSON message to document the structure, and commit that into our repo. If we had Elm, we'd definitely replace that approach with a type alias, because type aliases have additional benefits beyond documentation. Then, if we didn't have macros, we'd write boilerplate to translate the type alias into Json encoding/decoding by hand.
2. I immediately thought "this could totally solve the boilerplate concern I was seeing on my LocalChannel refactor." I'm not sure if that's true or not (or for that matter if said boilerplate concern can be resolved using existing tools), but my intuition is that this could make records in general significantly nicer to work with.
3. I agree that macros are cans of worms. However, for me the scary part of the can of worms is the "template" variety - looking at a nested function call and not being sure what's actually happening, because any of those nested calls could be macros instead of functions. That doesn't seem to apply in this case, because it's limited to the realm of types.
4. The "egregious abstraction" concern is valid, but I suspect in this case it's worthwhile even if it enables undesirable hacks.
5. The term deriving carries no relevant intuition for non-Haskellers, which suggests there's a better term for this concept out there.
6. I once saw Bela Fleck perform with Chick Corea in St. Louis, and it was totally rad.
   

[John Mayer]

Does anybody know if any other classes of macros, other than syntax templating and arbitrary code evaluation? Maybe something in between? I thought that the lisps are supposed to have some cool stuff going on with macros.

Also typo in the list email, genetics should be generics.

[Thomas Hunger]

My personal priority right now is around making a Promises API that'll take the place of the existing Http and WebSockets library, perhaps even becoming the general way to do any effectful stuff that cannot fit into Signals easily.

Some random thoughts:

I've been trying to figure out a nice API that deals with close/error for websockets for a while. Communicating errors/close would be doable with a union type but I'm not clear how reconnecting would work. I was thinking that .connect could take a error-handler-strategy argument, e.g. a function that deals with errors.

Do you have any plans / ideas for propagating HTTP errors with `andThen`?

Maybe the unpredictable nature of errors mean that this is outside the scope of elm and it'd be better to expose a lower-level API via ports? (that's how I'm currently doing it)

~

[Joey Eremondi]

@Evan: Like I said, this is by no means a priority. Enjoy your Holidays, and don't worry about this until Promises and such are finished. I doubt I'll have a chance to start looking at Haskelm again for another couple of weeks.

The description you link seems pretty solid. I'd be pretty insistent that ADTs be derivable, since otherwise there's really no benefit over Ports.

For John and Richard, my thought is that this would work best with the Generics style, and this is also a place where limiting things might be useful. (i.e. we might not want to open the can of worms for automatically deriving anything, if JSON is the only common use case).

It's also possible that we could completely hide this under ports. That is, every class that can have "deriving JSON" automatically does (the same way it works now for Show), and things are converted automatically over Ports. It also takes away some of the naming issues. Haskell's typeclasses mean you can have a polymorphic "fromJson", and it has the type annotation syntax that allows you to specify what kind of type you get out at the end.
We could just have it where Ports are the only means to convert to and from JSON, so when you specify the type of the Port it would know what type to convert to. This also solves the issue of "deriving" not meaning much to non-Haskellers.
FromJSON is easy, since it always returns the same type.

[John Mayer]

What's more important: getting automatic json, or getting a more general feature that enables automatic json?

There's this tension between practicality (json is this ubiquitous thing, from which all apps will benefit NOW) and pausing to solve the bigger problem (which would render any more narrow short term solution obsolete, but take longer).

[Jeff Smits]

I think the current Json.Encode/Decode is a good enough practical solution to allow pausing and solving the bigger problem.

[Alexey Zlobin]

I think layering is a way to do in between here.

Racket for example has 2 layers (exposed to users under name 'macros', many other features could be employed to customize syntax or semantics of the language). The first is arbitrary code transformation for advanced macros. And the second template-based, build entirely on top of the first, to keep things simple and accessible for mortals.

Scala macros I believe has similar structure just with more levels. Can't asset for sure, I lost the view on what's happening there.