Constructor function for extensible records / Json.Decoding of object hierarchies

[Gábor Varga]

Hello,

Let's say I have the following sort of type definitions:

type alias EntityBase =

    { id: Int

    , name : String

    }

-- Derived types

    

type alias PersonSpecfic entityBase =

    { entityBase 

        | age: Int

        , address : String

    }

type alias Person = PersonSpecfic EntityBase

Are there any constructor functions for the types Person / PersonSpecfic in the current version of Elm (0.16)?

(the compiler says "Cannot find variable `PersonSpecfic`")

It is relevant to have a constructor function to be able to create Json decoders for the type hierarchy.

Thanks,

Gabor

[Gábor Varga]

I also posted it to StackOverflow: http://stackoverflow.com/questions/35864401/constructor-function-for-elms-extensible-records-json-decoding-of-object-hier

[Janis Voigtländer]

No, there aren't. There were before 0.16.

--
You received this message because you are subscribed to the Google Groups "Elm Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elm-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Gábor Varga]

Thanks Janis.

Do you know of any best practices to Decode Json into such structures?

Right now I wrote my own constructor functions for the final record types I need to decode. 

Since they are taking 10-15 arguments and seem to be rather redundant, I was wondering if there is anything more elegant than that?

[Janis Voigtländer]

I'm afraid I don't have a better idea than writing one's own constructor functions (named or anonymous/inline).

[Maxime Dantec]

You can have a look to https://github.com/circuithub/elm-json-extra/blob/master/src/Json/Decode/Extra.elm

There is an apply function that allows you to build composable decoders, not limited to 8 fields.

[Janis Voigtländer]

This doesn’t address his problem at all. He is not looking for a way to handle more than eight fields, he is looking for a way to write down the “combination function” (like Location in the library’s example for (|:)) in the case where it’s not just a constructor for a non-extensible record type, but for an extensible one.

​

[Maxime Dantec]

"Since they are taking 10-15 arguments and seem to be rather redundant, I was wondering if there is anything more elegant than that?"

I answered that part.

[Janis Voigtländer]

No, you didn’t. He is looking for something more elegant than having to write his own constructor functions (with 10-15 arguments). Using the apply function from elm-json-extra doesn’t provide this. If you think it does, can you please give an example?

​

[Gábor Varga]

In that context "they" refer to the constructor functions that I have to write manually. I know about Json.Decode.Extra package and I use it. However, as Janis succinctly summarized, I have to have a constructor function which takes as arguments the fields of the record and produces the record itself. 

Such function is provided for non-extensible records, but apparently has to be written manually for extensible records. 

As a result of that I have to describe my API objects 3 times (from slightly different aspects):

1. the record types

2. the constructor function for the record types

3. the Json decoders

[Tiziano Santoro]

Not a direct answer to your question, but I also encountered similar issues, and ended up creating a protobuf generator for Elm (which happens to be written in Go): https://github.com/tiziano88/elm-protobuf . It generates types as well as JSON decoder/encoders, and I am also looking at adding support for Focus. It is still in early stages (API is not stable yet, and features are missing), but feel free to try it out and let me know if you find it useful.

Thanks,

Tiziano

[Evan Czaplicki]

Think about record extension vs. record nesting. My personal opinion is that nesting is always better, though I'm totally open to finding counter-examples. I just haven't yet.

You want to use extension. That's a valid approach. Say you instead try to represent it with record nesting though.

type alias Base =

    { id: Int

    , name : String

    }

type alias Specfics =

    { age: Int

    , address : String

    }

type alias Person =

    { base : Base

    , specifics : Specifics

    }

Seems like a fine way to represent things as well. This also has the benefit that you can write a decoder for the individual parts in a much clearer way. So far I don't see anything in this example that makes this a prohibitive solution.

[Gábor Varga]

Hi Evan,

Thanks for sharing your view on this.

I will keep it in mind.

Seeing Janis' thread on elm-dev (https://groups.google.com/d/msg/elm-dev/CPHvycRZyvI/ZqublhO-CAAJ), and reading your opinion inspired me to summarize my thoughts on the matter too.

First, my short response: I don't mind your approach, as long as the compiler takes care of it.

The longer version:

I would rather use nesting when the data I represent is inherently structured.

I prefer to use extension when there is a classification of things that we describe.

The most important reason is very practical and simple. Reading field names such as

technician.base.base.timeStatistics.avg

technician.base.base.timeStatistics.max 

would bother me:

1. it is too long and diverts my attention from the important information in the statement
2. is it 1 base or 2 bases or 3 bases that I need to put? I would forget,
3. it really is the technician that has timeStatistics not its abstract base -- it just happens to have other specific attributes that make only sense for technicians.

Mapping the object model/hierarchy produced by our (object oriented) backend just still feels more natural to do with record extensions. It does not feel natural with nesting.

Now, of course I very much like the simplicity of construction and destruction that nesting offers compared to extension.

1. I construct the "extended record" by constructing it's base and constructing it's extension separately -- and then I put them together. This is exactly how I want it!
2. Deconstruction: I can just take the base (or the specifics) to apply any functions which needs only that -- this is great too.

Back to my short response though, I think this could really be some very useful syntactic sugar. There needs to be only one restriction: an extension could not override fields in base records. (In my experience using covariant properties in object hierarchies is not too common, so I believe this should not be a major limitation.)

With that restriction, I believe the code for extendable records could compile to the exact same structures you suggested using nesting. But I have absolutely no experience with compilers, so...

To illustrate the above, please take a look at the following code snippet. I used very simplified models from our remote support solution, but I tried to keep it focused to the point.

-- Types

type alias EntityBase =

    { id : Int

    , name : String

    , closedSessions : Int

    , handlingTime : TimeStatistics

    , waitTime : TimeStatistics

    }

    

type alias TechnicianSpecific entityBase =

    { entityBase 

        | activePrivateSessions : Int

        , status : String

    }

type alias Technician = TechnicianSpecific EntityBase

type alias ChannelSpecific entityBase =

    { entityBase

        | activeSessions : Int

    }

type alias Channel = ChannelSpecific EntityBase

type EntityModel

    = TechnicianModel Technician   

    | ChannelModel 

    

    

-- Proposed destructors

justBase : EntityModel -> EntityBase

justBase entityModel =

    case entityModel of

        TechnicianModel (Technician _ entityBase) -> entityBase

        ChannelModel (Channel _ entityBase) -> entityBase

someValueFromSpecifics : EntityModel -> Int

someValueFromSpecifics entityModel =

    case entityModel of

        TechnicianModel (Technician techSpecifics _) -> techSpecifics.activePrivateSessions

        ChannelModel (Channel channelSpecifics _) -> channelSpecifics.activeSessions

        

-- Proposed ways of cunstruction

-- Simple

base =

    { id = 123

    , name = "Michael Simons"

    , closedSessions = 999999

    -- ....

    }

    

techSpecifics =

    { activePrivateSessions = 10

    , status = "online"

    }

mikeTheTechnician : Technician    

mikeTheTechnician =

    Technician base techSpecifics

mikeTheTechnician2 : Technician    

mikeTheTechnician2 =

    TechnicianSpecific 10 "online" base       

        

-- Json decoders

entityBase : Decoder EntityBase

entityBase =

    object5 EntityBase 

        ("id" := int)

        ("name" := string)

        ("closedSessions" := int)

        ("handlingTime" := timeStatistics)

        ("waitTime" := timeStatistics)

technicianSpecific : Decoder a -> Decoder (TechnicianSpecific a)        

technicianSpecific =

    object3 TechnicianSpecific

        ("activePrivateSessions" := int)

        ("status" := string)

technician : Decoder Technician

technician =

    technicianSpecific entityBase

    

        

As I mentioned before I do not have much experience with compilers or designing languages. Nevertheless, I hope the above opinion can be a valuable addition in designing on the future features of the record system in Elm. 

All the best,

Gabor Varga

[Kamil Durkiewicz]

Hi guys,

I've got another case when nesting is worse than extension. It comes from our current project. We're building a client application for an existing backend (which we cannot change). The backend returns large and nested documents (dozens of fields on each level) for GET requests and accepts documents of the same structure in POST requests when you want to do an update. In such case, the client application need to use the same document structure, otherwise it would be really confusing and error-prone at request construction level.

[Magnus Rundberget]

I'm not sure I buy that as a convincing argument for not using nesting.

I'm inferring (sorry if that's a strawman, but your post is a bit scarce in terms of details)  that you are saying that the view model(s) in elm must map 1-1 with the backend requests ? That to me is a bit like saying your backend model needs to map 1-1 with your database model isn't it ? I think it's to be expected to have to do mapping between abstraction layers and even more so between process boundaries.
Something like graphql might make that less the case for web apps, but I still believe there will be some level of mapping regardless.

I can't think of a single time I found me wanting to mirror the structure of my ui model  exactly like the structure of a large nested request, modify it and then send back the modified stuff to the backend

Maybe you could elaborate a little bit about your use case, to help me understand why the ui would/could be structured like the document structure from your backend documents ?


Just to be clear I'm not entirely convinced nesting is always the best solution (especially when Elm seems to favor making it difficult/cumbersome to perform nested record updates). However I haven't come up with a great example where record extension(inheritance in my still highly oo influnced brain) would clearly be better than nesting (aka composition?)

cheers
- magnus

[Leo Zhang]

Sorry to necropost, but this problem bit me recently. Here's my use case.

I'm working on an client that displays albums, static images, and animated images. The backend (which is third-party, so I can't change it) sends has lots of redundancy in the data models it sends me, so my models look something like this:

AnimatedImage is a superset of StaticImage is a superset of GalleryItem; Album is a superset of GalleryItem.

The way I've got this worked out using extension is:

type alias StaticImage = {

  -- static image fields here

}

type alias MakeAnimated staticImage = {

  staticImage |

    -- animated image fields here

}

type alias AnimatedImage = MakeAnimated StaticImage

Unfortunately, I can't figure out how to do JSON decoding this way because there is no type constructor for AnimatedImage. I'm working around this by nesting instead (i.e. having AnimatedImage contain a StaticImage field which contains a GalleryItem field), but this quickly gets unwieldy because working with AnimatedImages still requires me to access GalleryItem properties (e.g. all gallery items have a "url" field) a lot and nesting makes my code more noisy without adding semantic meaning.

[Aaron Strick]

I'd like to revisit @jvoigtlaender 's suggestion here (https://github.com/elm-lang/elm-compiler/issues/1308#issuecomment-194703200) about re-introducing constructors for extensible records only.

I bumped into a case where I believe just including constructors would have helped in achieving two of elm's core ideas, good errors and making impossible states impossible. It happens in decoding:

Imagine a type:

```

type alias Person =

  { id : Int

  , name : String

  , address : Address

  }

```

When we're decoding, we have nice insurance that all Person types have an address. But the world is complicated, and sometimes Person's are homeless and don't have an address :(.

We could represent this with Maybe Address, and decode a nil value from the backend as a Nothing - thus representing homeless as `address = Nothing`. But this isn't good because if the backed data is mistakenly nil, we have no way to raise an error to validate that the data is correct.

We could add an additional field `{ hasHome : Bool }`. During decoding, we could use `Result.map` to write a validator to ensure that when `hasHome = True`, we decode to Err if address is nil value. But this opens us up to the problem of having an invalid state: `hasHome == True && address == Nothing`.

Another solution, one which developers it seems tend towards, is using extensible types to represent the states:

```

type alias PersonBase a =

  { a

    | id : Int

    , name : Int

  }

type alias Person =

  PersonBase { address : Address }

type alias HomelessPerson =

  PersonBase {}

```

This enables us to give raise decoding errors, and to make impossible states impossible. It becomes unwieldy very fast to start creating the constructors for when the records grow in size.

Since this is something I ran into, I thought I'd bring it up, but also reframe the conversation in terms of the best way to achieve the design goals we strive towards.

[Witold Szczerba]

You said you cannot write a decoder because there is no constructor for:

type alias AnimatedImage = MakeAnimated StaticImage

This is true, but you still can create your own function. I know that would be a pain to keep that function in sync with those aliases (when they change) though.

Regards,

Witold Szczerba

--

You received this message because you are subscribed to the Google Groups "Elm Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elm-discuss+unsubscribe@googlegroups.com.

[Rupert Smith]

On Tuesday, February 28, 2017 at 1:24:32 AM UTC, Aaron Strick wrote:

Another solution, one which developers it seems tend towards, is using extensible types to represent the states:

```

type alias PersonBase a =

  { a

    | id : Int

    , name : Int

  }

type alias Person =

  PersonBase { address : Address }

type alias HomelessPerson =

  PersonBase {}

```

This enables us to give raise decoding errors, and to make impossible states impossible. It becomes unwieldy very fast to start creating the constructors for when the records grow in size.

I think trying to represent sub-types using extensible records in Elm is a mistake and is going to lead to all sorts of other difficulties in your code. I explored using extensible records for this purpose in this thread, and basically came to the conclusion that as elm does not support sub-typing (and cannot without losing full type inference), they should be avoided:

https://groups.google.com/forum/#!topic/elm-discuss/NVoWA0ZYZNM 

How about decoding to this data structure:

type Person = 

   Housed { id: Int, name: Int, address: Address }

 | Homeless { id: Int, name: Int }

or

type alias Named = { id: Int, name: Int }

type Person = 

   Housed Named Address

 | Homeless Named

?