Composability without extensible records

[Charles-Edouard Cady]

Greetings!

I originally posted this on https://github.com/elm-lang/elm-compiler/issues/985 which, as Richard Feldman pointed out, is probably not the best place to put it. I'll follow Richard's suggestion & be a bit less abstract.

I'm building an application to help ship captains create routes. My question is about the organization of the model of this application.

The application currently has one page containing three widgets:

1. A map showing the routes
2. A table showing the performances (eg. fuel consumption, duration...) of each route
3. A profile widget showing the speed profile on the selected route

It is structured in three layers of decreasing size (and increasing versatility):

1. The top layer, providing synchronization between the widgets and the layout of the page
2. The widget layer (all widgets are independent)
3. The elementary widget layer (containing sliders, date pickers...) ie. reusable components

The reusable components have their own model, which is independent from the rest of the application: their model is initialized by their containing widget.
The widgets, however, must share some information (eg. the list of routes) but not all (for example, the profile widget couldn't care less which route is hovered in the table widget, but the map widget does) because it complicates refactoring (changing the model one widget impacts the others) and makes it difficult to reuse widgets (eg. use the profile widget in a context where I do not need/have the fuel consumption).

Now in Elm's architecture tutorial, there are two extreme cases:

• The same model is shared by all widgets
• Each widget has its own independent model

I find myself somewhere in between: part of the model is shared by all widgets and is application-independent (eg. the waypoints of each route), part of it is shared only by two widgets and mostly concerns the layout (eg. route hovering) and part of it is not shared (and should not be) (eg. the sliders' state).

At the top-level (Page):

type alias Model a =
  { a
  | routes : List Route
  , hovered : Maybe Int
  , selected : Maybe Int
  , map : Map.InternalModel
  , table : Table.InternalModel
  , profile : Profile.InternalModel
  }

The Profile widget might use a part of this model:

type alias Model a =
  { a
  | route : List Route
  , selected : Maybe Int
  , profile : InternalModel
  }

while the Table widget uses another:

type alias Model a =
  { a
  | route : List Route
  , selected : Maybe Int
  , hovered : Maybe Int
  , table : InternalModel
  }

Just like @rgrempel in https://github.com/elm-lang/elm-compiler/issues/985, I want each module also provides its init function to initialize its part of Page's model. With extensible records, I could simply do (eg. in Table):

init : a -> Model a
init foo =
  {foo | table = initInternal}

and for Page I would have the very clean and composible chain:

init : Model
init =
  {routes = []}
  |> Table.init
  |> Profile.init

So the extensible records were an easy way for me to build composable applications. With the removal of this feature, the init function can no longer be type-parametrized. This is really important so let me emphasize a bit: no extensible records means init must know the full record it operates on.

So I decided to use @rgrempel's strategy, but in his case where all parts of his model were independent. The only workaround I found is to do the following for Page (top-level):

type alias Model a =
  { a
  | shared : Shared.Shared
  , map : Map.InternalModel
  , table : Table.InternalModel
  , profile : Profile.InternalModel
  }

init : Model
init =
  { shared = Shared.init
  , map = Map.init
  , table = Table.init
  , profile = Profile.init
  }


When you loose extensible records you have to put all shared parts in a Shared record, which essentially means you know in advance how your widget will be used.  For instance, the data shared by the Profile and the Table widgets is not the same as that used by the Profile and the Map widgets and if I add another widget, chances are I'll have to modify the Shared record. This makes me sad because it breaks separation of concern. With extensible records, I could simply add the fields I need to Page's model & in the specific widgets & they would simply be ignored by the other widgets. Whenever I modify what is shared, I'm modifying the Shared record that all widgets depend on &o if I decide to use eg. the Profile widget in another application, it will quickly become unmanageable.

As previously stated, as soon as you define an init function, the record it returns (or operates on) can no longer be type-parametrized (i.e. extensible), which means that if init returns the shared part, Shared is no longer extensible. If Shared is not extensible & you want to include Page in a bigger application, Page's model will have to be completely independent from the other widgets' model at the same level, ie. it will not be able to share part of its model with the other widgets.

With extensible records, you could apply the same pattern to any number of levels, but as soon as you put shared data in a shared field you're basically stating once and for all what is shared by all possible widgets: that information is only needed at the application level (at the Page level), but it dribbles down to all widgets (which shouldn't care whether they're being used in isolation or not).

Sorry to ramble on about this, but it's been a thorn in my side for a long time now.

I would really appreciate any thoughts on this.

[Janis Voigtländer]

Is it correct that your issue could be addressed by not bringing back arbitrary record extension in expressions via the pre-0.16 syntax { ... | ... = ... }, but only bringing back constructor functions for extensible records? That is, if from the table at https://github.com/elm-lang/elm-platform/blob/master/upgrade-docs/0.16.md#updating-syntax only language change in the row “record constructors that add fields” were reverted, but not the language changes in the other rows?

Also relevant in this context, then: https://github.com/elm-lang/elm-compiler/issues/1308.

--
You received this message because you are subscribed to the Google Groups "Elm Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elm-discuss+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Charles-Edouard Cady]

I'm not sure: I don't think I'll be able to define an init function like the following if I don't also have the "field addition" changes reverted:

init : a -> Model a
init old =
  { old | newField = 4 }

To unsubscribe from this group and stop receiving emails from it, send an email to elm-discuss...@googlegroups.com.

[Janis Voigtländer]

You should be able to do that.

If you have

type alias Model a =

   { a | newField : Int }

then

init : a -> Model a

init old = Model old 4

is exactly equivalent (in pre-0.16 Elm) to

init : a -> Model a
init old = { old | newField = 4 }

So where’s the problem?

​

To unsubscribe from this group and stop receiving emails from it, send an email to elm-discuss+unsubscribe@googlegroups.com.

[Janis Voigtländer]

Well, actually it would have been (swapped argument order compared to what I wrote before):

init : a -> Model a

init old = Model 4 old

But the point stands, that you would only need the row "record constructors that add fields" from https://github.com/elm-lang/elm-platform/blob/master/upgrade-docs/0.16.md#updating-syntax to be reverted.

[Dave Keen]

I've been using this pattern in a large application for a while now. The way I deal with the init issue is for each component to have two models - the main extensible model, plus it's own model (a locally scoped ComponentModel) which follows the normal nested Elm Architecture pattern. Anything in the shared model is assumed to be preinitialized by the top level Main module, and component's init functions, if they need them, return (ComponentModel, Cmd Msg), and are composed into the main model by the top level init function.

So far this pattern is serving me very well. I keep meaning to write up a blog series about it but haven't found a gap in work to do so yet!

[Charles-Edouard Cady]

Yes you're right, I hadn't seen it that way, thanks.

[Charles-Edouard Cady]

This sounds interesting: could you please give me an example?

[Richard Feldman]

 

I'm building an application to help ship captains create routes. My question is about the organization of the model of this application.

The application currently has one page containing three widgets

Sorry to ramble on about this, but it's been a thorn in my side for a long time now.

I would really appreciate any thoughts on this.

My main thought here is that I have had two kinds of experience building Elm applications:

1. Structuring my application as simply as possible - starting with one model, one view, and one msg type, not creating any "components" or distributing state management in any way - and not worrying about boilerplate or scaling problems I don't have.
2. Doing something fancier than #1.

I have had a wonderful time with #1, and it has scaled incredibly well at work.

#2 has brought me nothing but pain and unhappiness.

You are on the road to #2, so my macro-level suggestion would be to go back to #1. :)

[OvermindDL1]

On Tuesday, August 23, 2016 at 11:24:22 AM UTC-6, Richard Feldman wrote:

You are on the road to #2, so my macro-level suggestion would be to go back to #1. :)

#1 also ends up making utterly *huge* model and message unions though, with extremely non-reusable parts.  :-)

But yes, for a generic app #1 is best, however something like elm-mdl is a fantastic building of #2 that becomes almost trivial to use inside #1.  :-) 

[Richard Feldman]

On Tuesday, August 23, 2016 at 11:43:49 AM UTC-7, OvermindDL1 wrote:

On Tuesday, August 23, 2016 at 11:24:22 AM UTC-6, Richard Feldman wrote:

You are on the road to #2, so my macro-level suggestion would be to go back to #1. :)

#1 also ends up making utterly *huge* model and message unions though, with extremely non-reusable parts.  :-)

Regarding re-usable parts, I definitely recommend making things reusable on an as-needed basis. Splitting out a helper function takes about two seconds. ;)

As for big models, they are easy to maintain in Elm. I highly recommend big models!

At work we have 36,000 LoC of production Elm. One of our most complicated pages has a Model with 55 fields in it, and a Msg with 40 type constructors, and it feels nice to maintain. A year ago it was a bunch of nested React components and everyone dreaded touching it. Now we make changes to it all the time and it's no big deal.

Having that big a Model and that big a Msg sounds like it ought to be painful, but in practice, it's actually great. We don't talk about splitting it up because there's no pain point. I understand the reflex to eagerly split things up, but as counterintuitive as it sounds, I think it's the exact opposite of what leads to a good experience in Elm. :)

"If it ain't broke, don't fix it" is a good mantra here.

[OvermindDL1]

That is quite true, taking only the parts the function needs via matching out `{ parts, of, a, model }` is quite nice and extendable as needed.  :-)

[Richard Feldman]

For sure! :D

--
You received this message because you are subscribed to a topic in the Google Groups "Elm Discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elm-discuss/NPOcF4Dle2w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elm-discuss...@googlegroups.com.

[Charles-Edouard Cady]

While I understand that not breaking things up too often is sound advice in elm, I still think this doesn't answer my question: you're basically saying "you're doing it way too often" but you're not saying "don't do it at all" so I do need to find a scalable way of doing it a bit. The problem I describe is very real: I certainly don't have as much code as you but I do have about 5k lines so I'm not just describing some hypothetical problem. All I'm saying is extensibility would make my life easier and just dumping everything in a single module doesn't quite seem like the best thing to do: I did start out like that but I really do need to reuse parts of my code!

[Peter Damoc]

On Wed, Aug 24, 2016 at 12:30 AM, Richard Feldman <richard....@gmail.com> wrote:

At work we have 36,000 LoC of production Elm. One of our most complicated pages has a Model with 55 fields in it, and a Msg with 40 type constructors, and it feels nice to maintain. A year ago it was a bunch of nested React components and everyone dreaded touching it. Now we make changes to it all the time and it's no big deal.

Is there some public code that shows this pattern in action?

It would be wonderful to look at something that feels like it needs to be split up and see a version that is flat.

 

--

There is NO FATE, we are the creators.
blog: http://damoc.ro/

[Josh Adams]

Just break it into modules so your similar functions are grouped together in files.  Use them just like you already are.  We did a bit of this here: https://www.dailydrip.com/blog/elm-pair-programming-josh-adams-and-luke-westby-pairing-on-colluder

[Rex van der Spuy]

Just break it into modules so your similar functions are grouped together in files.  Use them just like you already are.  

The part I don't understand is: how do you create UI components that maintain their internal state without nesting them as child components?

[Richard Feldman]

While I understand that not breaking things up too often is sound advice in elm, I still think this doesn't answer my question

Here is concretely what I'd do:


type alias Model =
  { routes : List Route


  , hovered : Maybe Int
  , selected : Maybe Int


  , map : Map.InternalModel
  , table : Table.InternalModel
  , profile : Profile.InternalModel
  }


If Profile only needs a subset of the model, then its nicest implementation will be a standalone view function. I would not write type alias Model for profile; instead I would do this:

viewProfile : List Route -> Maybe Int -> InternalModel -> Html Msg


If you need to use it on multiple pages, i.e. its return type needs to be Html msg instead of Html msg , then give it an extra parameter so the caller can request its Msg of choice. For example, supposing profiles can set hovered (which is a Maybe Int) in response to a mouse event, I would write this:

viewProfile : (Maybe Int -> msg) -> List Route -> Maybe Int -> InternalModel -> Html msg

See my next answer for links on how to take this further iff you need to.

Just break it into modules so your similar functions are grouped together in files.  Use them just like you already are.  

The part I don't understand is: how do you create UI components that maintain their internal state without nesting them as child components?

You don't; since the Model is the single source of truth for application state, nesting is how you organize that state.

Check out elm-autocomplete and the API design session that went into it for the right way to do reusable widgets!

Is there some public code that shows this pattern in action?  

It would be wonderful to look at something that feels like it needs to be split up and see a version that is flat. 

I don't know of one offhand, but I'm realizing I should put in the effort to develop one so I can show what I mean instead of trying to describe it! :)

[Richard Feldman]

typo: should have written "i.e.its return type needs to be Html msg instead of Html Msg"

[Charles-Edouard Cady]

Thanks a lot, Richard, it makes much more sense now!

[Richard Feldman]

Awesome! Best of luck with it! <3

On Wed, Aug 24, 2016 at 1:43 PM Charles-Edouard Cady <charlesed...@gmail.com> wrote:

Thanks a lot, Richard, it makes much more sense now!

[Charles-Edouard Cady]

I have just one more tiny question: if the Profile module only has a view function, how do you handle Profile.Internal's update? More specifically, in the profile widget, each waypoint (there can be any number of them per route) has its own date picker which has an internal state (opened/closed, hovered/unhovered, etc.) and matching internal actions. Currently, Profile.update handles it which means the nitty-gritty of each component's update doesn't bubble up to the main app: how would you handle that if you only have a view function?

[Richard Feldman]

Follow up questions: What do Profile.update and Profile.Msg look like? Also, what kinds of different places do different profiles get updated?

[Charles-Edouard Cady]

The profile widget contains one slider and several date-time pickers wrapped in a WaypointConf component.
For Profile.update, I start by updating the internal model with the exposed model (possibly creating a fresh internal model if the selected route has changed):

internalUpdate : Msg -> Internal -> Internal
internalUpdate msg model =
  case msg of
    SliderMsg (Slider.ChangeValue newSpeed) ->
      { model | average_speed = newSpeed }
    WaypointConfMsg idx msg' ->
      let
        newOp : Maybe (WaypointConf.Model {})
        newOp =
          Maybe.map (WaypointConf.update msg') (Array.get idx model.waypointsConfs)
      in
        case newOp of
          Nothing ->
            model
          Just op ->
            { model | operating_points = Array.Extra.update idx (\_ -> op) model.waypointsConfs }


update : Msg -> Model a -> Model a
update msg model =
  let
    newInternal : Internal
    newInternal =
      internalUpdate msg <| toInternal model
  in
    case msg of
      SliderMsg (Slider.ChangeValue newSpeed) ->
        { model | profile = newInternal, shared = Trip.setAverageSpeed model.shared newSpeed }
      WaypointConfMsg idx msg' ->
        case msg' of
          WaypointConf.ChangeDateOrTime _ ->
            { model | profile = newInternal, shared = Trip.changeOpDate model.shared idx <| getDate idx model newInternal }
          WaypointConf.SameDateAndTime _ ->
            { model | profile = newInternal }

The Msg is a union type which contains one tag for the slider & one for the WaypointConf which contains the message itself and the index of the waypoint it applies to. It looks like this:

type Msg =
  WaypointConfMsg Int WaypointConf.Msg
  | SliderMsg Slider.Msg

The profile is updated either when the selected route changes, or when the waypoints are moved on the map widget or when the dates are changed in the profile widget or when the slider is moved on the profile widget.

[Richard Feldman]

Thanks! So let's start by talking about Slider for a sec.

      SliderMsg (Slider.ChangeValue newSpeed) ->
        { model | profile = newInternal, shared = Trip.setAverageSpeed model.shared newSpeed }

Because this code compiles, and it only covers the ChangeValue constructor, then assuming speed is an Int, I can infer that Slider's Msg looks like this:

type Msg = ChangeValue Int

A good question at this point is: how much of Slider's state must be synchronized with other state in the application?

For example, if Slider had some sort of visual flourish like "every time you drag it, it changes color," that state would not need any synchronization with other parts of the app. The slider would change color when you dragged it, and the rest of the application would neither know nor care.

The Slider's value is not independent like this. If you put a slider on a page and its value changes, but no other code ever finds out that its value changed, then the slider is useless! Its value needs to be synchronized with other state in order to be useful.

Since Slider has no other state besides this value, then Slider's state is 100% synchronized with other state. In a case like this, you have two options for keeping it in synchronized:

1. Give Slider its own Msg, and translate between its Msg and its caller's Msg

2. Do not give Slider its own Msg, just have its view accept a Msg constructor as an argument.

The second option is simpler. Here's how that would look:

viewSlider : (value -> msg) -> value -> Html msg

Now we can change Profile's Msg to this:

type Msg =
  WaypointConfMsg Int WaypointConf.Msg

  | SetSpeed Int

...and render the slider like this:

viewSlider SetSpeed currentSpeed

So now Slider no longer has its own Msg, it just says "hey caller, tell me how to wrap my values in your Msg type, whatever that might be, and we're cool." Now instead of Slider being a Model, a Msg, an update, and a view...it's just a view! :)

Let's look at Waypoint next:

WaypointConfMsg idx msg' ->
        case msg' of
          WaypointConf.ChangeDateOrTime _ ->
            { model | profile = newInternal, shared = Trip.changeOpDate model.shared idx <| getDate idx model newInternal }
          WaypointConf.SameDateAndTime _ ->
            { model | profile = newInternal }

Like Slider, its state is still 100% synchronized, meaning it also doesn't need its own Msg. We can convert Waypoint into a view function (which accepts a "here's how to wrap things as a Msg" function just like Slider did—or maybe two of those functions, since there are two things Waypoint can change), at which point we no longer need a Waypoint Msg, Model, or update. Only a view.

Finally, since Profile only has Waypoint and Slider inside it, and since Profile has no independent state to track either, we can give it exactly the same treatment as what we did for Waypoint and Slider.

To summarize how I'd recommend organizing things:

Current

WaypointConf - Model, Msg, update, view

Slider - Model, Msg, update, view

Profile - Model, Msg, update, view

App - Model, Msg, update, view

Proposed

WaypointConf - view

Slider - view

Profile - view

App - Model, Msg, update, view

Hope that helps! :)

[Mark Hamburg]

This suggests a generalization to the Elm update/view architecture pattern. The functions stay the same but the signatures change:

module Component exposing (InternalState, ExternalState, InternalMsg, init, update, view)

type InternalState = ... -- the purely internal information for the component

type ExternalState = ... -- the external value for the component (e.g., the value of a slider)

type InternalMsg = ... -- the messages sent to update the internal state

init : InternalState -- initial internal state based on an initial external state

update : InternalMsg -> InternalState -> (InternalState, Cmd InternalMsg) -- respond to internal messages

view : (ExternalState -> msg) -> (InternalMsg -> msg) -> ExternalState -> InternalState -> Html msg -- display

A further generalization would provide a standard way to allow the internal state to respond to external state changes.

The standard Elm architecture would be the special case where there was no external state. The slider example would be the case where there is no internal state. Evan's sorted table example would be a case where the internal state is simple enough that we can dispense with an update function for sorted tables and the table has no capacity to update the external state.

A lot of what these discussions around componentization seem to call for is more detail on how to stretch and contract the Elm architecture adding parameters in some cases and simplifying pieces away in others.

Mark

[Charles-Edouard Cady]

Brilliant! Thanks a lot!

To summarise what I understood:

- A component should only update its internal states

- All updates of shared states should be done at a higher level

which could be further summarised in:

A state should only be updated by its owner (i.e. the highest component where that state appears) & no state should appear twice in the component chain.

There's just a slight glitch in your previous answer: WaypointConf is *not* 100% synchronised: it needs its internal update to handle its hover state (for example, when hovering on a waypoint date date a button appears to open the corresponding date picker to modify it & the date picker has its own internal state such as hovering, open/closed etc.)

So while I do agree the slider can be reduced to a view, I don't think the WaypointConf can, which in turn means the Profile can't either.

Things get nasty when you try to synchronise shared states with internal ones, so don't do it (which, I think, is what you meant in a previous post with options 1 & 2).

A component can only operate on two kinds of states:

- states other components need to know: those are managed by a component higher up

- states others don't need to know: they should not depend on the previous ones & are managed by an internal update.

Am I on the path to enlightenment?