A more concrete question about API design

[Mark Hamburg]

I'm going to try to take the large app design questions and focus them on a more narrow and admittedly contrived example.

Say the people doing the client coding needed to be able to take a URL string and fetch a string via HTTP. (Yes, this is covered in the HTTL module. Bear with me. I'm trying to keep the example simple.) Dealing with tasks all over the place muddies up the client architecture that would otherwise focus on commands for external operations. So, we define:

getStringCommand :

    (Http.Error -> msg)

    -> (String -> msg)

    -> String

    -> Cmd msg

This is easy to write given the standard libraries.

But now it turns out we would like to execute these one at a time. We might generate any number of these commands during a single update call, but the mechanics of their execution demand that we not start the HTTP fetch for one until the HTTP fetch for the previous has finished. (I said it was contrived. Maybe we want to automatically fail subsequent commands if the first one fails.)

From what I understand of effect managers, we could write an effect manager to do this but the documentation around effect managers discourages reaching for them as a solution. They are identified as being for library writers and though this serialized string-fetcher seems a bit like a library in its usage, it also feels like a chunk of general app functionality. Or maybe the backend needs to use web sockets instead of HTTP and we would like to use the web sockets effects manager as part of the implementation.

One way to address this is to replace commands with requests, recognize string fetch requests when we reach a certain point in the model hierarchy, and process them accordingly generating commands as we move up the rest of the hierarchy. This has been covered in previous posts to the discussion list. The downside to this is that it doesn't interoperate well with code that wants to speak in terms of commands. One nice thing about effects managers is that the addressing of a command to a particular effect manager is essentially unseen by everything that handles it until we get to the app runner. Having lots of code need to switch from returning commands to returning requests is a very visible consequence of using this service that speaks via requests.

Another way to handle this is by changing update functions so that they still speak commands, but they now have a signature like:

update : Msg -> Model -> (Model, Cmd (Wrapped Msg))

We can then watch for wrapped commands and somehow unwrap the ones that really are looking for work by the sequencer code. That said, I'm waving my hands somewhat fast here and while we now continue to use commands, we don't use them in the way we're used to so I don't know that it's a big win over the requests approach.

Is there a better way to do this that I'm not seeing? The example is contrived but so are most examples. It feels like it gets at the sort of problem for which there ought to be a design pattern — i.e., structure your types and functions like this to solve this sort of problem.

Mark

[Nick H]

We might generate any number of these commands during a single update call, but the mechanics of their execution demand that we not start the HTTP fetch for one until the HTTP fetch for the previous has finished.

One solution that comes to mind is adding a command queue to your model. Something along these lines:

type alias Model =

  { pendingFetches : List (Cmd msg) }

update action model =

  case action of

    HTTPResponse value ->

      let

        newModel = processResponse value model

      in

        case newModel.pendingFetches of

          head :: tail ->

            ( { newModel | pendingFetches = tail }, head )

          [] ->

            ( newModel, Cmd.none )

--
You received this message because you are subscribed to the Google Groups "Elm Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elm-discuss+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mark Hamburg]

Yes, a command queue is the obvious implementation for what I identified as a contrived example. The core problem, however, is how items get into the command queue through the normal command routing mechanisms.

Mark

To unsubscribe from this group and stop receiving emails from it, send an email to elm-discuss...@googlegroups.com.

[Nick H]

OK, here I am going for the obvious implementation again. Sorry the formatting is a little nutty.

type Action

    = HTTPResponse String

| SomethingElse

type alias Model =

  { pendingFetches : List (Cmd msg) 

  , waitingForResponse : Bool

  }

  

doAThing : Model -> List (Cmd Action)

  

  

update : Action -> Model -> (Model, Cmd Action)

update action model =

  case action of

    SomethingElse ->

 case (model.waitingForResponse, doAThing model) of

 (True, newFetches) ->

   ( { model | pendingFetches = model.pendingFetches ++ newFetches }

, Cmd.none )

     (False, head :: tail) ->

   ( { model 

   | pendingFetches = model.pendingFetches ++ tail 

   , waitingForResponse = True

}

, head )

 (False, []) ->

   ( model, Cmd.none )

 

    HTTPResponse value ->

      let

        newModel = processResponse value model

      in

        case newModel.pendingFetches of

          head :: tail ->

            ( { newModel | pendingFetches = tail }, head )

          [] ->

            ( { newModel | waitingForResponse = False }, Cmd.none )

[Richard Feldman]

I'm going to try to take the large app design questions and focus them on a more narrow and admittedly contrived example.

From what I understand of effect managers, we could write an effect manager to do this but the documentation around effect managers discourages reaching for them as a solution.

It feels like it gets at the sort of problem for which there ought to be a design pattern — i.e., structure your types and functions like this to solve this sort of problem.

You seem very determined to invent a problem to solve, and then to ask the community for assistance solving it. My repeatedly pointing this out in other threads has obviously not dissuaded you.

Maybe that's fun for you, but I would hate to see a beginner read this post and think "maybe I need custom effect managers to write a scalable Elm application," which is insane.

Anyway, carry on - I just want to make it clear to anyone reading this thread that this is just the OP creating puzzles to solve. If anyone reading this is wondering whether it applies to you, don't worry: it doesn't.

[Mark Hamburg]

As I said, I know how to write the queue if that's what I really want.

My problem is that when a module uses the WebSocket.send function to create a command, I don't have to do anything special in my program to arrange for that command to make its way to the web socket effects manager. I just need to route it to my top level update function up through however many layers of the Elm architecture my app uses. When I want to use my HTTP fetch sequencer (again a contrived example that I had hoped would be simple enough not to get buried in discussions of how to build the sequencer), I now need knowledge of the sequencer to flow through the program in a way that effects managers avoid. I'm looking for a pattern that allows me to construct service APIs using the same sort of conventions used by effects managers and with the same sort of ability to not muck up the code with, for example, needing to change lots of uses of Cmd.map tagger to something more like Cmd.map (Wrapped.map tagger).

Mark

[Mark Hamburg]

I've been determined to try to use Elm as a solution for building potentially large applications. Maybe I should just be taking what you are saying as an indication that it's not useful for that and I should go back to looking at JavaScript. I'd hate to do that because I like the typed, pure functional approach but you seem insistent that the right approach to Elm is to write relatively monolithic pieces of code and then try to tease them apart if they become too complex. People create layer cake designs because they help manage systems complexity. What I'm hearing here is that Elm should only be considered for the very topmost layer. If it can't reach deeper, then it needs to compete with the technologies that sit below it because those will try to push upward.

My example here was contrived in an effort to keep things focused on the design issue. You dismiss this as inventing a problem to solve.

The problem I'm trying to solve is answering the question of whether Elm can be useful in building a large, complex application and if so how one should go about doing so since OO design patterns generally don't apply. If you want to tell me that Elm isn't useful for that and it's really just for hacking out a more reliable front end with the back end all coded in something else then I will take that advice seriously and start looking elsewhere for useful technologies.

Mark

--

[Nick H]

I now need knowledge of the sequencer to flow through the program in a way that effects managers avoid.

I disagree with this statement. I was hoping that by iterating through this solution, we would eventually come to an agreement re: the claim quoted above. But you are annoyed at me for treating your problem seriously, and you are annoyed at Richard for dismissing your problem, so I don't really know where to go from here.

[Richard Feldman]

you seem insistent that the right approach to Elm is to write relatively monolithic pieces of code and then try to tease them apart if they become too complex.

Yes, exactly!

Considering this is the most consistently successful approach to scaling in the entire history of software, I feel very comfortable endorsing it. ;)

[Josh Adams]

On Friday, August 26, 2016 at 1:24:15 PM UTC-5, Mark Hamburg wrote:

People create layer cake designs because they help manage systems complexity. What I'm hearing here is that Elm should only be considered for the very topmost layer. If it can't reach deeper, then it needs to compete with the technologies that sit below it because those will try to push upward.

People come to elm with designs that they relied on in languages/systems with entirely different semantics.  The core problem people keep bringing up here stems I think from a desire to take a not-fit-for-purpose 'architecture' and make Elm apps work that way.

You can!  There's some plumbing involved!  You can write functions to help with that plumbing or send out Cmd with tasks!  I don't recommend it mostly.

I've seen people with 5-deep nested trees of components that I would write flat.  It's a lot less code to do it that way.  The compiler helps manage the complexity.  Introducing the layers in these cases increases complexity.

--- THAT PART IS OVER, HERE'S ANOTHER THING

OK, so I do use component hierarchies in some elm applications.  I think if you're doing that, it's actually a good thing for each layer to handle the 'outbound things' and potentially transform them on the way up the chain.  If you don't need to do this, I think you probably don't need the hierarchy you built.  As an example, I have a Chat component.  It doesn't know that chat happens on websockets.  It sends out a `Maybe OutMsg` in response to some updates that is of type `Say String`.  It just says a thing.  The parent knows "aha!  The way I say things for him is to call this function on the websocket!  And since I set a 'callback' on this component I know that his outbound message should be wrapped with "sometopic"!"  If I changed the chat app to use longpolling or something, the Chat component doesn't need to change - just the piece that manages communication.  All the Chat component knows is it wanted to say something.

If you let children output parent messages in some way, rather than their own semantics, you make designs that are more rigid that they should be.

- Josh, just adding his random and probably amateur architectural ramblings to any thread he can.

[Mark Hamburg]

I'm not annoyed at you, Nick. I'm sorry if I came across that way. The queue logic is attractive because it's straightforward but it's also contrived so seeing the discussion veer toward how to write the queue feels like looking for ones keys under the streetlight because that's where the light is. I'm trying to focus the discussion on to the problem of how to generate a serialized command in one place, have it end up the queue, and then have the result get delivered back to the point where we tagged the command result to go.

I will admit that your more detailed queue code caused me to miss your generation of new commands in doAThing — which could readily generalize into being a call to an update function for an inner model — but it has the problem that then everything gets shoved into the queue. What if we only want to serialize some of the commands? For example, maybe what we're really running is a priority queue for HTTP fetches (a less contrived example) but we need commands that are trying to obtain the window size to run without queueing.

And to make matters worse, if we keep with the standard Elm architecture constructs, we get back a single command from an update and this is likely a batch command and we have no way to break that down into smaller commands. That may tell us that we need to route the commands on up to the app runtime and then have the serialized commands somehow send their payload back down without execution to get put into the queue. What are the type signatures that make that work particularly together with full compatibility with the tagging patterns in the Elm architecture?

Following the doAThing pattern, we could require the update function for an inner model making use of the queueing service to return not a command but rather a list of Queued/NotQueued commands:

type QNQ msg

    = Queued (Cmd msg)

    | NotQueued (Cmd msg)

type alias InnerUpdate innerModel innerMsg =

    innerMsg -> innerModel -> (innerModel, List (QNQ innerMsg))

Given that type signature, then we can readily figure out what needs to be delayed through the queue and what doesn't. That, however, moves away from the traditional Elm architecture code in the way update functions are written and it could run afoul of usage of Cmd.batch unless that was strongly discouraged in the codebase.

We could probably emulate the whole of the command architecture adding batch, none, and map functionality to QNQ. This would allow the code to look much the same as before but would force the widespread replacement of Cmd with our new QNQCmd. That's feasible but what if we have more than one of these sorts of services to support?

So, again, sorry for slamming your work on the queue. It's just that that's the part that is entirely contrived and hence is basically a matter of searching for a problem. A priority queue would have been less contrived but would also be more code and would have been an even bigger distraction. But the real meat here is the question of what it does to the general code structure to support this type of functionality.

Mark

[Mark Hamburg]

Thanks, Josh, That seems to confirm my impression that the answer may be "don't use commands for subcomponents". Use command-like things in similar patterns, but don't use commands. And I'd be fine with that if that were generally embraced. But then we also wouldn't need Cmd.map since commands would only exist at the top level of any application, so the Elm architecture seems to be built with an expectation of nesting.

Mark

--

[Nick H]

We could probably emulate the whole of the command architecture adding batch, none, and map functionality to QNQ. This would allow the code to look much the same as before but would force the widespread replacement of Cmd with our new QNQCmd. That's feasible but what if we have more than one of these sorts of services to support?

I think it's just as valid to ask, "what if we initially thought we would have to support more than one of these sorts of services, but in the end it turned out we didn't?" If that happens, all the effort that you put into architecting and building a more complicated system was wasted.

Worrying about scaling is akin to worrying about performance. In both cases, it's tempting to optimize as soon as you think you know your requirements. The reasoning is that if you solve your problems before they arise, development is smoother. This is a delusion. What actually ends up happening is:

• You misidentify the problems you are going to run into.
• You spend time solving problems that you won't run into.
• You over-engineered architecture slows down work on other things.
• You still run into scaling and performance problems that you didn't predict.
  

This is why we keep saying not to deal with scaling/complexity until it actually appears. That doesn't mean ignoring it until it becomes too big too ignore. It just means waiting until the "last responsible moment."

[Nick H]

I guess the way I've been building my main project, I do end up emulating the command architecture further down. It works pretty well so far! I am not using the Cmd type, but I think that is a good thing, because my internal APIs have nothing to do with Elm's platform API.

If you used the platform types internally at every level of your architecture, and then 0.18 came along and changed the platform API, all of your other stuff will break for no reason. ( I don't think this is actually going to happen with 0.18, but it did happen in 0.17! ).

So I agree that Cmd itself is not suitable for this more complex version of the queue problem. But that's fine. If you feel that the way Elm is being presented, that it is encouraging (or pressuring?) us to solve every problem with commands, then maybe we need to change the text of the guide/documentation/website.

[Richard Feldman]

Totally agree with Nick's advice. Well said! :)

[Alessandro Mencarini]

On Friday, August 26, 2016 at 9:38:23 PM UTC+1, Nick H wrote:

So I agree that Cmd itself is not suitable for this more complex version of the queue problem. But that's fine. If you feel that the way Elm is being presented, that it is encouraging (or pressuring?) us to solve every problem with commands, then maybe we need to change the text of the guide/documentation/website.

 

Newbie's opinion here, hopefully this can steer the discussion in a helpful way. 

In Elm tutorial you can read this:

In Elm, commands (Cmd) are how we tell the runtime to execute things that involve side effects. 

This caused me to look into Cmd as a first reaction when trying to build something where a child had to communicate state changes through websockets (only present in the parent). 

I hit a rubber wall and only found a good way of handing the situation when Josh pointed me to the "OutMsg pattern" he used.

As a newbie, I struggle to grasp whether I should tap into the Cmd module, and if so, how to do that.

[Erik Lott]

As a newbie, I struggle to grasp whether I should tap into the Cmd module, and if so, how to do that.

If by "tap into the Cmd module" you mean messing around with the Native Cmd code, or creating your own effects manager, than no, don't reach for that - it's not necessary to solve your problem. Without knowing more about your app architecture, my first thought would be this: if you really need access to the websocket in both the parent and the child (and there can be situations where you might), extract the websocket plumbing into a module, and include it in these other 2 modules (your parent and child).

The thing to watch out for is storing your main business/domain records is multiple places throughout the app - if you catch yourself doing this, you should stop and take a close look at your architecture.