Scaling Elm

[Peter Damoc]

Hello community, 

Scaling Elm apps seams to be a recurring topic. 

I was wondering if maybe we could negotiate a minimal set of functionality, something similar to ToDoMVC, that could be implemented using different approaches to explore what could be the best way to structure the code.

What should this minimal example cover and what this minimal example should be (topic)?

I'll start the list with some bits of functionality that I would like: 

- multiple pages with common structure (sidebar/navbar)

- navigation without reloading the app (SPA routing without the hash) 

- authentication 

- complex widget reuse (a module/widget that generates side-effects; e.g. a weather widget, some stock ticker, an ad provided by a third party)
- styling (CSS)

I would also like the example to cover real world concerns of: 

- using a build manager to integrate with other technologies 

- development mode - deployment build
- testing

As for topic, I was thinking about an interface to the MusicBrainz Database (a simplified interface).

What do you think?
What bits of functionality would you like to see exemplified?
Are you aware of any other project (in other languages) that exemplifies a minimal set of functionality and could be used as a template?  

--

There is NO FATE, we are the creators.
blog: http://damoc.ro/

[Oliver Searle-Barnes]

I think this would be fantastically useful :) I've kept out of discussions regarding architecture as they've been going in circles for the last 6 months or so. An example SPA seems like a great way to move forwards.

Something which every SPA I've ever written has included is a Store of some sort, an abstraction which mutates/queries for server side records and consolidates the results in the client. Perhaps it's my Ember.js background but from my perspective this is something that would need to be included.

[Duane Johnson]

On Wed, Apr 19, 2017 at 3:11 AM, Peter Damoc <pda...@gmail.com> wrote:

- complex widget reuse (a module/widget that generates side-effects; e.g. a weather widget, some stock ticker, an ad provided by a third party)

Selfishly, a rich text editor with custom elements would be awesome. I'm trying to figure out how to embed a non-elm widget inside elm (quilljs) and then elm views inside quilljs.

[art yerkes]

That's a pretty good list.  I'd add interacting with sound, files and focus, which require side-effect like interaction and haven't really been addressed well yet.

[Marek Fajkus]

I would like to help with this by contributing some version.

Anyway it's crucial to point out that this is really not easy thing to do. I like idea with MusicBrainz db but that doesn't include changing data on server. Also it's hard to demonstrate scaling without some noise (unrelated business logic) and the same time do not introduce over-engineering at the process anyone can use against whole idea. At least for me it's hard to imagine scope of this. I also believe this is the reason we there isn't something like this yet.

Honestly ultimate solution would be to start some actually useful OSS project which and solve some real use case. By that I mean something bigger. @Oliver mentioned he has Ember background. In ember there is Discourse and Hospital Run for instance.

Anyway now I feel I'm asking for too much and have doubts I'll be able to contribute for project like this... sadly... It is just idea that come to my mind.

[Oliver Searle-Barnes]

Now that I think of I remember that Josh Adams has been working on https://github.com/dailydrip/firestorm which is open source and has an Elm client. It's written as an SPA so it may serve as a good subject for this discussion.

[Erik Lott]

I'm a little pressed for time, but I'll try to give a general architecture outline that works really well for us. Our primary elm SPA is a customer facing administrative control panel that communicates with a backend api server and amazon S3. The app manages authentication, authorization, subscription status, and currently has 17 'pages'. This particular app is a Static SPA (currently served from s3-cloudfront combo).

Backend API Client

We have an api client written in elm that resides with the api server. The api-client elm module can be pulled into any elm project that needs to communicate with the backend server, and includes all http requests, model types, model json decoders, and any necessary model helpers. Importantly, this is where authorization and authentication is handled as well. A user who successfully signs-in via the api will be granted an authorization token. The authorization token must be provided with each following request to the api. 

example: https://gist.github.com/eriklott/812a63be9d9cfd6a949cccaa94e6790c

SPA

The folder structure for the majority of our elm SPA's end up as follows:

  - Component (folder)

  - Pages (folder)

  - Main.elm

  - Route.elm

  - Routing.elm

  .. any other app/infrastructure related modules

Route.elm

Module encapsulating functions relating to routes:

https://gist.github.com/eriklott/812a63be9d9cfd6a949cccaa94e6790c#file-route-elm

Routing.elm

The routing module holds the state of the current page, and isolates logic pertaining to which page is displayed for each route

https://gist.github.com/eriklott/812a63be9d9cfd6a949cccaa94e6790c#file-routing-elm

Main.elm

This is where the top level modules are wired together. It's difficult to create a generate example for this since your Main.elm will be application specific. Here is a hacked together example of a Main that you can sift through and try to understand what's going on in this specific application:

https://gist.github.com/eriklott/812a63be9d9cfd6a949cccaa94e6790c#file-main-elm

Some things to note: The client configuration is stored in the main model and passed down into the pages. When a page triggers a logout or login, that fact must travel back up to the main module so the client configuration state can be adjusted accordingly. If you need to know how to do this, I'm happy to explain.

Pages

Here is an example page:

https://gist.github.com/eriklott/812a63be9d9cfd6a949cccaa94e6790c#file-page-apples-elm

Components 

There are mostly shared headers, footers,menus, that are shared across various pages. Some have their own state, but most are simple reusable view helpers. 

Sorry for being short on descriptions. Hopefully the examples are enough to give you a sense of direction and organization. Happy to answer any questions.

[Oliver Searle-Barnes]

That appears to have much in common with our app. It seems useful to compare what people building SPAs are currently doing so here's a rough gist of the folder/file structure that we're using https://gist.github.com/opsb/d0977bcb30b42302f3f2dc3daf0befec. There's a few differences worth pulling out

1) We have the Store abstraction that I mentioned for data synchronisation

2) We have two different top level modules depending on whether or not you're logged in (AppState)

3) We split our larger pages into sections which function as separate mini apps 

I've found myself thinking in terms of mini apps a lot lately. Each Page and each page Section functions as a separate mini app, notably they don't interact with each other, the only communication between them is via data in the Store which they all share.

[Marek Fajkus]

That appears to have much in common with our app. It seems useful to compare what people building SPAs are currently doing so here's a rough gist of the folder/file structure that we're using https://gist.github.com/opsb/d0977bcb30b42302f3f2dc3daf0befec. There's a few differences worth pulling out

1) We have the Store abstraction that I mentioned for data synchronisation

2) We have two different top level modules depending on whether or not you're logged in (AppState)

3) We split our larger pages into sections which function as separate mini apps 

I've found myself thinking in terms of mini apps a lot lately. Each Page and each page Section functions as a separate mini app, notably they don't interact with each other, the only communication between them is via data in the Store which they all share.

We're using similar approach and are happy as well. One difference might be that we don't deal with sign-in in SPA itself. We're using Html.programWithFlags and passing user info to elm on embedding. Some more info here https://groups.google.com/forum/#!topic/elm-dev/iqErmKHnLDY and here https://groups.google.com/forum/#!topic/elm-discuss/Lo6bG96zotI

[Eric G]

I think this would be great to have and would be glad to contribute. 

Just to be clear, you are proposing we come up with one set of functionality, that then multiple people implement using different approaches? Like TodoMVC, but a larger example that also encompasses server communication and build management?

I think the scope has to be really narrow and well-defined, and the domain pretty familiar to people already, for this to work.  

I agree with Marek that it needs some aspect of updating remote data. Re. MusicBrainz idea, perhaps something like a bookmarking or "my favorites" feature.  Also, to simplify, perhaps there should be a 'reference server app' so that people don't have to implement that as well as the client side. I don't think we should assume 'serverless', i.e. client app talks directly to the service.

Personally, I would like to see an example based around a multi-user app -- i.e., where multiple users collaborate on editing the same entities (not necessarily in real time). For example, a UI that models a simple workflow like Alice creates a document that Bob can edit and then that Carla either can approve or send back to Alice and Bob to make further edits. Maybe some example can be built around MusicBrainz that involves collaboration so the domain is not quite so boring :)

My point is, TodoMVC and many other such examples are apps where each users owns his or her own data and there is no collaboration, and a minimal state machine. The weakness of these examples is they are utterly unlike most business applications. That's also their strength -- they are relatively easy to implement and compare :)   

[Erik Lott]

Oliver,

We have the Store abstraction that I mentioned for data synchronisation

Sure. I'm sure this isn't too different - conceptually - from what we're doing. Although our app manages some complicated processes, the communication with the backend server is as simple as it gets - json gets and posts.

We have two different top level modules depending on whether or not you're logged in (AppState)

It's interesting that you mention that, because we also experimented with separating our Auth and Public sides of the app via the type system. I can't remember why we chose to flatten the app instead... but we did. 

We split our larger pages into sections which function as separate mini apps 

Yeah, we do the same when necessary, although we use different naming. A "Page" in our applications always represent a single onscreen page - so routes and pages are always mapped one to one. When we have multiple components/widgets on a page, those are broken into separate modules (mini-apps as you say) and likely stored in the components/widget folder, or broken out into a standalone package. 

I've found myself thinking in terms of mini apps a lot lately. Each Page and each page Section functions as a separate mini app, notably they don't interact with each other, the only communication between them is via data in the Store which they all share.

Exactly. 

[Erik Lott]

Marek,

One difference might be that we don't deal with sign-in in SPA itself. We're using Html.programWithFlags and passing user info to elm on embedding. 

Good stuff. Less work when you don't have to worry about auth states.

[Marek Fajkus]

Marek,

One difference might be that we don't deal with sign-in in SPA itself. We're using Html.programWithFlags and passing user info to elm on embedding. 

Good stuff. Less work when you don't have to worry about auth states.

it's helping us a lot. Anyway this architecture is used even in js app and predates me joining project. Whole login happens on different domain with different service. Anyway it also means we have to do one XHR request before starting app.

this is example of idea I'm experimenting with: https://github.com/turboMaCk/elm-app-composition-example
that led to: https://github.com/turboMaCk/tea-component

we're not using exactly that right now. Also package itself is removed from package.elm-lang.org. It will be republished once I manage to change name + docs and I don't really enjoy doing that so I'm taking my time:D

[Erik Lott]

Marek, about the tea-component package: If you're discouraged by the boilerplate for nested update and view functions, you can clean-up the boilerplate with a single function each... no need to create an entire abstraction package.

[Marek Fajkus]

Erik,

Marek, about the tea-component package: If you're discouraged by the boilerplate for nested update and view functions, you can clean-up the boilerplate with a single function each... no need to create an entire abstraction package.

that sounds like what we do now. First of all that package is at this stage just experiment. Anyway I would like to see that kind of abstraction since it's both interesting thing to explore and might be useful in practice as well.

[Francesco Orsenigo]

I would add a dropdown.
Even a simple dropdown poses some interesting challenges to the architecture:
- Dropdowns are ubiquitous in web apps.
- Dropdowns interact: when opening one, any other dropdown must close.

- Implementing keyboard support requires side effects and state.

- Dropdowns can have many different options; it can be disabled, it can have a button to clear its selection, it can require a selected value or the selected value can be a Maybe.

- Dropdowns requires styling: how do you distribute style with an Elm package? More importantly, how do you organize it within the app?

We might want to also consider a numerical input element.
- How do I ensure that the input element always produces a number while, at the same time, allowing the user to use Backspace to delete the last cipher in the input?
There are going to be different ways of doing this, it would be very good to be able to compare examples.

[Lourens Rolograaf]

Hi Fransesco, did you read this? https://medium.com/elm-shorts/a-reusable-dropdown-in-elm-part-1-d7ac2d106f13 part 2 https://medium.com/elm-shorts/a-reusable-dropdown-in-elm-part-2-9659ef988441

Op donderdag 20 april 2017 23:45:07 UTC+2 schreef Francesco Orsenigo:

[Francesco Orsenigo]

Yes I did.
It does not support keyboard.
As I wrote, "implementing keyboard support requires side effects and state".

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Elm Discuss" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/elm-discuss/WDDrFq-uP58/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> elm-discuss...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Marek Fajkus]

I think having drop down component would be really valuable. Anyway I think that implementing some reusable component what ever complicated it might be is far from "scaling elm". Designing reusable parts of UI is not necessary same thing scaling application code base.

[Dwayne Crooks]

How about writing an Elm front-end for https://github.com/gothinkster/realworld.

On Wednesday, April 19, 2017 at 5:11:05 AM UTC-4, Peter Damoc wrote:

[Peter Damoc]

On Wed, Apr 26, 2017 at 7:13 PM, Dwayne Crooks <dwayne...@gmail.com> wrote:

How about writing an Elm front-end for https://github.com/gothinkster/realworld.

 

This is what I'm actually contemplating right now.
I'll explore it some more next week. 

[douglas smith]

Hey all,

Selfishly I would like to see a FUN and ENTERTAINING  instructional elm-programming application.

Something that would be useful to the WIDEST range of persons and experience possible. 

From middle-school age kids to experienced 'engineers'. 

It could be built concurrently by different groups of people, but have it all integrate together 

into a cohesive FUN and ENTERTAINING kxcd-ish style project. 

A mix of learning material and actual UI elements that teach using the technology itself.

A story of sorts that could be refactored over time as the language evolves.  

And of course it needs a repl.  :)

-Doug

[Marek Fajkus]

Petr,

I wonder have you managed to find some time to look more into or think more about this. I'm curious about new points you might have.

Douglas,

This is really wonderful idea but should be followed by question "how?". And that's really hard question to answer. I agree it would be nice to have something like that but I have doubts any of us really know what that really means and more importantly know how to do it in practice. I would say this thread is more about trying to find some solution for a problem rather than coming up with ambitious ideas. We're looking for interesting problem we know how to solve to demonstrate that solution.

[Peter Damoc]

On Sun, Apr 30, 2017 at 8:34 PM, Marek Fajkus <mare...@gmail.com> wrote:

Petr,

I wonder have you managed to find some time to look more into or think more about this. I'm curious about new points you might have.

 

I got home about an hour ago from a lovely but exhausting 4 days tango festival. 
I'll look into Conduit/RealWorld tomorrow.  

[Rex van der Spuy]

On Monday, May 1, 2017 at 1:33:27 PM UTC-4, Peter Damoc wrote:

 a lovely but exhausting 4 days tango festival. 

That's awesome!!!! :) 

[Marek Fajkus]

Hi folks!

Ossi just announced open-sourcing of one of their (Futurice) projects on twitter.

twit: https://twitter.com/ohanhi/status/859664021150158849
repository: https://github.com/Tradenomiliitto/tradenomiitti

these are statistics of that project (I'm using cloc)

github.com/AlDanial/cloc v 1.70  T=0.52 s (223.0 files/s, 20394.5 lines/s)

-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Elm                             41            531             26           4284
JavaScript                      42            358             41           2952
SASS                            18            168              4           1731
JSON                             4              0              0            151
YAML                             4             27              3            115
Markdown                         1             31              0             86
Bourne Shell                     4             13              2             36
INI                              1              2              0             23
HTML                             1              1              1             22
-------------------------------------------------------------------------------
SUM:                           116           1131             77           9400
-------------------------------------------------------------------------------

This is both server and frontend. It's far from being big but I still think it's relevant to this topic.

👏 both Futurice and Tradenomiliitto