Isomorphic Mithril

[Daniel Barreiro]

I like the idea of having the same application able to run both on the server side and the client side, as rendr does.  I like Mithril as well so I wondered whether I could mix the idea of an isomorphic application (one that can be rendered alike both on the client and the server) and Mithril.   I started doing that here and, so far, I am able to render the same app on both sides.  

It has long been possible to do that by using something like PhantomJS server side and picking the virtual DOM from it, but it is somehow costly as PhantomJS is a very complete emulator, too much for what I need.  I noticed that in the tests folder of the Mithril distribution there is a very small mock.js file which is all that Mithril requires to get a page rendered.  This file is far smaller than PhantomJS and, with very slight modifications, it served my purposes.

As the readme explains, so far, I got a piece of middleware for Express that can read a Mithril application and render it on the server side.  The demo I have so far produces two images of the same module, one produced on the client, the other on the server, just for me to check that both render the same.   Clicking the links (in the tabs) of the client-side window does not make the bottom one change as the redrawing occurs completely on the client side without ever reaching the server.  However, clicking on the links on server-side image, them being regular links, the request reaches the server and both images are then updated.  

My next step, and that is the reason for this message is regarding the next step, which is reviving the server-generated image once it reaches the client.   One simple possibility is simply forgetting about the server image and let the client produce a new one. That would still be quite useful.  Search engines would still be able to index the whole site, something that cannot happen with regular single-page-applications.   For small apps and since this happens only the first time the page is loaded, completely replacing the server-generated image with a new one might be an acceptable solution.   However, I would like to go a step further.

Up to this point I have not changed mithril.js (I am using the 'next' version, not the distribution one) except for a few logging statements to see what goes on. However, I think I will need to make a couple of internal variables public.

One of them is cellCache which is the internal copy of the DOM.   In order to revive the server-generated application I must provide it with this.  Normally, cellCache is generated as the first render takes place and every new image is compared against it to see what parts of the DOM actually need changing.  In my case, the first render happens on the server so, I need to send it to the client.  As long as I make it accessible, I can read it, pack it and send it to the client to insert it into the empty cellCache before the app gets rendered the first time.  With the cellCache filled from the server image, when the client tries to render it for the first time it should find that the cached version matches intended one and make no changes.  Better yet, I should be able to prevent the client to even attempt the first render via a small change in m.render. 

The cellCache on the server side contains references to the fake DOM nodes in mock.js in the nodes property for each node.  I would filter those out when sending it from the server and fill them in on the client side.  

This, however, is not enough.  There are functions that need reviving as well, the most common being the reference to m.route assigned to the config attribute on any link, as well as event listeners.   To do that I need the image produced by the call to the module's view method.   This I can pick from the second argument to m.render (argument 'cell') which I can store somewhere on the server side, transmit it to the client and then bind those functions to the client-side resources it expects and call them. I can probably pack the source of those functions in the cellCache so as to put everything in a single place.

So, I wonder, am I on the right track?  Would I be missing anything that I would need client-side to make revival possible?   Of course, the question later on would be if this makes sense at all in terms of performance.   Would reviving an existing DOM be faster than redoing it again?  I would assume it should. At least it would certainly avoid the flicker when the client generated image replaces the one that arrived from the server.

I'll keep digging into this.  I know the function stuff is somewhat tricky, but I believe I would be able to cover most cases.  A little introspection on the part of JavaScript would be great and I might have problems identifying function references. Eventually, I might have to impose some limits on the way functions can be called (no anonymous functions, all should be methods of the module except for m.route, which I can easily recognize or something like that).

So far I think I can get away without modifying mithril.js itself except for making cellCache and the cell argument to m.render accessible (something like m.$cellCache = cellCache), add a conditional to prevent the first client-side rendering and, I hope, nothing more.  

I'd appreciate any advice.

Thanks

Satyam

[Daniel Barreiro]

Any plans to do event delegation?   I think it would be a great idea and it wouldn't take much.   It would consume much less memory as there would actually be a single listener per event type (except for those that don't bubble that would need to be attached as they are now).   It would save even more memory than usual because right now every event listener has to be wrapped in calls to start and endComputation (in autoRedraw) each of which leaves a closure behind.  If they all bubbled to a single handler, these calls to start and endComputation would be done there.

My reason to ask this here is that in order to revive the code on the client side, avoiding the wrapping done by autoRedraw would give me access to the reference to the actual function (in the delegation table) instead of the wrapped function which I cannot see.

Satyam

[Leo Horie]

Hi Satyam

I tried running the code but it's throwing errors for me (first q/q-io are not declared in package.json, and after that, it appears Windows and path separators aren't playing nice), so I'm mostly going from the description on your message - so please correct me if I say something that is off the mark.

From my understanding, it sounds like you are pre-rendering a Mithril app on the server, and you want it to be downloaded as HTML (for SEO and whatnot) and want Mithril to pick things up on the client-side and run as a normal app.

There's one thread on the issue tracker dealing with this here ( https://github.com/lhorie/mithril.js/issues/60 ). I haven't had time to take @sanderhahn's proof of concept further, but there I suggested that the SEO case can be handled by wrapping the pre-rendered HTML in <noscript> tags. Pre-rendering a *fully functional* application with inline event handlers that does not download Mithril application code is, I think, not possible.

The way data binding works means that template functions must be callable from the client, which basically means that we have to run the Mithril application normally (i.e. download the js files and run them, irrespective of what HTML comes down the pipe with it). Here's a very simple example to illustrate:

```

function someView(ctrl) {

  return m("h1", ctrl.title)

}

```

This code would yield something like this:

```

<h1>Hello</h1>

```

But let's say some other part of the app changes `ctrl.title` when you click on some button. In plain HTML, there's no way to express the binding between `ctrl.title` and the innerHTML of the H1 unless your markup has some sort of syntax to indicate that such a binding exists... and this is exactly what the `someView` templating function does in the first place.

Going back to the SEO case, I don't really see a reason to download both static HTML and dynamic application code to a single client (i.e. you can choose what to serve based on user agent, for example) and crawlers either run JS or they don't.

in terms of running the application on the server (for CI tests or what have you), yes, mock.js gives you an example of how you can essentially mock the browser from underneath Mithril so you can actually pretend to make XMLHttpRequests and other asynchronous things in a way that gives you control over the execution context. With that being said, that API is undocumented and may change without notice, so I wouldn't rely on it for anything production-level yet.

[Jake Verbaten]

> I don't really see a reason to download both static HTML and dynamic application code to a single client

Performance. Performance is king. First no cache pay load is what we measure.

The fetch for the JS file can take seconds dependening on mobile network. Sending server rendered HTML with the initial payload is how you get the "time until seeing content" to an absolute minimum.

This optimization buys you 50ms ~ 250ms ~ 2s depending on the internet quality of your audience,

Also remember that if you serve your JS from a CDN ( a second host ) the DNS lookup also add 100s of milliseconds.

At real production scale decreasing the "time until user sees non-interactive content" down by 200ms saves a ton of money.

--
You received this message because you are subscribed to the Google Groups "mithril" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mithriljs+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[eddyystop]

Here's some info from Google about first page timing:

Mobile Analysis in PageSpeed Insights

Optimizing the Critical Path Rendering

[Satyam]

On 14/06/14 02:48, Leo Horie wrote:

Hi Satyam

I tried running the code but it's throwing errors for me (first q/q-io are not declared in package.json, and after that, it appears Windows and path separators aren't playing nice), so I'm mostly going from the description on your message - so please correct me if I say something that is off the mark.

Sorry, I forgot I had mode a folder with its own package.json and its own dependencies (q and q-io) under the one I uploaded to GitHub.  Now I placed all the dependencies in the top package.json.   I installed it myself from the zipped download from GitHub and it now works, at least in Ubuntu.

From my understanding, it sounds like you are pre-rendering a Mithril app on the server, and you want it to be downloaded as HTML (for SEO and whatnot) and want Mithril to pick things up on the client-side and run as a normal app.

Yes, that is correct.  The 'whatnot' being the 'time to first page view'  which was longer to mention than the SEO stuff.

There's one thread on the issue tracker dealing with this here ( https://github.com/lhorie/mithril.js/issues/60 ). I haven't had time to take @sanderhahn's proof of concept further, but there I suggested that the SEO case can be handled by wrapping the pre-rendered HTML in <noscript> tags. Pre-rendering a *fully functional* application with inline event handlers that does not download Mithril application code is, I think, not possible.

I agree, that would be fine if it was only a matter of SEO, not so for the 'time to first page view'.  As a matter of fact, providing both the active application plus the <noscript> copy for SEO would actually make matters worst though, as you say below, detecting the user agent would allow for sending one or the other in which case, the <noscript> would be pointless anyway.

The way data binding works means that template functions must be callable from the client, ....

The way I meant to do it is,

• Send the rendered HTML be reading out the DOM tree produced in mock.js. (this already works)
• Send the contents of `cellCache` without the functions and node references as JSON. (I can do that right now though it just shows in the server-side console)
• On the client side, traverse both the received cellCache and the DOM, filling it the `nodes` property of `cellCache` with the actual nodes produced by the HTML.   I am not doing this yet, because I'm not yet done with cellCache (more on that later).  This should easy your concern about building the cellCache out of the DOM produced from the HTML.  Indeed, the DOM has too many properties and you wouldn't know which ones are relevant for your cellCache so, I prefer to rely on the cellCache sent from the server which only needs the cross references to the actual nodes the describe.
• I am also showing in the server console a JSON encoded version of the `cell` argument that `m.render()` receives.   I want to pick the functions from there, the ones that respond to UI events, the calls to `m.withAttr`, or to `m.route`. 
  
• The later(`m.route`) is easy to pick and easy to revive since it doesn't take any arguments.  I can easily fill in the reference to that function to the `m.route` on the client side and it will work.  
  
• For `m.withAttr` it is not so easy since it does take varying arguments.  It would seem I would have to monkey-patch that one as well to fill in those arguments.
• For event listeners, matters are harder, there is no easy solution. If they are inline, I can take the toString() of the function, however, references to variables in outer closures, a likely occurrence, would be lost.  As it is now with the sample app the toString() of most of those functions shows [native code]  as the body of the function since each of them is wrapped in a call to `bind` to adjust its context.  This also makes it impossible for me to compare references as I do with m.route.  m.route is never wrapped in bind so when I compare the function reference in the view with a reference to m.route, I can find matches.  If I do the same with the event listeners, I never get a match because the reference in the cell is to the wrapper, not the function itself.  

This is why I am not yet transmitting and resolving `cellCache`.  I would like to send some sort of encoded references to those functions, either their class and method names or the source itself) along cellCache, so that they can be resolved to the existing functions on the client side. I would like to change as little as possible of Mithril itself and of the style of its modules but it would certainly be much easier if I placed restrictions on event listeners, at least for isomorphic apps.  For example, if event listening functions were called in the context of the controller, wrapping them in a call to bind would not be necessary and I could easily compare references.  Then, for every function I find in a cell I would compare the function reference to the functions in Mithril, in the controller or in the module and send them like, say `{"where":"controller", "method":"onClick"}` and have them resolved and reattached in the client.

That is also why using event delegation would be great as there would be far less references to resolve on reviving all these in the client and the scope could more easily adjusted in a central dispatcher than in the miriad of `autoRedraw` closures spread all over. 

Satyam

[Leo Horie]

Ok, but as I said, merely having a static JSON representation of the app is not enough to revive it. The H1 example I gave wouldn't diff correctly if you needed to change its value after page load because the relationship between `ctrl.title` and the location in the DOM tree is only present in the template function, and this relationship is not preserved anywhere in the `cell` data structure. So even if you managed to revive all the event handlers, you would still not have a way to handle interpolations. For example, in this snippet - http://jsfiddle.net/ZFnY6/ - there's no way to revive the "H1-updates-on-input" aspect from only JSON structures unless the server parses the javascript AST of the view function to determine where bindings occur.

But at that point, I think we're over-complicating, and it's easier to just run the application normally, and modify Mithril so that it plays nicer with pre-generated markup (as per @sanderhahn's proof of concept)

[Daniel Barreiro]

The proof of the pudding is in the eating, they say so I made a pudding.

To install:

• Download the zip from https://github.com/Satyam/mithril-isomorphic and extract the contents
  
• Run npm install in the min folder
  
• run node server.js
  

As the prompt sugests, go to localhost:8000.   You should see a simple tabbed screen.

• You will notice the text is all uppercased.  That is the means to tell server-generated content from client one.
  
• Click on the `Settings` tab.  You will see that the server-generated content is now active since the screen does switch to the requested tab. 
  
• The content on that tab is generated on the client side so the text is now lowercase.  The tabs, though, remain in uppercase since they have not been changed.
  
• As expected both the checkbox and the buttons work since they have been generated on the client side.
  
• Press F5 or Ctrl-R to refresh the settings page.   
  
• The page now reloads from the server and it now shows all in uppercase.
  
• Play with the controls, they still work.  I have to fix the checkbox which appears checked the first time around, however, press it twice (once to un-check it and once again to check it) and it will work
  
• Got to the `list` tab again and you will see the contents now in lowercase.
• Move around, the tabs will always remain uppercase but the contents will change depending on whether it has been revived from the image sent from the server or has been generated on the client side.

Does it have limitations?  Certainly, I cannot revive any random function, but I believe the limitations are quite acceptable.  The code is messy because I tried to alter the Mitrhil code as little as possible so I monkey-patched things and exposed some internal functions in funny ways.   However, the little I did change of Mithril itself would still work if used without the rest of my code.  I used the Mithril version from the next branch, downloaded a few days ago.  

[Daniel Barreiro]

I fixed a couple of errors I had but one seems to remain, which is not really an error.   

If you load the `about` tab from the server, it will all show in uppercase.  If you change the contents of the textarea and then switch to the `list` tab to see it changed, you will see all `undefined - undefined`.   This is not because the reviver is failing it is, in fact, working fine.  The problem is that since I switched all the text coming from the server to uppercase to tell it apart from that generated on the client, when the `list` tries to show it, it will look for properties `author` and `quote`, however, due to the uppercasing, the properties are now `AUTHOR` and `QUOTE`, their lowercase versions are now undefined, which is what the `list` tab shows.

[Daniel Barreiro]

It was silly to leave that last error there, though it somehow proves the code works, but it required too many pointless explanations like the one above.  It was easier to fix it.

Anyway, the application can successfully revive several different types of function calls.  It manages to revive:

• a call to `m.withAttr()`, which later calls a method in the controller
  
• references to methods in the controller, 
  
• references to `m.route`
  
• an inline, anonymous function.
  

This might not cover the whole range of possible actions that a raw Mithril app could use but it is a pretty decent assortment.  I didn't tailor the demo app to work with the reviver, I just used the demo, which I picked from elsewhere, and revived whatever was in it.  

The app also uses `m.request()` to bring content from the server (the quotes themselves) and it does so both in the client, as expected, and in the server.

[Satyam]

So, the original question still stands.  Could it be integrated into Mithril?  Should it?   Looking at the mailing list statistics, it seems plenty of people are interested, it has plenty of views.

My brief demo is in no way a real product.  It combines a collection of patches to Mithril, a minimally modified Mithril (just enough to expose a couple of internal data structures) and a few things that could be better done in a deployment script using grunt, gulp or whatever.  However, it does prove that content generated in the server can be successfully revived on the client side.

After all the earlier talk about sending JSON encoded data structures and what not, I found out that the task was much easier than I originally thought.  By intercepting the `m.render()` method the first time around, the reviver simply builds the `cellCache` (the internal copy used for diffing) based on what it finds.  Basically, the `cellCache` is an almost exact copy of the data structure that the `view` method provides, with the addition of references to the nodes that represents that view on the screen.  Since at that point the DOM is the product of the very same view with the very same parameters except that it was produced on the server, it is just a matter of traversing the two and providing the links in between them.  I do not use the DOM as the basis, each node has too many properties that I don't care about.  I use the data structure from the `view` as the basis, which is very brief and straight to the point. 

The other big thing is the actual reviving of the active content, that is, all the functions that are linked to `onXxxx` attributes or the `config` pseudo-attribute.  Though initially I made a great fuzz of trying to handle each type appropriately (I had about 5 or 6 different types), it turned out that most of them follow the same pattern, thus, it turned out to be just two kinds, `config` attributes referencing `m.route` and all the rest.  

All this could be best done if integrated straight into Mithril itself and, being so simple, I am quite sure it would not significantly increase its size. Besides, part of it can be loaded conditionally, as I did with the `revive` function which is concatenated on the fly at the server to supplement the basic Mithril which is otherwise (mostly) left alone. However, this is beyond my abilities.  There is a lot of wizardry in Mithril, and no comments on a piece of code, the `build` function, that at 150 lines, it represents almost a fourth of all the code in Mithril.  I dug into bits and pieces of it and plenty of the code I added is a copy of parts of Mithril itself. Plenty of lines on `revive` deal with traversing the data structure, something the `build` function already does.  If all that were to be integrated into Mithril, a good part of the duplication in my patches would not need to be there.  My guess is that an extra argument to `build` called `revive` or some such and a few conditionals for that flag here and there would be all that is needed.  Where to put those? I don't know.   Leo, what do you think?

Satyam

[Jake Verbaten]

I've done a bit of research into server side rendering and client side hydration of virtual DOM apps.

You shouldn't need to bake it into the framework.

There seems to be three steps

 - render a mithril app to a string, this can be a module used on the server external from mithril

 - hydrate / revive / virtualize a server side rendered DOM into a virtual tree. 

At this point you probably want to somehow build up a cellCache from the server side rendered DOM.
You then want to re-render your mithril app on the client and instead of doing a diff+patch on an empty

cellCache you do it on a pre-populated cell cache. the process of recreating the virtual DOM on the 

client and diffing it into existing HTML will invoke all the config and m.route and event handlers and

what not. 

Since your only creating the virtual DOM on the client and not creating new DOM nodes this is really

cheap and you can re-use all the diff / patch machinery to ensure that the config and event handlers

and what not are added back properly.

 - deal with the transient state problem.

If between the time it takes for the server HTML to get rendered and the javascript to get loaded a 

user selects an <input> and starts typing, he will have typed before event listeners were added.

This means that when the client javascripts revives the mithril app it will render over the <input>

with a value of ''. This is a really painful problem to solve.

Note the first two steps I mentioned can be implemented as separate modules without any changes

to mithril. In fact I would love to collaborate on those modules and support both the mithril and vtree

virtual DOM data structures since they are very similar. We already have https://github.com/marcelklehr/vdom-virtualize

--

[Satyam]

Though my initial question was how to do it, that went unanswered so I went ahead and did it. Right now, I'm not asking how to do it, which you are trying to answer below, I've already done it. You may try it out. What I'm asking is how to do it better and whether it is a worthwhile feature.

On Sunday, June 29, 2014 12:41:21 AM UTC+2, Jake Verbaten wrote:

I've done a bit of research into server side rendering and client side hydration of virtual DOM apps.

You shouldn't need to bake it into the framework.

 

Unfortunately, yes, I have to.   The `cellCache` you mention and many others are local variables within the closure that returns `m` and inaccessible from the outside.  If you look at the code, my only change to Mithril  is to call the reviver if it is present and expose a number of internal variables.

There seems to be three steps

 - render a mithril app to a string, this can be a module used on the server external from mithril 

 - hydrate / revive / virtualize a server side rendered DOM into a virtual tree. 

At this point you probably want to somehow build up a cellCache from the server side rendered DOM.

 

Nope, you don't want to do that.   DOM nodes have too much information and there is no way to know which parts are relevant and which aren't.

You then want to re-render your mithril app on the client

I do call the `view` of the app in the client, if that is what you mean by re-render.  I am not re-rendering the DOM at all, that is the whole point.

 

and instead of doing a diff+patch on an empty

cellCache you do it on a pre-populated cell cache. the process of recreating the virtual DOM on the 

client and diffing it into existing HTML will invoke all the config and m.route and event handlers and

what not. 

This already happens.  I have this working already.    

Since your only creating the virtual DOM on the client and not creating new DOM nodes this is really

cheap and you can re-use all the diff / patch machinery to ensure that the config and event handlers

and what not are added back properly.

Well, that is sort-of the point.   I would like to re-use that diff /patch machinery that you mention, namely, the `build` function which is a behemoth that takes about a 20% of all the lines of code in Mithirl, has no comments and is a private, hidden function which cannot be overridden from outside since it is not exposed.

If you look at the reviver function, most of it is devoted to traversing the tree, which `build` already does.  There are very few parts that would need to be added to `build`, one is the function detection and reviving, the other is associating the cellCache with the DOM nodes (also here though it seems there is some needless redundancy, which I have to respect since Mithril expects it).

 I feel that those few changes would not add that many lines to Mithril itself and might be worth the trade-off.  The problem is that I get lost in such a big piece of code and don't know exactly where to put the changes in, unsure of what I might be breaking.  Thus, I cannot measure how big is that trade-off.  Would it add 10, 20 or 50 lines?   Once minimized, would it be 100, 200 or 500 more bytes?  Would that be acceptable?  If on the higher range, perhaps some part can be put outside of the core and send to the client only when reviving is needed, such as the reviver function is sent right now.

 - deal with the transient state problem.

If between the time it takes for the server HTML to get rendered and the javascript to get loaded a 

user selects an <input> and starts typing, he will have typed before event listeners were added.

This means that when the client javascripts revives the mithril app it will render over the <input>

with a value of ''. This is a really painful problem to solve.

Indeed, this is a problem, but your example is not the issue.   Since the DOM is not recreated, the text in the input is not lost.   If the user tabs away from the input and there is an onchange even yet to be attached, the action associated with that, most likely an update on the model would be lost, however, I would expect that  few users would be that fast.  If an input is validated or character types restricted by onkeypress or some such, those keystrokes would go through unchecked.  onclick events on buttons would be inactive, which the user would simply repeat.  Clicking on an anchor would lead to the page being refreshed from the server side instead of on the client side, it would be slower, but not fatal.

Note the first two steps I mentioned can be implemented as separate modules without any changes

to mithril.

As I said that, unfortunately, is not possible. 

In fact I would love to collaborate on those modules and support both the mithril and vtree

virtual DOM data structures since they are very similar. We already have https://github.com/marcelklehr/vdom-virtualize

 As part of Mithirl testing, Leo uses mock.js which does all the virtualizing it needs, which I use myself, slightly modified from the original. Instead of a generic virtual DOM, this one only provides the methods that Mithril actually calls, making it small and fast.  Anyway, this one runs in the server side, it does not travel through the wire to the client so it does not add weight to the client app.

I am sorry if my reply sounds a little negative.  I appreciate your help but it comes somewhat late.   The point now is, I found it is not that hard to add this feature to Mithril.  How big a change would it be and, once that is known, would you accept the extra weight?  

Thanks

 

[Jake Verbaten]

Hi Satyam,

It's great that you've already implemented most of this, wanted to give a second neutral opinion so you can double check.

> The `cellCache` you mention and many others are local variables within the closure that returns `m` and inaccessible from the outside. 

> namely, the `build` function which is a [...] hidden function which cannot be overridden from outside since it is not exposed.

It sounds like build and cellCache should be exposed somehow such that you can work with them

>  I would expect that  few users would be that fast.

If your javascript takes a few seconds to load on first uncached page request on a lower quality internet connection in india then this will be a problem.
You might not expect that users would be this fast, but you might also be spoiled by how snappy your internet is as a developer.

Now admittedly I think this only really a problem at scale.  note this problem does not exist if you do purely client side rendering because there will

never be an <input> without onchange listeners. 

> As I said that, unfortunately, is not possible. 

Then the bug is probably that mithril does not diff itself properly against an existing DOM. You will have to make sure that calling `m.render()` on a client

side created virtual tree (i.e. rendering your top level controller) does the correct thing, i.e. minimal actual DOM manipulation and calling all

the config props and adding all the event handlers.

It looks like your revive implementation does just that and would have to be merged into build.

--

[Satyam]

On Sunday, June 29, 2014 11:08:35 AM UTC+2, Jake Verbaten wrote:

Hi Satyam,

It's great that you've already implemented most of this, wanted to give a second neutral opinion so you can double check.

Thanks 

> The `cellCache` you mention and many others are local variables within the closure that returns `m` and inaccessible from the outside. 

> namely, the `build` function which is a [...] hidden function which cannot be overridden from outside since it is not exposed.

It sounds like build and cellCache should be exposed somehow such that you can work with them

Indeed.  Mithril is a blackbox from the outside, very little of it is exposed.  In the future others may also find hard to add features due to this lack of transparency.  

>  I would expect that  few users would be that fast.

If your javascript takes a few seconds to load on first uncached page request on a lower quality internet connection in india then this will be a problem.
You might not expect that users would be this fast, but you might also be spoiled by how snappy your internet is as a developer.

I suffered such slow speeds myself, and I know you simply pace yourself into a rhythm that the system can handle.  Anyway, having the page displayed in front of your eyes fast, even if you cannot fully interact with it, is better than waiting for the whole thing.  Another trick is to send the page with an extra className in the BODY that will make the input controls look inactive and then take it out once the JS code is active and ready.  

Now admittedly I think this only really a problem at scale.  note this problem does not exist if you do purely client side rendering because there will

never be an <input> without onchange listeners. 

> As I said that, unfortunately, is not possible. 

Then the bug is probably that mithril does not diff itself properly against an existing DOM.

 

As far as I can tell, Mithril doesn't do any diffing against the DOM, it compares the data structure that the `view` method produces with the previous instance of that data structure (the `cellCache`) and when it finds a difference, it redoes the parts of the DOM that correspond to that difference.  That is why the `cellCache` is full of references to DOM nodes, so that when it finds a difference it knows right away what must be touched.

 

You will have to make sure that calling `m.render()` on a client

side created virtual tree (i.e. rendering your top level controller) does the correct thing, i.e. minimal actual DOM manipulation and calling all

the config props and adding all the event handlers.

I just assume that since the DOM created by the HTML received from the server was produced by the same Mithril app, the data structure returned by the `view` method, the expected tree, should correspond to it. So, the only DOM changes I make are the attachment of the event listeners, which don't carry through from the server.  The revival does require a full traversal of the DOM to get the references to the nodes, but no changes.  Neither of them requires a redrawing of the screen, no flickering, no reflow.  

Though the reviving process might take almost as long as Mithril would normally take right now, the difference is that the user would have been reading the screen all through it, since even before Mithril itself got loaded.

[Leo Horie]

Hi Satyam

Did you get a chance to look at sanderhahn's code from the link in my first comment? ( https://github.com/sanderhahn/mithril.js/commit/973bbba7eb4505d464d20809d6961151670cd76c )

No offense, but I think this thread is kinda just leading back to everything that was said in that thread :)

Anyways, I've been meaning to look into this, but I've just been very busy and imho this feature is lower priority than the other things I was looking into (bugs, lifecycle management, keys, etc).

[Satyam]

On Sunday, June 29, 2014 5:31:12 PM UTC+2, Leo Horie wrote:

Hi Satyam

Did you get a chance to look at sanderhahn's code from the link in my first comment? ( https://github.com/sanderhahn/mithril.js/commit/973bbba7eb4505d464d20809d6961151670cd76c )

No offense, but I think this thread is kinda just leading back to everything that was said in that thread :)

Yes, I read through it.  That is why I took the time to code an even simpler solution, otherwise it is just opinions going back and forth.

Anyways, I've been meaning to look into this, but I've just been very busy and imho this feature is lower priority than the other things I was looking into (bugs, lifecycle management, keys, etc).

Ok, I can accept that, no problem. I'm happy knowing that you are not dismissing the issue altogether, as the thread you mention seemed to imply.

Thanks 

[Steven Garcia]

I was about to post a question about this and was pleasantly surprised by all the great work being done by Satyam. Just curious to see if this could be on the roadmap for Mithril itself, how much it would need to be patched, or if the whole isomorphic aspect could be declared as an external dependency.

As a footnote to all this, I think this could be a potential game changer for the framework. I already love it for its functional purity and believe that it could provide competition to a limited niche currently occupied by like 4 frameworks (all of which are built on lame-ass Express)

More food for thought: perhaps we can also add some generator goodness, a la KoaJS?

Curious to hear any thoughts on these topics

[Leo Horie]

Hi Steven

Server-side is not in the roadmap for Mithril core, but I think it provides enough of a foundation for a full-stack framework (specifically, the issue of rendering correctly after server-side template generation was addressed a few releases ago, and the undocumented m.deps can be used to integrate mithril to non-browser environments.)

Basically I want to keep core simple and small. Pierre-Yves mentioned he is working on an app that he can fit in not just a single HTTP request, but in a single TCP packet. It's a level of performance optimization that is pretty much unheard of elsewhere. I think this type of experiment is very exciting and it can only happen by keeping Mithril core small.

With that being said, I think full stack frameworks can definitely use some disruption. I think Meteor is probably the state of art in that space right now, but it currently has some hard dependencies that are just not feasible for a lot of people (e.g. pushing something like MongoDB to a corporate client's IT dept is hard because doing so can severely damage the client relationship). IMHO, Mithril is definitely fit to be a piece in the equation to crack that problem

Re: generators - I think they are very cool, but they're still far from being usable in the client-side world. The appeal of a Mithril-based full stack framework would be the write-once-run-everywhere slogan. Trying to supporting generators would, at a minimum, add complexity to the build process, if it's at all possible.

[Satyam]

As my demo pointed out earlier in the discussion, the problem with Mithril going isomorphic is how little it exposes of its internals.  `m.deps` is not enough.  Without making Mithril more complex, it would benefit if some of its internals were visible from the outside.  An obvious requirement would be to get`cellCache` exposed or somehow reachable.  There were others which I don't remember any more, but the point is, making out-of-the-box Mithril isomorphic is not possible.  Making it possible does not require lots of changes or making it bigger, just making a few of its crucial internal data structures accessible.

[eddyystop]

@Satyam, perhaps you can make a PR showing what you need at a minimum. 

[Leo Horie]

@Satyam, just to clarify, in my last post, I was just talking about using Mithril as part of a bigger framework, basically similar to the proof of concept I did for the Mithril+Meteor article on the blog.

Talking specifically about a Mithril-only isomorphic system, there are still some aspects that I don't quite understand how they would work, for example validation, SQL integration, using 3rd party libraries via config, etc. Do you have any insights on how you would handle these framework "edges"?

[Satyam]

Leo,

I don't think you should concern yourself with making Mithril isomorphic yourself but with allowing others to make it possible and deal with those considerations themselves.  Eventually, some combination of the issues you mention might not make it possible in some circumstances.  

The problem as I see it now is that Mithril keeps all its cards too close to its chest.  Too many things remain completely unreachable within its main closure.  Turning `cellChache` into `m._cellCache` would not greatly hurt either size or performance.  Likewise with the `controllers` array.   You can see my patch here. The $reviver function you see there is here .  It has a lot of code copied over from the `build` function which is otherwise inaccessible from the outside. I am sure the `build` function could handle what `$reviver` does with a couple of conditionals here and there. 

Anyway, that would get you involved in making Mithril isomorphic which, 

I understand, you prefer not to.  But if only you allowed it to happen by making accessible (though declared protected) and documenting some of Mithril internals, someone could take care of it.  For example, on the server side, since for the first page I don't care about `cellCache` I didn't need to change Mithril itself because all I needed, the `m.render()` method, was accessible so I could just change it. But on the client side, I had to feed Mithril with a valid `cellCache` so I had to touch Mithril itself.  I would not have needed to do it if both `cellCache` and `controllers` were accessible.  Allowing access to a few utility functions would also be good but not that crucial.

[Leo Horie]

Well, my instance is that if a project is interesting, I don't mind getting involved (although I do have time constraints that often prevents me from contributing as much as I'd like).

My concern was mostly one of adding random hooks for every project that wants to use Mithril in a not-so-standard way (the death-by-a-thousand-paper-cuts problem, if you will). Given that isomorphic-mithril is highly experimental, I think it's reasonable for you to just fork Mithril and hack on it (as you have done). Exposing cellCache and controllers is something that I think is straightforward enough for one to do themselves in a fork, and if the experimental project reaches a level of stability that can actually be used in a real world scenario, I think we could, at that point in time, discuss the benefits of merging the fork back to the Mithril production branch.

[Satyam]

I also said (boldface added):

 But if only you allowed it to happen by making accessible (though declared protected) and documenting some of Mithril internals

Adding some comments would also help. Some of the tricks in there are hard to figure out even while debugging.   Turning plain strings into String instances with a nodes property is hard to pick even when single stepping through the code.  It is hard to fork and modify something when you don't even know what it is. 

And merging the fork back into the project might not be the ultimate goal for someone.   Someone might want to turn Mithril isomorphic for a particular project where it might be well suited, with no show-stoppers, but not make it generic enough to make it part of a generic Mitrhil.  In such a circumstance, both forks would be forever apart, but the fork might still benefit from updates to the original.  In that case, it would be safer for the isomorphic version not to alter Mithril at all but override its methods, as I tried to do, so that it is safe to keep Mithril updated.

[Leo Horie]

I thought you only needed to expose data structures like cellCache and controllers. Doing that is just a matter of changing references to those variables to m.$cellCache /m.$controllers, and those are all in relatively straightforward areas of the code. Are there other places you need to hook into?

I've recently added some comments to the source code of the build function (basically describing what is the purpose of the arguments, and a high level explanation of the keys algorithm), but I could try writing up a beefier explanation of what build does, if it helps. 

[Satyam]

Actually, I don't need anything.  I gave up on Mithril some time ago because of the lack of information and access to the internals.  I am not sure why I kept subscribed to this list but suddenly I saw the subject I started two months ago and was curious. Not sure why I bothered replying. Sorry.  

[Leo Horie]

That's too bad. I still want to look at this when I get a chance. If you want to give it a go again, I'm always happy to answer questions too.

[Flemming Hansen]

@satyam has a valid point with the documentation. 

Not just for this isomorphic issue, but future-wise in general, as it prevents people from contributing. 

It's reminiscent of any start-up entrepreneurship's challenge of growing beyond the scope of a one-man show. 

So no offense meant, Leo, but perhaps this issue is more important than it seems.

[Leo Horie]

@Flemming yeah, I was actually having this exact same discussion with @jsguy on the issue tracker. I think it's definitely getting to that point. FYI, I added some more comments there to explain the cached data structure and some more one-liners here and there in places where I've seen people try to hack on.

I think my problem is that some things that are not so easy to grok look obvious to me just because I'm so close to the code. So, if you're having trouble understanding what's going on while reading the source, by all means, let me know, so I can write more helpful comments.

[cla...@gmail.com]

I know many people using react now just by the isomorphic feature, It's really a key feature and even angularjs guys are using react right now for isomorphic components...I don't know why it is not a priority for mitrhril: yes, fix the bugs are priority but mithril seems pretty fast and stable now, maybe is the moment for add this feature

this would extend the interest in mithril and increase the collaboration, just know, although mithril is pretty interesting I keep using reactjs because I need its isomorphic feature..would be interesting check the community opinion...

again, this seems the most promising framework for the nearly future, I hope the best and I hope than this grown and include "the good parts" from react and maybe ember..thanks!!

[Jon Nyman]

This is a bit off topic but similar. Per SEO and rendering only on the front-end, I know google can now search SPA’s. I don’t know if they are 100% at it but They found my static SPA blog. Before searching bilby.js lenses and my post would show up with !# in the url. I have since changed to created html files on each page in order to pass the SoundGecko test, that is, if you don’t like reading or like listening while reading you can use SoundGecko to convert it to audio. But the pages are very minimal and tell the person to turn on JavaScript with CSS.

If anyone is interested in looking at the static SPA blog built with Mithril you can go to http://jon.prescottprogrammers.com or https://github.com/jon49/programming-blog. It’s not perfect, I’ve been changing things as I learn more but it works!

​

[berg...@gmail.com]

Hi guys,

I've implemented an isomorphic framework that uses mithril, and renders server side first:

https://github.com/jsguy/misojs

Here is a list of some of the features:

* Lightweight footprint (~15kb gzip+minified)
* Full-stack isomorphic framework
* SPA that renders the initial page server side
* Data models with isomorphic validation
* Auto-reload browser on changes

It solves some issues such as the m.request promise by using an integrated data handling routine, and is relatively easy to get up and running.



On Thursday, June 12, 2014 at 1:23:15 AM UTC+10, Satyam wrote:
> I like the idea of having the same application able to run both on the server side and the client side, as rendr does.  I like Mithril as well so I wondered whether I could mix the idea of an isomorphic application (one that can be rendered alike both on the client and the server) and Mithril.   I started doing that here and, so far, I am able to render the same app on both sides.  
>
>
> It has long been possible to do that by using something like PhantomJS server side and picking the virtual DOM from it, but it is somehow costly as PhantomJS is a very complete emulator, too much for what I need.  I noticed that in the tests folder of the Mithril distribution there is a very small mock.js file which is all that Mithril requires to get a page rendered.  This file is far smaller than PhantomJS and, with very slight modifications, it served my purposes.
>
>
> As the readme explains, so far, I got a piece of middleware for Express that can read a Mithril application and render it on the server side.  The demo I have so far produces two images of the same module, one produced on the client, the other on the server, just for me to check that both render the same.   Clicking the links (in the tabs) of the client-side window does not make the bottom one change as the redrawing occurs completely on the client side without ever reaching the server.  However, clicking on the links on server-side image, them being regular links, the request reaches the server and both images are then updated.  
>
>
> My next step, and that is the reason for this message is regarding the next step, which is reviving the server-generated image once it reaches the client.   One simple possibility is simply forgetting about the server image and let the client produce a new one. That would still be quite useful.  Search engines would still be able to index the whole site, something that cannot happen with regular single-page-applications.   For small apps and since this happens only the first time the page is loaded, completely replacing the server-generated image with a new one might be an acceptable solution.   However, I would like to go a step further.
>
>
> Up to this point I have not changed mithril.js (I am using the 'next' version, not the distribution one) except for a few logging statements to see what goes on. However, I think I will need to make a couple of internal variables public.
>
>
> One of them is cellCache which is the internal copy of the DOM.   In order to revive the server-generated application I must provide it with this.  Normally, cellCache is generated as the first render takes place and every new image is compared against it to see what parts of the DOM actually need changing.  In my case, the first render happens on the server so, I need to send it to the client.  As long as I make it accessible, I can read it, pack it and send it to the client to insert it into the empty cellCache before the app gets rendered the first time.  With the cellCache filled from the server image, when the client tries to render it for the first time it should find that the cached version matches intended one and make no changes.  Better yet, I should be able to prevent the client to even attempt the first render via a small change in m.render. 
>
>
> The cellCache on the server side contains references to the fake DOM nodes in mock.js in the nodes property for each node.  I would filter those out when sending it from the server and fill them in on the client side.  
>
>
> This, however, is not enough.  There are functions that need reviving as well, the most common being the reference to m.route assigned to the config attribute on any link, as well as event listeners.   To do that I need the image produced by the call to the module's view method.   This I can pick from the second argument to m.render (argument 'cell') which I can store somewhere on the server side, transmit it to the client and then bind those functions to the client-side resources it expects and call them. I can probably pack the source of those functions in the cellCache so as to put everything in a single place.
>
>
> So, I wonder, am I on the right track?  Would I be missing anything that I would need client-side to make revival possible?   Of course, the question later on would be if this makes sense at all in terms of performance.   Would reviving an existing DOM be faster than redoing it again?  I would assume it should. At least it would certainly avoid the flicker when the client generated image replaces the one that arrived from the server.
>
>
> I'll keep digging into this.  I know the function stuff is somewhat tricky, but I believe I would be able to cover most cases.  A little introspection on the part of JavaScript would be great and I might have problems identifying function references. Eventually, I might have to impose some limits on the way functions can be called (no anonymous functions, all should be methods of the module except for m.route, which I can easily recognize or something like that).
>
>
> So far I think I can get away without modifying mithril.js itself except for making cellCache and the cell argument to m.render accessible (something like m.$cellCache = cellCache), add a conditional to prevent the first client-side rendering and, I hope, nothing more.  
>
>
> I'd appreciate any advice.
>
>
> Thanks
>
>
> Satyam

[Leo Horie]

Nice, looks promising.

I noticed the README says it's not ready for production yet. How stable is it?

[berg...@gmail.com]

On Wednesday, February 4, 2015 at 5:33:08 AM UTC+11, Leo Horie wrote:
> Nice, looks promising.
>
>
> I noticed the README says it's not ready for production yet. How stable is it?

Thanks :) Yeah I hope it will turn into the kind of framework that people will want to use - I'm trying my best to keep it un-opinionated, and flexible whilst providing something useful.

It is actually quite stable in so far that it works with what I've done with it, ie: a simple todo app (single url SPA), and a user management app (multiple url SPA), that uses mongodb for storage.

See: https://github.com/jsguy/misojs/wiki/Patterns for the two distinct patterns (single and multiple url SPA)

I want to be sure that I don't need to make too many drastic changes before we release this to the world, the areas of concern for me are:

* How I've tackled the server-side timing issue for initialising modules.
I've added a binding that delays the rendering server side, so that our API request can finish before rendering: https://github.com/jsguy/misojs/blob/master/system/index.js#L192 This works quite well, but it would be great to get you to look at the pattern and validate that it will do what we need.

* How the adaptor pattern works. For the database access, I've created a lightweight "adaptor pattern" which essentially allows you to create an adaptor for a data store, and expose methods within miso that the user can call - it generates shims for both the client and server, which it turn hooks into the delay binding mentioned above. It works pretty well, and abstracts the problem away from the user. Note: this is an un-opinionated pattern, so we can create any methods on the API that we see fit - ie: it is not an abstraction - I recognise the fact that not all database APIs can be neatly squeezed into a single set of methods...

* Documenting how people need to write their code for it to work properly - it's pretty close to normal mithril, but there are some rules you'll need to follow for the (optional) automatic stuff to work, eg: automatic routing. you can of course manually add routes and so on, but there is much benefit to following the automatic patterns. This one I can do, it will just take a little time :)

Thanks for looking at this - I hope to address the above, with your help, and then it will be ready for public consumption!

[berg...@gmail.com]

Just to be clear, there are two places that the ready binder is being used:

1. In the adaptor:

https://github.com/jsguy/misojs/blob/master/system/adaptor/adaptor.js#L52

2. In the route:

https://github.com/jsguy/misojs/blob/master/system/index.js#L192

The generated code only runs on the server - mithril handles the client side just fine.
I think this pattern is an "ok" solution to the problem, but I've had to add a pointer to the binding in the scope of the executing action, which is a little fiddly.

Anyway, it would be good to get some more eyes on it, before getting too many more people involved...

[berg...@gmail.com]

Ok, I've mitigated my concerns by adding another adaptor, (flat-file-db), which enabled me to prove that it should work just fine .

v 0.0.5 is now out, please check it out:

npm install misojs -g

no need to install mongodb anymore, as the default adaptos is flat-file-db.

[view.t...@gmail.com]

I'm probably way behind the ball on this conversation, but I've been using a few tools as of late for isomorphic mithril. I still view my compilation as a hackish mix of some tools (including Stephan Hoyer's) that I glued together and sugared up a bit, but regardless, I've built a few powerful things so far off of it. I have plans to continue to iterate and make it an out of the box, full-stack solution. In case anyone finds it helpful....

https://github.com/ezramorse/mens

[Per Sommer]

this looks interesting. I'm wondering why you choose to implement sockets as a rest replacement. i mean, speed is the obvious gain, but if a server has many simultaneous users i guess it would cause a considerable strain? 

[view.t...@gmail.com]

On Friday, April 1, 2016 at 5:26:48 AM UTC-7, Per Sommer wrote:
> this looks interesting. I'm wondering why you choose to implement sockets as a rest replacement. i mean, speed is the obvious gain, but if a server has many simultaneous users i guess it would cause a considerable strain?

1) Ok, we can both agree about speed. :)
2) I use it every day at work, so I'm used to it and like it!
3) I use it every day....and have seen its proven ability to handle massive concurrent connections and scale when necessary.
4) It allows you to do things very easily with pushing data that modern web applications now rely on, like sharing select session variables or template updates or server side data mutations or communications from other peers...
5) Using long polling or periodic polling for pushing data client side, paired with significant ajax, is terribly inefficient and causes simultaneous connections
6) There is no significant penalty for stacking simultaneous data requests (since its utilizing the same socket), whereas this is not the same scenario with stacking ajax calls.
7) Significant load is reduced with isomorphism, such that I now have further room to open up socket connections.
8) Code reduction vs creating api endpoints

I do understand that there are tradeoffs here, but the traditional usage of AJAX is not entirely efficient on its own, and it gets really muddy when you toss long-polling into the mix. In contrast, I'm taking the approach of just letting the client run the entire website, and opening a single, consistent socket for all communication (instead of multiple http requests flying every which way).

But the real advantage here is #4. The reality is that anyone can make a Hello World website. If that's all we were doing here, why even use mithril? Its 2016 already, and I wrote with a bit of a different paradigm in mind: its time we all *share* the cloud together, simultaneously. Models aren't something that live on a server and get replicated outwards on screens. Models are living, breathing data that can tangibly exist in a mutable fashion for everyone at the same time. We should build beautiful applications that espouse these ideas, and allow interactivity by teams and communities on such shared data. And that is where socket.io shines (and shines further paired with mithril).

[Per Sommer]

Thanks for the extended response.

It's on my watch list now and I'll be checking out what you'll be adding in the future. I'll give it a go myself one of these days.

I'm still a bit vary of how many concurrent users it can handle unless there are some tested numbers behind it. I've tried digging into how many simultaneous connections socket.io can handle but results are not conclusive enough.

[Ezra Morse]

What socket.io can handle depends entirely on the size of your instances and what you need to do besides maintaining connections. But if you're hitting the wall at 10K concurrent connections, its what I like to call a "good problem to have". If your traffic demands that you must scale socket.io instances horizontally, then that should indicate a site with monetization potential to support such scaling.

OTOH, if someone has no use for server side communications and push updates (via socket.io or traditional long-polling/comet), then this is probably all around not the right tool for the job when spinning up a static Jekyll might be better suited. In the end, it comes down to use-case.

I'm interested in a very specific type of web development that allows people to communicate and collaborate in real-time with shared data. So that's what I'm trying to build a tool for in the end. :)

I find mithril to definitely fit the bill when it comes to quickly prototyping interfaces that facilitate these types of real-time, multi-user interactions. That's my use case and what I'm trying to ultimately solve on the server side. VDOM + Isomorphism + Websockets is the ultimate solution for problems that have plagued me for a while!