Virtual DOM for Knockout?

[Evan Worley]

Hi All,

I'm sitting here at FluentSF listening to a talk about React.js. The speaker mentioned Ember, Angular, and Knockout and then showed how complex this toy example was in Ember/Angular compared to React. What he didn't do was contrast it with a Knockout implementation. The truth is, the toy example implemented in knockout would've been very simple, maybe even simpler than the React example.

The one place where React blows knockout out of the water, is performance, due to the virtual DOM implementation. Has anyone thought of overhauling knockout to apply all bindings to a virtual DOM and then apply a mapping layer between the virtual DOM and the actual DOM?

I strongly believe in Knockout and love how it has evolved. The simplicity of the bindings, but extensibility of the whole system is just fantastic. But as long as it's directly applied to the DOM, and we have inefficient bindings (such as foreach), we will not be able to compete with similar libraries that use a virtual DOM approach. The array change tracking helps a lot, but it hasn't been implemented across the board, most notably in the foreach binding.

Does anyone think a radical change here could be just the thing that makes Knockout become the #1 view model / UI binding toolkit? I'd be happy to help with something like this, but last I heard Knockout can not accept work from Non-microsoft employees :(.

Thoughts?

Evan Worley

[rpn]

Hi Evan-

I just wanted to clarify that it is only Knockout-Projections (https://github.com/SteveSanderson/knockout-projections/) that can not currently accept external contributions.  Knockout itself does not have any restrictions (other than signing a CLA: http://knockoutjs.com/contributing.html).

The virtual DOM piece of React is something that has definitely piqued my interest. There are no plans currently in the works to implement something like this in Knockout.  I wonder though how extensible/reusable their implementation might be?  I wonder if there would be some way for KO to use it in an optional way (abstract any DOM interactions in current KO code).  Regardless, it sounds like some pretty substantial changes would be necessary.  I would also wonder about browser support (React says IE8+).

-Ryan

[Evan Worley]

Hi Ryan,

Thanks for taking the time to reply and consider my thoughts. Also thanks for clarifying about knockout-projections, that is good to know!

I haven't looked in detail at the React virtual DOM, but it could be worth considering reusing it. I have mixed feelings on browser support going before IE9. On one hand I don't think there is a good reason why IE8 can't be supported, but on the other hand I seen less and less technology supporting IE8-, and am super anxious for the near future reality where all browsers auto-update themselves and there is no more of these limitations.

I agree it would be a rather radical change, perhaps even better to prototype as a stand alone project using knockout semantics. I'd focus on what I think are some of the most important features first, and also those that demand performance, such as templatized foreach expansions.

Kind of hoping to rally some troops here, to create enough momentum to get this going.

- Evan

[Patrick Steele]

Unfortunately, IE8 still appears to have a little over 20% of the market:

http://www.netmarketshare.com/browser-market-share.aspx?qprid=2&qpcustomd=0

---
Patrick Steele
http://weblogs.asp.net/psteele

--
You received this message because you are subscribed to the Google Groups "KnockoutJS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to knockoutjs+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[rpn]

Several of the client projects that I have worked on it the last year required IE7 support with IE8 being the minimum in all of the others.

With that said, if this type of functionality was a possibility, then I would potentially see it as an extensible option. If DOM manipulation was abstracted, then it could be possible to swap in some type "DOM provider" that used something like React's virtual DOM.  This provider could require a newer browser, as necessary.  None of this sounds easy or trivial in any way, but certainly would be interesting to have people explore it as a possibility.

-Ryan

[Baz]

What is a virtual dom fundamentally anyway - is it more than just a hierarchical structure of objects? For example, is this a virtual dom:

body {

   table: {

      tr: {

         td: {

            span: 'cell value #1'

         }

      }

   }

}

[Evan Worley]

At the simplest level it's an in memory representation of a DOM document. It would need to be a bit more complex than your example, to imitate a real W3C dom. Ideally it would have very similar methods to an actual DOM, the primary benefit of using it being that it's fast, compared to manipulating the actual DOM.

Next is all about its use. Each node in the virtual DOM would have a direct mapping to a node in the actual DOM. This way whenever the virtual DOM is manipulated, it could at some point update the actual DOM. This allows for the observers that manipulate the virtual DOM to be very fast, and allows us to batch updates to the DOM. Kind of like throttling in knockout but on a whole other level.

--
You received this message because you are subscribed to a topic in the Google Groups "KnockoutJS" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/knockoutjs/RBx623pYCXg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to knockoutjs+...@googlegroups.com.

[Baz]

It seems like just a node-map could be enough. What do you think of something like this:

1. On load, create initial hierarchical in-memory structure of nodes (virtual dom)
2. Intercept KO events, and create new virtual dom representing changes
3. Compare actual virtual dom with new virtual dom
4. Calculate and apply most efficient commands to update real dom with differences
5. Replace actual virtual dom with new virtual dom for future events

The challenge is going to/from the virtual dom, but the virtual dom itself could be pretty dumb, aside from a few functions to get/set nodes. What types of things are you thinking of that need closer feature-parity to a W3C dom?

[giedriu...@gmail.com]

I have just tested React on one area in my website, where I have the biggest performance problems and improvement is staggering. Couple hundred song list hooked up with variety of features, previously rendered in 2-4s, with react it takes no longer than split second, it is even hard to notice when the DOM changes.
While I do understand that there is still a need for support of older IE, these days need for performance is just as important. With time performance will be more and more important so in my personal opinion virtual DOM is the feature to look at.

[Baz]

Might you keep it in for production?

On Mon, Mar 24, 2014 at 1:53 PM, <giedriu...@gmail.com> wrote:

I have just tested React on one area in my website, where I have the biggest performance problems and improvement is staggering. Couple hundred song list hooked up with variety of features, previously rendered in 2-4s, with react it takes no longer than split second, it is even hard to notice when the DOM changes.
While I do understand that there is still a need for support of older IE, these days need for performance is just as important. With time performance will be more and more important so in my personal opinion virtual DOM is the feature to look at.

--

[Evan Worley]

Thanks for sharing! Can you talk more about what that page uses from knockout? Did you have a templatized foreach binding by chance? Also what type of template. How big are the models (# observables, # computed observables).

Also did you look at the memory footprint differences?

--
You received this message because you are subscribed to a topic in the Google Groups "KnockoutJS" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/knockoutjs/RBx623pYCXg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to knockoutjs+...@googlegroups.com.

[giedriu...@gmail.com]

I would like to keep it. Though to make it work properly I need to spent more time on it. I'm loading data from server straight to knockout observables now, but to use it with react I need simple values, so there is too much overhead at this moment, even though it is much faster. Also same data is used in many places and I wouldn't like to have same data in two different types.
But this will eat all my evenings now until I'll feel confident to use it in production.

[Evan Worley]

Wouldn't it be amazing if you could use the simplicity and elegance of knockout bindings, with the speed and performance of a virtual DOM? (Massive plug for virtual DOM support to knockout, which as we know is virtually a re-write preserving only semantics).

--
You received this message because you are subscribed to a topic in the Google Groups "KnockoutJS" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/knockoutjs/RBx623pYCXg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to knockoutjs+...@googlegroups.com.

[giedriu...@gmail.com]

I have extended observables connected to adaptor which loads data from server. All it is just a computed which updates observable and observableArray values. Yes I'm using forEach templates and models are quite big and contains all types of observables, plus there is quite a lot of functionality connected to them, list elements are selectable and draggable, using "knockout-sortable" plugin. But the main hiccup knockout has is that text and certain elements inside the list are observables, this forces hundreds of repaintings on load.
On the page I'm using almost all knockout features, and knockout from the very beginning was the backbone of application, so any changes now is very expensive to make.
Memory consumption is the thing that surprised me the most, while I was expecting speed improvement, I did not expected memory improvements, because I'm still using knockout to load the data, so essentially I have now duplicated data, but while knockout only version didn't played nicely with chrome memory, especially when starting to navigate between long lists containing over 200 items, with react memory was growing very slowly and just few seconds after I stopped navigating between lists, memory dropped back to previous levels, which is very pleasant to see.

And I totally agree that having knockout simplicity with react speed, would really make a lot of people happy. That would like something really properly designed for large web-apps.

[Ω Alisson]

Evan, Vue.js is just that :D

[Baz]

Half the case for react is the performance angle, the other half, is the simplicity of being able to program statelessly. This allows you to throw away and replace data models on every change. For example, if you had a list of 100 items, and 1 item changed, in react you would just discard the whole list, replace it with the new list, and have the component recalculate the full virtual dom with the new list. The final effects on the real dom will be to only change the single list-item, but that's outside the coder's scope. Given that, if a virtual dom were introduced in KO, would it still make sense to have bindings (aside from it being good for legacy projects)? I'm not trying to make a case for react here, I think there is a good answer to the question, but I don't fully see it.

[Winston Fassett]

Does VueJS have a virtual DOM?  This comment by the author suggests otherwise.

[Ω Alisson]

It doesn't but it has a comparable speed :)

[Winston Fassett]

I'm still trying to get my head around where the real performance benefits are with ReactJS. 

I think this StackOverflow question has an answer that sums it up:

• Batched DOM read/write operations.
• Efficient update of sub-tree only.

and this blog post from Stoyan Stefanov at FB adds

• DOM updates are usually executed in a requestAnimationFrame tick
• Rather than messing with lots of DOM event handlers, they use synthetic events

It seems to me that one can do these things in any HTML app, without requiring a virtual DOM.  Perhaps Knockout can, too?  

KO has options for synthetic events, but not for batching UI updates, as far as I can tell.  I've often wanted to do things like:

• pause all UI updates and resume later
  
• batch UI updates by requestAnimationFrame

Throttle is helpful, but not when you want to throttle the whole UI.

This reminds me of knockout-deferred-updates, which I haven't used.  Would it help with any of this?

[Evan Worley]

Hi Winston,

There are other things to consider as well, such as how some of the knockout bindings are implemented. For example the foreach binding slows down linearly with the number of elements in the bound array. This is due to the binding not using the new "arrayChange" functionality, which allows array changes to be tracked more intelligently such that detecting "what's new" is constant time, not O(n). Additionally the expansion of the foreach binding also runs over all the existing DOM nodes, which is quite expensive. These performance issues are independent of a virtual DOM, though some of them (like traversing and comparing DOM nodes) would be made faster by dropping in a virtual DOM.

- Evan

[steven....@gmail.com]

There are examples of using ReactJS within AngularJS directives. 

Seems to me that perhaps a knockout binding handler could be a proxy to the underlying ReactJS component ?

KO doesn't need to implement this themselves do they ?

[Evan Worley]

Hi Winston,

Sorry for the extremely late reply. "arrayChange" functionality is great, but indeed foreach does not take advantage of it. Some discussion about foreach performance is here - https://groups.google.com/forum/#!searchin/knockoutjs/foreach/knockoutjs/iw15cn1BakU/VfAWhZYgag0J.

You're right that even optimized foreach creates a lot of DOM churn. That's my main motivation for a virtual DOM, as this will bundle DOM updates more efficiently. That said, we all know rendering hundreds or thousands of DOM nodes will always be inefficient, so for large lists virtual scrolling needs to come into play. That's what I'm doing for one of my problems, where the observableArray that is expanded by a foreach binding only contains ~50 nodes at a time (from a set of thousands).

Having deferred DOM updates may help in some cases, but I essentially achieve this by batching up my observableArray changes (http://www.knockmeout.net/2012/04/knockoutjs-performance-gotcha.html)

So in summary, I think most of your points are valid optimizations to the problem, but they I'd imagine they will only have linear improvements. Radically changing the underpinnings, by using a virtual DOM and efficient update handlers could achieve a exponentially smaller update time that would be constant regardless of the number of elements in the array.

My two cents,

Evan 

[Winston Fassett]

Funny coincidence, I was about to post this video from Pete Hunt talking about the "Secret's of React's Virtual DOM", just to continue the conversation.  

https://www.youtube.com/watch?v=-DX3vJiqxm4

It's excellent, with readable examples contrasting their approach with that of the "key-value observable" (KVO) approaches of Angular, Knockout and Ember (although he only has examples in Angular and Ember, that I saw).  

One big takeaway I had was that the Virtual DOM was not about performance, but about simplicity.   He cites Rich Hickey's quote, "What matters for simplicity is that their's no interleaving."  He found the key-value observable pattern led to a lot of interleaving, making it unintuitive and difficult to reason about.  He has good examples.    

Yes, he does brag about React performance relative to Ember and Angular (but not KO), specifically, at the beginning of the talk, but later he explicitly takes the focus off of performance.  He even has a counter example, where React is 10x slower than the KVO approach, but he says "it doesn't matter because it's a worst case scenario and it's still under a frame slower".  They trade a frame of performance for some very attractive programming properties, he says.

He also mentions that, out of the box, React does re-rendering synchronously, making it really aggressive with repetitive render calls (no DOM rendering, but a lot of programmatic rendering and diffing), whereas Om (a ClojureScript project that uses React), does it in a RequestAnimationFrame.  

So, for those of us who are comfortable with Knockout and the KVO approach, I'm not sure how a Virtual DOM would help, as it's fundamentally about handling state differently to support programming in a different style.  I have trouble separating it from React.  

[Tony Leeper]

I don't think Knockout is compatible with the concept of a Virtual DOM like ReactJS.  It's a different approach entirely.  I think however that DOM updates triggered from Knockout observables would benefit from being batched to prevent layout thrashing (the main problem ReactJS is trying to solve with the Virtual DOM).  I recently came across this excellent little library that can do just that: https://github.com/wilsonpage/fastdom.  Would be awesome to see something like this included in a future version of Knockout.  The author also wrote a blog post on it here: http://wilsonpage.co.uk/preventing-layout-thrashing/

[dan.an...@gmail.com]

Any more interest on batching DOM updates or a virtual DOM from the core KO devs?  I'm going to play around with the ideas in this thread this weekend, see if I can make anything work.