KnockoutJS performance with large amounts of data

[blu...@gmail.com]

Hi All

Does anyone know if there any plans to increase the speed performance of Knockout when dealing with large amounts of data (150+ rows with 10 columns per row) that needs to be displayed on screen at once?

[Ian Yates]

That's a scenario where you can make KO perform really poorly - but you could make just about any framework perform poorly.

Could you post a rough outline of how you're doing the binding?  It should be quite OK to make KO render that much.  If you're using <table> elements you can save a lot of rendering time by using fixed layout, etc rather than letting the browser automatically determine a layout too.

On Sunday, March 29, 2015 at 3:19:56 PM UTC+10, blu...@gmail.com wrote:

[Ralph Becket]

I've re-written an application showing about an order of magnitude more data than this, updating every few seconds.  I'm about to rewrite the bulk of the page to not use Knockout, since this is about an order of magnitude past Knockout's design parameters!  (The rest of the logic will stay Knockout-based).

That said, here are some things which have helped:

(1) make sure your computeds are doing the absolute minimum amount of recomputation.  You may need to selectively override .equalityComparer on observables/computeds to perform better equality checking before issuing update notifications.

(2) Make updates propagate incrementally (say, update twenty rows every twenty milliseconds rather than the whole lot in one go).

(3) Constantly measure memory and CPU performance using whatever profiling tools you have.

(4) Try your application out regularly using different browsers.

Hope this helps.

[Ian Yates]

All excellent points!

Another one, to reinforce points (1) and (2), is to avoid doing lots of .push() and .pop() on observable arrays.  If you have a lot of that to do, grab a copy of the underlying array, do your pushes and pops to it, and then tell the observable array that it's valueHasMutated().

[o.viau...@gmail.com]

The default knockout template engine is not the perfect choice when dealing with large amounts of data.

What you can do is extend knockout to use the underscorejs templating engine, which is way faster. I that in a calendar application where 365 cells already posed a problem to the default knockout template engine. With the underscore template engine its incredible fast.

You should find everything here:

http://stackoverflow.com/questions/18410055/using-underscore-template-with-knockout-using-interpolate-due-to-asp-net

(or google for using underscorejs template engine with knockout)

that should do the trick.

[Ralph Becket]

Just wanted to report on what happened going off the Knockout reservation: it worked like a charm.

To put this in context, my task was to display

- dozens to hundreds of tables

- each with up to fifty columns and six hundred rows

- with each cell having data, dynamic CSS classes, and hover and click behaviours

- supporting sorting of tables by column and by row

- the whole lot being udpated every three seconds or so.

My first attempt using Knockout worked, but was woefully unresponsive, even after several rounds of re-engineering for optimisation.

The new version directly generates and updates the many thousands of DIV cells.  The only concession to responsiveness is to limit the number of updates per "timeout" to around five hundred.  This gives the illusion of instant response times with no perceivable ripple effect as updates propagate.  The hover and click logic is handled by a pair of functions bound to the top-level container DIV.  All the other controls on the page, templating etc., is still handled nicely using Knockout.  Interestingly, the price I've paid for this direct approach is a modest reduction in the size of my code!

[Jean-Sebastien Lemay]

Hi Ralph,

Do you mind giving me more details on how you accomplished this? Specifically, what did you do when you say "The new version directly generates and updates the many thousands of DIV cells." Are you using a library other than Knockout to generate the HTML/DOM, yet still use Knockout to create observables that shape the resulting HTML/DOM?

[thatsas...@gmail.com]

I know this thread is long gone, but I wanted to see if you had any details on how you sped up the rendering while keep the knockout templates.

[levani]

I'm having a similar problem. In my app I need to display about 50 tables with 10-15 rows and about 10 columns each. Rows and columns are generated dynamically for each table. It takes about 4-5 seconds to show those tables in chrome. That's too much time I think.

Currently the only 'solution' I was able to find is to display first few (not more than 10) tables right away and the rest in a 200ms timeout. This creates an illusion of a more responsiveness but my app is still unusable for that 5 seconds, until the render is fully over.

What would be a more proper solution to this? Will switching to other rendering engine make any difference?

[Slavko Parezanin]

What about on document ready 

- first render tables visible in ViewPort 

- after that render other tables

https://emberway.io/ember-js-detecting-if-a-dom-element-is-in-the-viewport-eafcc77a6f86#.2l52so9z4

[levani]

That's what I'm doing right now but this solution is still far from being ideal. The problem in this case is that the whole page "freezes" before the render in complete. Not even the scroll is working during that 4-5 seconds period.

[Gunnar Liljas]

Unfortunately that freeze is expected, when creating many or large tables which are very dynamic. Having dynamic columns is a complete disaster for DOM based rendering, so if at all possible, try to make that part slightly more static. I serve dynamic templates from the server to accomplish this, but it should be possible to e.g use a knockout component to create the template at runtime.

/G

--
You received this message because you are subscribed to the Google Groups "KnockoutJS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to knockoutjs+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Gustavo Hispagnol]

Have you tried knockout-repeat binding? It did the work for me, almost instant rendering times with 100-200 resonably complex items.

[Ilya Kutukov]

We solved a problem with paged displaying and filtering/managing tens of thousands of records by using throttled computed arrays with filtering logic inside to which DOM was binded.

http://knockoutjs.com/documentation/throttle-extender.html

Then we took mobile and desktop application version and tweaked throttling values to make GUI not very laggy using browser profiler and prints inside computed signalling that recomputation occur. And got a sweet result.

There are some not significant performance problems in framework itself but our major problem was using granular instances/arrays update that is followed by unnecessary computations and reflows during data update phase.

[Hasen el Judy]

I've been able to display  > 10k rows

There are a few tricks I used

First, use the repeat binding (3rd party) instead of foreach.

Second, don't put a scrollbar on the table or its container. Put the scrollbar on a separate div. Listen to changes in the scrolling position and use it to derive the visible portion of the table.

For example, if you decide each row takes up 30 pixels, and the scrollTop is at 300, then you start displaying rows from index 10.

For this I had a simple custom binding to map scrollTop to an observable. Also another binding so that wheel events on the table would change the scrollTop observable (and it would reflect immediately on the scrollbar).

Apply the same logic to columns.

The result is that it's pretty fast. There's also not much upfront initialization time.

The only downside (if you consider it so) is that when you scroll down, you only scroll one row at a time, kind of like a spreadsheet.