UI virtualization in Knockout

[Itay Neeman]

Hi Everyone,

I tried to find info for this on the docs, but couldn't find anything.

Is there a way to do UI virtualization (using the WPF terminology) in
Knockout? Say I have a list with a million elements in it - I don't
want it rendered all at once, I only want the parts that are visible
(plus some delta above and below) to be rendered. Does Knockout have a
good way to handle this (I've seen it done in JS before - Uki does it
for their framework)?

Thanks!

Itay

[rpn]

Hello-

I think that this is an interesting topic.  I would love to try to work up a sample when I get a chance.

I think that the basic idea would be that you would have an observableArray of data and then a dependentObservable that is a slice of that data.  This is similar to how you would do paging in Knockout.  The dependentObservable would depend on the observableArray of data and it would depend on another observable that told it the index to start slicing data from (and potentially another observable to tell it how many to grab or that could be static).

Then, I suppose you would want to watch the scrolling events to figure out if the user is at the top or bottom of the section to trigger a change in the start index,  You would also likely want to trigger AJAX requests to fetch more data, if you are doing it incrementally.

Should definitely be possible.  I hope this gives you some ideas.  I will give it a shot too, when I get a chance. 

[Itay Neeman]

Ryan,

Thanks for the feedback. I'm very new to Knockout, so I'm still trying to figure my way around. Some thoughts and comments:

1. In a sense, doing UI virtualization for lists only makes sense to me when you have something like a template binding. So you might have something like:

data-bind="template { data: someList, virtualize: true, virtualizeCount:60, name: someTemplate, itemTemplate: someItemTemplate }"

This is because Knockout needs to know how to "re-use" a template in order to change up which specific items are bound to it.

2. Another thing to note is that while virtualization is similar to paging, it is continuous rather than discrete. So if you have 100 items and are currently only displaying items 10-20, then if you scroll one down, you should be displaying items 11-21, etc. Given that I wonder if continuously copying a slice of the observable array is not too expensive.

3. In an ideal world, Knockout would automatically figure out how many items can fit on the screen at once for that list, and multiply it by 3 (one for the visible part, and one each for above and below so you have smooth scrolling). That way, it would always render only what's necessary, rather than you having to figure this out for yourself.

4. You're absolutely right that you need to hook into scroll events to figure out when a certain items goes out of view, so that you can adjust your item offset by 1. At that point, you'd also need to do some interesting things if you wanted to reuse DOM elements rather than keep creating and destroying them.

5. The notion of using AJAX is a great idea - that would actually fall under data virtualization, since you're allowing less than the complete set of data to be present. I'd love to see this too, but first let's start with just UI virtualization.

If you combine data and UI virtualization you can get very performant apps that work great on both mobile and desktop browsers, since you're significantly reducing the amount of memory that's required.

Anyway, I'd love to hear if anyone else has more thoughts. I'll try and see if I can figure out enough of Knockout to know what to do (and I'm certainly a newb when it comes to the DOM), but if anyone else wants to take a crack, feel free.

Itay

[rpn]

I agree with your thoughts.

I don't time to work on it now, but here are a couple previous samples that might give you ideas:

Infinite scroll: http://jsfiddle.net/rniemeyer/gCh8x/  

Infinite scroll w/ajax: http://jsfiddle.net/rniemeyer/swxJt/

Basic paging: http://jsfiddle.net/rniemeyer/5Xr2X/

SimpleGrid from KO website: http://knockoutjs.com/examples/grid.html   source of binding here: http://knockoutjs.com/examples/resources/knockout.simpleGrid.js

[Joel Dart]

this idea has also been on my "someday" list. For a great example of
virtualization to get you started, check out slickgrid:
http://mleibman.github.com/SlickGrid/examples/example4-model.html

Another wrench but one I think is still possible, is floated templated
items and nested templates, but definitely walk before you run.

[rpn]

I was looking at SlickGrid last night a little bit when looking for a sample.  Looks pretty nice.  I was thinking it would also be interesting to bind to SlickGrid from Knockout.

[Itay Neeman]

Thanks for the comments and suggestions. 

Joel, I tried to look at the source for SlickGrid, but it was a bit too much for me :) There's a lot of code in there.

Ryan, I think the main difficulty is figuring out when a single element has scrolled partially into view. If you're currently viewing only elements 10-20, and you scroll a little bit so that element 21 is slightly visible, then you should update the original observable array to have elements 1-31 instead of 0-30. I'm not quite sure how to achieve this in JS + DOM world, though. Part of the problem is that I can't seem to find a way to go for an element in an observableArray to the DOM element that is currently rendering it.

While in theory this problem shares a lot with infinite scrolling, it's quite a bit more tricky because you need to either re-use DOM elements or continuously remove them and add new ones. The easiest way that I can think of implementing it would be using the dependentObservable technique that Ryan mentioned, together with being able to figure out how, given a list, a new element is scrolled partially into view.

[Itay Neeman]

I gave it a quick shot on jsfiddle, but no luck:

http://jsfiddle.net/sFGtY/6/

This will kind of continuously append more items (1 at a time) as you scroll down, and move the "viewport" of the array.

The main problem is that inserting new times changes the scroll, and so what you think the last viewable item is and what is actually the last viewable item is wrong. This is because I believe Knockout is causing the div to be re-rendered, as it removed an element and added a new one at the bottom, which in turn changes the scroll offsets, which in turn throws off what you think is the last element.

I'm open to suggestions on how to do it better.

Itay

[Itay Neeman]

So I had a thought and fixed part of the issue above:

http://jsfiddle.net/8vK9q/1/

This will adjust the scroll offset by the amount of the item that we're removing, so that we're constantly scrolled at the same amount.

The issues that remain have to do with the edge cases of what happens when you have no more items to add. My code also only deals with virtualizing on the way down - it does not virtualize scrolling up. I'll update here if I fix anymore issues.

Itay

[Itay Neeman]

A slightly improved example:

http://jsfiddle.net/gz3r8/2/

If you just keep hitting the down or up arrows on the scroll bar, this will more or less scroll infinitely while always keeping 30 items in the list. The scrolling should be smooth as well, as the offsets are adjusted for you. A few issues remain:

1. Handling the edge case where there aren't enough items.

2. As you can see, the scroll bar is kind of jerky because you keep shifting the offsets on it. I think this is probably OK.

3. If you scroll really fast (which you can do by picking up the scroll bar), you can cause it so the "last element" is never scrolled through, and so the logic to adjust things doesn't kick in). This is by far the biggest issue.

Any thoughts are welcome!

Itay

[Sally Xu]

Hi,

Have you found out a way to do DataBinding with SlickGrid from
knockout? Can you display an observableArray in SlickGrid? It didn't
work when I tried to send an observableArray to the SlickGrid
constructor:

var data = ko.observableArray();
// Populate data

grid = new Slick.Grid($("#myGrid"), data , columns, options);

[rpn]

Hello Sally-

I haven't done much with SlickGrid, but you would want to pass the underlying array in for your data.  So, you would want to pass data() in your case.

I haven't started looking at the SlickGrid events to see what could be bound to get some two-way binding with Knockout going, but here is a sample where it gets initialized from a binding:

http://jsfiddle.net/rniemeyer/A9NrP/

[Sally Xu]

Thank you for the link. I got it working.