Optimize ng-repeat for large and static dataset (i.e. printing)

[David Hjelle]

Hi!

Is there a way to speed up ng-repeat or the interpolation or compiling processes for displaying large static datasets since I don't need 2-way data binding? Or perhaps another technique that I'm not aware of?

The background: I've an app in which I display a large dataset (potentially a couple thousand rows) using virtualized rows, inspired by Timothy Sweet's ng-grid. This works great. I'd also like to let the user print the data. Unfortunately, the rendering time required if I use the same template and increase the number of "virtual" rows to the entire data set is much longer than I'd like it to be (starting to approach a minute or more of client-side processing). I was about to assume that the delay was displaying that many DOM nodes in a browser, but I can save the computed HTML from the web inspector, and that renders identically in sub-second times. I've started profiling, but have quite a bit of Angular internals yet to learn.

Thanks for any suggestions or pointers on where to start! Happy to write my own directives if necessary, but I would love to learn from the wisdom of the experts before I start. :-)

David

[Peter Bacon Darwin]

You could easily write your own version of the repeat directive...

Pete
...from my mobile.

--
You received this message because you are subscribed to the Google Groups "AngularJS" group.
To post to this group, send email to ang...@googlegroups.com.
To unsubscribe from this group, send email to angular+u...@googlegroups.com.
Visit this group at http://groups.google.com/group/angular?hl=en-US.
 
 

[David Hjelle]

I'll give it a shot. Thanks for nudging me in that direction!

[Sergey Chico]

I have similar problem. I want to write my own directive that concatenates thousands of rows into one html string and then compiles it and inserts it into <tbody>.

Can someone help and show very basic example representing right way to start?

Here is a start point http://jsfiddle.net/chico_xyzzy/Ufa3C/1/

Custom description must be a text input with apply button sending XHR (contains item id and description) to server. Do I need bi-directional binding for this?

ngRepeat works VERY slow with thousands of items in list.

воскресенье, 25 ноября 2012 г., 23:16:29 UTC+4 пользователь David Hjelle написал:

[Peter Bacon Darwin]

Do your users actually read through and modify over a thousand lines of data in one screen?  I feel sorry for them.

How about using paging or virtual scrolling to limit the number of items on the page?

Pete

--

[Sergey Chico]

Yes I really need to show the full list without pagination. Actually I have some sort of play/stop button for each item.

вторник, 27 ноября 2012 г., 16:27:31 UTC+4 пользователь Peter Bacon Darwin написал:

[Sergey Chico]

For now we are just run loading tests. There will be limit of items setted on server side. Besides that our application should not prevent user to insert as many data as he wants. By the way it is nearly impossible and it is not a recommended way to organize user data. There will be more less than 100 items per list actually. But I still think I can use custom directive that is more fast than ngRepeat for static data + one custom dropdown button per item because as I can see using ngRepeat makes application works exponentially slower the bigger item list length is.

вторник, 27 ноября 2012 г., 16:27:31 UTC+4 пользователь Peter Bacon Darwin написал:

Do your users actually read through and modify over a thousand lines of data in one screen?  I feel sorry for them.

[Witold Szczerba]

I would write very very simple directive with angular.forEach and elm.append which would insert plain markup row by row. Simple and it would work very fast.

Regards,
Witold Szczerba
---
Sent from my mobile phone.

--

[clayton collie]

I'm currently working on yet another social network, and there can be an unbounded size of items for a user's feed. I'm having to handle this in a directive using a lot of jQuery and delegate and less Angular bindings.

[Sergey Chico]

Yep. That is what I told about and exactly what I did. But there is still an issue with my custom kinda-play/stop button. I'll add what I did at jsFiddle tomorrow cause it's already late evening here in Moscow.

вторник, 27 ноября 2012 г., 19:27:21 UTC+4 пользователь Witold Szczerba написал:

[Konstantin Cherkasoff]

Hi!

Do your users actually read through and modify over a thousand lines of data in one screen?  I feel sorry for them.

How about using paging or virtual scrolling to limit the number of items on the page?

Pete

In AdWords one can easily get 500 ads or keywords in the single screen.

It's quite normal to display 500-1000 items per page for UI's like that. Ctrl+F is much more effective than endless pagination or server-side search.

[Peter Bacon Darwin]

But the joy of an AngularJS app is that you can have a search box in the page - like docs.angularjs.org - that means you don't have to display them all at once.

[Konstantin Cherkasoff]

But the joy of an AngularJS app is that you can have a search box in the page - like docs.angularjs.org - that means you don't have to display them all at once.

Yes but in docs.angularjs.org all items are dispayed at the page load.

[Peter Bacon Darwin]

Yes, but they don't have to be.  I could imagine an app where when you first load the page it displays you the top, say 20, items.  If you scroll down it would load up items that become visible and remove items that are not visible.  There could be a search/filter box which would filter the overall list (both visible and invisible) and the screen would update accordingly.  A bit like GMail, where if you do a search it brings up a paginated filtered list, not just a filter of what was visible.

It may not be what you are looking for.

On 28 November 2012 10:20, Konstantin Cherkasoff <k.cher...@gmail.com> wrote:

But the joy of an AngularJS app is that you can have a search box in the page - like docs.angularjs.org - that means you don't have to display them all at once.

Yes but in docs.angularjs.org all items are dispayed at the page load.

--

[Witold Szczerba]

Show us the issue you have with the button, I am curious :)

Regards,
Witold Szczerba

[Sergey Chico]

I had resolve the issue simply by calling $compile function onto the button directive. Nothing special.

среда, 28 ноября 2012 г., 14:28:42 UTC+4 пользователь Witold Szczerba написал:

[Tim Sweet]

I would use a grid with virtualization.

you could also take a look at the way I do virtualization having a canvas and a viewport to scroll so that the user doesn't have thousands of items on screen or stuck with pagination

I have tested this on to 1 Million rows +

http://angular-ui.github.com/ng-grid/

[Robert Aikins]

Did you ever find something that did this successfully? I have a need for something exactly like that. 

[David Hjelle]

What I ended up doing was basically as Tim Sweet suggested: a grid with virtualization. I would have used his ng-grid http://angular-ui.github.com/ng-grid/, except that I was looking to display hierarchical data, which ng-grid didn't support at the time.

I've not yet made a plunk of how to do this—let me know if that'd be helpful!—but here's basically what I did, strongly influenced by ng-grid. There are obviously a million ways to do this. And, frankly, I'd love to know if there are ways to improve performance even further.

1. Make a HTML template grid using ng-repeat.
2. Populate this with a subset of the data, based on how much would actually fit on the screen plus a couple buffer rows at the bottom (and possibly top).
1. This subset should be calculated in the controller from the main data set. I think only putting the subset of data on the $scope reduces the time the dirty checking takes, though I didn't experimentally verify that.
2. Note that you'll be recalculating this subset as the user scrolls, so you'll want a fast function to calculate it.
3. If you have any processing to do on the data (I had angular filters I wanted to apply), you'll probably want to do this and cache the result before doing anything else. That way, you don't need to run the filters as you scroll.
3. Put the grid into two divs: a "viewport" that is the displayed height of the grid, and inside that a "data" div. The grid will live inside the data div.
1. The data div should be absolutely positioned with a "top" corresponding to where the first row of the subset of data ought to be (something like $index * row_height).
2. The viewport should be set to overflow:scroll.
3. I actually absolutely positioned each row. I don't remember exactly why off the top of my head. The top of each row was pre-calculated.
4. Set up a onscroll on the viewport.
1. This should recalculate the subset of data displayed and update the $scope—including the data subset, the top of the data div, etc.
2. By default, since Angular doesn't have a scroll event yet, you'll have to tell Angular to fire the $digest on the $scope yourself. Something like "if (!$scope.$$phase) { $scope.$digest(); } should do the trick. ($digest is much faster than $apply)

Now, I'm still not quite satisfied with this solution. It works—and wasn't nearly as hard as I expected to roll my own—but I'm not yet able to keep framerates at a consistent 30 fps.

It's also possible that something like http://ngmodules.org/modules/abourget-angular might serve your needs better than virtualized grids. It looks—though I've not used it—like it supports one-way bindings instead of Angular's default two-way bindings.

Hope that helps someone!

David

[David Hjelle]

I realized I forgot to explain what I did for the printing side of things. I ended up using document.write into a new window, basically duplicating all the template, etc., code. I ended up with this as IE8 was not performant using any other method of creating HTML that I could find (such as building the DOM with DOM methods or jQuery).

I'm pretty sure one could hand-parse the angular template and then use $.appendTo or document.write or whatever to spit out the appropriate HTML, but I haven't had time to investigate that yet.

David

On Friday, January 25, 2013 5:18:28 PM UTC-6, Robert Aikins wrote:

[Eddie Huang]

Have you done some profiling? From my experience, ng-repeat is usually not the problem, it's if you nest expensive directive inside ng-repeat that's where problem starts.

However, if you are trying to generate a table with massive number of binding, the angular compile time just takes too long. (eg: generate a million divs with a simple binding in each, takes minutes!)

Do it server side? :(