Performance of Large Grids

[Matt Fowles]

All~

I have started a small project, the end goal of which is to have something like a crossword puzzle.  I have a basic grid that updates; however, the performance degrades considerably with large grids (I would like to be able to handle sizes around 100x100).  The current implementation is extraordinarily naive and takes ~3s to update a 50x50 grid.  I am almost entirely new to the world of web and js stuff, so I am looking for suggestions about how to change the basic design to make it scalable.

https://github.com/fowles/meteor-crossword

Thanks,

Matt

[Dror Matalon]

with 50x50 you're dealing with 2500 items, with 100x100, 10,000.
Rather than set up 2500 or 10,000 event listners, use a single
listener for the board, which you are doing, and when an item is
clicked, figure out which item was clicked.

> --
>
>

[Matt Fowles]

Dror~

When I make the change you are suggesting, I see no appreciable difference in the speed of refreshes.

b6cd5362f0222822a02b1d6ce052626d50d94a80

Matt

--

[Leopold ODonnell]

Is it possible that you still have event listeners on each of your game cells? Have you created one listener with a simple hit-test?

Otherwise it's time to do some debugging/tracing

Good Luck,

Leo

[David Greenspan]

Meteor will set up one listener as long as the event is attached to Template.board, so you can feel free to use a specific selector and take advantage of the data context.  Using a sequence of classes could be buggy because the browser could re-order them.

As for the larger question of speed... this is new territory for Spark, but the biggest algorithmic gain would come from being able to invalidate different squares of the board individually -- either putting them in different documents in the database, or doing something fancy with Deps.  I'm not sure exactly what contributes the most to the current method being slow, though.

-- David

--

[Matt Fowles]

All~

I tried pushing the elements into a collection and then reference them via id from the board.  This does make redraw for clicking individual cells speedy; however, the initial draw of the table slows down even further to a painful degree.  Throwing chrome's profiler at it, I see

34.26% 64.78% LocalCollection.Cursor._getRawObjects

20.19% 36.15% LocalCollection.Cursor.forEach

20.19% 36.15% LocalCollection.Cursor.fetch

20.19% 36.15% LocalCollection.findOne

20.19% 36.15% _.extend.findOne

20.19% 36.15% Template.cell.elem

which looks to me like findOne() is taking most of the time.  Is LocalCollection.findOne() doing a linear search of items?

57d741bfbff82e184cc2a265015179832996c01e

Matt

--
 
 

[David Glasser]

Findone is doing a linear search unless the query is by id: minimongo does not yet have indexes.

--
 
 

[Matt Fowles]

David~

The find one call in question is:

    return Elements.findOne({_id:this})

interestingly, when I profile the version that does not use a separate Element collection, my profile tells me that I spend >60% of my time in "(program)".  My searching seem to imply this happens when the dom is manipulated in place too much and the better thing to do is built the dom separate from the window and then tie it into the main dom once.  Not sure if that is an accurate read of things though.

Matt

--
 
 

[David Glasser]

I think that {_id: this} might be less optimized than just putting the string in, ie Elements.findOne(this). (Obviously this should be improved.)  (Also, there might be a slight issue where when a string is put into "this" it gets converted into a js "String Object", which is annoying thing that doesn't work how you expect, so you might need Elements.findOne(this.toString()) which converts back from "String Object" to "string literal".)

--
 
 

[David Greenspan]

I agree you should write Elements.findOne(String(this)) in case it's faster than Elements.findOne({_id: this}).  (The String function converts its argument to a string primitive.)

Notes:

The main hotspot is responding to the DDP messages from the server.  (mongo-livedata/collection.js, registerStore({update:...}))

When changing the board size, the calls to Elements.insert also take a while.

--
 
 

[Matt Fowles]

David~

I couldn't make Elemenets.findOne(this) work (cause I didn't know about the String Object this).  Changing to Elements.findOne(this.toString()) fixes that.

With that optimization the performance of storing Element references in the board becomes similar to the performance of storing literal elements in the board.  This also allows updating individual element very efficiently.

It still takes ~2-5s to load an existing board of 50x50.  The profiler in chrome attributes most of the time cost to "(program)".

52.28% 52.28% (program)

6.70% 6.70% set innerHTML

6.03% 6.76% LocalCollection._compileSelector

5.58% 5.58% (garbage collector)

4.30% 10.99% DomUtils.htmlToFragment

2.22% 23.98% _.extend.materialize

Matt

--