Very slow (non linear) performance with knockout.mapping on large item set

[ronan thibaudau]

Hello,
First of, thanks for this great library.
I have a somewhat large-ish dataset (2400 products, each is an object
with 5 properties), using knockout.mapping.fromJS(data,model) on it is
"very" slow. I've made a small timing method and time grows non
linearly as i select more items which surprised me, what's happening?
is it a bug in the algo making it non linear?
You can see exemple calls and time is growing fast relative to items
count making it very quick at first, reasonable on most data sets, but
having an unreasonable worst case for us
http://rt-informationtechnology.com/KnockoutMappingTimes.png
For information when using it against our whole set it takes approx
108 000 ms (for 2400 items, vs 7 for 1 item!)

[ronan thibaudau]

I've made some progress in finding the issue, it seems that the
mapping plugin checks every item against every other item in the
array, which i could understand if it was updating a 200 array
observable with 200 elements, but not so much as it's simply updating
a new array with 200 elements (the viewmodel is empty at this point)
As you can see here : http://rt-informationtechnology.com/PowerOfTwo.png
We're down to 200 calls (for 200 elements) and suddently jump to 40
000 calls for an inner function taking most of the time (and 200 * 200
just so happens to be 40 000).
This would totally explain the non linear progression making the
mapping plugin great but unusable for large datasets, i'm afraid i'm
not quite proficient enought in knockout to fix this as i don't know
the structure of the mapping plugin at all, is this a quick fix
someone could commit to or could someone guide me toward the necessary
steps?
Best regards.

[Scott Messinger]

Great work identifying the slowness. Hopefully Roy or Steve will be able to provide some suggestions.

-Scott

[Roy Jacobs]

Ronan mailed me directly to notify me of this thread, here's a summary of my advice so far:

Hi Ronan,

Updating large numbers of arrays is not really what the mapping plugin is designed for, but have you looked at using the create/update callbacks? Using them, you can specify per-property how you would like to create/update them. In your case, you could override the creation of the array.

Let's say your array is called 'items'. Then, your mapping options would be something like:

var options = {

   items: {

      create: function(options) { return customCreateItems(options.data); }

   }

};

Inside this callback, you could do something like:

var customCreateItems = function(items) {

   var result = [];

   for (var t=0;t<items.length;t++) {

      result.push(ko.mapping.fromJS(items[t]); /* This maps a single item */

   };

   return result;

};

In other words, you would be bypassing the array mechanism of the mapping plugin and manage the array yourself. Your update callback would be identical, probably.

However, normally this only works for properties, so what you can do is wrap the array in a regular property and then use the callbacks.

I hope this helps! I will still try to look at solving the power-of-two complexity in the plugin, but for now this should also work.

If you need more suggestions, let me know.

With regards,

Roy

[Scott Messinger]

Thanks, Roy! That’s helpful!

[Claudio Pallone]

Hi Roy,

Could you please create a simple sample solution that we can downlod
that shows you solution?

Cheers

C

[Roy Jacobs]

Sure, I'll try to put up a jsfiddle or a blog post that illustrates this approach.

Roy

[Claudio Pallone]

Thank Roy,

We really appreciate that.

Cheers

C

On Mar 8, 2:17 pm, Roy Jacobs <roy.jac...@gmail.com> wrote:
> Sure, I'll try to put up a jsfiddle or a blog post that illustrates this
> approach.
>
> Roy
>

> > > Roy- Hide quoted text -
>
> - Show quoted text -

[Claudio Pallone]

Hi Roy,

Have you been able to put a sample so that we can understand how to
deal with the situation above?

Thanks

C

On Mar 8, 3:17 pm, Roy Jacobs <roy.jac...@gmail.com> wrote:
> Sure, I'll try to put up a jsfiddle or a blog post that illustrates this
> approach.
>
> Roy
>

[Anu]

Hi,

I am facing the similar issue with plug-in. Were you able to figure it out the issue. Appreciate your response.

Thanks in advance.

Regards

Anu