[ANN] knockout.wrap a super fast ko.mapping replacement

[Anders Rune Jensen]

Hello

Motivation:

The performance of ko.mapping.fromJS(JSObject) is very slow. A simple
test wrapping 500 simple elements takes 700ms in firefox. This seems
to be a known issue[1], that is still there.

Futhermore I don't have the need for ko.mappings ability to update
already mapped objects. This shaves off a huge chunk of the
complexity.

Solution:

A new plugin knockout.wrap with almost the same interface as
ko.mapping (fromJS, fromJSON, toJS and toJSON). The plugin can do
simple mappings from JS to observables and the other way again.
Because the code is much simpler it is super fast:

ko.mappin.fromJS(500JSObjects): 737ms
ko.wrap.fromJS(500JSObjects): 16ms

ko.mappin.toJS(500WrappedObjects): 22ms
ko.wrap.toJS(500WrappedObjects): 5ms

The code is available here: https://github.com/arj03/knockout.wrap

ko.mapping can attach computed functions while it is wrapping. ko.wrap
can do this as well, the syntax is a bit different:

function populateArray()
{
var t = {elements: []};

for (var i = 0; i < 500; ++i)
t.elements.push({id: i, name: "hello" + i});

return t;
}

var computedFunctions = {
"/elements": function(e) {
e.nameLength = ko.computed(function() {
return e.name().length;
}, e);
return e;
}
};

ko.wrap.fromJS(t, computedFunctions);

and:

function populateArray2()
{
var t = {elements: []};

for (var i = 0; i < 500; ++i)
t.elements.push({id: i, data: { name: "hello" + i} });

return t;
}

var computedFunctions2 = {
"/elements/data": function(e) {
e.nameLength = ko.computed(function() {
return e.name().length;
}, e);
return e;
}
};

ko.wrap.fromJS(t2, computedFunctions2);

[1]: https://groups.google.com/forum/#!msg/knockoutjs/NuKs_tawI2M/Mw3HAaXSv60J

[Roy Jacobs]

Hi Anders,

Excellent work! Based on the earlier thread I had been doing some optimization work recently but I hadn't yet committed it. I have now commited it to the branch "arrayperf".

https://github.com/SteveSanderson/knockout.mapping/tree/arrayperf 

Maybe you can also try a small benchmark with the new version? It works best if you supply a key callback. In "spec\issues.js" there's a test that adds 5000 items to an array, maybe you can base a test on that.

https://raw.github.com/SteveSanderson/knockout.mapping/arrayperf/build/output/knockout.mapping-latest.debug.js

Roy

[anders.ru...@gmail.com]

On Thursday, 7 June 2012 18:33:13 UTC+2, Roy Jacobs wrote:

Hi Anders,

Excellent work!

Thanks :-)
 

Sure, the performance is a bit better. Now it is only about 26x slower ;-)

Btw. I can see that you still use the same hashtable (objectLookup) implementation.

http://people.iola.dk/arj/2012/05/30/hashtables-in-javascript/

[Roy Jacobs]

Sure, the performance is a bit better. Now it is only about 26x slower ;-)

Well yes, that's why it wasn't released yet. :) Also I agree the mapping plugin will never reach the same speed as 'wrap' due to all the extra features, but any improvements are probably welcome.

 

Btw. I can see that you still use the same hashtable (objectLookup) implementation.
http://people.iola.dk/arj/2012/05/30/hashtables-in-javascript/

Thanks, that's a useful link!

Can you put the benchmark code up somewhere as well? I'd like to spend some more time optimizing the code and having a target to shoot for always helps.

Roy

 

[Roy Jacobs]

Anders, I've used the pointers on your blog to improve the speed of objectLookup in the mapping plugin as well. It now JSON stringifies the key and uses that to index into a list of buckets that use the old objectLookup. This solves the issues with duplicate keys for different objects and again gave a very nice performance boost.

So thanks again for the info!

Roy

[anders.ru...@gmail.com]

Sure, it's available here:

https://dl.dropbox.com/u/195530/test-knockout.zip

A totally unrelated thing, that I have been thinking about is: to wrap object or not to wrap object. The first time I used knockout and knockout.mapping I found it a bit strange that objects are not wrapped in an observable, while everything else is. It is a bit strange that you sometimes have to call with () and sometimes not. Thinking about it, I can't think of a use case where one would use the fact that an object is observable. But I like that the syntax is uniform. Do you have any thoughts on the subject?

[Roy Jacobs]

Sure, it's available here:

https://dl.dropbox.com/u/195530/test-knockout.zip

Thanks! 

A totally unrelated thing, that I have been thinking about is: to wrap object or not to wrap object. The first time I used knockout and knockout.mapping I found it a bit strange that objects are not wrapped in an observable, while everything else is. It is a bit strange that you sometimes have to call with () and sometimes not. Thinking about it, I can't think of a use case where one would use the fact that an object is observable. But I like that the syntax is uniform. Do you have any thoughts on the subject?

The reasoning behind it is that you'd want to have as few observables as possible for a tree and also to ease the amount of () you'd need to type. However, since the plugin is now being used for highly complex models this is not as important anymore. The reason it's still in is mainly for backwards compatibility and also because it does handle everything in a consistent manner, at least. It's not as 'discoverable' perhaps, but there you go :)

[Mike]

Wow, that's a significant performance gain. I will give it a try. On a somewhat related note here are a few other relating things to mapping:

I am using very cool hash table class for javascript, see  http://www.timdown.co.uk/jshashtable/ and seems to be very fast and I love the API.  If you need a hash table for quick lookups this seems to be the "trick".

Also, I sometimes have circular references in my object model, this library  https://github.com/substack/js-traverse can handle those, and when it see's a leaf or node it's seen before elsewhere in the tree it stops walking that path. I actually took it and chopped it up to its tiny core for just that feature. I know that sometimes people have been burned using mapping with circular references.

Anyway, thanks for the perf tweak ...

[Roy Jacobs]

Mike,

To avoid the circular references the mapping plugin is already marking leafs as visited but it was not doing this in a hugely efficient manner. In his blog post Anders refers to the exact same hashtable implementation as you are now referring to :) I've taken some ideas from both these sources to speed this up.

Roy

[Mike]

Ahh, cool, well, then I'm definitely switching to your mapping plugin. 

[Roy Jacobs]

Ahh, cool, well, then I'm definitely switching to your mapping plugin. 

Okay, but please note that I am *not* Anders who wrote ko.wrap. I'm the developer of ko.mapping. :) I just did some performance tweaks inspired by Anders' work on ko.wrap.

[anders.ru...@gmail.com]

On Friday, 8 June 2012 16:11:38 UTC+2, Mike wrote:

Also, I sometimes have circular references in my object model, this library  https://github.com/substack/js-traverse can handle those, and when it see's a leaf or node it's seen before elsewhere in the tree it stops walking that path. I actually took it and chopped it up to its tiny core for just that feature. I know that sometimes people have been burned using mapping with circular references.

Glad you find it interesting.

Do you have an example of a circular reference object model?

[Mike]

Well, I'm working with (and author if entityspaces.js) and it an ORM for ko. It represents your database in a full hierarchical object model, it's hooked up both ways, from top to bottom and bottom to top.

For instance:

1) Employees.OrdersCollection
2) Order.Employee 

Of you walk into "Employees.OrdersCollection[0].Order.Employee" you can then acccess it's OrdersCollection and walk right back into where you started

[Mike]

Sorry for typo's (repost)

Well, I'm working with (and author of entityspaces.js) and it's an ORM for ko. It represents your database in a full hierarchical object model, it's hooked up both ways, from top-to-bottom and bottom-to-top.

For instance:

1) Employee.OrdersCollection

2) Order.Employee 

If you walk into "Employee.OrdersCollection[0].Order.Employee" you can then acccess it's OrdersCollection and walk right back into where you started ...

[anders.ru...@gmail.com]

I have just pushed a new version that can handle cyclic data structures. Thanks for the example :-)

[x00...@gmail.com]

Hello Andres. Thank you for your plugin, it's doing mapping exactly as I expected. I have only one suggestion: when computedFunction is avaliable don't wrap argument before pass it to function. 

Let me demonstrate it on real example:

JSON of Grid Widget:  

• Settings
• Level
• Column[ ColumnData1, ColumnData2,ColumnData3,..., ColumnData5000 ]
• Column[ ColumnData1, ColumnData2,ColumnData3,..., ColumnData5000 ]
  
• Column[ ColumnData1, ColumnData2,ColumnData3,..., ColumnData5000 ]
  
• Level
  
• Column[ ColumnData1, ColumnData2,ColumnData3,..., ColumnData5000 ]
  
• Column[ ColumnData1, ColumnData2,ColumnData3,..., ColumnData5000 ]
  
• Column[ ColumnData1, ColumnData2,ColumnData3,..., ColumnData5000 ]

And if I don't need to have ColumnData as Observable I need to unwrap already wrapped data.

I believe more flexible way is to allow user to map data manually in compudetFunction if it is required.

Again thank you for doing this job and special thanks for MIT license )

[Anders Rune Jensen]

Hello

Yes I understand that it might not be deseriable to wrap everything.
It would be quite easy to add skip support by modelling it the same
way as computedFunctions. I thought about adding it, but decided
against it because its much cleaner to have everything be an
observable, and we can get away with this because the performance is
so much better than ko.mapping :-)