Memory usage of JavaScript object properties

[joh...@grob.org]

I'm trying to figure out how much memory different objects take up in Chrome. To be more specific, I'm interested in objects that were parsed from JSON. I've been using the DevTools memory profiler for that, and I've found some interesting things. However, I'm getting really confused about the memory overhead incurred by each object property/key (including the key string).

One of the experiments I did was create two objects with the same number of properties and the same amount of data in the property values, but different property key lengths (attachments obj1.jpg and obj2.jpg). Note that both keys and values are random strings, so there are no duplicate key names. The memory profile reports that both objects have the exact same retained size...weird!

So I made a number of objects with different amounts of properties to see how the memory the profile attributes to "properties" changes (attachment obj3.jpg). So there seems to be sudden jump in the size of "properties", in this case between when having more than 85 properties. The same behavior is observable with less or more properties: the size of "properties" remains at a certain level and then suddenly jumps by a lot when some magic number of properties is hit. That suggests that Chrome is allocating a block of memory to store the current and potential future properties, and when it runs out of memory in that block, it increases the size of the block significantly (factor of 4?).

Ok, so 1564 bytes for 85-ish properties, that's about 18 bytes per property. With keys of 10 characters (as in my experiment), that seems to make a lot of sense. So if I double the key size to 20 characters, the jump from 1564 to 6172 bytes should happen somewhere around 43 properties, right? But no! Even with keys twice as long, the jump still happens at 85 properties (see obj4.jpg). Same happens with keys of 50 characters.

Now I'm at loss for answers. Where are the property names stored? Is the required memory accounted for at all in the memory profiler? Any ideas?

Best,
Joe

(sorry for the crummy screenshots btw)

[Ilya Tikhonovsky]

If you interesting in the V8 internals you can watch Lars Bak video about v8 internals.

http://www.youtube.com/watch?v=hWhMKalEicY

The structure of a V8 object is very complex and may change on the fly.

As example the internal representation of an array might be converted from FixedArray into Dictionary if the array has many holes when you insert new item into it.

Also the internal representation of a dictionary with number only keys might be converted into FixedArray just for space and performance.

There are many other optimizations in v8. If you are interesting in the topic you can dig into the sources. https://code.google.com/p/v8/

Regards,
Tim.

--
You received this message because you are subscribed to the Google Groups "Google Chrome Developer Tools" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-chrome-develo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-chrome-developer-tools/3cfcd97a-47c7-42fe-9a4e-e11f0b705db1%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[joh...@grob.org]

Thanks for the video link, Tim! I haven't seen this one yet, although I watched a couple of others about optimization for V8. Seems like a topic about which a lot can be said :-)

By using random strings for both property names and values (and having lots of properties in general), I was hoping to prevent optimized object representations from being applicable and force the object to Dictionary mode. But in any case, I guess the engine always has to keep the full property names around, since the object may need to fall back to Dictionary mode. That's why I think the storage for the property names should be accounted for somehow.

Best,
Joe

Regards,
Tim.

To unsubscribe from this group and stop receiving emails from it, send an email to google-chrome-developer-tools+unsub...@googlegroups.com.

[al...@chromium.org]

Hi Joe,

On Friday, November 1, 2013 2:59:00 PM UTC+4, joh...@grob.org wrote:

I'm trying to figure out how much memory different objects take up in Chrome. To be more specific, I'm interested in objects that were parsed from JSON. I've been using the DevTools memory profiler for that, and I've found some interesting things. However, I'm getting really confused about the memory overhead incurred by each object property/key (including the key string).

One of the experiments I did was create two objects with the same number of properties and the same amount of data in the property values, but different property key lengths (attachments obj1.jpg and obj2.jpg). Note that both keys and values are random strings, so there are no duplicate key names. The memory profile reports that both objects have the exact same retained size...weird!

So I made a number of objects with different amounts of properties to see how the memory the profile attributes to "properties" changes (attachment obj3.jpg). So there seems to be sudden jump in the size of "properties", in this case between when having more than 85 properties. The same behavior is observable with less or more properties: the size of "properties" remains at a certain level and then suddenly jumps by a lot when some magic number of properties is hit. That suggests that Chrome is allocating a block of memory to store the current and potential future properties, and when it runs out of memory in that block, it increases the size of the block significantly (factor of 4?).

Ok, so 1564 bytes for 85-ish properties, that's about 18 bytes per property. With keys of 10 characters (as in my experiment), that seems to make a lot of sense. So if I double the key size to 20 characters, the jump from 1564 to 6172 bytes should happen somewhere around 43 properties, right? But no! Even with keys twice as long, the jump

There seems to be a flaw in the calculations. A 10 character string takes 24 bytes (I can see you're on a 32-bit platform, right), so for 85 keys it'd need at least 85*24=2040 bytes, i.e. 1564 won't fit all the keys. I wonder you are keeping other references to the keys from somewhere else, e.g. from referenceObjParsed object? If yes the strings won't be counted as a retained by the properties array.

-Alexei

[joh...@grob.org]

Hi Alexei

There seems to be a flaw in the calculations. A 10 character string takes 24 bytes (I can see you're on a 32-bit platform, right), so for 85 keys it'd need at least 85*24=2040 bytes, i.e. 1564 won't fit all the keys. I wonder you are keeping other references to the keys from somewhere else, e.g. from referenceObjParsed object? If yes the strings won't be counted as a retained by the properties array.

Sort of: referenceObjParsed is a JSON stringified and parsed version of referenceObj. Is V8 able to handle that smartly?

If I don't create such a copy, things look a little different, but even more confusing (to me, at least). For example, "properties" has a different shallow and retained size in that case (obj5.jpg). I would understand if that were the case when the object had been copied (like in the examples I've pasted previously), but in this case, it hasn't. Also, I was surprised to see that sometimes an object with more properties uses less memory than an object with fewer properties (obj6.jpg and obj7.jpg).

I've pasted the source code that generates referenceObj and referenceObjParsed below.

Cheers,
Joe

        var stringChars = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ';
        function randomString(length) {
            var result = '';
            for (var i = length; i > 0; --i) result += stringChars[Math.round(Math.random() * (stringChars.length - 1))];
            var a = {a: result};
            return JSON.parse(JSON.stringify(a)).a;
        }

        window.referenceObj = {};
        window.referenceObj['key5'] = {};
        for (var i = 0; i < 100; i++) {
            window.referenceObj['key5'][randomString(5)] = randomString(100);
        }
        window.referenceObj['key10'] = {};
        for (var i = 0; i < 100; i++) {
            window.referenceObj['key10'][randomString(10)] = randomString(100);
        }

        var keyLength = 10;
        for (var i = 10; i <= 100; i+=5) {
            window.referenceObj['props'+i] = {};
            for (var j = 0; j < i; j++)
                window.referenceObj['props'+i][randomString(keyLength)] = randomString(10);
        }
        window.referenceObjParsed = JSON.parse(JSON.stringify(window.referenceObj));

 

[al...@chromium.org]

On Friday, November 1, 2013 6:14:20 PM UTC+4, joh...@grob.org wrote:

Hi Alexei

There seems to be a flaw in the calculations. A 10 character string takes 24 bytes (I can see you're on a 32-bit platform, right), so for 85 keys it'd need at least 85*24=2040 bytes, i.e. 1564 won't fit all the keys. I wonder you are keeping other references to the keys from somewhere else, e.g. from referenceObjParsed object? If yes the strings won't be counted as a retained by the properties array.

Sort of: referenceObjParsed is a JSON stringified and parsed version of referenceObj. Is V8 able to handle that smartly?

I'm not sure what you mean under "handle smartly". But I can see that after stringify-parse a string gets copied.

 

If I don't create such a copy, things look a little different, but even more confusing (to me, at least). For example, "properties" has a different shallow and retained size in that case (obj5.jpg).

Shallow size is the size of the properties array itself (an array of references to both keys and values). Retained size adds sizes of all objects referenced from this array (and not references from elsewhere, to be precise). These are strings in your case.

 

I would understand if that were the case when the object had been copied (like in the examples I've pasted previously), but in this case, it hasn't. Also, I was surprised to see that sometimes an object with more properties uses less memory than an object with fewer properties (obj6.jpg and obj7.jpg).

This is an internal magic of how v8 allocates internal arrays for a dictionary. Feel free to look at exact implementation details at https://code.google.com/p/v8/source/browse/trunk/src/objects.h

[joh...@grob.org]

If I don't create such a copy, things look a little different, but even more confusing (to me, at least). For example, "properties" has a different shallow and retained size in that case (obj5.jpg).

Shallow size is the size of the properties array itself (an array of references to both keys and values). Retained size adds sizes of all objects referenced from this array (and not references from elsewhere, to be precise). These are strings in your case.

Aha, that makes perfect sense! However, with that in mind I noticed an issue: in Chrome 30, the retained size properly takes the size of the object payload into account as you described, regardless of whether I create a stringify-parse copy of the object or not. In Chrome 32, this is also the case if I don't create the stringify-parse copy. If the copy is created, however, the profiler reports Shallow Size == Retained Size == size of the properties array only, for both referenceObj and referenceObjParsed. This is how I obtained the first batch of screenshots in this thread (obj1-3.jpg), and I assume it's what you meant in your earlier post when you said the calculation was flawed.

Thanks for your explanations!

Joe