Total Size in Heap Snapshot != Memory in Task Manager

[Ramki Gaddipati]

Hi,

I am trying to optimize a javascript app I developed for Chrome.
However, I am not able to understand the chrome memory usage
reporting.
In the task manager, I see that my application is consuming 140MB of
memory. Whereas, when I take a heap dump in the Web Inspector, the
Total memory consumption reported in 37MB (5.45 for code and 31.61 for
Objects).
What does Objects memory size, as reported in Heap Snapshot of Web
Inspector, include? How do I know where the rest of the memory is
being consumed? Can I know, how much of it is being consumed by DOM?

thanks,
+ramki

[Anton Muhin]

Ramki,

Heap profiler reports JS engine memory usage and doesn't account for
C++ allocated objects (like WebKit DOM objects, strings, etc.) There
is ongoing work to report how much memory is retained from JS objects
in C++ DOM world, but it's not in yet.

yours,
anton.

> --
> Chromium Developers mailing list: chromi...@chromium.org
> View archives, change email options, or unsubscribe:
>    http://groups.google.com/a/chromium.org/group/chromium-dev
>

[Mikhail Naganov]

That's right.
In Chrome's Task manager, you can reveal "JavaScript memory" column
(by right-clicking on column titles), it will provide you memory size
allocated to JavaScript virtual machine.

[Ramki Gaddipati]

thanks. The tool, that is in development, will be of great help.

+ramki

[Davit Barbakadze]

On Thursday, April 7, 2011 4:24:09 PM UTC+4, Anton Muhin wrote:

There is ongoing work to report how much memory is retained from JS objects
in C++ DOM world, but it's not in yet.

Was there any progress on this?

 

[Anton Muhin]

+ various people who worked on this

[PhistucK]

From what I can see, they are constantly adding more and more stuff in this regard.

I am following this (a feed) and constantly seeing progress.

☆PhistucK

 

--

[Davit Barbakadze]

I'm having huge problems here analyzing my app. I'm trying to show client-side thumbs for files chosen with input[type="file"]. So what I do is, preload each file with FileReader, resize it with canvas and then display as dataUrl. Memory in Task Manager sometimes goes over 2gb, while Heap size hardly reaches 8mb. No idea how to troubleshoot this. It's ok with 10-20 thumbs, but quickly chokes for numbers over 100. I'm nulling and reusing objects wherever possible, but it doesn't help. Any ideas? Is it logical to assume that Chrome currently is not capable of doing this? 

[Daniel Murphy]

If I've added a feature that keeps it's own in-memory cache (up to 6 Mb currently), should I be contacting someone and including this in some memory stats?

- Daniel

[Alexei Filippov]

There's an experimental effort on reporting memory usage breakdown by components. It is still under development, but if you want to look at it do the following:

1. grab the latest chrome canary build

2. run it with --enable-devtools-experiments

3. open the devtools and under its setting (a small gear at the bottom right) you should see the experiments tab.

4. enable the 'Live native memory chart' option

5. restart the devtools (just close and open its window)

You should now see the native memory chart on the profiles tab.

[Davit Barbakadze]

Thanks for useful tip. Very nice info there. But still not helpful in my case. What I see that every time I preload more image thumbs "Memory cache resources" gets pushed up and pushes "Other" as well. But after mere minutes or some action on the page "Memory cache resources" gets dropped back to initial value, while "Other" stays :|

Now what exactly constitutes "Other"?

[Alexei Filippov]

Thanks for trying it. There was a problem with reporting cached images properly. Perhaps it is related to your case. It has been fixed a couple days ago, and the fix might not yet get into the canary build (I can't check it right now). Anyway it would be interesting to take a look at your page. Can you share the url?

Other size corresponds to the memory that is not yet instrumented. Ideally if everything is instrumented it should be zero.

[Alexei Filippov]

Yes, could you please point us to the code location or revision so we can take a look. Thank you!

[Dai Mikurube]

Sorry for my off-topic.

The native memory chart is very nice.  Do you have some design doc, or could you tell me a good point to start reading code?  I wonder if we could share the status sometime.  I'm working on memory profiling, too.

--

Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

--

Dai MIKURUBE

   dmik...@chromium.org

[Davit Barbakadze]

Here is the app itself: http://goo.gl/emE2m. Might be hard to follow. It is meant to become multi-functional file-uploader on top of XHR2 and FileAPI multi-runtime pollyfills. Trying to optimize it now, avoid circular references, memory leaks, etc. When on the page choose thumbnails view and add some files. Logic flow underneath is quite intricate, but I think simpler app with the same logic might encounter the same problem. Meanhwile I will try to wrap up a simpler example.

[Yury Semikhatsky]

On Fri, Aug 10, 2012 at 12:53 PM, Davit Barbakadze <jay...@gmail.com> wrote:

Here is the app itself: http://goo.gl/emE2m. Might be hard to follow. It is meant to become multi-functional file-uploader on top of XHR2 and FileAPI multi-runtime pollyfills. Trying to optimize it now, avoid circular references, memory leaks, etc. When on the page choose thumbnails view and add some files. Logic flow underneath is quite intricate, but I think simpler app with the same logic might encounter the same problem. Meanhwile I will try to wrap up a simpler example.

Note that FileAPI related stuff is not instrumented yet and I suspect that it might account for some considerable part of the process memory in case of your application. I'm going to play with your App and see what data is missing in the memory instrumentation. Having a good example where the Other part grows big would be very helpful for us. This way we could better understand which parts of WebKit may take a lot of memory and instrument them.

Yury

 

On Friday, August 10, 2012 12:04:15 PM UTC+4, Alexei Filippov wrote:

Thanks for trying it. There was a problem with reporting cached images properly. Perhaps it is related to your case. It has been fixed a couple days ago, and the fix might not yet get into the canary build (I can't check it right now). Anyway it would be interesting to take a look at your page. Can you share the url?

Other size corresponds to the memory that is not yet instrumented. Ideally if everything is instrumented it should be zero.

On Friday, August 10, 2012 11:32:48 AM UTC+4, Davit Barbakadze wrote:

Thanks for useful tip. Very nice info there. But still not helpful in my case. What I see that every time I preload more image thumbs "Memory cache resources" gets pushed up and pushes "Other" as well. But after mere minutes or some action on the page "Memory cache resources" gets dropped back to initial value, while "Other" stays :|

Now what exactly constitutes "Other"?

On Thursday, August 9, 2012 10:05:51 PM UTC+4, Alexei Filippov wrote:

On Thursday, August 9, 2012 9:03:03 PM UTC+4, Davit Barbakadze wrote:

On Thursday, April 7, 2011 4:24:09 PM UTC+4, Anton Muhin wrote:

There is ongoing work to report how much memory is retained from JS objects
in C++ DOM world, but it's not in yet.

Was there any progress on this?

 

There's an experimental effort on reporting memory usage breakdown by components. It is still under development, but if you want to look at it do the following:

1. grab the latest chrome canary build

2. run it with --enable-devtools-experiments

3. open the devtools and under its setting (a small gear at the bottom right) you should see the experiments tab.

4. enable the 'Live native memory chart' option

5. restart the devtools (just close and open its window)

You should now see the native memory chart on the profiles tab.

--

[Yury Semikhatsky]

On Fri, Aug 10, 2012 at 12:11 PM, Dai Mikurube <dmik...@chromium.org> wrote:

Sorry for my off-topic.

The native memory chart is very nice.  Do you have some design doc, or could you tell me a good point to start reading code?  I wonder if we could share the status sometime.  I'm working on memory profiling, too.

There is no design doc. All the instrumentation calls go through Source/WebCore/dom/MemoryInstrumentation.h which may be a good start point. Also there is an umbrella bug for the effort where you can track changes: webk.it/87262

Yury

[Davit Barbakadze]

Here is a simplified version: http://goo.gl/bzHV7. Although I should say now, that in this simplified example, setting resources to null (something that I thought I was doing in original app as well), definitely triggers some garbage collection, and "Other" periodically drops down, quite noticeably actually, and while memory usage increases as one adds more images, it is not that dramatic overall (in comparison with original problem). Probably I leak in there somewhere after all. Not sure how to track it down though... :|

"Other" drop even more noticeably if I use canvas element as a thumb directly, but I might require to export it to another Image eventually. Anyway it is not about the flaw in the app (which is obviously there), it's about how to leverage the power of all those tools to track that flaw down. 

[Davit Barbakadze]

Looks like I was keeping a reference to the canvas element all way through, buried deep in the object...

How one is supposed to track such leakage down with Chrome tools?

[Yury Semikhatsky]

On Fri, Aug 10, 2012 at 7:54 PM, Davit Barbakadze <jay...@gmail.com> wrote:

Looks like I was keeping a reference to the canvas element all way through, buried deep in the object...

How one is supposed to track such leakage down with Chrome tools?

You should be able to find live HTMLCanvasElement and their retainers using heap profiler. Shallow size of such objects will include only the size of the JS wrapper not the size of the native object though.

Yury

[Davit Barbakadze]

But this basically means that they will effectively slip one's attention. There should be a way to report the size of the objects as they occupy it in "real" memory?

[Dai Mikurube]

Thanks, Yury.  I'm taking a look at MemoryInstumentation.h.

--

Dai MIKURUBE

   dmik...@google.com

[Arijit chattopadhyay]

Is there someway that I can automatically dump the entire heap into a file from the C++ code?

Thanks,

Arijit.