Garbage collection tuning

[Talbot McInnis]

We have a memory intensive web application and V8's Javascript Garbage Collection frequency is proving to be too relaxed for our needs. We can watch the memory in the Timeline (on Dev Tools) grow as we perform memory intensive DOM operations in Javascript, however it can take several minutes for large sets of memory (50+ MB) to be garbage collected. There is no leak, we always return back to baseline memory eventually and we can expedite that by pressing the "Collect Garbage" button on the Dev Tools.

The real problem is that our application is so memory intensive that users can push it beyond it's 1.2GB process memory limit which leads to crashing the render process.

I have tried many of the javascript flags to tweak GC, but none provided satisfactory results.

Is there a way to adjust the garbage collector to become more aggressive, or failing that we could setup a monitor thread in our host application to monitor memory on the render process and trigger whatever method the "Collect Garbage" button triggers in the Dev Tools. We have tried exposing "window.gc()" which seems to help a bit, but does not appear to be as aggressive as clicking the "Collect Garbage" button. Is there a way to access the method that this button calls?

We are using CefGlue.  I am willing to build a custom libcef.dll but I find it hard to believe there isn't a better way.  Shouldn't the cef project protect itself against javascript?  Couldntthe GC force a pause and collect when memory exceeds 700MB?

[Demetrios Papadopoulos]

Besides trying to tweak the GC, have you considered using any
techniques to reduce the memory required by the application. Even if
you manage to tweak the GC to make it work as needed, you can't expect
real users to be running Chrome with the same flags.

A very interesting and elegant approach I have found are resource
pools, see detailed article at
http://www.html5rocks.com/en/tutorials/speed/static-mem-pools/. Not
all applications are good targets for applying this technique, but it
might be worth checking it out.

> --
> --
> Chromium Discussion mailing list: chromium...@chromium.org
> View archives, change email options, or unsubscribe:
> http://groups.google.com/a/chromium.org/group/chromium-discuss
>

[PhistucK]

Does your application work fine in normal Chrome? If it does, then perhaps you should post this to the CEF group.

If not, then you may want to file an issue at crbug.com (or maybe V8) for it?

Also, Chrome now has a 64 bit edition for Windows (already) and Macintosh (soon), perhaps that will let you leverage more memory?

☆PhistucK

--

--
Chromium Discussion mailing list: chromium...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-discuss

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-discu...@chromium.org.

[PhistucK]

Also, you may want to split your work into a number of Workers. Assuming the user has enough memory, each worker will increase the total memory available for your application, if I remember correctly.

☆PhistucK

[PhistucK]

​It sounds like a bug... waiting a few minutes for 50 MB to be collected sounds like a lot of time.​

The browser strives to be the platform for any and all of the applications (pretty much). If memory intensive applications are crashing, it makes browsers a non viable platform for such applications, which is clearly not the intention.

☆PhistucK

On Thu, Oct 30, 2014 at 9:24 PM, Siva P Thumma <siva...@gmail.com> wrote:

PhistucK, Can this be treated as a bug to be frank?

[Talbot McInnis]

Great article.  Interesting graph there too.  I would love to see the GC recollecting that often!  We have reduced our memory allocation a little, but in the majority of cases we are at the mercy of jquery.  It seems like even simple DOM objects being added through jquery require huhge amounts of memory.  We are dealing with selectable, scrollable lists of around 1300 items and usage grows up to 60MB each time we change the list.

[Talbot McInnis]

No, we see the exact same behaviour in normal chrome.  Ill explore the bug report avenue.  Thanks.

[PhistucK]

Frankly, since it looks like you are targeting Chrome (or other modern browsers) specifically, I would not even use jQuery.

While it may simplify some work, it has a huge performance (and memory, as you experience) impact.

For example, on mobile - projects on which I am working, do not use jQuery at all.

The standard platform has most of the APIs already. Specifically, querySelector and querySelectorAll brings much of the functionality, when combined with Array.prototype.map/forEach/filter and the rest.

I add them to the prototype of the result of querySelectorAll. NodeList.prototype.map = Array.prototype.map and so on and you can just use them like this - document.querySelectorAll("bla").forEach(...), it is very convenient (I also have $.query or whatever as a shortcut for document.querySelectorAll, so the extra code length ceases to be an issue).

I am curious - what does it simplify so much that makes you prefer it over vanilla JavaScript solutions?

Or do the various jQuery plugins make it worth it (there is possibly a vanilla JavaScript library as well)?

☆PhistucK

On Thu, Oct 30, 2014 at 9:36 PM, Talbot McInnis <talbot...@gmail.com> wrote:

Great article.  Interesting graph there too.  I would love to see the GC recollecting that often!  We have reduced our memory allocation a little, but in the majority of cases we are at the mercy of jquery.  It seems like even simple DOM objects being added through jquery require huhge amounts of memory.  We are dealing with selectable, scrollable lists of around 1300 items and usage grows up to 60MB each time we change the list.

--
--
Chromium Discussion mailing list: chromium...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-discuss

[Ian Wizard]

I'm experiencing a similar issue just using pure JS.  I have seen it get up to 2.7GB before getting collected.  There seems to be sufficient idle time for a GC to happen, but it just doesn't.  Also, I notice that the JS heap is remaining relatively small (below 50MB).  Nothing in task manager shows more than 50MB despite the tab itself reading upwards of 2GB.  I am creating many ArrayBuffers per second, but they also die within 1 second in most cases (I can't readily track them for re-use).  Whenever a GC happens, it resolves the issue (despite only collecting ~1MB, the tabs usage drops back to baseline), but I'm seeing up to 10 seconds or more between GC runs.

[Primiano Tucci]

+jochen (this smells like something we've seen recently).

Ian, I would be extremely grateful if you had some repro steps that make chrome fall into this state. I think I've seen this happening on random pages but never managed to reproduce reliably myself. 

Could you file a bug on crbug.com possibly with a small html attachment (or just a code snippet) which reproduces the problem? That would be a pretty good help for us and for chrome!

Thanks,

Primiano

--

[Jochen Eisinger]

yes, please file a bug with reproduction steps.

As for the CEF issue - are you saying that you enforce a 1.2GB limit on the renderer process size? Did you communicate this limit to V8? The default behavior is to give the entire available memory to V8 (https://code.google.com/p/chromium/codesearch#chromium/src/gin/isolate_holder.cc&l=38) and it tunes its GC heuristics to stay within those limits.

[Aravind]

Hello Guys,

We are experiencing a very similar problem of what has been stated here. Has there been a bug raised already for this issue ? Could someone please mention the bug id created for this ?

The issue we are experiencing is the Chrome extension we have created has been crashing as the GC is not get triggered when the memory becomes low. We had used the chrome.system.memory.getInfo to print the available memory. From that we see the available memory gets lower. It starts from about ~4GB and gets as low as upto ~1.2GB. Once it gets lower than this the extension crashes. When the "Collect Garbage" button in the Timeline window of DevTools is clicked the available memory increases to the original ~4GB level.

This behavior clearly shows the GC is not getting triggered when it has to. We have tuned our code to the extent as much as we can. We are using plain JS and no other JS libraries. We are seeing this issue only on Chrome on Windows. Chrome on MacOS X is not having this GC issue. Chrome version 48 & 49 on Windows 7 & 10 is having this issue.

Can some one throw light on if this is an accepted Chrome bug ?

Regards,
Aravind

[PhistucK]

This sounds like a real bug.

You can search crbug.com for an existing issue and star it. If you cannot find one, file a new issue using the "New issue" link on the same page.

Please, do not add a "+1" or "Me too" or "Confirmed" (or similar) comment. It just wastes the time of Chrome engineers and sends unnecessary e-mails to all of the people who starred the issue.

You can reply with a link to the found or created issue and might get triaged (and fixed) faster.

Thank you.

☆PhistucK

--

--
Chromium Discussion mailing list: chromium...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-discuss

---
You received this message because you are subscribed to the Google Groups "Chromium-discuss" group.

[Aravind]

Thank you for the reply.

I do have a bug created in the link provided. Please find it at https://bugs.chromium.org/p/chromium/issues/detail?id=595342 .

The information about the extension and the steps to be reproduced have been mentioned in the bug along with the attachment.