JavaScript heap out of memory issue

[Abhishek Singh]

Hi,

I noticed a V8 crash with following message:

<--- Last few GCs --->

  251206 ms: Mark-sweep 1400.2 (1456.1) -> 1400.2 (1456.1) MB, 3688.5 / 0.0 ms [allocation failure] [GC in old space requested].

  254854 ms: Mark-sweep 1400.2 (1456.1) -> 1400.2 (1456.1) MB, 3648.2 / 0.0 ms [allocation failure] [GC in old space requested].

  258562 ms: Mark-sweep 1400.2 (1456.1) -> 1400.2 (1426.1) MB, 3707.6 / 0.0 ms [last resort gc].

  262315 ms: Mark-sweep 1400.2 (1426.1) -> 1400.2 (1426.1) MB, 3752.6 / 0.0 ms [last resort gc].

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x3c20e87ca141 <JS Object>

    1: OnHTTPGet [0x3c20e8704311 <undefined>:~64] [pc=0x328074967a8e](this=0x94116b170d1 <JS Global Object>,req=0x6ba99d91ed1 <an Object with map 0x73dc030c4f9>,res=0x6ba99d91f29 <JS Object>)

==== Details ================================================

[1]: OnHTTPGet [0x3c20e8704311 <undefined>:~64] [pc=0x328074967a8e](this=0x94116b170d1 <JS Global Object>,req=0x6ba99d91ed1 <an Object with m...

What the application does is - it executes JS function call when an end user fires a HTTP request and response object is written back to the end user. Call volume was around 10K HTTP requests/sec(JS code being executed is trivial) - snippet of it:

function OnHTTPGet(req, res) {

  res.body.msg = “Hello world”; // res is nested map exposed from C++ to JS world

}

===

I tried simulating behaviour on a standalone environment and could replicate the problem, here is code to do that: https://gist.github.com/abhi-bit/505538ac45d4b11014c3681e3bb74a2b

What I noticed that my machine has probably 4GB of memory left but V8’s memory footprint was around 1.6GB before the crash. To summarise my questions:

• Is 10K request/sec(JS function calls/sec) too much for GC to handle in this scenario? Any tunable to control this behaviour or am I doing wrong in standalone code?
• Why V8 mem_size could only grow upto 1.6GB, even when system has 4GB of memory? I’ve seen similar problem in couchdb(map-reduce index based on top of V8) as well, where app V8 binding crashed when footprint when close to 1.6GB(but that with V8 3.23) - error dump looked like this:

[ns_server:info,2016-09-03T2:28:27.006,babysitte...@127.0.0.1:<0.68.0>:ns_port_server:log:169]ns_server<0.68.0>:

ns_server<0.68.0>: #

ns_server<0.68.0>: # Fatal error in CodeRange::GetNextAllocationBlock

ns_server<0.68.0>: # Allocation failed - process out of memory

ns_server<0.68.0>: #

ns_server<0.68.0>:

Thanks,

Abhishek

[Ben Noordhuis]

On Fri, Sep 23, 2016 at 12:54 PM, Abhishek Singh
<singhabh...@gmail.com> wrote:
> Hi,
>
> I noticed a V8 crash with following message:
>
>
> <--- Last few GCs --->
>
> 251206 ms: Mark-sweep 1400.2 (1456.1) -> 1400.2 (1456.1) MB, 3688.5 / 0.0 ms [allocation failure] [GC in old space requested].
> 254854 ms: Mark-sweep 1400.2 (1456.1) -> 1400.2 (1456.1) MB, 3648.2 / 0.0 ms [allocation failure] [GC in old space requested].
> 258562 ms: Mark-sweep 1400.2 (1456.1) -> 1400.2 (1426.1) MB, 3707.6 / 0.0 ms [last resort gc].
> 262315 ms: Mark-sweep 1400.2 (1426.1) -> 1400.2 (1426.1) MB, 3752.6 / 0.0 ms [last resort gc].
>
>
> <--- JS stacktrace --->
>
> ==== JS stack trace =========================================
>
> Security context: 0x3c20e87ca141 <JS Object>
> 1: OnHTTPGet [0x3c20e8704311 <undefined>:~64] [pc=0x328074967a8e](this=0x94116b170d1 <JS Global Object>,req=0x6ba99d91ed1 <an Object with map 0x73dc030c4f9>,res=0x6ba99d91f29 <JS Object>)
>
> ==== Details ================================================
>
> [1]: OnHTTPGet [0x3c20e8704311 <undefined>:~64] [pc=0x328074967a8e](this=0x94116b170d1 <JS Global Object>,req=0x6ba99d91ed1 <an Object with m...
>
>
> What the application does is - it executes JS function call when an end user fires a HTTP request and response object is written back to the end user. Call volume was around 10K HTTP requests/sec(JS code being executed is trivial) - snippet of it:
>
> function OnHTTPGet(req, res) {
> res.body.msg = “Hello world”; // res is nested map exposed from C++ to JS world
> }
>
> ===
>
> I tried simulating behaviour on a standalone environment and could replicate the problem, here is code to do that: https://gist.github.com/abhi-bit/505538ac45d4b11014c3681e3bb74a2b
>
> What I noticed that my machine has probably 4GB of memory left but V8’s memory footprint was around 1.6GB before the crash. To summarise my questions:
>
> Is 10K request/sec(JS function calls/sec) too much for GC to handle in this scenario? Any tunable to control this behaviour or am I doing wrong in standalone code?

Modulo bugs in the garbage collector the request rate shouldn't
matter. Your application can not allocate faster than the collector
can reclaim because it will simply block your application until it
catches up. Not necessarily true for a truly concurrent GC but V8's
is still stop-the-world when it has to be.

A more likely cause is that you have a memory leak somewhere. Try
making snapshots of the heap that you can inspect in Chrome DevTools;
see HeapProfiler::TakeHeapSnapshot() in include/v8-profiler.h or
https://github.com/bnoordhuis/node-heapdump for a more full-fledged
example.

> Why V8 mem_size could only grow upto 1.6GB, even when system has 4GB of memory? I’ve seen similar problem in couchdb(map-reduce index based on top of V8) as well, where app V8 binding crashed when footprint when close to 1.6GB(but that with V8 3.23) - error dump looked like this:

The default limit is ~1.6 GB. You can turn it up with
--max_old_space_size or through
Isolate::CreateParams::constraints::set_max_old_space_size().

[Abhishek Singh]

Hi Ben,

It turned out to be memory leak in my C++ wrapper on top of V8 - fixed it. Thanks for the suggestion around using v8::HeapProfiler::TakeHeapSnapshot().

> --
> --
> v8-users mailing list
> v8-u...@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to the Google Groups "v8-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.