Setting up memory parameters for testing in Node.js

[John Dallman]

I'm porting a mathematical modelling library, compiled from C and C++ code, to WebAssembly. The test harness for it is also written in C and C++, and is quite complex; replacing it is far too big a job to be considered. C is a large majority of the code, both in the library and in the test harness. I'm compiling for 32-bit; I have one other 32-bit platform, Windows on x86 with 32-bit addressing, but all my other platforms are 64-bit. 

I have some test cases that need too much RAM for 32-bit addressing, so those are not run on 32-bit platforms. But I have about a dozen test cases that work on 32-bit Windows but run out of memory on WebAssembly. This is out of about 130,000 tests, so it isn't a big problem, but it does seem that something is using memory less efficiently under WebAssembly than under Windows. We run 32-bit Windows testing under 64-bit Windows, which provides almost 4GB of address space to 32-bit processes, if you pull the right levers, and we do that.  

When I look at my Node.js processes that are running my tests, they tend to be using about 13GB of address space, with about 3.5GB resident. 

Given those two things, I'm wondering if something about memory growth isn't being handled efficiently, and I should change my linking flags or my command-line arguments to node? 

I link with:  

-sMAXIMUM_MEMORY=4000MB -sALLOW_MEMORY_GROWTH \

-sINITIAL_MEMORY=256MB -sSTACK_SIZE=64MB \

-sSUPPORT_LONGJMP=1 -sDEFAULT_PTHREAD_STACK_SIZE=1MB \

-sNO_DISABLE_EXCEPTION_CATCHING \

 -sEMULATE_FUNCTION_POINTER_CASTS=1 \

-sBINARYEN_EXTRA_PASSES=--pass-arg=max-func-params@55

My Node.js startup options are:

--max-old-space-size=4000 --stack-size=16384 --stack-trace-limit=20

 Should I increase INITIAL_MEMORY to 4000MB? Something else?  

I'm well aware that one can't get nearly that much memory in a browser. This is about showing that the WebAssembly build can pass the same tests as the native code platforms.  

Thanks in advance,

John Dallman

[Mike]

Not saying this will fix it, but one thing you could try doing would be increasing the max memory to 4GB instead of 4000 and then dropping the initial memory down to a more realistic number since setting the initial memory doesn't increase the actual addressable memory

-sALLOW_MEMORY_GROWTH=1 -sINITIAL_MEMORY=256MB -sMAXIMUM_MEMORY=4GB -sSTACK_SIZE=16MB -sDEFAULT_PTHREAD_STACK_SIZE=1MB

You can also add in some tooling to track the memory usage 

#include <emscripten/heap.h> 

#include <inttypes.h> 

#include <stdio.h>

 

 

static void report_wasm_memory(void) {

const size_t current = emscripten_get_heap_size();

const size_t maximum = emscripten_get_heap_max();

const uintptr_t dynamic_top = *emscripten_get_sbrk_ptr(); 

fprintf(stderr, "wasm memory: current=%zu MiB, maximum=%zu MiB, sbrk=%" PRIuPTR "\n", current / (1024 * 1024), maximum / (1024 * 1024), dynamic_top); 

}

Hope this helps..  

Then call report_wasm_memory() before the point where the failure happens

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqg%3DKGofDusq2NiANLyEHDvWk_S3kJbQoyurzVwhPLaEO-A%40mail.gmail.com.

[John Dallman]

I've had an idea about where the large amounts of non-resident virtual memory come from. They appear immediately when the node process starts, before my test harness starts consuming memory to put data in. I think they come from reserving 4GB of address space for each WebAssembly module in the process, per https://spidermonkey.dev/blog/2025/01/15/is-memory64-actually-worth-using.html. 

I'm only using one module, statically linking my product to its test harness, but it seems rather likely there are a few others in a Node.js process. I'm running node on a 64-bit OS, of course. 

Given that, reducing fragmentation possibilities with:

 -sALLOW_MEMORY_GROWTH \

-sINITIAL_MEMORY=3GB \

-sMAXIMUM_MEMORY=4GB \

-sSTACK_SIZE=16MB \

-sDEFAULT_PTHREAD_STACK_SIZE=2MB

seems like a plausible idea. Comments will be very much welcome!

John

To view this discussion visit https://groups.google.com/d/msgid/emscripten-discuss/CACD-exuTU9WiubQ-OfkMf8pZ3m6gfvMXViOMdZfWvBvN8ZXc8A%40mail.gmail.com.

[Sam Clegg]

On Thu, Jul 30, 2026 at 12:06 PM John Dallman <jgdats...@gmail.com> wrote:

Actually the amount of memory available under wasm32 and wasm64 are likely to be the same in the browser as they are under node. 

For wasm32 you should be able to get close to 4GB of linear memory.    And for wasm64 you should able to get up to 12gb these days.

Perhaps you could look at just one of your failing tests to see why it is running out of memory?   If you suspect `ALLOW_MEMORY_GROWTH` is an issue then perhaps just remove that completely and set `INITIAL_MEMORY=4gb`. 

That certainly seems to work for me:

$ emcc ~/test/hello.c -sINITIAL_MEMORY=4gb
$ node ./a.out.js
Hello, world!
$ wasm-objdump -x a.out.wasm | grep memory.*pages
 - memory[0] pages: initial=65536 max=65536

Perhaps one reason is that the native program uses a different heap allocator?   But honestly I would expect the available memory on the native 32-bit build to be less (and not more), since its has to use at least some of the 4gb range for the program code pages, etc.

 

This is about showing that the WebAssembly build can pass the same tests as the native code platforms.  

Thanks in advance,

John Dallman