Measuring WasmGC heap usage

[Wolfgang]

Hello,

I'm running some numbers for my masters thesis.

I'd like to measure the heap usage of Node.js when running a WasmGC module.

Specifically, I'd like to measure only the memory allocated due to WasmGC (excluding everything JS related).

E.g. the total memory allocated and the maximum observed throughout execution would be interesting for me.

Is it possible to measure the heap usage due to WasmGC separately?

(I only use WasmGC's structs (and i31s, but they are not stored on the heap I guess))

Thanks in advance.

Best,

Wolfgang

[Matthias Liedtke]

Hi,

> Specifically, I'd like to measure only the memory allocated due to WasmGC (excluding everything JS related).

If you take a heap snapshot, these snapshots include wasm-gc objects. At least in DevTools wasm types look like "$type5316 (wasm)". (If you don't have source maps, otherwise the name should be nicer.)
So you should be able to take a heap snapshot and then process it to filter it to the types that include "(wasm)" to get the wasm-gc heap usage.

> Is it possible to measure the heap usage due to WasmGC separately?

No, you'll need to extract that information unfortunately (or see --trace-gc-object-stats below).

> (I only use WasmGC's structs (and i31s, but they are not stored on the heap I guess))

Exactly, i31refs do not take extra heap usage (other than the memory for the reference itself, e.g. when stored in a struct / array.

Getting the maximum memory would be more difficult. You can get aggregated data with the command line flag --trace-gc-object-stats.
There you'd need to mainly check for WASM_ARRAY_TYPE and WASM_STRUCT_TYPE.
This might be even easier than filtering the heap snapshots and as it logs these stats on every mark compact of the GC it should give you a good indication of what the maximum heap usage for wasm is, assuming there aren't that many large JS allocations happening in between.
(You might want to make sure to trigger a manual GC at the end of the script, not sure if things like --expose-gc work in node.js to do that.)

Best regards,
Matthias

--
--
v8-dev mailing list
v8-...@googlegroups.com
http://groups.google.com/group/v8-dev
---
You received this message because you are subscribed to the Google Groups "v8-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to v8-dev+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/v8-dev/a48c3ded-c738-4f79-a0fa-8d838d629570n%40googlegroups.com.

[Jakob Kummerow]

E.g. the total memory allocated

I don't think we have a way to extract that information, even indirectly, because --trace-gc-object-stats doesn't affect the minor GC, so won't include short-lived objects.

From --trace-gc you can compute the overall total allocated size, but without any distinction on the type of object.

 

and the maximum observed throughout execution would be interesting for me.

Keep in mind that the maximum size of the managed heap is largely a function of the configuration/heuristics of the garbage collector, as opposed to the running application. For example, V8's GC will try to stay out of the way by allowing the heap to grow pretty large when the application is busy, and spending more time on GC activity when the application is idle. Simply measuring the maximum heap size encountered this way will yield a number that doesn't have much to do with the application's peak "live set". Looking only at surviving objects after a major GC isn't a reliable way to measure the maximum either, because that major GC might have happened long after the peak size of the live set.

To determine an application's actual required peak memory consumption, you'll probably have to binary-search maximum heap sizes (--max-old-space-size) until you find the setting that just barely avoids crashing with OOM.

[Wolfgang]

Thanks for the explanations!

> To determine an application's actual required peak memory consumption, you'll probably have to binary-search maximum heap sizes (--max-old-space-size) until you find the setting that just barely avoids crashing with OOM.

I think this is what I want, I'll try that. Thanks!

--

--
v8-dev mailing list
v8-...@googlegroups.com
http://groups.google.com/group/v8-dev
---
You received this message because you are subscribed to the Google Groups "v8-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to v8-dev+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/v8-dev/CAKSzg3QJ%2B%3D6bPq3FK7%3D_Ubi6i23mGkV4kQ6cC7hue-gM%3DNtN4Q%40mail.gmail.com.