WASM memory access from JS

[Felipe Gasper]

Hello,

Is there a good, stable means of accessing the WASM memory from JavaScript?

I’m trying to compile GNU AWK (gawk) to WASM. gawk expects to run via main(), so that’s what I’m doing via callMain(). The safest approach is to load/compile the WASM every time gawk runs, but that’s quite slow.

To speed things up I’m trying to run callMain() multiple times with the same WASM state. gawk, though, leaves a bunch of globals around, which need to be reset each time callMain() is called.

On my workstation I’m using Emscripten 3.1.31, which exposes Module.asm.memory. Using that I can save the memory’s state before the first invocation, then restore it before each successive one. This works well enough. Apparently in newer Emscripten releases, though, `Module.asm` no longer exists; thus, my little trick breaks.

I’d ideally like not to have to supply my own memory; right now I’m deferring to Emscripten’s defaults, and I’d like to retain that behaviour. Is there, or could there be, a reliable way to clear WASM’s memory from JavaScript?

Thank you!

cheers,
-Felipe Gasper
Toronto

[Sam Clegg]

To access the wasmMemory from the outside you should be able to do `-sEXPORTED_RUNTIME_METHODS=wasmMemory` and then access it via `Module.wasmMemory`.

You can also use the `.buffer` property of any of the HEAPXX exports if you export any of those?

BTW, I'm curious about the "The safest approach is to load/compile the WASM every time gawk runs, but that’s quite slow." quote.   Have you measured this?   What platform are you running on?  Ideally we would like to make repeated running like this fast so folks like you do not have to do work arounds, e.g. via caching the re-using as much as possible.   Have you looked into using the `-sMODULARIZE` and then using the module constructor each time you want to run the program?

cheers,

sam

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/884ED095-86DB-4306-9968-A924BB3189C1%40felipegasper.com.

[Felipe Gasper]

> On May 21, 2024, at 1:52 PM, 'Sam Clegg' via emscripten-discuss <emscripte...@googlegroups.com> wrote:
>
> To access the wasmMemory from the outside you should be able to do `-sEXPORTED_RUNTIME_METHODS=wasmMemory` and then access it via `Module.wasmMemory`.
>
> You can also use the `.buffer` property of any of the HEAPXX exports if you export any of those?

Ah OK. Which of these approaches is likely the more stable?

> BTW, I'm curious about the "The safest approach is to load/compile the WASM every time gawk runs, but that’s quite slow." quote. Have you measured this? What platform are you running on? Ideally we would like to make repeated running like this fast so folks like you do not have to do work arounds, e.g. via caching the re-using as much as possible. Have you looked into using the `-sMODULARIZE` and then using the module constructor each time you want to run the program?

I’m already using MODULARIZE, actually. From what I could tell, calling the module constructor instantiates WASM. I have a simple test that runs gawk 1,000 times (to check for memory leaks); that test ran fairly slowly when I had it call the module constructor for each invocation. By switching to just clearing the memory between callMain() invocations I realized a visible speedup.

Thank you!

-FG

[Sam Clegg]

On Tue, May 21, 2024 at 11:18 AM 'Felipe Gasper' via emscripten-discuss <emscripte...@googlegroups.com> wrote:

> On May 21, 2024, at 1:52 PM, 'Sam Clegg' via emscripten-discuss <emscripte...@googlegroups.com> wrote:
>
> To access the wasmMemory from the outside you should be able to do `-sEXPORTED_RUNTIME_METHODS=wasmMemory` and then access it via `Module.wasmMemory`.
>
> You can also use the `.buffer` property of any of the HEAPXX exports if you export any of those?

Ah OK. Which of these approaches is likely the more stable?

I don't imagine either of them will change any time soon.   However if you are already exporting one or more of the HEAPXX views then that would be the simpler/small approach I think.

 

> BTW, I'm curious about the "The safest approach is to load/compile the WASM every time gawk runs, but that’s quite slow." quote.   Have you measured this?   What platform are you running on?  Ideally we would like to make repeated running like this fast so folks like you do not have to do work arounds, e.g. via caching the re-using as much as possible.   Have you looked into using the `-sMODULARIZE` and then using the module constructor each time you want to run the program?

I’m already using MODULARIZE, actually. From what I could tell, calling the module constructor instantiates WASM. I have a simple test that runs gawk 1,000 times (to check for memory leaks); that test ran fairly slowly when I had it call the module constructor for each invocation. By switching to just clearing the memory between callMain() invocations I realized a visible speedup.

I see.   Thanks for the feedback.   I have couple of followup thoughts:

1. I wonder which part of the module constructor is taking time here

2. I wonder if we can improve this?  Perhaps there is more caching we can do?

3.  I wonder how significant this difference is in real world usage, outside of a microbenchmark that runs gawk 1000 times?

Thank you!

-FG

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/B4D356B6-5137-4156-9E58-0E3DB9DA4787%40felipegasper.com.

[Felipe Gasper]

> On May 21, 2024, at 6:52 PM, 'Sam Clegg' via emscripten-discuss <emscripte...@googlegroups.com> wrote:
>
> On Tue, May 21, 2024 at 11:18 AM 'Felipe Gasper' via emscripten-discuss <emscripte...@googlegroups.com> wrote:
>
>
> > On May 21, 2024, at 1:52 PM, 'Sam Clegg' via emscripten-discuss <emscripte...@googlegroups.com> wrote:
> >
> > To access the wasmMemory from the outside you should be able to do `-sEXPORTED_RUNTIME_METHODS=wasmMemory` and then access it via `Module.wasmMemory`.
> >
> > You can also use the `.buffer` property of any of the HEAPXX exports if you export any of those?
>
> Ah OK. Which of these approaches is likely the more stable?
>
> I don't imagine either of them will change any time soon. However if you are already exporting one or more of the HEAPXX views then that would be the simpler/small approach I think.

I’ll give that a shot, then. Thank you!

>
> > BTW, I'm curious about the "The safest approach is to load/compile the WASM every time gawk runs, but that’s quite slow." quote. Have you measured this? What platform are you running on? Ideally we would like to make repeated running like this fast so folks like you do not have to do work arounds, e.g. via caching the re-using as much as possible. Have you looked into using the `-sMODULARIZE` and then using the module constructor each time you want to run the program?
>
> I’m already using MODULARIZE, actually. From what I could tell, calling the module constructor instantiates WASM. I have a simple test that runs gawk 1,000 times (to check for memory leaks); that test ran fairly slowly when I had it call the module constructor for each invocation. By switching to just clearing the memory between callMain() invocations I realized a visible speedup.
>
> I see. Thanks for the feedback. I have couple of followup thoughts:
>
> 1. I wonder which part of the module constructor is taking time here

I would _guess_ it’s the WASM parse/compile. But I don’t know enough to say for certain.

> 2. I wonder if we can improve this? Perhaps there is more caching we can do?

I would think the WebAssembly Module and Instance, at least, could be cached. Maybe the Memory, too, if the size doesn’t change?

> 3. I wonder how significant this difference is in real world usage, outside of a microbenchmark that runs gawk 1000 times?

I’m not sure, really.

My own concern is that I’m looking at using this in a log viewer. The viewer stores the log lines in a JS array, so it’s easiest to run WASM (gawk, jq, etc.) separately on each line.

Obviously it’s more sensible just to run WASM once, but that would mean bigger architectural changes for the log viewer, so I’m trying to avoid that.

cheers,
-Felipe Gasper