Memory growth issues and clarifications

[Shachar Langbeheim]

Hi, 
I have experienced these errors, which I understand that are related to holding array buffers after memory growth:
TypeError: Cannot perform %TypedArray%.prototype.set on a neutered ArrayBuffer
    at Uint8Array.set (<anonymous>)
    at Object.mmap (appWASM.js:2645)
    at Object.mmap (appWASM.js:4347)
    at __emscripten_syscall_mmap2 (appWASM.js:5232)
    at ___syscall192 (appWASM.js:5242)
    at ___mmap (:1234/wasm-function[10088]:208)

Reading about this for a bit, I have a couple of questions. I know those are a lot of questions, but I'm about to write a pretty memory-intensive application, and I want to know what are my constraints and requirements. If instead of answers there's some good reading materials about WASM memory growth that is accessible to a layman I'd really appreciate a link.

1. What should I do when memory growth happens? 
1. How do I know that a memory growth event happened? 
2. Can I know what objects and buffers do I need to regenerate or move? 
3. Can I know what triggered memory growth?
4. What happens if a memory growth event happens on another thread, while native code is running?
2.  What exactly happens to current memory and pointers during memory growh?
1. Might some of my native code pointers be made unusable? Can I know what?
2. In the native code, can these neutered arrays hold stack memory, or only heap memory?
3. Is there a way to check for WASM memory leaks in a running web page? 
1. Is there some way to know how the WASM memory is allocated? Some kind of memory map?

Thanks in advance!

[Alon Zakai]

The key thing is that after memory growth in asm.js (but not wasm!) the typed array views get "neutered". Using them will then throw an error.

To avoid this, the simplest thing is to never keep typed array views around in your code. Instead, look at the global HEAP8 etc. views, and use them on demand. That is, avoid

  MyLongTermStorage.view = HEAP8.subarray(x, y);

where that .view is going to be used over time. Instead, just do HEAP8.subarray(x, y) right where you need it.

In wasm things are slightly different. Wasm memory growth will keep typed arrays alive to the previous size. The only problem is the views don't grow. So if you want to look at an object in the newly-allocated memory, a problem happens. The solution is, again, to avoid keeping around views, and to create views right when you need them etc.

- Alon

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CA%2B_KjGY6jxFBAfixfEC9kdp_Mwx4gdviR1PiR0ObQEcWcXvMGw%40mail.gmail.com.

[Shachar Langbeheim]

Thanks!
This is interesting - I'm compiling with the WASM=1 flag, so I didn't expect to still have asm.js code. I assume that there is still asm.js code might be caused by me using fastcomp and not the upstream compiler?
Additionally, I'm pretty sure that the code that I've written doesn't create typed array views - my JS only interacts with wasm using objects created defined using Embind. Could some code generated by emscripten keep those typed arrays?

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpSDJ6cPJ1%2BJzvSTcL7AUFrLSCrA%3DUSGf%2BHqY1D0Pjs6-A%40mail.gmail.com.

[Alon Zakai]

If you compile with WASM then there is no asm.js code, so nothing to worry about there.

If emscripten creates any typed arrays, it should handle them correctly with memory growth - there could be bugs, though, please file anything you find and we'll fix it!

- Alon

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CA%2B_KjGYw-p7xdH-HLSRerAMcewwnYrTrK4%2B6BtivcHg0btVrzA%40mail.gmail.com.