Best practices for growable heap access in JS library code?

[Brion Vibber]

I've been updating my ogv.js codecs to support growable memory in all my variant builds, including JS, Wasm, and Wasm+pthreads.

My JS library mix-ins need access to memory to transfer data in and out, and for pthreads this means they must handle the case where HEAPU8 etc are out of date because of growth from another thread. Since I'm mostly creating TypedArray views and copying them in or out, the easiest thing seems to be create views around wasmMemory.buffer, which always has the current-sized backing ArrayBuffer, and just not worry about HEAPU8, HEAPU32, etc.

(I don't think this works with fastcomp, but it does work with WASM=0 output on upstream backend using wasm2js.)

There are wrapper functions in growableHeap.js for individual loads/stores and creation of views such as GROWABLE_HEAP_VIEW_U8 which do the same thing, but as I understand this only gets included on pthread builds. Is there a preferred method of access to these functions, or are they meant only to be used as transformations? (The JS optimizer transforms individual accesses into HEAP* into GROWABLE_HEAP_LOAD_* and *_STORE_* functions so I think it might already be handling those magically, but doesn't seem to transform anything to the view creation functions.)

-- brion

[Brion Vibber]

Doing further testing, I think there's still some bugs in the standard library when using threading together with growable memory; I found my threaded decoders sometimes hang a few seconds in, shortly after growing memory.

I filed https://github.com/emscripten-core/emscripten/issues/9963 with a workaround for emscripten_futex_wake failing when the views are out of date, though I think there are other similar bugs remaining.

-- brion

[Alon Zakai]

In general those GROWABLE_* functions are intended for automatic transformation, correct. They could be used directly too, but most code that could use them should already be transformed automatically, I think. But I guess you could export them, and use them outside.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CAFnWYT%3DjeWi9ODQSeqCnPc6cf0hbyYOqDUS%3DfPdFMvnp_L9KxA%40mail.gmail.com.