managing views after heap realloc

[Brian Craft]

Copy overhead for large data is too high, so I'm hoping to keep the data in the wasm heap and provide access from js via typed array views.

This seems to work well except when the heap grows, invalidating the views. Are there any good ways of dealing with this? I'm hoping to avoid putting knowledge of the invalidation all over the js code.

I tried writing Proxy which delegates to a view, and updates the view as necessary, but using traps kills the performance of the views. E.g. every access, like view[i] hits the javascript trap, so iterating the view becomes painfully slow.

[Alon Zakai]

If you don't capture references to views, then emscripten will update Module.HEAP8 etc. when memory grows. So only using those should be fine.

Otherwise, perhaps there should be an option to listen for memory growth events, so that you can be notified when you need to recreate the views, but this gets very hard with pthreads (see https://github.com/WebAssembly/design/issues/1271 )

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Brian Craft]

In my use case it's not hard to catch the memory growth. There are only a few entry points, and I can check the allocated views afterward. The problem is managing the references to the views that have spread throughout the application state.

I can write a Proxy to a view that can swap out the underlying view, but as I mentioned, it's too slow. Turns out it's more performant to subclass the view. The underlying view can be swapped out by setting __proto__. Not a great solution, and still 10x slower to access than the view w/o subclassing, but faster than a Proxy.

The main places in the code holding references are memoized functions, so another approach might be to add special handling to the memoizers, to swap out the views in cache when they are invalidated.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

[Eric Mandel]

I might be really useful to add a sentence to the "Access Memory from Javascript" section of https://emscripten.org/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.html, referencing your illuminating memory growth note https://gist.github.com/kripken/949eab99b7bc34f67c12140814d2b595. It's a good example of the requirement that "you know what you are doing", especially for C programmers who can't wait to get a buffer pointer back into Javascript.

[Beuc]

Hi,

Would you like to propose a pull request? :)

https://github.com/emscripten-core/emscripten/blob/incoming/site/source/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.rst
https://github.com/emscripten-core/emscripten/edit/incoming/site/source/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.rst

- Sylvain

[Eric Mandel]

Sure, but after the Event Horizon Telescope press conference, starting in a few minutes!

[Eric Mandel]

PR made … let me know what you need.

> --
> You received this message because you are subscribed to a topic in the Google Groups "emscripten-discuss" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/emscripten-discuss/hX1KzStIa4E/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to emscripten-disc...@googlegroups.com.