About std:: unordered_map and wasm

[eri0o o]

Hello,

I am using std::unordered_map<const char*,int32_t> and my wasm result has a really slow insert time. Is there something to that I should be aware of? My code crashes after 25 inserts for lack of memory, no matter what.

It's a bit crazy because everything else works great.

Regards,

Érico

[eri0o o]

If code is needed this can be seen here:

https://ericoporto.github.io/agsjs/

Code that was used to generate that: https://github.com/ericoporto/ags/tree/ags3-emscripten

You can see it crash after 25 allocations of managed objects, which is this line here:

https://github.com/ericoporto/ags/blob/9e9229fc26fcce9f365e2a60f0f87e20a298113c/Engine/ac/dynobj/managedobjectpool.cpp#L158

[Mehdi Sabwat]

Hi,

problem seems to be here : https://github.com/ericoporto/ags/blob/ags3-emscripten/CMakeLists.txt#L94

linear memory is an array buffer that has a limited size, you need to explicitly allow growth if you allocate dynamically (=1) and did not specify TOTAL_MEMORY .
more info here:
https://emscripten.org/docs/optimizing/Optimizing-Code.html#memory-growth

Regards,

Mehdi

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CAG1KTJdxCi2PE6xZLMwWeyBdmMd%2BzQ_ed6x-HNe0nOB0kqkq5A%40mail.gmail.com.

[eri0o o]

Thanks, I changed and with the leak detection on the console is possible to see it's trying to allocate a lot of memory.

It should be only a insert of a pointer / integer in the unordered map, but for some reason it tries to allocate a lot of memory for that - until it tries to allocate more memory than it's possible.

It still crashes around the same point.

[eri0o o]

Hey, it appears it's something wrong with allocators in the unordered_map resulting code. I tried to change to std::map, but the error comes even sooner and happens at the second allocation - at least this gives a hint it's in the allocation and not in the hashing? I step on the Webassembly code in the chrome dev tools but was going nowhere reading the source from the STL.

It was here (for unordered map) :

:std::__2::pair<std::__2::__hash_iterator<std::__2::__hash_node<std::__2::__hash_value_type<unsigned\20long\2c\20int>\2c\20void*>*>\2c\20bool>\20std::__2::__hash_table<std::__2::__hash_value_type<unsigned\20long\2c\20int>\2c\20std::__2::__unordered_map_hasher<unsigned\20long\2c\20std::__2::__hash_value_type<unsigned\20long\2c\20int>\2c\20std::__2::hash<unsigned\20long>\2c\20true>\2c\20std::__2::__unordered_map_equal<unsigned\20long\2c\20std::__2::__hash_value_type<unsigned\20long\2c\20int>\2c\20std::__2::equal_to<unsigned\20long>\2c\20true>\2c\20std::__2::allocator<std::__2::__hash_value_type<unsigned\20long\2c\20int>\20>\20>::__insert_unique<std::__2::pair<unsigned\20long\2c\20int>&\2c\20void>\28std::__2::pair<unsigned\20long\2c\20int>&\29

For now I replaced this with a simple for that iterates in the array of object until it finds the corresponding address and it's working.

I also tried to turn on EMULATE_FUNCTION_POINTER_CASTS but this gave me an error message in the console at the module initialization saying it could not convert 1 to BigInt , I then tried to disable WASM_BIGINT but it didn't work. Function pointer casts are necessary for the plugin interface in the code so it would be cool to get them, otherwise I need to figure a way to rewrite that part of the engine - but this should be ok for now.

I do still have some problems (threaded sounds and a blocking while) but I think I can solve them with SDL threads and asincfy, or other approaches, those are fine and can easily be solved once I have more time.

Anyway, it's pretty amazing seeing a simple game bootup and run in the browser, great work people.

[Sam Clegg]

While its not inconceivable that there exists a bug in unordered_map or in the default allocator (dlmalloc) it is highly unlikely since these are both very heavily tested.

It seems more likely that there is some kind of memory corruption bug occuring in your code.  You can try running with `-fsanitize=address` or with `-s SAFE_HEAP` to try to debug them.

You might also want to look at assumptions being made about the alignment malloc.  Emscripten is somewhat naughty today in that it returns pointers from malloc that are only 8 byte aligned whereas `max_align_t` is aligned to 16 bytes.

cheers,

sam

--

You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CAG1KTJetOArwP6VVUdNSOQsAvGzDvy0%3DHwtA%2Bz60f5hVueZxkQ%40mail.gmail.com.