PTHREAD_POOL_SIZE questions...

17 views
Skip to first unread message

Mark Sibly

unread,
Apr 4, 2023, 10:45:51 PM4/4/23
to emscripten-discuss
I'm building a 3rd party c++ lib in emscripten that uses pthreads, and I'm finding that I have to use -sPTHREAD_POOL_SIZE=23 for it to work, otherwise it hangs.

I did think leaving out -sPTHREAD_POOL_SIZE (or setting it to 0) would create threads on demand, but I just found this on the emscripten pthreads docs page and suspect this is what's happening as the lib's API is just a single function that appears to execute synchronously:

"you cannot call pthread_create and then keep running code synchronously that expects the worker to start running"

So OK, I'll need to manually specify a pool size, but what value should I use? The lib bases it's thread usage on std::thread::hardware_concurrency() (and seems to be a bit off as I have 16 but need a pool size of 23) but I can't know that at link time so what should I use? I'll also need to hard code this as a maximum into the lib of course but that's OK. Is there a way to determine PTHREAD_POOL_SIZE at runtime?

Can I just use 256 or something crazy, or would that break on some low specced computers, ie: how 'heavyweight' are these WebWorkers? I'm sort of feeling an '8' here for some reason...

One other thing, what exactly does -sPTHREAD_POOL_SIZE_STRICT do? I'm finding setting it to '0' causes my app to hang if I  haven't set thread pool size high enough, and setting it to '2' causes an exception to be thrown instead.

So I've just set it to '2', but am I missing something with '0'? Are there any other useful values? Note that there's nothing about -sPTHREAD_POOL_SIZE_STRICT on the emscripten pthreads docs page.

Finally, the lib makes quite complex use of threads - there are std::futures and std::promises all over the place and I initally thought I'd never get it going in emscripten, but once I found the PTHREAD_POOL_SIZE fix everything just worked! This is a very impressive achievement for emscripten IMO!

Bye!
Mark

Alon Zakai

unread,
Apr 5, 2023, 1:21:17 PM4/5/23
to emscripte...@googlegroups.com
On Tue, Apr 4, 2023 at 7:45 PM Mark Sibly <mark...@gmail.com> wrote:
I'm building a 3rd party c++ lib in emscripten that uses pthreads, and I'm finding that I have to use -sPTHREAD_POOL_SIZE=23 for it to work, otherwise it hangs.

I did think leaving out -sPTHREAD_POOL_SIZE (or setting it to 0) would create threads on demand, but I just found this on the emscripten pthreads docs page and suspect this is what's happening as the lib's API is just a single function that appears to execute synchronously:

"you cannot call pthread_create and then keep running code synchronously that expects the worker to start running"

So OK, I'll need to manually specify a pool size, but what value should I use? The lib bases it's thread usage on std::thread::hardware_concurrency() (and seems to be a bit off as I have 16 but need a pool size of 23) but I can't know that at link time so what should I use? I'll also need to hard code this as a maximum into the lib of course but that's OK. Is there a way to determine PTHREAD_POOL_SIZE at runtime?

You can specify it during startup, so you can tell it to use the browser's thread count (which is reflected to C++ in thread::hardware_concurrency()). See this note:

 

Can I just use 256 or something crazy, or would that break on some low specced computers, ie: how 'heavyweight' are these WebWorkers? I'm sort of feeling an '8' here for some reason...

Each worker is a full JS context, so it does take several MB at least. It might make sense to tell the application to use a maximum of say 8 or so, yeah, unless it can really benefit from more...
 

One other thing, what exactly does -sPTHREAD_POOL_SIZE_STRICT do? I'm finding setting it to '0' causes my app to hang if I  haven't set thread pool size high enough, and setting it to '2' causes an exception to be thrown instead.

So I've just set it to '2', but am I missing something with '0'? Are there any other useful values? Note that there's nothing about -sPTHREAD_POOL_SIZE_STRICT on the emscripten pthreads docs page.



Finally, the lib makes quite complex use of threads - there are std::futures and std::promises all over the place and I initally thought I'd never get it going in emscripten, but once I found the PTHREAD_POOL_SIZE fix everything just worked! This is a very impressive achievement for emscripten IMO!


Great! :)

Yeah, there are quite a lot of weird things with how the Web does threads, and so we have to use a lot of hacks and things, but often a lot of code ends up just working with one or two settings adjustments, thanks to a great deal of work that's gone into this...

- Alon

 
Bye!
Mark

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/db07f3cc-0080-4c1f-946c-d57a3fef4cd5n%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages