Hi Rich,
On Sun, Oct 25, 2015 at 8:29 PM, Rich Oliver <
rzi...@gmail.com> wrote:
> From what I've read native and java should be able to handle thousands of
> threads and possibly tens of thousands, without locking up, with a modern
> processor and JVM, as long as most of the threads are blocked. So the
> conventional one thread per connection model should scale up quite some way.
The benefits of asynchronous (non-blocking) programming are that you
are able to suspend with less state than when you send a thread to
sleep keeping its whole stack. So, in the optimal case, you only have
as many threads as CPU cores and then you run your processes to a
point where they need to wait for something and then prepare a
continuation context that contains as little information as possible
to continue (in the best case of a stateless protocol, this could be
just a reference to a function). It is basically a cooperative
multitasking system. You can have lots of logical processes to be
waiting for something without consuming excessive memory.
As an example process take an actor which waits for the next message
while keeping only its actor state (a few hundred bytes + any user
data it keeps). This way you can run millions of concurrent (mostly
waiting) processes on a few OS threads.
Compare that to a JVM thread which consumes at least the fixed amount
of stack space that it was configured to run with. (In theory, you
could configure small stack sizes and structure your program in a way
where the blocking waits would be "on a small stack" but in doing so
you might create a system which is very similar to an event loop or a
similar asynchronous runtime infrastructure).
Using less resources means you can fit more users on the same
hardware. It could also mean that your system is actual performing
better than the synchronous variant. That's because keeping your
working set small may lead to indirect performance benefits (CPU cache
usage, etc.). Also, context-switching between threads (and AFAIK,
especially, waking up JVM threads) has a real cost that will make your
system using CPU resources for management tasks instead of working on
actual computations.
Blocking threads may also have other undesirable side-effects like
keeping locks if you are not careful.
So, I guess YMMV but I wouldn't usually suggest a thread per
connection model in most cases.
--
Johannes
-----------------------------------------------
Johannes Rudolph
http://virtual-void.net