Java / Scala vs Linux native for non blocking (http) io

216 views
Skip to first unread message

Rich Oliver

unread,
Oct 12, 2015, 10:24:06 AM10/12/15
to scala-user
I've put the following question on stackoverflow, which hopefully might be of some interest to Scala users: Stackoverflow

I'm not sure if that link will work. If it doesn't it can be found by the title: Java / Scala vs Linux native for non blocking (http) io

Eric Richardson

unread,
Oct 19, 2015, 6:48:38 PM10/19/15
to scala-user
Hi Rich,

You might want to contribute to Akka HTTP as an alternative to building something from scratch.

You can read about Spray as well and it's role.

Eventually, I would think that this creation will replace the Netty based http in Play.

I think this is pretty cool stuff and gets you a whole platform of asynchronous bits.

Hope this help,
Eric

Johannes Rudolph

unread,
Oct 20, 2015, 3:18:57 AM10/20/15
to Rich Oliver, scala-user
Hi Rich,

some time ago I answer a similar question on the spray mailing list
which may contain some information that is relevant to your question.
See here: https://groups.google.com/d/msg/spray-user/avYhEFYsvcg/fC_vky_vzQQJ

I try to give you an answer to your SO question "How do the facilities
compare for using Linux native to Java /Scala for writing non blocking
io?". The common way to do non-blocking network IO in Linux is using
epoll.

There is no way around some way of polling when doing asynchronous
programming. You can make polling efficient if your polling logic is
able to multiplex over many (all) the polls in your program so that
you spend only one thread waiting for input. You can hide polling
behind other infrastructure to make asynchronous programming more
convenient. In Linux, epoll offers a quite efficient polling
mechanism.

Java NIO uses epoll behind the scenes, so you are using the same
technique. From my experience, it is hard to get better performance by
trying to roll your own epoll wrapper. The reason is that JNI / native
calls are extremely expensive in Java and every bit of optimization
that helps preventing native calls will help performance quite a lot.
It seems the Java NIO implementation is not too bad (even if the API
is a pain).

It is questionable if the performance you can get with epoll is good
enough for 10Gbit networks or e.g. 1M connections per server. The
article you link in your SO question is based on the idea to
completely circumvent the OS network stack and roll your own network
stack that talks directly to the NIC (which, however, means that you
need to dedicated a (virtual) NIC to your application).

The link in one of the answers to your question points to seastar
which contains a page that lists some of the options you currently
have for fast alternative network stacks in Linux:

http://www.seastar-project.org/networking/

In any case, I'd profile your use case before taking drastic measures
(I consider writing JNI code a drastic measure). The existing
infrastructure may already be fast enough for your use case. (See
http://www.techempower.com/benchmarks/ for recent benchmarks about
what kind of performance is possible.)

HTH
Johannes

Rich Oliver

unread,
Oct 25, 2015, 3:29:11 PM10/25/15
to scala-user
Thanks for the responses and the information. It was one of those tricky questions where you're trying to work out where to prioritise your learning, but you don't really know anything yet to make a judgement on. Anyway I realise that for the time being I don't need to use native or java.nio. It seems Java nio is not necessarily quicker than standard io. Its an additional facility rather than a replacement. And on the other hand if one is going Native and really wants performance and you are running your own server there is no need to limit yourself to Linux and Epoll, BSD and even more niche open source Unix Operating Sytems are available.

From what I've read native and java should be able to handle thousands of threads and possibly tens of thousands, without locking up, with a modern processor and JVM, as long as most of the threads are blocked. So the conventional one thread per connection model should scale up quite some way.

I have done a bit of JNI, But if I do go down the native root I'm interested in experimenting with a separate native process and communicating from the JVM process via dbus / kdbus, I don't know if this would work, but I'm also interested in it as a possible way to run a wayland applications for the desktop. Certain Mobile OSes are using Wayland as well as desktop distro's.

Johannes Rudolph

unread,
Oct 26, 2015, 5:00:53 AM10/26/15
to Rich Oliver, scala-user
Hi Rich,

On Sun, Oct 25, 2015 at 8:29 PM, Rich Oliver <rzi...@gmail.com> wrote:
> From what I've read native and java should be able to handle thousands of
> threads and possibly tens of thousands, without locking up, with a modern
> processor and JVM, as long as most of the threads are blocked. So the
> conventional one thread per connection model should scale up quite some way.

The benefits of asynchronous (non-blocking) programming are that you
are able to suspend with less state than when you send a thread to
sleep keeping its whole stack. So, in the optimal case, you only have
as many threads as CPU cores and then you run your processes to a
point where they need to wait for something and then prepare a
continuation context that contains as little information as possible
to continue (in the best case of a stateless protocol, this could be
just a reference to a function). It is basically a cooperative
multitasking system. You can have lots of logical processes to be
waiting for something without consuming excessive memory.

As an example process take an actor which waits for the next message
while keeping only its actor state (a few hundred bytes + any user
data it keeps). This way you can run millions of concurrent (mostly
waiting) processes on a few OS threads.

Compare that to a JVM thread which consumes at least the fixed amount
of stack space that it was configured to run with. (In theory, you
could configure small stack sizes and structure your program in a way
where the blocking waits would be "on a small stack" but in doing so
you might create a system which is very similar to an event loop or a
similar asynchronous runtime infrastructure).

Using less resources means you can fit more users on the same
hardware. It could also mean that your system is actual performing
better than the synchronous variant. That's because keeping your
working set small may lead to indirect performance benefits (CPU cache
usage, etc.). Also, context-switching between threads (and AFAIK,
especially, waking up JVM threads) has a real cost that will make your
system using CPU resources for management tasks instead of working on
actual computations.

Blocking threads may also have other undesirable side-effects like
keeping locks if you are not careful.

So, I guess YMMV but I wouldn't usually suggest a thread per
connection model in most cases.

--
Johannes

-----------------------------------------------
Johannes Rudolph
http://virtual-void.net

Rich Oliver

unread,
Oct 27, 2015, 4:16:20 PM10/27/15
to scala-user
I'm feeling Akka typed actors is the way to go. Watching Venkat Subrahamaniam has persuaded me of the immense dangers of using threads directly. On the other hand I've got "Learning Concurrent programming in Scala" on order, so hopefully that will give me a better understanding of the use cases for different forms of concurrency. His (Alexander Prokopec's) PhD thesis presentation: https://www.youtube.com/watch?v=mbvdj4qnD-4 is brilliant so hopefully the book should be good.

First off all though, I'm going to try and get a multi application server using long polling without concurrency working. If I close all the requests after sending the response and just keep the long polls open, then I should be able to do it on one thread. That way I won't be confusing programming functionality with the complexities of actors.

Reply all
Reply to author
Forward
0 new messages