NettyServerBuilder difference between executor() and workerEventLoopGroup()

[Pierre]

Hello,

What is the difference between setting an executor(Threadpool) and setting a workerEventLoopGroup(EventLoopThreadPool) on NettyServerBuilder.

If I understand, all the IO operations (accept connexions, read/send packet...) are done by the bossEventLoopGroup thread.

But what about the RPC call themself ? Are they done by executor threadpool or workerEventLoop threads ?

[Carl Mastrangelo]

There are three main threadpools that gRPC Java uses in Server mode:

* Boss Event Loop Group  (a.k.a. bossEventLoopGroup() )

* Worker Event Loop Group ( a.k.a. workerEventLoopGroup() )

* Application Executor (a.k.a. executor() )

The Boss group can be the same as the worker group.  It's purpose is to accept calls from the network, and create Netty channels (not gRPC Channels) to handle the socket.   Effectively, this is the thread pool that calls listen() and accept().   

Once the Netty channel has been created it gets passes to the Worker Event Loop Group.  This is the threadpool dedicating to doing socket read() and write() calls.  This is often called the "network thread".   In addition to doing the direct reads and writes, it can also do small amounts of processing, such as turn a collection of bytes into an HTTP/2 request, or do SSL / TLS encryption.  The main thing to know about this group is that it must never block.  That is, it cannot call wait(), or sleep(), or await results from a Future, or things like that.  The reason is that doing this prevents the worker thread (the "event loop") from servicing other requests.

The last thread group is the application executor, also called the "app thread".  This is where the gRPC stubs do their main work.  It is for handling the callbacks that bubble up from the network thread.  For example, when a packet comes in, the worker thread decrypts it, decodes the HTTP/2 code, and then notifies the app thread that data has arrived.  It immediately goes back to doing other network work.  The App thread notices that there is data available, and converts the data into (usually) a Protobuf, and then invokes your stub.   

Note that the Application executor could have no threads at all.  This would make the network thread immediately do the work of handling the request.   On the builder this is the directExecutor() function.   The reason for using this is that it saves a thread hop from worker event loop to the app thread, or about 15 microseconds.  In very latency sensitive environments this is a good idea.  However, it makes it very easy to accidentally do blocking work, which is deadlock prone.   It is rarely correct to do.

As for usages:   Most people should use either reuse the same boss event loop group as the worker group.   Barring this, the boss eventloop group should be a single thread, since it does very little work.  For the app thread, users should provide a fixed size thread pool.  By default, we use an unbounded cached threadpool, but this is just for safety.  It is not the most efficient, and can be dangerous in some circumstances.  It just happens to be the safest default, given no other information.       

[Uli Bubenheimer]

Most people should use either reuse the same boss event loop group as the worker group.

That's very useful to know. On Linux, can it make sense to use a shared EpollEventLoopGroup for boss/worker, or should it be NioEventLoopGroup to make sense for sharing?

[Eric Anderson]

Use Epoll if using the Epoll channel, Nio if Nio channel. By default we use Nio channel, but we have discussed maybe auto-detecting and stuff. If you provide your own executor, it's good to specify the channel so you know everything is compatible.

On Thu, Oct 5, 2017 at 9:56 AM, Uli Bubenheimer <ubu...@gmail.com> wrote:

Most people should use either reuse the same boss event loop group as the worker group.

That's very useful to know. On Linux, can it make sense to use a shared EpollEventLoopGroup for boss/worker, or should it be NioEventLoopGroup to make sense for sharing?

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscribe@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/c9d0dbf3-64a7-4de2-9656-7882bde90d1a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[stefan.g...@gmail.com]

Hi,

Thank you for the detailed information. Can you explain how a message is passed through the thread pools in the opposite direction (when StreamObserver#onNext(Message) is called)? 

Suppose we have a publish/subscribe application. A client subscribes for a topic. On the server the StreamObserver instance is cached, thus for each topic there are multiple StreamObserver instances. When an update for a topic arrives, the associated StreamObserver instances are iterated and for each of them #onNext(message) is called. Does #onNext(Message) blocks until the message is transferred to the client or it is queue somewhere in the network thread? Can a slower connection to a specific client influence the communication to other clients if the StreamObservers are iterated sequentially. If this is the case, then maybe a separate thread pool should be used, where each thread is associated with a single StreamObserver and calls its #onNext(Message) method?

Thanks,

Stefan

[Ethan Cheng]

Have a question for this one:

Once the Netty channel has been created it gets passes to the Worker Event Loop Group.  This is the threadpool dedicating to doing socket read() and write() calls.  This is often called the "network thread".   In addition to doing the direct reads and writes, it can also do small amounts of processing, such as turn a collection of bytes into an HTTP/2 request, or do SSL / TLS encryption.  The main thing to know about this group is that it must never block.  That is, it cannot call wait(), or sleep(), or await results from a Future, or things like that.  The reason is that doing this prevents the worker thread (the "event loop") from servicing other requests.

Is this a channel any gRPC service developer will ever touch the worker Event Loop Group? I thought all the network thread are working under the hood and application developer will just need to deal with application thread which can be block. I know that things happened in network thread like HTTP/2 request, encryption that may be blocking, but are they also work under the hood? Can you give an example where application developer need to code something at the network thread level?

[Eric Anderson]

Please do not comment on three-year-old threads. Instead, start a new email thread and just refer back to the original.

The event loops are behind-the-scenes. You only need to mess with them if you want to change their sizes (how many threads are used) or their types or are using directExecutor(). For example, today to use Unix Domain Sockets you have to specify an epoll event loop. executor() is the main thread pool to think about, since that's where your application code runs.

--

You received this message because you are subscribed to the Google Groups "grpc.io" group.

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/c2dec0f4-f71b-42e9-b8a1-9aa143ee0634n%40googlegroups.com.