Question about what netty worker threads should do

[Parikshit Samant]

Hi,

I had this basic question about what activities Netty worker threads should or should not do, as a guideline/convention.

1. Should not block in IO: I understand that. Because if workers block in IO, there is nobody to service requests even though CPU is idle and we have memory. And creating too many worker threads (hundreds or thousands) is also not appropriate for context switch overhead reasons, in fact, Netty allows us to solve that problem by multiplexing thousands of channels on just a few worker threads.

2. Should not block in heavy CPU work: Say, deserialization, lookups, even sending asynchronous requests to other services, etc. Not sure about whether this is true or not. Someone has to do the CPU work anyways, say via some other thread, while Netty worker just puts incoming records on a buffer that is emptied by other threads. Even then, say, if those threads are going to be scheduled by the OS anyways, it is very likely that netty workers don't get a chance to run because these other threads are doing some CPU work. So, adding that additional level of indirection through an additional buffer has not helped much. In fact, might add a bit of latency. The netty worker could very well have done that work. So, is it then ok for Netty workers to do some more work other instead of just putting off the requests in another buffer for handling by other threads?

Wanted to know whether my opinion on the above two points look correct.

Regards,

--Parikshit N. Samant.

[이희승 (Trustin Lee)]

Sorry I'm late.

 

On Tue, Jan 19, 2016, at 10:37 PM, Parikshit Samant wrote:

Hi,

 

I had this basic question about what activities Netty worker threads should or should not do, as a guideline/convention.

 

1. Should not block in IO: I understand that. Because if workers block in IO, there is nobody to service requests even though CPU is idle and we have memory. And creating too many worker threads (hundreds or thousands) is also not appropriate for context switch overhead reasons, in fact, Netty allows us to solve that problem by multiplexing thousands of channels on just a few worker threads.

 

Looks correct to me.

 

2. Should not block in heavy CPU work: Say, deserialization, lookups, even sending asynchronous requests to other services, etc. Not sure about whether this is true or not. Someone has to do the CPU work anyways, say via some other thread, while Netty worker just puts incoming records on a buffer that is emptied by other threads. Even then, say, if those threads are going to be scheduled by the OS anyways, it is very likely that netty workers don't get a chance to run because these other threads are doing some CPU work. So, adding that additional level of indirection through an additional buffer has not helped much. In fact, might add a bit of latency. The netty worker could very well have done that work. So, is it then ok for Netty workers to do some more work other instead of just putting off the requests in another buffer for handling by other threads?

 

A task can be run in an I/O thread if it does not block and does not run for too long, CPU-consuming tasks are a good example of tasks that can be run in an I/O thread. However, if the task takes too long to finish, the I/O tasks won't be able to handle I/O because they have to wait until the long-running CPU task is finished.

 

So, what's more important is the ratio between the time spent for Netty's I/O tasks and the time spent for your logic. Try to measure that and see if it's skewed.

 

HTH,

T

 

--

https://twitter.com/trustin

https://twitter.com/trustin_ko

https://twitter.com/netty_project

 

[erb...@gmail.com]

I don't see any reason to offload work from Netty I/O thread to other threads, just because the work is slow. The thread handoff is unnecessary. If your server is too slow, eventually you won't be able to keep up with the traffic, and create more threads won't solve the problem.

The problem that having different threadpool solves, is to provide separation and decrease the blast radius of slow requests. For example, if some of your requests are fast & important, some other requests are slow & not-important, it makes sense to create different threadpool for them.

Thanks!
Erben

[Gaurav Sharma]

Wouldn't you want to offload blocking remote calls to say MySQL away from the Netty I/O worker threads?