Backpressure on accept connections

[James Roper]

Hi all,

I just want to validate my understanding of Netty here and how server connections are accepted.  A typical Netty server has two conceptual channels, one channel is effectively a channel of channels, representing a stream of incoming connections, and that one creates a new channel pipeline to handle each incoming connection.

On the server channel of channels, if you set auto read to false, then this will allow you to backpressure on the incoming stream of connections, which would allow you to use this to implement a connection limit.  In that case, if you did back pressure (ie, by not invoking read() when you receive a read complete event), Netty would stop accepting new connections, which would cause any subsequent connections received to fill up the OS connection backlog, until the SO_BACKLOG limit is reached, at which point, the OS will start rejecting connections outright.

Is my understanding here correct?  Before we go and implement it ourselves, does Netty provide any built in channel handlers that implement connection limit logic?  I imagine it wouldn't be hard to do - you just need a way to communicate from the child channel pipelines when a channel has been closed so that the connection counter can be decremented and read can be invoked again on the channel receiving the connections.

And finally, all this is in the context of Netty 4 - does the same apply to Netty 3, using Channel.setReadable() to implement the backpressure?  Play of course is on Netty 4, but we have a very large user who are asking about this for an older version of Play which is on Netty 3.

Cheers,

James

[Norman Maurer]

Hey James,

comments inline.

On 10. May 2017 at 04:22:20, James Roper (jro...@gmail.com) wrote:

Hi all,

I just want to validate my understanding of Netty here and how server connections are accepted.  A typical Netty server has two conceptual channels, one channel is effectively a channel of channels, representing a stream of incoming connections, and that one creates a new channel pipeline to handle each incoming connection.

Correct.

On the server channel of channels, if you set auto read to false, then this will allow you to backpressure on the incoming stream of connections, which would allow you to use this to implement a connection limit.  In that case, if you did back pressure (ie, by not invoking read() when you receive a read complete event), Netty would stop accepting new connections, which would cause any subsequent connections received to fill up the OS connection backlog, until the SO_BACKLOG limit is reached, at which point, the OS will start rejecting connections outright.

Correct, that its not a real “rejection” but more like “dropping on the floor”.

Is my understanding here correct?  Before we go and implement it ourselves, does Netty provide any built in channel handlers that implement connection limit logic?  I imagine it wouldn't be hard to do - you just need a way to communicate from the child channel pipelines when a channel has been closed so that the connection counter can be decremented and read can be invoked again on the channel receiving the connections.

There is no ChannelHandler doing this at the moment but its on my ever-growing to-do list. As you stated it should not be too hard to do. Just have a ChannelHandler that is shared between accepted connections and have a counter that is increased / decreased. Once you hit the limit call “channel.parent().config.setAutoRead(false)” and once you want to accept more use “channel.parent().confing.setAutoRead(true)”.

And finally, all this is in the context of Netty 4 - does the same apply to Netty 3, using Channel.setReadable() to implement the backpressure?  Play of course is on Netty 4, but we have a very large user who are asking about this for an older version of Play which is on Netty 3.

In theory yes, that said I would look again into the netty 3 code base if there are any gotchas.

Cheers,

James

--
You received this message because you are subscribed to the Google Groups "Netty discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to netty+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/netty/43802c13-afb0-40e9-99fc-ff0b52ed3f9a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Julien Viet]

wouldn’t it be better to implement this at the protocol level instead of TCP ?

for example with a 503 response for a web server.

To view this discussion on the web visit https://groups.google.com/d/msgid/netty/CAKrGPjwCsERNG-qEPzgk%2BCdD7A81sJvMAuqAHvg5YRwamEPXNg%40mail.gmail.com.

[James Roper]

On 2 Jun. 2017 11:10 am, "Julien Viet" <jul...@julienviet.com> wrote:

wouldn’t it be better to implement this at the protocol level instead of TCP ?

for example with a 503 response for a web server.

Sending a 503 consumes significantly more resources than not accepting the connection in the first place - it depends on your use case. If you want to prevent an already overloaded server from becoming even more overwhelmed, this distinction might matter.

On May 10, 2017, at 8:31 AM, 'Norman Maurer' via Netty discussions <ne...@googlegroups.com> wrote:

Hey James,

comments inline.

On 10. May 2017 at 04:22:20, James Roper (jro...@gmail.com) wrote:

Hi all,

I just want to validate my understanding of Netty here and how server connections are accepted.  A typical Netty server has two conceptual channels, one channel is effectively a channel of channels, representing a stream of incoming connections, and that one creates a new channel pipeline to handle each incoming connection.

Correct.

On the server channel of channels, if you set auto read to false, then this will allow you to backpressure on the incoming stream of connections, which would allow you to use this to implement a connection limit.  In that case, if you did back pressure (ie, by not invoking read() when you receive a read complete event), Netty would stop accepting new connections, which would cause any subsequent connections received to fill up the OS connection backlog, until the SO_BACKLOG limit is reached, at which point, the OS will start rejecting connections outright.

Correct, that its not a real “rejection” but more like “dropping on the floor”.

Is my understanding here correct?  Before we go and implement it ourselves, does Netty provide any built in channel handlers that implement connection limit logic?  I imagine it wouldn't be hard to do - you just need a way to communicate from the child channel pipelines when a channel has been closed so that the connection counter can be decremented and read can be invoked again on the channel receiving the connections.

There is no ChannelHandler doing this at the moment but its on my ever-growing to-do list. As you stated it should not be too hard to do. Just have a ChannelHandler that is shared between accepted connections and have a counter that is increased / decreased. Once you hit the limit call “channel.parent().config.setAutoRead(false)” and once you want to accept more use “channel.parent().confing.setAutoRead(true)”.

And finally, all this is in the context of Netty 4 - does the same apply to Netty 3, using Channel.setReadable() to implement the backpressure?  Play of course is on Netty 4, but we have a very large user who are asking about this for an older version of Play which is on Netty 3.

In theory yes, that said I would look again into the netty 3 code base if there are any gotchas.

Cheers,

James

--
You received this message because you are subscribed to the Google Groups "Netty discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to netty+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/netty/43802c13-afb0-40e9-99fc-ff0b52ed3f9a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "Netty discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to netty+unsubscribe@googlegroups.com.

[Julien Viet]

agreed, it can be done using a soft and hard limits, above the soft limit return an HTTP error, above the hard limit, refuse the connection.