how to use back-pressure?

867 views
Skip to first unread message

Nicolas Ocquidant

unread,
Mar 30, 2017, 7:09:31 AM3/30/17
to vert.x
Hi,

I've just finished reading the vertx documentation, and regarding Pump and back-pressure, I need an advice ;)
I develop a middleware which receives HTTP requests, do some transforms (using Buffers), then sends TCP requests to a server with NetSockets. 
What options do I have here to prevent a memory overflow (if I receive more HTTP requests than I could send TCP ones)? I would prefer a "clean" exception than an OutOfMemory one...

Thanks
--nick

Julien Viet

unread,
Mar 30, 2017, 12:07:28 PM3/30/17
to ve...@googlegroups.com
Hi,

the max memory consumed by the file transfer depends on the intermediary buffer queues in the pipeline of the pump.

For each of these you need to know their settings (possibly setting them explicitly):

- HttpServer : HttpServerOptions#setReceiveBufferSize / HttpServerOptions#setSendBufferSize (perhaps not used much but still counts)
- NetClient : NetClientOptions#setSendBufferSize / NetClientOptions#setReceiveBufferSize (perhaps not used much but still counts)
- NetSocket#setWriteQueueMaxSize (default is 64k)

So you can estimate a buffer size per connection and the pump should not consumer more than this size.

You could set a total memory budget that you can account for each connection. When you accept a new http request, if the memory budget is limited, you should avoid this request by returning an appropriate HTTP status response.

I recommend writing a work load test and do proper testing and monitor the JVM heap metrics and GC.

Julien

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/d5abeeb4-8f57-4a05-a480-cca4ae49c0fd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Nicolas Ocquidant

unread,
Mar 30, 2017, 3:46:33 PM3/30/17
to vert.x
Thank you very much for your answer.

The pump you are referring to is the internal mechanism of vertx or do you suggest me to add a pump somewhere?

If I need a pump, I don't know where to put it exactly in my code below...

@Override
public void start(Future<Void> fut) {
   Router router = Router.router(vertx);
   router.route().handler(BodyHandler.create());
   router.post("/").handler(this::handlePost).produces("application/json");

   // Create the HTTP server
   int port = CONF_SERVER_PORT.getInteger(env);
   vertx.createHttpServer().requestHandler(router::accept).listen(port, result -> {...});
}

private void handlePost(RoutingContext routingContext) {
   HttpServerResponse response = routingContext.response();
   JsonObject command = routingContext.getBodyAsJson();
   
   NetClientOptions options = new NetClientOptions().setConnectTimeout(10_000).setIdleTimeout(10);
   NetClient client = vertx.createNetClient(options);
   client.connect(port, host, res -> {
      if (res.succeeded()) { // Connected!
         NetSocket socket = res.result();
         // Send request
         socket.write(Buffer.buffer(CodecUtils.encode(command)));
         // Handle client
         socket.handler(new NetResponsePipeline(vertx, env, socket, response));
         // Close client when socket is closed
         socket.closeHandler(e -> client.close());
      } else {...}
   });
}

OK, so I understand that I must do a work load test and adjust the size of the buffers accordingly.

Is there a simple way to limit the number of opened connection either for the HTTP server or the TCP client, or do I have to handle it myself using a CountDownLatch?

Thanks again
--nick

Julien Viet

unread,
Mar 30, 2017, 4:07:39 PM3/30/17
to ve...@googlegroups.com
On Mar 30, 2017, at 9:46 PM, Nicolas Ocquidant <nocqu...@gmail.com> wrote:

Thank you very much for your answer.

The pump you are referring to is the internal mechanism of vertx or do you suggest me to add a pump somewhere?

If I need a pump, I don't know where to put it exactly in my code below…

I’m referring to Vert.x Pump that you setup after connect is succeeded()

a simple long is enough:

1/ check the value before client.connect and increment it
2/ if after connect res.failed() : decrement it 
3/ if after connect res.succeeded() : decrement it in socket close handler

looking at your code : don’t create a NetClient for every post, just create one single NetClient in start(Future<Void>) instead

Jochen Mader

unread,
Mar 31, 2017, 3:27:59 AM3/31/17
to ve...@googlegroups.com
I hope I read your description correctly.
The problem is a limited amount of available Connections to your backend.
In that case it might be better to also do rate limiting.
Let me explain:
Bacpressure will cause an open connection to not accept further content for an unspecified amount of time. This can cause trouble with some HTTP-clients.
I would DEFINITELY do backpressure but I would also limit the amount of parallel HTTP-requests.
If the number of HTTP requests exceeds the number of available NetSockets you send back HTTP 429 to signal the client to come back later.
With this you won't waste resources on both, server and the client side and the client will retry later on.


To unsubscribe from this group and stop receiving emails from it, send an email to vertx+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+unsubscribe@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/56702FA1-4A60-40DD-BA02-478F2F338579%40julienviet.com.

For more options, visit https://groups.google.com/d/optout.



--
Jochen Mader | Lead IT Consultant

codecentric AG | Elsenheimerstr. 55a | 80687 München | Deutschland
tel: +49 89 215486633 | fax: +49 89 215486699 | mobil: +49 152 51862390
www.codecentric.de | blog.codecentric.de | www.meettheexperts.de

Sitz der Gesellschaft: Düsseldorf | HRB 63043 | Amtsgericht Düsseldorf
Vorstand: Michael Hochgürtel . Rainer Vehns
Aufsichtsrat: Patric Fedlmeier (Vorsitzender) . Klaus Jäger . Jürgen Schütz

Nicolas Ocquidant

unread,
Mar 31, 2017, 9:36:10 AM3/31/17
to vert.x
I intended to put this in place. 
Thank you for your input. 

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages