Send Http request on one event loop and receive response on another

[Michael Pog]

I'm building an application with Vertx, which does Http multi proxying. In other words I have multiple verticles, each on a separate thread/eventloop. Each verticle is an http server and http client. 

When an incoming request comes in, one of the verticles takes that request (handled by Vert.x) and sends out many outbound requests. 

Because I send out the requests on the same thread/even loop, the responses come back on the same thread/event loop as well.

However I believe the performance would be much better if I could receive the responses on any of the verticle instances - more parallelism (Just like when you listen on the same port with an http server, in multiple verticles, you may receive the request on any of the listening verticles)

Is there a way I could do that or will there be an option like that down the road?

Thanks

[Julien Viet]

for which reason do you believe it would it be more performant ?

the bottleneck you need to care about for http client parallelism is rather the connection pool size and if you are using keep-alive / pipelining

i.e most of the time your requests to the remote host are sent on the same connection for performance reason, so you believe they are sent in parallel but in reality the requests are serialized on the connections.

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/ccdc3096-cc77-4755-9b3e-6f30c16ea094%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Michael Pog]

The application I have is sending many outbound requests through its HTTP client, the requests are for different services/hosts. The responses come on different connections, therefor it would be more performant if I could receive those responses on different threads, in parallel, instead of on the same thread from which the requests were sent.

[Julien Viet]

Hi,

when you scale a Vert.x HttpServer on several verticles, each verticles will have its own HttpClient instance using the same event loop.

So if your verticles uses as you said an HttpServer + an HttpClient, this verticle’s HttpClient will be on the same event loop than the HttpServer. If you deploy 3 instances, there will be 3 event loops using the HttpServer+HttpClient combo.

Julien

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/9f7032d0-bd69-4275-b3eb-3260c11866fd%40googlegroups.com.

[Michael Pog]

Hi, 

Thanks for replying to my question.

That's precisely what I'm doing, I'm creating a verticle of http server + http client, and running as many instances of that verticle as the number of cores.

That works great, in theory and in benchmarking, where I send the same request to the application that "fan out" into a fixed number of outbound requests. 

In that case, all threads are equally busy, and there's no asymmetry. The load is even. 

However in practice/production, the number of outbound requests (to different hosts and services) depends on the incoming request and logic. So I may get one request, on one thread, that fans out into 30 outbound requests, and another verticle instance/thread gets a request that fans out into 2 outbound  requests. In that case the load is uneven. Some thread will be very busy at times, where others will be almost idle at times.

So that's why I think if I could at least receive the responses on different threads I could spread the load more evenly. 

Say I have 2 verticle instances, one sends out 30 requests, and another sends out 2. If I could got all 32 of the responses in the same time, it would be much better if one thread handled 16 and another 16, instead of the first thread handling 30 sequentially, and the other thread just handling its 2 responses.

I hope I make more sense now.

Thanks

[Julien Viet]

yes your response clarifies.

the use case you described can be handled using the EventBus by moving the HttpClient logic in a EventBus handler and then you can control how many handlers are deployed.

Julien

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/baadc7f9-401b-4042-857b-f688c10287cb%40googlegroups.com.

[Tim Fox]

>> However in practice/production, the number of outbound requests (to different hosts and services) depends on the incoming request and logic. So I may get one request, on one thread, that fans out into 30 outbound requests, and another verticle instance/thread gets a request that fans out into 2 outbound  requests. In that case the load is uneven. Some thread will be very busy at times, where others will be almost idle at times.

That's only going to be true in the case where you have a small number of concurrent inbound requests. If your server is processing many requests at once the load will be distributed across the verticles, so while some may fan out to 30 outbound requests and others to only 2, that will even out over all inbound requests so all cores will be approximately evenly utilised. It's only in the case where you have very little traffic where you would see asymmetry, and in that case you might find performance is good enough anyway.

With this kind of thing I'd do some tests first with the traffic you expect to handle before committing to architectural choices that might be unnecessary :)

[Jez P]

+1. Don't try and over-optimise prematurely. You might be making your architecture more complex than necessary, for little or no benefit. 

Premature optimization on the basis of what "may" happen is almost always a mistake. Build performance tests based on real traffic expectations as trials for your architecture and design. Honestly, the bottlenecks are far more likely to prove to be where you're not looking, not where you are. If you know you will have elements of really heavy logic for some requests, use the eventbus to push that logic to worker verticles or use executeBlocking to take them off the event loop anyway. If they're not heavy enough to be taken off the eventloop, there's probably little to worry about.

[Jez P]

That should say "if you know for sure". But rule 1 of optimization is always "measure, don't guess". Right now, you're doing more of the latter than the former.

[Julien Viet]

and before measuring, define your throughput / latencies expectations :-)

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/413c3a58-9061-432e-abb2-0aa39fb90132%40googlegroups.com.

[Michael Pog]

Yeah, that makes sense.

Thank you guys!!! That was very helpful.