What makes Vertx http client asynchronous?

[Igor Spasić]

The vertx HTTP client must be executed on some thread to call the target and wait for response.

Which thread is it? It is not event loop, obviously. It is not the working thread (as far as I figured).

So which thread is executing this?

Or even more: how can I make my own client asynchronous in the same way vertx is? Just calling executeBlocking is not enoguh?

Thank you!

[Igor Spasić]

What I meant is : non-blocking.

[ad...@cs.miami.edu]

Hi,

It is a large amount of effort to make a non-blocking http client from scratch.  It is all based on Java NIO (which is fairly lowlevel).  A number of good libraries have been built to assist in building NIO java applications (Grizzly, Mina, Netty, etc).  Vertx is based on Netty.  You an use Netty to build your own NIO http clients the same way as vertx.  You may want to check out the first few chapters of Netty in Action : https://www.amazon.com/Netty-Action-Norman-Maurer/dp/1617291471  to get a sense on how NIO works in general.

I found this summary online and thought it was a pretty good summary so I will re-post it:

Java NIO enables you to do non-blocking IO. For instance, a thread can ask a channel to read data into a buffer. While the channel reads data into the buffer, the thread can do something else. Once data is read into the buffer, the thread can then continue processing it. The same is true for writing data to channels.

The flow is something like the following (as I understand it):

a) main threads make an NIO call for data with callback
b) the main thread continues doing stuff
c) a small number of NIO threads wait on the data (selectors) and then execute the callback when there is data to process

Since step c can use a small number of threads to observe many channels, this scales very well to many "main threads".

> Just calling executeBlocking is not enoguh?

Often it is.  Is there a particular senario you are thinking about?

Best,

-Adam

[Igor Spasić]

Thanx Adam!

Well, the issue I am having is with nested http calls - app gets frozen because handler A depends on handler B that is after him in the event loop. So I am exploring the right way to deal with this situation.

[Mumuney Abdlquadri]

If I understand you well,

http://vertx.io/docs/vertx-core/java/#_async_coordination

The link might help you.

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/a0eee843-5bbd-476b-b5a1-7ff2c991d128%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Julien Viet]

The http client makes the connection as soon as sendHead(), write(..) or end(..) is called, the connection request is sent using the current vertx thread or if you are calling from a main, a vertx thread obtained (it uses getOrCreateContext()). The callback will be done on this thread.

you should wrap your code in a Verticle, it is very simple : https://github.com/vert-x3/vertx-examples/blob/master/core-examples/src/main/java/io/vertx/example/core/http/simple/Client.java

Julien

--
You received this message because you are subscribed to the Google Groups "vert.x" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.

Visit this group at https://groups.google.com/group/vertx.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/da9622b6-ccf0-4db9-b4c1-99b116bfc2d9%40googlegroups.com.

[Igor Spasić]

Thanx Julien! Forgot to tell that my code is executed already from verticle.

Will check it out!

[Mumuney Abdlquadri]

Hi Igor, Sorry I read that as nested async calls... Take to what other commenters have said.

Probably show the code you already have so you can be better assisted.

On Wed, Oct 5, 2016 at 8:42 AM, Julien Viet <jul...@julienviet.com> wrote:

The http client makes the connection as soon as sendHead(), write(..) or end(..) is called, the connection request is sent using the current vertx thread or if you are calling from a main, a vertx thread obtained (it uses getOrCreateContext()). The callback will be done on this thread.

you should wrap your code in a Verticle, it is very simple : https://github.com/vert-x3/vertx-examples/blob/master/core-examples/src/main/java/io/vertx/example/core/http/simple/Client.java

Julien

On Oct 4, 2016, at 11:54 PM, Igor Spasić <igor....@gmail.com> wrote:

The vertx HTTP client must be executed on some thread to call the target and wait for response.

Which thread is it? It is not event loop, obviously. It is not the working thread (as far as I figured).

So which thread is executing this?

Or even more: how can I make my own client asynchronous in the same way vertx is? Just calling executeBlocking is not enoguh?

Thank you!

--
You received this message because you are subscribed to the Google Groups "vert.x" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/da9622b6-ccf0-4db9-b4c1-99b116bfc2d9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "vert.x" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/vertx.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/16C7AFE5-C7B8-453C-8761-2366DE10CCD5%40julienviet.com.

[Igor Spasić]

Well basically this is what we have.

+ some reasonable number of verticles (cpu#*2)

+ on each event loop we have a handler that calls other URL on the server using executeBlocking. It is a nested call.

+ This other URL is on the same server. It might execute more nested calls.

What happens is that if we have 20 working loops, as soon as 20-ish requests come in the same time, the occupy all working threads; Each calls the nested call on the same server, but since there is no working thread left to process it, server gets frozen.

I use 'false' argument when executeBlocking, of course.

For me this is an microservice usage anti-pattern (invoking itself), however, I have to fix it ;)

On Wednesday, October 5, 2016 at 9:48:20 AM UTC+2, Mumuney Abdlquadri wrote:

Hi Igor, Sorry I read that as nested async calls... Take to what other commenters have said.

Probably show the code you already have so you can be better assisted.

On Wed, Oct 5, 2016 at 8:42 AM, Julien Viet <jul...@julienviet.com> wrote:

The http client makes the connection as soon as sendHead(), write(..) or end(..) is called, the connection request is sent using the current vertx thread or if you are calling from a main, a vertx thread obtained (it uses getOrCreateContext()). The callback will be done on this thread.

you should wrap your code in a Verticle, it is very simple : https://github.com/vert-x3/vertx-examples/blob/master/core-examples/src/main/java/io/vertx/example/core/http/simple/Client.java

Julien

On Oct 4, 2016, at 11:54 PM, Igor Spasić <igor....@gmail.com> wrote:

The vertx HTTP client must be executed on some thread to call the target and wait for response.

Which thread is it? It is not event loop, obviously. It is not the working thread (as far as I figured).

So which thread is executing this?

Or even more: how can I make my own client asynchronous in the same way vertx is? Just calling executeBlocking is not enoguh?

Thank you!

--
You received this message because you are subscribed to the Google Groups "vert.x" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.

Visit this group at https://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/da9622b6-ccf0-4db9-b4c1-99b116bfc2d9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "vert.x" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.

Visit this group at https://groups.google.com/group/vertx.

[Julien Viet]

why are you using executeBlocking at all ?

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/d1397ff6-b1d3-483c-8f38-1b71981c137c%40googlegroups.com.

[Igor Spasić]

because the target URLs handler (the nested one) calls DB to do some checks.

And we use custom http client, which in blocking one

Lets say we can't change this at the moment.

[Mumuney Abdlquadri]

Are you using any of these http://vertx.io/docs/#data_access

If yes, you still do not executeblocking in your code.

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/vertx.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/d4cd8596-77af-404d-b21a-4f0e2b73b67c%40googlegroups.com.

[Igor Spasić]

Its ElasticSearch in there.

but lets say this can not be changed. That is the answer I always get; but lets say we can't change this (for various reason we have).

I still want to find the best way to call the nested call and not to get server freeze.

Currently, if I have 20 verticles, if I fire 21 requests at the same time, I got the server freeze, as explained above.

[Igor Spasić]

Sorry not 20 verticles, but the size of working thread pool

[Mumuney Abdlquadri]

So basically, you have architected you system to block by default. 

If I may ask how are you calling Elastic search. I think it has some HTTP API...

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/vertx.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/d5262885-090c-418d-9007-123619ad1c47%40googlegroups.com.

[Igor Spasić]

Yeah, ES has a java client, and it is HTTP as it is on the other server.

To recap: we have a set of handlers that are blocking, and that call it selfs (nested calls) 

I dont think we can remove blocking nature of these operations; and they dont want to remove the usage of nested http calls.

I just wonder if there is anything smart I can do for this set of handlers, to work better? I was thinking into not using executeBlocking, but a common java executors thread pool just for nested calling. If we have e.g. max 3-level deep nested calls, then having a pool of eg 2x<number_of_working_threads> threads would sound fine.

[Mumuney Abdlquadri]

I don't know if the Java Client for ES is blocking by default.

But these are things you might do. Remove the executeBlocking() calls from the HTTP call level into the level that interacts with ES. I mean only the code that talks to ES should be run in executeBlocking()...

Then try to use Futures to coordinate async operations.

Please do not manage threads by yourself in a vert.x app.

I am still curious to see a snippet of code you have on this.

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/vertx.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/a2125c78-9014-414c-a806-3023a6dad201%40googlegroups.com.

[Jez P]

If you're doing blocking code you're going to be constrained by the worker thread pool. You could make the worker thread pool huge so you can do a lot of concurrent blocking operations, but this will cause a lot of context switching and hammer your scalability. The whole point of doing non-blocking i/o and running a small number of eventloop threads is to keep the total thread count small to minimise context switching between them. Blocking i/o is in direct conflict with this approach.

Since blocking code blocks a thread, you must take it off the event loop (or your problems will be even worse). Therefore because you have blocking code, which must be run on a worker pool, you are going to be constrained by the size of that worker pool. Sorry for being the bearer of bad news, but if you can't change the blocking code to be non-blocking, then your only real option is to sacrifice scalability and boost the size of the worker pool. If you really must do this (because you can't make your code non-blocking) I'd suggest your best option is to distribute your workload over several machines using worker verticles, so that you effectively have one non-blocking server which delegates to n instances (via the eventbus) of your blocking code executor worker verticle - if you distribute n can be much larger without sacrificing overall throughput, but obviously requires more machines, so there is an associated cost.