Cannot use HttpClient in a worker verticle

[Trevor S]

Hi, is using httpclients in single threaded worker verticles restricited in vertx-3 for some reason?   Today, we use vertx httpclients in single threaded worker verticles in vertx2 extensively.

Thanks

SEVERE: Cannot use HttpClient in a worker verticle

java.lang.IllegalStateException: Cannot use HttpClient in a worker verticle

at io.vertx.core.http.impl.HttpClientImpl.<init>(HttpClientImpl.java:102)

at io.vertx.core.impl.VertxImpl.createHttpClient(VertxImpl.java:258)

[Trevor S]

Hi, any word on this? This is quite a breaking change from vertx2, and would make migrating to vertx3 very difficult.

Httpclient is a core vertx functionality.

I was thinking mAybe u meant to restrict multithreaded worker vertices, and not single threaded ones...

[Tim Fox]

Hi Trevor-

Can you explain the use case for why you want to do this in some more
detail?

[Trevor S]

HI Tim,

its the same use case for which vertx-3 now has executeBlocking.  You need to intersperse blocking and non-blocking code.  In vertx2, you do this in a worker verticle and this has some advantages as well ... 

we typically do everything in worker verticles and standard verticles are essentially just doing http routing to business logic on the opposite end of an event bus call.  This is great, as it enables us to distribute work across many processes.  In our worker verticles, we inject spring beans, make blocking calls, do some http GETS, POSTS, etc, do some more blocking calls (call db, do calculation), and return back over the eventbus.  We've standardized this approach to the point where we have a base class developers just extend, inject spring code and do their thing.   we can also size worker vertical thread pools, and associate some classes exclusively to some pools/verticle-instances, and and such we can can isolate/size certain logic - that is we can give our "auth" code 2 threads and "BusinessLogic" 10 threads/verticle-instances, lets say, because that allows better usage of the db.  as far as i can see, you cannot size executeBlocking or have different thread pools (worker verticles) for different things.

we use vertx httpclient as our httpclient so everything plays well with vertx...i.e. we don't want to use apache httpclient to make http calls when the vertx client works well.

i could convert all worker vertices to standard verticles and then wrap all blocking code in executeBlocking ... but now i am not only wrapping http calls with callbacks, but basically everything else as well.  Thats a whole nother later of callbacks which is not needed if you are already in a worker vertical.

Thanks!

ps my mac is trying to autocorrect "verticle" to "vertices"...uhg..

[Tim Fox]

Alright, thanks Trevor,

Let me investigate

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tim Fox]

IIRC,  one of the main reasons we don't allow HttpClients in worker verticles is they don't play nicely together.

Let's say you're in a worker and you're executing an HTTP request, and presumably you're blocking waiting for the result before you do the next thing, something like:

CountDownLatch latch = new CountDownLatch(1);

client.getNow(80, "blah.com", "/some-uri", resp -> {

      // Do something with response
      latch.countDown();
});

latch.await();

//Now do the next thing...

The trouble with the above is that I think it will block for ever, even if a response is received from the server.

This is because of the nature of verticles - Vert.x guarantees they are never executed by more than one thread at a time, so if the thread is blocked on the latch.await() then the thread is still executing in that verticle, so the response handler won't be called until the awaiting thread exits otherwise you'd have more than one thread executing in the verticle, which would break the Vert.x concurrency guarantees.

I think the same is true in Vert.x 2, so I am curious how you are getting this to work in 2...

On 07/05/15 13:05, Trevor S wrote:

[Trevor S]

We don't use it like that, because of the reasons u mention.

We'd have blocking calls before the http call then inside the http callback...not after. So, we'd nest appropriately. Works well, scales well.

Dbcall1
With result, make http call.
Inside httpcallback, save to db.

Or something to that effect. Very common use case IMO. Don't mind the nested callbacks, as we don't go too deep.

[Trevor S]

[Tim Fox]

Trevor,

I have made some changes which will now enable httpclient/server and
netclient/server to be used in worker verticles (and execute blocking).

The changes are waiting for review here
https://github.com/eclipse/vert.x/pull/1034

[Trevor S]

Thanks Tim!!

I can help test if needed, and contribute any documentation or short tutorial on how to use (and what to avoid) when using in a worker verticle, let me know.

[Dev Null Fin]

Hi,

I am unable to get HttpClient working in Worker Verticle. I am currently invoking EventBus call from Verticle and EventBus handler is running in worker verticle. 

Here is the code for HttpClient I am using in the Event Bus Handler. My server is not getting the request from client.

If I run the same code from Verticle, then it works. Is there any issue with running HttpClient in worker verticle?

val client : HttpClient = vertx.createHttpClient().setHost("hostname").setPort(12002);

val proxyReq : HttpClientRequest = client.request("get", "/rest/url",
  new Handler[HttpClientResponse]{
  override def handle(event: HttpClientResponse): Unit = {
    System.out.println("Proxying response: " + event.statusCode())
    event.dataHandler(new Handler[Buffer] {
      override def handle(buffer: Buffer): Unit = {
        System.out.println("Got Response")
        message.reply(buffer);
      }
    })
  }
})

proxyReq.headers().set("Accept", message.body().getString("Accept"))
proxyReq.headers().set("Accept-Language", message.body().getString("Accept-Language"))
proxyReq.headers().set("Accept-Encoding", message.body().getString("Accept-Encoding"))
proxyReq.headers().set("Cookie", message.body().getString("Cookie"))
System.out.println("EventBus Cookie : " + message.body().getString("Cookie"))
System.out.println("Client End")
proxyReq.end()

Am I doing anything wrong?

[Dev Null Fin]

I am using Vertx 2.

[Tim Fox]

Most probably you are blocking waiting for the result, but without seeing your actual code I can't tell.

--

[Trevor S]

Not sure about your code, but the httpclient works fine in worker verticles in vertx2.