Does it make sense to use worker verticle as http server?

[Ted C]

what the difference between the following two options ? 

  1. deploy 16 standard verticles as http server, and submit blocking jobs to a worker verticle pool

  2. deploy 16 worker verticles as http server, and submit blocking jobs to another worker verticle pool

I've done some pressure test for the two options, and I found the performances are very close. If there's no much difference, what is the advantage of using standard verticle and eventloop?

[Julien Viet]

Hi,

I think about 2. it depends if the worker is waiting for the job
submission to happen or not. Can you clarify ?

> --
> You received this message because you are subscribed to the Google Groups "vert.x" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/c3c5fa10-b4ec-4d50-bbb5-ddcb61a71bafn%40googlegroups.com.

[Blake]

Hey Ted,

I think it's more about the expectation? 

On a standard (non-blocking) verticle you don't expect operations to block and on worker verticles you DO expect operations to block for extended periods of time.

I believe there's nothing really different between what you're doing between the standard/worker verticle as you're basically treating the worker verticles as you are treating the standard verticle (i.e., you're offloading the blocking operation on both of them to another "blocking" pool).

The http server accepts a handler that is called when the request is ready (i.e., all of the data for the request has been received and now you can process it) if you were running your blocking operation on the http server verticle (standard or worker) then that thread cannot run any more operations until the blocking operation completes.

That being said, I'm really not sure where the performance will break down in terms if you had EVERY request running some blocking operation would it be better to just run it all on the verticle's thread anyway or is there still a benefit to passing them off to a worker verticle? I could speculate both ways on it, but would have to look at source more and probably just benchmark it.

[Ted C]

Hi julienviet & Blake, 

I'm so glad to receive your reply! 

I'll give some demo code to clarify my question. The code is as follow: 

 1. I write a `HttpVerticle` as http server, it handle http requests and submit blocking jobs to another `BlockVerticle` via event bus request message.

 2. In the `BlockVerticle` I simulate a blocking job by sleep 100 ms and reply the message.

 3. I deply 10 BlockVerticles(worker mode) and 4 HttpVerticles(standard/worker mode).

I performed a stress test by jmeter with 1000 request threads. 

Then I found whether I set the HttpVerticle as a worker or not, the performances are quite close.

The report is as follow: 

(HttpVerticle in standard mode)

(HttpVerticle in worker mode)

Question: 

If like what Blake said, 'there's nothing really different', why do we need the eventloop design and what is the advantage of using it. 

Maybe we can simply prepare some different thread pools for different purpose(eg. for blocking jobs and non-blocking jobs)? 

Looking forward to your reply And thanks a lot !!!

[Julien Viet]

Hi,

your HTTP server in this case is not blocking since it is only the event bus verticle that blocks, so indeed setting it as a worker is not needed.

When you set the HTTP server as worker, even if it is not blocking, it requires for execution to borrow a thread from the worker pool which is a more limited resource than if it were using the event loop. When the pool is heavily solicited then the HTTP server would wait for a worker to be available to handle the request.

In addition your HTTP server might handle other requests that could use non blocking clients. In this situation you can server some requests using non blocking and only use blocking for requests that need a blocking API exposed via the event-bus.

HTH

Julien

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/03679c9b-0ee2-4036-a7a8-42d062749d20n%40googlegroups.com.

[Ted C]

Hi Julien, 

Thanks for your reply!  

After trying to google for some information, I still don't understand why borrowing a thread from a worker thread pool (say have 16 threads in total) is more expensive than using an event loop thread. 

In the book Netty in Action, it says,

  Pooling and reusing threads is an improvement over creating and destroying a thread with each task, but it doesn’t eliminate the cost of context switching, which quickly becomes apparent as the number of threads increases and can be severe under heavy load. In addition, other thread-related problems can arise during the lifetime of a project simply because of the overall complexity or concurrency requirements of an application.

But however, 

1. if we use a separate thread pool with a small number of threads for non-blocking jobs, it should not cause serious context switching problem. 

2. we can still avoid concurrency problems, like what have done in the worker verticle model.

If it is true, what is the essential difference between using event loop and using a separate thread pool?

[Julien Viet]

Hi,

I am not saying it is expensive, I am saying that it is a shared resource.

If you have a worker pool of 16 and all threads are already borrowed to execute event-bus blocking actions, then when a new HTTP requests is processed, the HTTP server request handler will have to wait until a worker thread is available to process the request, even though it does not block.

I hope it makes sense

Julien

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/9ec1b1c6-b029-4a8d-8c76-c9f9d1e7f295n%40googlegroups.com.

[Ted C]

Hi Julien,  

Thanks for your patient explain.  

But If I use a separate thread pool to handle non-blocking requests (there's another thread pool to handle those blocking jobs), then what you said should not be a problem. 

Is it right?

[Julien Viet]

it might be a problem with itself, but it should not.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/69156ee3-b9f2-47c4-85ab-393e36126422n%40googlegroups.com.

[Ted C]

hmmmm...  what do you mean by 'it might be a problem with itself'?

[Julien Viet]

if you are using a worker pool on the HTTP server and block the thread for some time then fast requests will be impacted by slow requests that block the worker thread.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/4a312021-219c-4f81-9e12-b68b7c4f858fn%40googlegroups.com.

[Ted C]

but we can create a different thread pool for http server, then fast requests will not be impacted by slow requests. what the difference between using a different thread pool and using event loop?

[Ted C]

I just don't understand that why not simply create a different thread pool to handle those fast requests instead of using event loop.