Dedicated jetty thread pool for latency-sensitive endpoints.

[Pedro Cardoso]

Hello all,

I have a dropwizard application where one of the endpoints is meant to be requested orders of magnitude more than other endpoints, let's call it endpoint x, which is in the critical path of a latency-sensitive workload.

This endpoint will be hit 5k+ times per second at peak times with average rates being anywhere from 100 - 500 times per second and is meant to respond within 2-5 milliseconds.

At the same time, I have other endpoints which are infrequently hit but which access databases and applies logic which is substantially slower (think operational management of a distributed set of machines and applying changes over this cluster). 

Is there a way to define this endpoint x to have a dedicated thread pool and jetty instance such that blocking operations on other endpoints (operational and even dropwizard's metrics endpoints) do not affect the application's ability to respond to the more important one?

Additionally, do you have any advice on how I can disable deserialization logic to not pay that cost? I already handle serialization/deserialization in another part of the codebase and if I could simply send bytes down the wire it would be perfect

I can't seem to find any documentation or general information about this. Can someone help me out?

Thank you for your time.

Have a great rest of day!

[Ryan Kennedy]

Pedro:

 rather than have additional Jetty thread pools, I recommend having a look at the JAX-RS AsyncResponse mechanism for processing requests asynchronously. This will allow you to push your resource processing off to other thread pools (e.g. via an ExecutorService). This then causes your service to use the Jetty thread pool momentarily while you attempt to submit tasks to the ExecutorService, which will complete the request via the AsyncResponse object. In this way you can create separate thread pools for different activities within your service. I did this previously to push my service to make heavy use of CompletableFuture composition, performing synchronous, blocking tasks (e.g. JDBC calls) in a dedicated thread pool.

A nice feature of this is that you can cap the size of the additional thread pools, allowing you to control the maximum concurrency. If you fail to submit a task to the ExecutorService because the underlying thread pool is full, you can return an HTTP 503 with a Retry-After response header to indicate to the client that you're busy and that they should try again after a brief pause.

As for your other question regarding serialization, I believe your REST resource methods can accept an InputStream parameter, which will give you raw access to the bytes of the request body. This would allow you to bypass any deserialization that would otherwise be performed by JAX-RS. You may need to take extra special care, however, to ensure that the InputStream is read fully. I've seen a number of service/client implementations suffer due to InputStreams not being fully read. It looks like you may also be able to have a byte[] parameter (via Stack Overflow).

Ryan

The content of this email is confidential and intended for the recipient specified in message only. It is strictly prohibited to share any part of this message with any third party, without a written consent of the sender. If you received this message by mistake, please reply to this message and follow with its deletion, so that we can ensure such a mistake does not occur in the future.

--
You received this message because you are subscribed to the Google Groups "dropwizard-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dropwizard-us...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dropwizard-user/7ebfeb6d-3f13-41d7-a8f5-8c3614b263een%40googlegroups.com.