Questions on gRPC Java

[Carl Mastrangelo]

Hello!

I have been using gRPC a lot recently, and after having worked on it too, I have encountered some some questions I couldn't find the documentation for.  I was hoping to have the expected behavior confirmed here, and maybe get the answers into the documentation.

1.  What thread do ServerInterceptors run on?   I would assume it's the ServerBuilder executor, but that presents a problem.  To limit the number of threads, we need to limit the number of calls.  The ServerInterceptor is the natural place to do this, but if the executor is used to enforce the call limit, it's too late.  We already spent a thread on it.

2.  What happens when there is both a Context deadline and an AbstractStub deadline?  I couldn't find in the documentation which one takes precedence, or how they are combined.   I assume the Context deadline fires anyways if present, but that means that using AbstractStub.withDeadlineAfter() may be misleading.   The deadline would not be able to be increased.

Carl

[Eric Anderson]

Heyo!

On Tue, Jul 21, 2026 at 3:31 PM 'Carl Mastrangelo' via grpc.io <grp...@googlegroups.com> wrote:

1.  What thread do ServerInterceptors run on?   I would assume it's the ServerBuilder executor, but that presents a problem.  To limit the number of threads, we need to limit the number of calls.  The ServerInterceptor is the natural place to do this, but if the executor is used to enforce the call limit, it's too late.  We already spent a thread on it.

Yes, the ServerBuilder executor. You can track the concurrency present and fail the RPC within the interceptor. Yes, you used a thread, but for a short period of time. This is pretty easy and reasonably effective.

If you limit the number of threads, then you've moved the problem to the queue that forms. Ordinarily you'd configure the executor to throw RejectExecutionException, but that isn't supported. Somewhere I remember mentioning the idea of having yet another executor, and only use it for cancelling RPCs. It could be a single thread. But that's not implemented, although it wouldn't be that hard.

We added ServerBuilder.callExecutor() which lets you change the executor for an RPC. This is great if you want to segment your services to separate pools. ServerCallExecutorSupplier is called on the ServerBuilder executor as well, because we have to look up the MethodDescriptor before we can create the ServerCall to provide context. ServerBuilder executor being used is fine from a resource usage perspective. But it can increase latency. If you don't have a ServerBuilder.fallbackHandlerRegistry() or you know it executes quickly, then you can use callExecutor(directExecutor()) and always return an executor from the ServerCallExecutorSupplier (to avoid using directExecutor() for application processing), without paying two thread hops.

For FIFO (based on RPC concurrently limit, not queue length), ServerCallExecutorSupplier could increment for each RPC and use an interceptor to decrement when complete. If this new RPC would exceed a threshold, fail the RPC, and either return null or a "cleanup" executor for the fast-running cancellation processing.

Part of the problem with RejectedExecutionException in Java is you can't implement LIFO to reduce average latencies. For LIFO, you have to get "fancy."

For LIFO (based on "queue length"), ServerCallExecutorSupplier can store call information in the returned executor to annotate the Runnables for that specific RPC. If it integrates with the executor's queue stack and wants to then kill RPCs at the bottom of the stack, it can, because it can identify where those Runnables came from and it knows the application is currently not receiving callbacks. It still has to run the Runnables to release resources, and because grpc-java doesn't support cancellation on server-side, I think you still need to pair this with an interceptor. It is probably much easier for unary RPCs since there is no need for potentially-slow application cleanup. I suspect nobody has actually tried this approach.

2.  What happens when there is both a Context deadline and an AbstractStub deadline?  I couldn't find in the documentation which one takes precedence, or how they are combined.   I assume the Context deadline fires anyways if present, but that means that using AbstractStub.withDeadlineAfter() may be misleading.   The deadline would not be able to be increased.

They don't impact each other; both apply. So the shortest "wins." This is no different than Context.withDeadlineAfter() on a Context that already has a deadline, which is also not documented, but due to cancellation cascading that's the only possible implementation. The child context can't outlive the parent.

[Carl Mastrangelo]

Hey Eric

Thanks for the reply.  A few inline responses:

On Tue, Jul 21, 2026 at 4:29 PM Eric Anderson <ej...@google.com> wrote:

Heyo!

On Tue, Jul 21, 2026 at 3:31 PM 'Carl Mastrangelo' via grpc.io <grp...@googlegroups.com> wrote:

1.  What thread do ServerInterceptors run on?   I would assume it's the ServerBuilder executor, but that presents a problem.  To limit the number of threads, we need to limit the number of calls.  The ServerInterceptor is the natural place to do this, but if the executor is used to enforce the call limit, it's too late.  We already spent a thread on it.

Yes, the ServerBuilder executor. You can track the concurrency present and fail the RPC within the interceptor. Yes, you used a thread, but for a short period of time. This is pretty easy and reasonably effective.

If you limit the number of threads, then you've moved the problem to the queue that forms. Ordinarily you'd configure the executor to throw RejectExecutionException, but that isn't supported. Somewhere I remember mentioning the idea of having yet another executor, and only use it for cancelling RPCs. It could be a single thread. But that's not implemented, although it wouldn't be that hard.

Yes, I was hoping that other executor would basically run on the network thread, with cancellation being a cheap thing.  Basically a concurrency limiter.  A hacky solution I have is a custom executor that checks for concurrency limits, and then funnels the callbacks to a secondary, thread pool executor.  Workable but less than ideal.

Minor request: I think having Javadocs on https://grpc.github.io/grpc-java/javadoc/io/grpc/ServerBuilder.html#executor(java.util.concurrent.Executor). would help clarify the usage.  I couldn't remember if this covered ServerCall methods, or the Interceptor methods too.  Actually it's kinda vauge what specifically the executors are intended to be used for.

 

We added ServerBuilder.callExecutor() which lets you change the executor for an RPC. This is great if you want to segment your services to separate pools. ServerCallExecutorSupplier is called on the ServerBuilder executor as well, because we have to look up the MethodDescriptor before we can create the ServerCall to provide context. ServerBuilder executor being used is fine from a resource usage perspective. But it can increase latency. If you don't have a ServerBuilder.fallbackHandlerRegistry() or you know it executes quickly, then you can use callExecutor(directExecutor()) and always return an executor from the ServerCallExecutorSupplier (to avoid using directExecutor() for application processing), without paying two thread hops.

For FIFO (based on RPC concurrently limit, not queue length), ServerCallExecutorSupplier could increment for each RPC and use an interceptor to decrement when complete. If this new RPC would exceed a threshold, fail the RPC, and either return null or a "cleanup" executor for the fast-running cancellation processing.

Part of the problem with RejectedExecutionException in Java is you can't implement LIFO to reduce average latencies. For LIFO, you have to get "fancy."

For LIFO (based on "queue length"), ServerCallExecutorSupplier can store call information in the returned executor to annotate the Runnables for that specific RPC. If it integrates with the executor's queue stack and wants to then kill RPCs at the bottom of the stack, it can, because it can identify where those Runnables came from and it knows the application is currently not receiving callbacks. It still has to run the Runnables to release resources, and because grpc-java doesn't support cancellation on server-side, I think you still need to pair this with an interceptor. It is probably much easier for unary RPCs since there is no need for potentially-slow application cleanup. I suspect nobody has actually tried this approach.

2.  What happens when there is both a Context deadline and an AbstractStub deadline?  I couldn't find in the documentation which one takes precedence, or how they are combined.   I assume the Context deadline fires anyways if present, but that means that using AbstractStub.withDeadlineAfter() may be misleading.   The deadline would not be able to be increased.

They don't impact each other; both apply. So the shortest "wins." This is no different than Context.withDeadlineAfter() on a Context that already has a deadline, which is also not documented, but due to cancellation cascading that's the only possible implementation. The child context can't outlive the parent.

That's kinda what I remembered too, but then it made me think that there wasn't a way to increase it.  Another request:  [Javadoc] guidance on when to use the Stub, CallOption, or Context for setting deadlines.   

One other idea:  Is it possible to fork a context with the current key-values, but drop the cancellation propagation?  Something like Context.ROOT.withAllKeys(Context.current()).attach().

 

[Eric Anderson]

Just a quick reply.

On Wed, Jul 22, 2026 at 1:26 PM Carl Mastrangelo <cmastr...@netflix.com> wrote:

That's kinda what I remembered too, but then it made me think that there wasn't a way to increase it.

Correct, you don't increase it. (except by fork(). see below)

Another request:  [Javadoc] guidance on when to use the Stub, CallOption, or Context for setting deadlines.   

There's no preference. Context is general purpose, so works even when not immediately calling gRPC, but is more annoying because you have to provide it a ScheduledExecutorService and cancel it yourself. Stub is a convenience to set CallOption.

One other idea:  Is it possible to fork a context with the current key-values, but drop the cancellation propagation?  Something like Context.ROOT.withAllKeys(Context.current()).attach().

Context.fork()