DEADLINE - What rule of thumb do you use for high throughput

[Nicolas Cheutin]

Hi,

We have dozens of services chatting with GRPC, they all have high throughput, up to 5-6K RPS @10ms P99 per instance.

As we tune our services for always better efficiency (which means substantial Cloud costs savings), there is always the deadline triggering discussions.

If we take our services with highest throughput, we stand at:

- 10ms P99

- 20ms P999

- 1 retry

- Deadline: 80ms

Question often comes back on whether we should be more aggressive on deadline and have more retries VS higher deadline and less retries.

What are you folks using as strategy? any rule that helps you define appropriate deadline?

Cheers,

Nico