client-side retries and idempotency

1,272 views
Skip to first unread message

Michael Day

unread,
Aug 3, 2016, 8:43:08 PM8/3/16
to grpc.io
Hi everyone,

I'm attempting to implement client-side retries using the existing retry interceptors in open pull requests.  One of the interceptors (https://github.com/grpc/grpc-java/pull/1570) relies on MethodDescriptor's idempotent field, yet I'm not sure how to set this via my proto definitions.  I'm aware that I could probably work around it using custom MethodOptions, but I thought I would ask the list in case there's something obvious that I'm missing.

Josh Humphries

unread,
Aug 3, 2016, 9:17:13 PM8/3/16
to Michael Day, grpc.io
A custom method option in the proto file is the way we're doing this at Square. We also use other custom method options to influence other kinds of retry policies not implemented in that pull request: latency-triggered retries, where another attempt is made before the first call completes if it's taking too long.

Michael Day

unread,
Aug 4, 2016, 10:30:19 AM8/4/16
to grpc.io, m...@clickverge.com
Thanks. Do you guys also do most of your retries on the client-side? For example, for optimistic locking exceptions, we return ABORTED status. In general, do you think it's better to automatically retry those failures on client-side?

Josh Humphries

unread,
Aug 4, 2016, 10:34:08 AM8/4/16
to Michael Day, grpc.io
It depends on what is being retried. We definitely do client-side retries of RPCs. We have rules on what response codes are safe to retry (which is greatly expanded when the method is flagged as idempotent). But some optimistic concurrency operations, like storage transactions that can abort due to interference, will also get retried on the server (with limited retry count in both contexts). We also propagate deadlines (e.g. an outbound RPC will "inherit" its currently deadline if we are currently handling an inbound RPC) to prevent unnecessary retries.


----
Josh Humphries
Payments Engineering
Atlanta, GA  |  678-400-4867

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscribe@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/2ffab03b-fad5-49e4-b099-9865356ba6a9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Reply all
Reply to author
Forward
0 new messages