Questions about RetryPolicy and WriteType

[Stefano Ortolani]

Hi, 

I am looking at the different RetryPolicies in the python driver and while RetryTimeout and WriteTimeout are clearly handled by "on_read_timeout' and "on_write_timeout" methods, I was wondering how could I deal with "OperationTimedOut" (timeouts in the driver). 

Is there a way to specify a policy or my only chance is increasing its value?

Also, I was looking at the "DowngradingConsistencyRetryPolicy" and in particular at its behavior when "on_write_timeout" is triggered.

In that case, if WriteType = SIMPLE, the timeout is ignored. I can't seem to understand the rationale why it is safe especially if ConsistencyLevel is set to 1.

How can I know that the write (simple INSERT/UPDATE/DELETE) took indeed place and the replica didn't die in the meanwhile?

Cheers,

Stefano

[Adam Holmberg]

OperationTimedOut is up to the application to handle. Increasing the timeout is commonly the best way to accommodate things that might take longer. There is also the option to catch OperationTimedOut and try again, but that could lead to retry explosions. Furthermore, retrying a failed operation under similar load, with the same consistency and timeout, could easily fail again.

Regarding DowngradingConsistencyRetryPolicy: you are correct that the documentation is inconsistent with the implementation. This is actually an issue that was raised recently:

https://datastax-oss.atlassian.net/browse/PYTHON-338

It will be in the next release.

Regards,

Adam Holmberg

To unsubscribe from this group and stop receiving emails from it, send an email to python-driver-u...@lists.datastax.com.

--

[Stefano Ortolani]

Thanks a lot Adam, everything is now crystal clear!

Cheers,

Stefano

[Stefano Ortolani]

I am looking at the git diff; so basically when the write_type is a WriteType.Simple, and when "received_response == 0", I can reasonably be sure that the write did not take place.

Would it be safe in your opinion to RETRY instead of RETHROW?

Cheers,

Stefano

On Thursday, July 16, 2015 at 6:24:01 PM UTC+1, Adam Holmberg wrote:

[Adam Holmberg]

We don't retry for these write types because the driver has no way of knowing if the write succeeded, and they are not necessarily idempotent.

For an easy example, consider a counter increment: if I put a counter increment request on the wire and get a timeout, all I know is I'm not sure if it *did* happen. I do not know for sure that it *did not* happen. If I implicitly retry, there's a chance of double-incrementing something. There are analogous cases in simple writes (list appends, for example). It may be the case that certain applications are okay with double-updating some things, but that is up to application semantics -- we can't apply that as a general rule. In those cases, it's perfectly valid to catch these rethrown exceptions and retry from the application level -- the application knows which statements are okay to run multiple times, and which are not.

I hope this explains the logic a little more. Thanks for the discussion.

Adam