What's the recommended heartbeat timeout negotiation technique?

[vitaly numenta]

It appears that pika (and perhaps some other clients) picks the min of the one proposed by the server and the one proposed by the client's user. Thus, there is no way for the user to get a larger timeout. This is especially critical since RabbitMQ 3.5.5 reduced default heartbeat timeout from 580 to 60 seconds.

[Michael Klishin]

Java client picks the greatest value. Other clients usually copy what it does. Pika should,
too:
https://github.com/rabbitmq/rabbitmq-java-client/blob/master/src/com/rabbitmq/client/impl/AMQConnection.java#L367

I don't think values higher than 60 seconds make a lot of sense. You probably want a value
of 10 or so. We picked 60 as new default because 10 can be too low but 580 just wasn't reasonable
and most people eventually ended up lowering it significantly. 
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[vitaly numenta]

Thank you Michael, I opened https://github.com/pika/pika/issues/665 to address this discrepancy.

[vitaly numenta]

Since pika.BlockingConnection presently doesn't doesn't service heartbeats in the background, the ability to set a larger heartbeat timeout, but not disable it entirely, is preferable.

[Ján Kianička]

In our project we had issue with network connections interuptions due to very long processing loops caused by Oracle database load.
We are using single threaded C applications and rabbitmq-c client.
The issue had to be mitigated by setting bigger heartbeat values on server side:
heartbeat variable in rabbitmq.config
In this case clients had wide range and bigger flexibility.
Jan

[vitaly numenta]

I've been giving this some more thought and am having trouble rationalizing picking the max value of broker's and app's heartbeat timeout during negotiation.

It seems that the value configured on the broker should serve as the default value for any connection, where the AMQP app doesn't provide an explicit heartbeat timeout value.

However, if the AMQP app provides an explicit heartbeat timeout value, it seems that the AMQP app knows the properties of its application best, and the values provided by the app should be respected verbatim (not the max).

For example, if the AMQP app has certain norms about how long processing of requests in the app should take (during which time communication may be blocked), it might set the heartbeat timeout at a value that would cause the connection to be dropped if the processing takes an unreasonably long to complete, based on the app's own understanding of "unreasonably long".

[vitaly numenta]

Okay, I think I convinced myself why max makes sense for heartbeat timeout negotiation (except the zero case, more on that below). Per https://www.rabbitmq.com/resources/specs/amqp0-9-1.pdf,

*Both peers negotiate the limits to the lowest agreed value* as follows:

•  The server MUST tell the client what limits it proposes.

• *The client responds and MAY reduce those limits* for its connection

When negotiating heartbeat timeout value, the reasoning about reducing needs to be reversed. The way I think it makes sense to interpret this rule for heartbeat timeouts is that the consumable resource is the frequency of heartbeats, which is the inverse of the timeout. More frequent heartbeats consume more resources than less frequent heartbeats. So, when both heartbeat timeouts are non-zero, we should pick the max heartbeat timeout rather than the min (to "reduce those limits"). The heartbeat timeout value 0 (zero) has a special meaning - it's supposed to disable the timeout. This makes zero a setting for the least frequent heartbeats (i.e., never); therefore, if any (or both) of the two is zero, then the above rules would suggest that the negotiation should yield 0 value for heartbeat, effectively turning it off.