pubsub: change in Receive flow control

[Jonathan Amsterdam]

In v0.19.0, the latest release of the Google Cloud libraries for Go, there is a change in the behavior of pubsub.Subscription.Receive.

Receive controls the number of messages being processed by consulting ReceiveSettings.MaxOutstandingMessages/Bytes. Previously, the processing of a message was considered finished when Receive's callback function returned. As of this release, it is considered finished when Ack or Nack is called on the message.

Programs that ack messages just before the callback returns should be unaffected. If your callback does significant work after calling Ack, you may observe more concurrent callback invocations, increasing throughput, CPU and memory consumption. If you defer the Ack call until after the callback returns, you may observe decreased throughput.

If this change negatively affects you, you can adjust MaxOutstandingBytes/Messages to compensate. You may also wish to disable the client's flow control by setting both MaxOutstanding values to -1, and use your own flow control. We recommend a weighted semaphore for that purpose; that is what we use.

See issue #870 for more information.