Configure Time to leave(Expiry) of a PubsubMessage

[Manikandan Nagarajan]

Hi,

I want to know if there is a way to configure the expiry of a Pubsub Message

I have a scenario wherein, I use the poll mechanism to retrieve the messages that I publish.

I want the GooglePubsub to retry indefinitely when the subscriber fails to process a message due to a valid error scenario ---> GooglePubsub does this as I do not ack this

But, for some bad requests, I dont want it to retry indefinitely.

Is there a way to configure the expiry of a message when I publish it

[Rohit Khare]

At present, the 7day maximum retention time is often the minimum as well. We are interested in a design that works well for specifying a variable garbage collection policy, particularly if that maximum rises. 

What is your use case and specific duration that would help, to the degree you can share with the list (or reply to me directly)?

Thanks,

  Rohit Khare

  Product Manager, Google Cloud Pub/Sub

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/aa9afa69-9a09-4061-865b-d8327c0bcbae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Manikandan Nagarajan]

I wish to set a time to the message when I do a publish. 

After polling, when there is a failure during the processing of the message, I do not ack the message. So this message retries indefinitely. This mechanism is not desirable all the time for my usecase.

If there's a way to set a TimeToLeave(Expiry) for the message when I publish, I dont want the message to stay in the system even if it is currently retrying due to a failure in the processing.

This will stop that particular message from retrying again and it will be removed from the system after it's expiry.

I want this configurable for each message I am publishing. The time may vary from 2 minutes to 7 days.

On Thursday, 27 August 2015 11:58:49 UTC-7, Rohit Khare wrote:

At present, the 7day maximum retention time is often the minimum as well. We are interested in a design that works well for specifying a variable garbage collection policy, particularly if that maximum rises. 

What is your use case and specific duration that would help, to the degree you can share with the list (or reply to me directly)?

Thanks,

  Rohit Khare

  Product Manager, Google Cloud Pub/Sub

On Thu, Aug 27, 2015, 11:40 AM Manikandan Nagarajan <manikand...@gmail.com> wrote:

Hi,

I want to know if there is a way to configure the expiry of a Pubsub Message

I have a scenario wherein, I use the poll mechanism to retrieve the messages that I publish.

I want the GooglePubsub to retry indefinitely when the subscriber fails to process a message due to a valid error scenario ---> GooglePubsub does this as I do not ack this

But, for some bad requests, I dont want it to retry indefinitely.

Is there a way to configure the expiry of a message when I publish it

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

[Takashi Matsuo]

Hi Manikandan,

There is a workaround in the user land. You can add timestamp as an attribute and check before processing it, then throw away if it's too old. You can also add other criteria for throwing away.

Are you OK if those messages are thrown away just because you have a big backlog and the consumer can not catch up?

On Thu, Aug 27, 2015 at 1:35 PM, Manikandan Nagarajan <manikand...@gmail.com> wrote:

I wish to set a time to the message when I do a publish. 

After polling, when there is a failure during the processing of the message, I do not ack the message. So this message retries indefinitely. This mechanism is not desirable all the time for my usecase.

If there's a way to set a TimeToLeave(Expiry) for the message when I publish, I dont want the message to stay in the system even if it is currently retrying due to a failure in the processing.

This will stop that particular message from retrying again and it will be removed from the system after it's expiry.

I want this configurable for each message I am publishing. The time may vary from 2 minutes to 7 days.

On Thursday, 27 August 2015 11:58:49 UTC-7, Rohit Khare wrote:

At present, the 7day maximum retention time is often the minimum as well. We are interested in a design that works well for specifying a variable garbage collection policy, particularly if that maximum rises. 

What is your use case and specific duration that would help, to the degree you can share with the list (or reply to me directly)?

Thanks,

  Rohit Khare

  Product Manager, Google Cloud Pub/Sub

On Thu, Aug 27, 2015, 11:40 AM Manikandan Nagarajan <manikand...@gmail.com> wrote:

Hi,

I want to know if there is a way to configure the expiry of a Pubsub Message

I have a scenario wherein, I use the poll mechanism to retrieve the messages that I publish.

I want the GooglePubsub to retry indefinitely when the subscriber fails to process a message due to a valid error scenario ---> GooglePubsub does this as I do not ack this

But, for some bad requests, I dont want it to retry indefinitely.

Is there a way to configure the expiry of a message when I publish it

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/aa9afa69-9a09-4061-865b-d8327c0bcbae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/9f745b96-344f-4d2d-b714-5ca8d4d7a2a5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Takashi Matsuo | Developers Programs Engineer | tma...@google.com