"You have passed a subscription that does not belong to the given ack ID"

[Igor Clark]

Hi folks,

I sometimes get this message, along with "Error: Resource not found (resource=<my-subscription-id>)". It's a bit hard to track down where in the code this is happening as it's coming back via an UnhandledPromiseRejectionWarning in the node.js app I'm working on. They're both happening at times in the app where I wouldn't expect them - as far as I can tell, the subscription should be live and running just fine, but every so often these just get thrown.

Trying to track them down, I wonder if anyone has any kind of feel for the kind of circumstances that might throw these? Obviously I need to go through with a fine-tooth comb and check all my error handling cases, but in case it gives me a clue or even somewhere to start looking, are there any particular operational or other issues that are most likely to repond with "Resource not found" or "You have passed a subscription that does not belong to the given ack ID" ?

Thanks for any ideas,

Igor

[Alex Mordkovich]

Hi Igor,

Regarding the "Resource not found (resource=<my-subscription-id>)" error: are you deleting (and re-creating) subscriptions while using them (i.e,. while pull()'ing and acknowledge()'ing messages on them)? Subscriptions are persistent resources that shouldn't just "disappear".

Regarding "You have passed a subscription that does not belong to the given ack ID": when you pull a message from a subscription, you get an ack id along with the message, and that ack id is valid only for the subscription from which that message was pulled. If you send an acknowledge() request with that ack id and a different subscription, you will get this error.

One possibility that could result in these two errors being somehow correlated: are you by any chance generating subscription names dynamically when making pull() or acknowledge() requests? If so, it is possible that you're generating incorrect subscription names -- in some cases, this results in the "Resource not found" error; in other cases, the subscription name corresponds to an existing subscription, but not the one from which you pull()'d the message, resulting in the other error.

So, I would check the logic that populates subscription names in your requests.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/7e1b53cb-d379-44b3-b15a-a208a0a49c96%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Igor Clark]

HI Alex, thanks for getting back to me so quickly!

- I have multiple clients, each subscribed to multiple topics.

- I am dynamically generating subscription names, at creation time; each client has a UUID, and I'm using that to generate the subscription name, but only once per client/subscription pair.

- I am deleting subscriptions on each client, but (in theory!) only after I've stopped using them.

- I'm only creating a subscription once on each client and keeping a reference to it which I use solely to delete it with later. I'm using "autoAck: true" option on the topic.subscribe() call and setting a subscription listener, which is why it seemed strange that I'd get the ack ID error, as I'm not dealing with ack IDs at all, but leaving it entirely to the library.

So your message has given me an idea - I wonder if I'm somehow leaking a reference to the subscription, thus keeping it hanging around and trying to ack a message immediately after the subscription has been deleted. I'll go through all the code and see what I can find. Will come back with any findings!

Thanks again,

Igor

[Igor Clark]

Looks like that might have been it - I tracked down a cleverly disguised leaky reference ;-) and it seems to have stopped. Will see how that goes.

It's not at all strange that the error never really happened when testing locally against the emulator, as I know that behaves quite differently in some ways. What I did find hard to get my head round was that it never really happened when running the app locally against the live PubSub service either, but only when running in GKE against the live service. I guess it could have just been because of the differences in latency and response time over the connection. Which makes me think I need to go over the overall design again as I don't want to be relying on accidents of timing!

Regardless, it seems to be working for the moment, so I'll keep a close eye on that and come back if it happens again. For now it seems like like your ideas pushed me in the right direction, so thank you very much Alex :-)

Best,

Igor

[Igor Clark]

Actually, it's still happening. I don't think what I changed made any difference.

It doesn't happen at all in the emulator, or when running the app locally against GCP PubSub. It happens reliably when running the app on Container Enginer against live GCP Pubsub.

Any ideas as to why that might be would be extremely welcome!

Thanks,

Igor

[Alex Mordkovich]

Hi Igor,

It sounds like there's some sort of race condition in you client code that manifests itself only when run locally. Perhaps your client code is running across more cores (more parallelism) when on GKE, thereby exposing the race.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsubscrib...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/7e1b53cb-d379-44b3-b15a-a208a0a49c96%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/5f1f6b43-7e4d-40b4-8b29-a5fd4e9da451%40googlegroups.com.

[Igor Clark]

Hi Alex,

I think you're right, it does seem like race conditions. I'm still investigating. Thanks for your input :-)

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/7e1b53cb-d379-44b3-b15a-a208a0a49c96%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alex Mordkovich]

On Mon, Jan 23, 2017 at 3:08 PM, Igor Clark <igor....@gmail.com> wrote:

It sounds like there's some sort of race condition in you client code that manifests itself only when run locally

Meant to say: only when run on GKE.

[Chris Hiestand]

3 years later than OP, I'm seeing the same errors when running integration tests from my laptop.

"Error: Resource not found (resource=<my-subscription-id>)".

"You have passed a subscription that does not belong to the given ack ID"

The tests would run fine the first time, and on subsequent runs they would work sometimes and not others. Smells like a race condition.

My hypothesis based on the behavior I see is that when you delete a subscription, the Pub/Sub API will return a 200 before it finishes actually deleting the subscription. If you then recreate a subscription with the same name and try to use it, you may get these errors (race conditions).