Subscription processes hangs forever

[Suman Biswas]

Hi,

Publish process publishes only 2 messages. Ran subscription process on my local and messaged processed successfully. 2nd time I ran the subscription process once again on my local and there is no messages but the subscription process hangs forever. Can you please explain why?

I am using google pub-sub version 0.21.1-beta.

Thanks..

[Alex Martelli]

On Thu, Aug 17, 2017 at 12:56 PM, Suman Biswas <suman....@gmail.com> wrote:

Hi,

Publish process publishes only 2 messages. Ran subscription process on my local and messaged processed successfully. 2nd time I ran the subscription process once again on my local and there is no messages but the subscription process hangs forever. Can you please explain why?

When you call `pull` on a subscription object, the default is that it will block and wait indefinitely for messages to be available (so, "forever" if no messages are available and no more will be ever sent). To return immediately when no messages are available, you must pass to the `pull` call, for example, `return_immediately=True` in Python (of course you don't tell us which programming language you use, making it impossible for us to guess that).

Alex

 

I am using google pub-sub version 0.21.1-beta.

Thanks..

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/509c40bb-94d0-4a14-9030-0840d3beffc4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Suman Biswas]

Hi Alex, 

Thanks for the update. I am using Java. Can you please let me know the java parameter name?

Thanks,

Suman

Sent from my iPhone

[Kenworth (Google Cloud Platform)]

The parameter will be the same regardless of the language. 

For Remote Procedure Call (RPC):

return_immediately=True

or a sample JSON representation: 

{
"returnImmediately": True
}

Here is the documentation for RPC and REST API.

[Kamal Aboul-Hosn]

Actually, the client library does not have a "pull" method nor the ability to set returnImmediately. The client library is designed to take care of these features itself. When you say you ran the subscriber again, did you publish new messages for it? If not, then it is likely that your first subscriber acknowledged the messages and so there were no more messages to deliver to your subscriber. In that case, the subscriber will run indefinitely, calling your provided MessageReceiver on new messages until you call stopAsync on it. 

--

You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/e4cb1359-8f83-42d0-9153-fcc40cf7d8bd%40googlegroups.com.

[Suman Biswas]

Hi Kenworth,

Thanks for the update. As I stated,  I am using google pub-sub version 0.21.1-beta (Apache Maven) where I would not able to find out the place holder for parameter "return_immediately". In PollingSubscriberConnection.java this parameter is set to false by default . How can I set this parameter? Can you please give me an example.

[Suman Biswas]

Hi Kamal,

Thanks for the update but it does not sound good. Client does not has any control other than stopAsync. Is it possible to add back this properties?

Thanks,

Suman

On Friday, August 18, 2017 at 5:17:51 AM UTC-7, Kamal Aboul-Hosn wrote:

Actually, the client library does not have a "pull" method nor the ability to set returnImmediately. The client library is designed to take care of these features itself. When you say you ran the subscriber again, did you publish new messages for it? If not, then it is likely that your first subscriber acknowledged the messages and so there were no more messages to deliver to your subscriber. In that case, the subscriber will run indefinitely, calling your provided MessageReceiver on new messages until you call stopAsync on it. 

On Thu, Aug 17, 2017 at 10:04 PM 'Kenworth (Google Cloud Platform)' via Google Cloud Pub/Sub Discussions <cloud-pubs...@googlegroups.com> wrote:

The parameter will be the same regardless of the language. 

For Remote Procedure Call (RPC):

return_immediately=True

or a sample JSON representation: 

{
"returnImmediately": True
}

Here is the documentation for RPC and REST API.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

[Kamal Aboul-Hosn]

Hi, Suman,

Can you give a little more detail about the property you want? What is the use case you are trying to solve? Is it that you want to start up the subscriber, get two messages you know you want to receive, and then shut down the subscriber? Prior to using the 0.21.1-beta client library, how were you accomplishing the behavior you wanted?

Kamal 

On Fri, Aug 18, 2017 at 11:22 AM Suman Biswas <suman....@gmail.com> wrote:

Hi Kamal,

Thanks for the update but it does not sound good. Client does not has any control other than stopAsync. Is it possible to add back this properties?

Thanks,

Suman

On Friday, August 18, 2017 at 5:17:51 AM UTC-7, Kamal Aboul-Hosn wrote:

Actually, the client library does not have a "pull" method nor the ability to set returnImmediately. The client library is designed to take care of these features itself. When you say you ran the subscriber again, did you publish new messages for it? If not, then it is likely that your first subscriber acknowledged the messages and so there were no more messages to deliver to your subscriber. In that case, the subscriber will run indefinitely, calling your provided MessageReceiver on new messages until you call stopAsync on it. 

On Thu, Aug 17, 2017 at 10:04 PM 'Kenworth (Google Cloud Platform)' via Google Cloud Pub/Sub Discussions <cloud-pubs...@googlegroups.com> wrote:

The parameter will be the same regardless of the language. 

For Remote Procedure Call (RPC):

return_immediately=True

or a sample JSON representation: 

{
"returnImmediately": True
}

Here is the documentation for RPC and REST API.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/e4cb1359-8f83-42d0-9153-fcc40cf7d8bd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/c87f97e0-47d6-47ee-b934-330b26600ab0%40googlegroups.com.

[Suman Biswas]

Hi Kamal,

Here is our use case

-- In a day (4:00 AM to 6:00PM UTC ) there will be 50 messages and based on content different set of processes will kick off. Since we will get messages between 4:00 AM to 6:00PM UTC we planed to 
    run subscriber from 3:00AM till 8:00PM UTC. 

We plan to move into google cloud and trying to do some POC.

Thanks,

Suman

On Friday, August 18, 2017 at 8:39:34 AM UTC-7, Kamal Aboul-Hosn wrote:

Hi, Suman,

Can you give a little more detail about the property you want? What is the use case you are trying to solve? Is it that you want to start up the subscriber, get two messages you know you want to receive, and then shut down the subscriber? Prior to using the 0.21.1-beta client library, how were you accomplishing the behavior you wanted?

Kamal 

On Fri, Aug 18, 2017 at 11:22 AM Suman Biswas <suman....@gmail.com> wrote:

Hi Kamal,

Thanks for the update but it does not sound good. Client does not has any control other than stopAsync. Is it possible to add back this properties?

Thanks,

Suman

On Friday, August 18, 2017 at 5:17:51 AM UTC-7, Kamal Aboul-Hosn wrote:

Actually, the client library does not have a "pull" method nor the ability to set returnImmediately. The client library is designed to take care of these features itself. When you say you ran the subscriber again, did you publish new messages for it? If not, then it is likely that your first subscriber acknowledged the messages and so there were no more messages to deliver to your subscriber. In that case, the subscriber will run indefinitely, calling your provided MessageReceiver on new messages until you call stopAsync on it. 

On Thu, Aug 17, 2017 at 10:04 PM 'Kenworth (Google Cloud Platform)' via Google Cloud Pub/Sub Discussions <cloud-pubs...@googlegroups.com> wrote:

The parameter will be the same regardless of the language. 

For Remote Procedure Call (RPC):

return_immediately=True

or a sample JSON representation: 

{
"returnImmediately": True
}

Here is the documentation for RPC and REST API.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/e4cb1359-8f83-42d0-9153-fcc40cf7d8bd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

[Kamal Aboul-Hosn]

Hi, Suman,

You could use stopAsync to accomplish this behavior. If your wish to bring down the subscriber is time-based, you can set a timer that executes calls stopAsync at the desired time. On your main thread, you can call awaitTerminated() and then your subscriber will only shut down when the timer that shut down the subscriber has completed.

The returnImmediately property that was described is not a property of the subscriber, but of an individual request to get messages (which is what forms the building blocks of the client library), so adding it into the client would not actually do what you want.

Kamal

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/e4cb1359-8f83-42d0-9153-fcc40cf7d8bd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/c87f97e0-47d6-47ee-b934-330b26600ab0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/f6b43188-9277-49bf-9969-f0de9de12f33%40googlegroups.com.

[Kir Titievsky]

Another route is to use Cloud Functions with a push subscription. This way you don't have to worry about scheduling the subscribers. The asynchronous vs. sync issues go away. If you are not comfortable with JavaScript, using AppEngine HTTP end points in one of the many languages it supported will work as well. This is a great path forward if your jobs take less than a minute. It's not bad for longer jobs as well.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/CAAGPqmkRfc4wB2XHHAj3miTH2irjmVCQz7K4J7ZEA_9we1PBmg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Kir Titievsky | Product Manager | Google Cloud Pub/Sub

[Suman Biswas]

Hi Kir,

Thanks for the update. It looks like cloud function approach is the perfect fit for us. 

Thanks,

Suman

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/e4cb1359-8f83-42d0-9153-fcc40cf7d8bd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/c87f97e0-47d6-47ee-b934-330b26600ab0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/f6b43188-9277-49bf-9969-f0de9de12f33%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/CAAGPqmkRfc4wB2XHHAj3miTH2irjmVCQz7K4J7ZEA_9we1PBmg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Kir Titievsky]

Suman,

If you want to synchronously retrieve messages one at at time, you have several options:

1. Use a push subscriber and an HTTPS webhook to do your processing. Cloud Functions and App Engine are excellent, auto-scaling, managed services to run these kinds of jobs. This is the best way to go if your processing steps don't take more than 30-40 seconds. 

2. Use the synchronous interface in the Client Library you already use, where you would set the returnImmediately parameter here.  We are still working on full documentation for that, but the interface works and is supported. 

3. Use the API Client library, which is automatically generated against our synchronous REST/HTTP1 API.

Does this help at all?

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsubscrib...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/e4cb1359-8f83-42d0-9153-fcc40cf7d8bd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsubscrib...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/c87f97e0-47d6-47ee-b934-330b26600ab0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsubscrib...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/f6b43188-9277-49bf-9969-f0de9de12f33%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsubscrib...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/CAAGPqmkRfc4wB2XHHAj3miTH2irjmVCQz7K4J7ZEA_9we1PBmg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Kir Titievsky | Product Manager | Google Cloud Pub/Sub

--

You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/2ebffe8e-6539-4c2e-adce-31bcdd002d8c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Suman Biswas]

Thanks a lot for your suggestion. It's really a very good suggestion. 

Thanks,

Suman

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/e4cb1359-8f83-42d0-9153-fcc40cf7d8bd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/c87f97e0-47d6-47ee-b934-330b26600ab0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/f6b43188-9277-49bf-9969-f0de9de12f33%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/CAAGPqmkRfc4wB2XHHAj3miTH2irjmVCQz7K4J7ZEA_9we1PBmg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Kir Titievsky | Product Manager | Google Cloud Pub/Sub

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/2ebffe8e-6539-4c2e-adce-31bcdd002d8c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.