Instability on pubsub using Python Client Library

[Paul Garlick]

Hi guys,

Please could someone help me with the pubsub python client library.  I am following the tutorials closely and seem to get unprompted errors.  I have a simple script call "sendmessage.py" that sends a text message with a random number appended so that I can tell messages apart.  The subscriber code looks like this:

from google.cloud import pubsub_v1

def callback(message):
    print('Received message: {}'.format(message))
    message.ack()

def listen_for_errors(project, subscription_name):

    subscriber = pubsub_v1.SubscriberClient()
    subscription_path = subscriber.subscription_path('<my-project-name-here>', 'test-subscription')

    subscription = subscriber.subscribe(subscription_path, callback=callback)

    try:
        subscription.future.result()
    except Exception as e:
        print(
            'Listening for messages on {} threw an Exception: {}.'.format( 'test-subscription', e))
        raise

A screenshot of the send/receive running on two compute instances is attached.  The system seems to work fine for the first minute or so, then the subscriber seems to trip up with the following error message:

Exception in thread Thread-ConsumeBidirectionalStream:Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/usr/lib/python2.7/threading.py", line 754, in run self.__target(*self.__args, **self.__kwargs) File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/subscriber/_consumer.py", line 363, in _blocking_consume request_generator, response_generator) File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/subscriber/_consumer.py", line 275, in _stop_request_generator if not response_generator.done():AttributeError: '_StreamingResponseIterator' object has no attribute 'done'

This happens after a short time (less than a few minutes) even if no messages are sent.  Once it has crashed, there is no way to recover - e.g. by pressing enter, typing quit(), pressing CTRL+C, etc, so I have to shutdown the instance and start over.

I find it a bit strange that I am following the tutorials so closely and yet there are unprompted errors when my code is running.  Please would it be possible for someone to point out where I have gone wrong or suggest a robust workaround to ignore the error and keep listening for messages?

Kind regards and thank you to anyone who can help,

Paul

[Jordan (Cloud Platform Support)]

I see that this issue has been discussed before and fixed by the engineering team, so make sure you are using the latest Client Library as the Pub/Sub API is now at 0.32.1. 

Note that Google Groups is reserved for general product discussions and is not for technical support. If you are still seeing this issue after updating it is recommended to report this to the engineering team in their Issue Tracker. 

[Paul Garlick]

Hi Jordan,

Thank you for your response.  The issue was indeed related to the way in which I had set up the google-cloud client library on my compute engine instances, so I have corrected my boot code and now the system is stable.  I have provided the original/corrected code below in case it helps others, but if you would prefer to just delete/close the thread I understand!

The original boot script code featured the lines:

sudo apt-get update
sudo apt-get -yq install python-pip
sudo apt-get install python-dev
sudo pip install --upgrade google-cloud-storage
sudo pip install --upgrade google-cloud-pubsub

I wonder if the last line should have said google-cloud-pubsub_v1 but in any case simplified the code to:

sudo apt-get update
sudo apt-get -yq install python-pip
sudo pip install --upgrade google-cloud

And the system is now working well.

Kind regards,

Paul