Re: [cloud-pubsub-discuss] Python pubsub.

583 views
Skip to first unread message

Kir Titievsky

unread,
Nov 13, 2017, 2:53:16 PM11/13/17
to dmo...@protoprod.com, Google Cloud Pub/Sub Discussions
Daniel, In what context are you running this? For example, is this an App Engine 1.9 local emulator run? 


On Mon, Nov 13, 2017 at 1:15 PM Daniel Morgan <dmo...@protoprod.com> wrote:
I'm currently just trying to import the pubsub library into my project. I'm getting an import error on it. If I comment out the import the rest of the code loads fine. I'm not finding a solution to this issue. 


 (/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py:263)
Traceback (most recent call last):
 
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
    handler
= _config_handle.add_wsgi_middleware(self._LoadHandler())
 
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
    handler
, path, err = LoadObject(self._handler)
 
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 85, in LoadObject
    obj
= __import__(path[0])
 
File
"/base/data/home/apps/s~hyperwave-168417/pubsub-processor:20171113t125819.405497509582670372/main.py", line 23, in <module>
   
from google.cloud import pubsub_v1
 
File
"/base/data/home/apps/s~hyperwave-168417/pubsub-processor:20171113t125819.405497509582670372/lib/google/cloud/pubsub_v1/__init__.py", line 17, in <module>
   
from google.cloud.pubsub_v1 import types
 
File
"/base/data/home/apps/s~hyperwave-168417/pubsub-processor:20171113t125819.405497509582670372/lib/google/cloud/pubsub_v1/types.py", line 20, in <module>
   
import psutil
 
File
"/base/data/home/apps/s~hyperwave-168417/pubsub-processor:20171113t125819.405497509582670372/lib/psutil/__init__.py", line 40, in <module>
   
from . import _common
ImportError: cannot import name _common



--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/16e16000-a5e5-4030-9176-dd098d9b8205%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Kir Titievsky | Product Manager | Google Cloud Pub/Sub 

Daniel Morgan

unread,
Nov 13, 2017, 3:04:11 PM11/13/17
to Google Cloud Pub/Sub Discussions
I figured it out...It was because I was loading into standard. Which help me solve another problem I had. Learning Dev, thanks



On Monday, November 13, 2017 at 2:53:16 PM UTC-5, Kir Titievsky wrote:
Daniel, In what context are you running this? For example, is this an App Engine 1.9 local emulator run? 


On Mon, Nov 13, 2017 at 1:15 PM Daniel Morgan <dmo...@protoprod.com> wrote:
I'm currently just trying to import the pubsub library into my project. I'm getting an import error on it. If I comment out the import the rest of the code loads fine. I'm not finding a solution to this issue. 


 (/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py:263)
Traceback (most recent call last):
 
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
    handler
= _config_handle.add_wsgi_middleware(self._LoadHandler())
 
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
    handler
, path, err = LoadObject(self._handler)
 
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 85, in LoadObject
    obj
= __import__(path[0])
 
File
"/base/data/home/apps/s~hyperwave-168417/pubsub-processor:20171113t125819.405497509582670372/main.py", line 23, in <module>
   
from google.cloud import pubsub_v1
 
File
"/base/data/home/apps/s~hyperwave-168417/pubsub-processor:20171113t125819.405497509582670372/lib/google/cloud/pubsub_v1/__init__.py", line 17, in <module>
   
from google.cloud.pubsub_v1 import types
 
File
"/base/data/home/apps/s~hyperwave-168417/pubsub-processor:20171113t125819.405497509582670372/lib/google/cloud/pubsub_v1/types.py", line 20, in <module>
   
import psutil
 
File
"/base/data/home/apps/s~hyperwave-168417/pubsub-processor:20171113t125819.405497509582670372/lib/psutil/__init__.py", line 40, in <module>
   
from . import _common
ImportError: cannot import name _common



--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

Paul Harwood

unread,
May 23, 2018, 10:24:31 AM5/23/18
to Google Cloud Pub/Sub Discussions
Can you explain the answer a bit more - I have hit the same problem


On Monday, 13 November 2017 20:04:11 UTC, Daniel Morgan wrote:
I figured it out...It was because I was loading into standard. Which help me solve another problem I had. Learning Dev, thanks



On Monday, November 13, 2017 at 2:53:16 PM UTC-5, Kir Titievsky wrote:
Daniel, In what context are you running this? For example, is this an App Engine 1.9 local emulator run? 


On Mon, Nov 13, 2017 at 1:15 PM Daniel Morgan <dmo...@protoprod.com> wrote:
I'm currently just trying to import the pubsub library into my project. I'm getting an import error on it. If I comment out the import the rest of the code loads fine. I'm not finding a solution to this issue. 


 (/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py:263)
Traceback (most recent call last):
 
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
    handler
= _config_handle.add_wsgi_middleware(self._LoadHandler())
 
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
    handler
, path, err = LoadObject(self._handler)
 
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 85, in LoadObject
    obj
= __import__(path[0])
 
File
"/base/data/home/apps/s~hyperwave-168417/pubsub-processor:20171113t125819.405497509582670372/main.py", line 23, in <module>
   
from google.cloud import pubsub_v1
 
File
"/base/data/home/apps/s~hyperwave-168417/pubsub-processor:20171113t125819.405497509582670372/lib/google/cloud/pubsub_v1/__init__.py", line 17, in <module>
   
from google.cloud.pubsub_v1 import types
 
File
"/base/data/home/apps/s~hyperwave-168417/pubsub-processor:20171113t125819.405497509582670372/lib/google/cloud/pubsub_v1/types.py", line 20, in <module>
   
import psutil
 
File
"/base/data/home/apps/s~hyperwave-168417/pubsub-processor:20171113t125819.405497509582670372/lib/psutil/__init__.py", line 40, in <module>
   
from . import _common
ImportError: cannot import name _common



--

Kamran (Google Cloud Support)

unread,
May 23, 2018, 7:40:57 PM5/23/18
to Google Cloud Pub/Sub Discussions
Hello Paul,

Would you elaborate further on describing your issue? Is your setup a local development? Is it possible to post reproduction steps and code? 

In the meantime, for information on how to get started with Cloud Client Libraries for the Cloud Pub/Sub API and some examples, please visit this article

Yendry Machado

unread,
May 24, 2018, 4:56:14 PM5/24/18
to Google Cloud Pub/Sub Discussions
I have exactly the same problem. Any update ???


On Monday, November 13, 2017 at 2:53:16 PM UTC-5, Kir Titievsky wrote:
Daniel, In what context are you running this? For example, is this an App Engine 1.9 local emulator run? 


On Mon, Nov 13, 2017 at 1:15 PM Daniel Morgan <dmo...@protoprod.com> wrote:
I'm currently just trying to import the pubsub library into my project. I'm getting an import error on it. If I comment out the import the rest of the code loads fine. I'm not finding a solution to this issue. 


 (/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py:263)
Traceback (most recent call last):
 
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
    handler
= _config_handle.add_wsgi_middleware(self._LoadHandler())
 
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
    handler
, path, err = LoadObject(self._handler)
 
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 85, in LoadObject
    obj
= __import__(path[0])
 
File
"/base/data/home/apps/s~hyperwave-168417/pubsub-processor:20171113t125819.405497509582670372/main.py", line 23, in <module>
   
from google.cloud import pubsub_v1
 
File
"/base/data/home/apps/s~hyperwave-168417/pubsub-processor:20171113t125819.405497509582670372/lib/google/cloud/pubsub_v1/__init__.py", line 17, in <module>
   
from google.cloud.pubsub_v1 import types
 
File
"/base/data/home/apps/s~hyperwave-168417/pubsub-processor:20171113t125819.405497509582670372/lib/google/cloud/pubsub_v1/types.py", line 20, in <module>
   
import psutil
 
File
"/base/data/home/apps/s~hyperwave-168417/pubsub-processor:20171113t125819.405497509582670372/lib/psutil/__init__.py", line 40, in <module>
   
from . import _common
ImportError: cannot import name _common



--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

Kir Titievsky

unread,
May 24, 2018, 5:09:20 PM5/24/18
to machad...@gmail.com, Google Cloud Pub/Sub Discussions
Taking a guess here: Pub/Sub has two client libraries for most languages, as described here. The "Cloud Client Library in language X" is the recommended client library for most situations as it is generally higher level and more performant. But it is not supported on AppEngine Standard.  For that environment, we have "API Client Libraries."  So what I think is happening is that you are attempting to use the "Cloud Client Library" on "AppEngine Standard."  You have two options: either use AppEngine Flex instead of Standard OR use "API Client Library" instead of Cloud Client library. We are working to make this better. 

Gory details are here.


To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.
--
Kir Titievsky | Product Manager | Google Cloud Pub/Sub 

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/d8a9af93-5b99-4770-b088-17dc561ffeaf%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Paul Harwood

unread,
May 24, 2018, 5:57:35 PM5/24/18
to Kir Titievsky, machad...@gmail.com, Google Cloud Pub/Sub Discussions
I got the answer and when I did - the previous comment also made sense.

That is that the Google Cloud Client Libraries do not work in Google Appengine Standard Environment. 

That could be a lot clearer in the chaotic documentation. Things like the article that you linked me to.

The whole idea that the basic, standard Google Cloud APIs are not part of the standard Google Appengine Environment is so braindead that I did not even consider it. It is like something that you would expect from Microsoft!

It seems that the recommendation is to use the google-api-client-libraries - which are basic and there seems to be actual no example - anywhere on the web - of how to use those libraries to use the pubsub service which is what I want.

And anyway - that also appears to be broken since the way to do service account authorisation in appengine depends on a module in oath2client (oauth2client.contrib.appengine - that is specifically for appengine) is not in the version of oauth2client in the SDK and therefore I would assume thus also not in the standard environment. It also cannot be vendored in since there IS a version of oauth2client in the SDK (please, please correct me on any of this if I am wrong). 

This all makes me think it would actually be easier to access the Google Cloud Services by running on SOMEONE ELSE's cloud platform !!! I would say that it looks like no one is in charge - but as a Xoogler myself I know that there is no one in charge - just a whole load of over worked SWE's running from ball to ball - trying to keep them all in the air!

Kir Titievsky

unread,
May 24, 2018, 6:20:30 PM5/24/18
to run...@gmail.com, Yendry Machado, Google Cloud Pub/Sub Discussions
Paul, I can understand the source of your frustration and can tell you that I, and many others, are trying to make this better.  In the short term, it sounds like what's missing here is proper documentation on how to use the API Client Libraries on AppEngine standard.  Does that sound right?

Paul Harwood

unread,
May 24, 2018, 6:41:38 PM5/24/18
to Kir Titievsky, Yendry Machado, Google Cloud Pub/Sub Discussions
That sounds perfectly right :) And some changes to the documentation I was going to try to Feedback ( the difference between "we don't recommend" and  "these libraries WILL NOT WORK in the standard environment" is huge :)

Yendry Machado

unread,
May 25, 2018, 10:26:34 AM5/25/18
to Google Cloud Pub/Sub Discussions
thanks, worked correctly
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.
--
Kir Titievsky | Product Manager | Google Cloud Pub/Sub 

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

Paul Harwood

unread,
May 26, 2018, 12:09:43 PM5/26/18
to Yendry Machado, Google Cloud Pub/Sub Discussions
I got it working - thanks.

For anyone reading this who wants pointers to making it work - this is a good start :


It would be nice if this were referenced in the actual docs.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.
--
Kir Titievsky | Product Manager | Google Cloud Pub/Sub 

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.


--
Kir Titievsky | Product Manager | Google Cloud Pub/Sub 

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/fb3b715b-d0da-43c8-a7cc-27c8cbc15943%40googlegroups.com.

Daniel Morgan

unread,
May 29, 2018, 8:16:10 AM5/29/18
to Google Cloud Pub/Sub Discussions

This was the exact issue it should be loaded into Flex instead of standard. I did mention it above when I first posted this issue. Glad it helped someone else though! 



On Thursday, May 24, 2018 at 5:09:20 PM UTC-4, Kir Titievsky wrote:
Reply all
Reply to author
Forward
0 new messages