Pub/Sub in GAE Standard Environment

[Richard Cheesmar]

Questions:

1. Is Pub/Sub still in official Beta status?

2. Is it officially documented that you cannot use Pub/Sub with GAE standard?
    
    Details:
        Import of psutils fails in a standard GAE environment in Pycharm, but not in a vanilla python environment, both using exact same virtualenv.

        Pub/Sub uses psutils via types

             from google.cloud import pubsub_v1
            
             pubsub_v1 imports types which imports psutils

            see https://github.com/GoogleCloudPlatform/google-cloud-python/issues/3892

3. Will you be able to use Pub/Sub in a GAE Standard environment or will you have to migrate to Flex?

[George (Cloud Platform Support)]

Hi Richard, 

1) Pub/sub is not in beta, even if some related gcloud commands contain the "beta" word. 

2) You can use Pub/sub with GAE Standard, as documented on the "Cloud Pub/Sub Client Libraries" page. The old posting you refer to refers another thread where the issue is closed. 

3) You may decide upon the best environment for your app independently of the way Pub/sub is to be implemented for each environment. The above linked page recommends using the older Google APIs Client Libraries if running on Google App Engine standard environment. You are encouraged to read more about the client libraries for Cloud APIs in Client Libraries Explained.

[Richard Cheesmar]

Hi, I tried using it in PyCharm and although everything is loaded in the virtual env that I am using, and is accessible, psutil will not in a PyCharm GAE environment, aka the following call

     from google.cloud import pubsub_v1

This imports types which in turn imports psutil and gives an import error

However, psutil it loads in a standard vanilla Python environment using the same virtualenv

To make sure it wasn't something more sinister I did a simple test of each:

Standard PyCharm Vanilla Python Project:

main.py contains

import psutil def x(): print "hello world" if __name__ == "__main__": x()

Google App Engine Project -
main.py contains

import webapp2 import psutil class MainHandler(webapp2.RequestHandler): def get(self): self.response.write('Hello world!') app = webapp2.WSGIApplication([ ('/', MainHandler) ], debug=True)

The import in the vanilla python project works fine, but the import in the GAE app returns the traceback:

Traceback (most recent call last): File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle handler = _config_handle.add_wsgi_middleware(self._LoadHandler()) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler handler, path, err = LoadObject(self._handler) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject obj = __import__(path[0]) File "/Users/chez 1/google-app-engine-projects/untitled/main.py", line 18, in <module> import psutil ImportError: No module named psutil

[George (Cloud Platform Support)]

Hi Richard, 

PyCharm supports all major Google App Engine development practices. This feature is supported in the Professional edition only. Do you actually use the pro edition of PyCharm? 

You may find detailed instructions on PyCharm's page for Google App Engine. Their technical support may prove helpful as well. 

This forum is meant for general discussions and voicing of opinions. If strictly coding is involved, you have better chances of getting help on stackoverflow or similar forums, where specialized programmers are ready to help with targeted advice. 

[Richard Cheesmar]

Yes, I have been using it 5 years

[Richard Cheesmar]

Actually, if you look at the traceback here: google App Engine seems to be messing with the import:

Traceback (most recent call last):
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
    handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
    handler, path, err = LoadObject(self._handler)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
    obj = __import__(path[0])
  File "/Users/chez 1/google-app-engine-projects/untitled/main.py", line 18, in <module>
    import psutil
ImportError: No module named psutil

On Thursday, October 19, 2017 at 9:20:39 PM UTC+3, Richard Cheesmar wrote:

[George (Cloud Platform Support)]

Hi Richard, 

To import various Python libraries in the standard environment, it is better to follow the recommended way, as detailed in the "Using third-party libraries" online document. 

[Richard Cheesmar]

Tried that too I just cannot get it to work, end of.

  File "/Users/chez 1/google-app-engine-projects/pitpit-gae/supplier/api/v1/handlers.py", line 9, in <module>
    import psutil
  File "/Users/chez 1/google-app-engine-projects/pitpit-gae/external/psutil/__init__.py", line 149, in <module>
    from . import _psosx as _psplatform
  File "/Users/chez 1/google-app-engine-projects/pitpit-gae/external/psutil/_psosx.py", line 16, in <module>
    from . import _psutil_osx as cext
  File "/usr/local/google_appengine/google/appengine/tools/devappserver2/python/sandbox.py", line 965, in load_module
    raise ImportError('No module named %s' % fullname)

[Josh Moore]

Hi there!  Let me jump in here and add some clarification to your original questions.

> 1. Is Pub/Sub still in official Beta status?

Pub/Sub is a GA product, however the google-cloud-pubsub python client library is in beta.  You can use the google api client library for python, which is GA.  

> 2. Is it officially documented that you cannot use Pub/Sub with GAE standard?     

> ... snip ...
>            see https://github.com/GoogleCloudPlatform/google-cloud-python/issues/3892

It's referenced that the google api client libraries are the preferred way to connect in GAE Standard, but not that the google-cloud-python libraries don't work at all.  I'll file a docbug to make this more clear.

> 3. Will you be able to use Pub/Sub in a GAE Standard environment or will you have to migrate to Flex?

You can absolutely use Pub/Sub in the GAE Standard environment.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscribe@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/170378aa-95dd-45a6-aa0b-477d469b2c4d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Josh Moore |

Cloud Solutions Engineer |

josh...@google.com |

[ccc...@actionapp.io]

Regarding 2. I tried to use google-cloud-pubsub python client library in GAE but end up having import error because google-cloud-pubsub has c extension and GAE standard cannot have 3rd party library with c extension.

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.

Visit this group at https://groups.google.com/group/google-appengine.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/170378aa-95dd-45a6-aa0b-477d469b2c4d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[ccc...@actionapp.io]

I've tried using google-cloud-pubsub but end up having import error. It seems that google-cloud-pubsub python library is using c extesion, which is not supported in GAE standard.

On Friday, October 20, 2017 at 8:40:48 AM UTC-7, joshuamo wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.

Visit this group at https://groups.google.com/group/google-appengine.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/170378aa-95dd-45a6-aa0b-477d469b2c4d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.