Gcloud App Engine does not include django; Intentional?

[Evan Jones]

It appears that the documentation now recommends using the version of App Engine shipped with gcloud (via gcloud components install app-engine-python). However, this version does not include any of the built-in versions of Django, which can be loaded by adding references to the "libraries" section. Is this an intentional change, or is this a bug? It is annoying because it means that an app that works with the standalone version of dev_appserver.py may not work with the gcloud version. It also means the documentation is wrong in many places. Its relatively easy to fix by vendoring django, but it would be nice to fix the documentation and make the two distributions be the same.

Details:

Here is the diff -u output between ls google-cloud-sdk/platform/google_appengine/lib and ls google_appengine/lib. You can see that the gcloud version is missing all versions of django, and graphy.

--- appengine 2016-09-30 11:43:22.000000000 -0400

+++ gcloud 2016-09-30 11:42:07.000000000 -0400

@@ -8,17 +8,10 @@

 concurrent

 deprecated_enum

 distutils

-django-0.96

-django-1.2

-django-1.3

-django-1.4

-django-1.5

-django-1.9

 docker

 endpoints-1.0

 fancy_urllib

 google-api-python-client

-graphy

 grizzled

 httplib2

 ipaddr

[Adam (Cloud Platform Support)]

Thanks for reporting this. I'll do some investigation and will update the thread when I have more info.

[Adam (Cloud Platform Support)]

It looks like there was an issue with the Django libraries being left out in Gcloud 128.0.0 which was fixed in 129.0.0. Please let us know if you're still having issues.

[Evan Jones]

I just did a clean install of gcloud 129.0.0 and this has not changed: the lib directory still does not include any versions of django, and the following example application does not work. If I download the standalone Python 1.9.40 distribution, it does work:

hello.py:

import django

import webapp2

class HelloHandler(webapp2.RequestHandler):

    def get(self):

        self.response.write('ok')

app = webapp2.WSGIApplication([

        ('/', HelloHandler),

    ],

    debug=True

)

hello.yaml:

runtime: python27

api_version: 1

threadsafe: true

handlers:

- url: /

  script: hello.app

libraries:

- name: django

  version: "1.4"

dev_appserver.py output:

$ ./google-cloud-sdk/bin/dev_appserver.py hello.yaml 

INFO     2016-10-07 21:17:36,663 devappserver2.py:769] Skipping SDK update check.

INFO     2016-10-07 21:17:36,735 api_server.py:205] Starting API server at: http://localhost:64103

INFO     2016-10-07 21:17:36,739 dispatcher.py:197] Starting module "default" running at: http://localhost:8080

INFO     2016-10-07 21:17:36,743 admin_server.py:116] Starting admin server at: http://localhost:8000

ERROR    2016-10-07 21:17:42,274 wsgi.py:263] 

Traceback (most recent call last):

  File "/Users/evanjones/wtf/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle

    handler = _config_handle.add_wsgi_middleware(self._LoadHandler())

  File "/Users/evanjones/wtf/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler

    handler, path, err = LoadObject(self._handler)

  File "/Users/evanjones/wtf/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject

    obj = __import__(path[0])

  File "/Users/evanjones/wtf/hello.py", line 1, in <module>

    import django

ImportError: No module named django

[Adam (Cloud Platform Support)]

Yes, I see this too after updating. I'll ask about this again and try to get some conclusive information this time.

[Adam (Cloud Platform Support)]

It looks like the django libs have been moved to an external package, as the size of django has a significant effect on the install time. It still can be installed with the following command:

gcloud components update app-engine-python-extras

I've requested that the documentation gets updated accordingly.

[Evan Jones]

Thank you so much for digging into this! That solves our problem.

I agree that distributing app engine via gcloud is superior to the previous approach. I also agree that people should just follow the "vendoring" directions for pure python dependencies, rather than relying on the app engine specific "libraries" section in app.yaml, so I think these are positive changes. However, there is also a bunch of existing app engine code out there, so I'm happy that we've figure out how to fix this backwards compatibility issue.

Evan