What is the state of 3rd party Python libraries in GAE standard environment?

[Я Юь]

This is the list of Python libraries available on GAE from:

https://cloud.google.com/appengine/docs/python/tools/built-in-libraries-27

listed here with version numbers, years of their release, and current versions:

django 1.5, 2013 3 years ago, 1.10

jinja2 2.6, 2011 5 years ago, 2.8

lxml 2.3.5, 2012 4 years ago, 3.6.4

numpy 1.6.1, 2011 5 years ago, 1.11.2

PIL 1.1.7, 2010 6 years ago, 3.4.2

pycrypto 2.6, 2012 4 years ago, none abandoned

webapp2 2.5.2, 2012 4 years ago, 3.0.0 beta

webob 1.2.3, 2012 4 years ago, 1.6.2

They haven't been updated in years. Requests for updates on the issue tracker are acknowledged but left unresolved, like other reported bugs.

(Yes, I know that there are alternative solutions, but I still prefer to know what is the state of Python on GAE.)

[Evan Jones]

You may already be aware of this, but the documentation has started encouraging users to "vendor" third-party dependencies by including them directly yourself. This will work for any pure python library. From the list above, the ones where this will NOT work are lxml, numpy, PIL and pycrypto. For django, jinja2, webapp2, and webob, it is pretty easy to pull in any version you would like. See the instructions:

https://cloud.google.com/appengine/docs/python/tools/using-libraries-python-27#installing_a_third-party_library

[PK]

Correct. I have been successfully using this technique for many many 3rd party libraries for a while now. 

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/2032aeac-1be9-4f3b-8e50-97cbe354ac88%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jon Wayne Parrott]

Indeed, vendoring is the recommended approach. App Engine was started before vendoring was really possible which is one of the reasons we provided bundled libraries. Now that pure Python libraries can be vendored, that is the best way forward.

Note that we are updating security related bundled libraries that can not be vendored (we recently updated SSL to 2.7.11).

[Я Юь]

What about other libraries that cannot be vendored in the Standard Environment? See the second post above for the list.

(I am aware of Flex Env / Comp Engine)

[Jon Wayne Parrott]

As mentioned, we are working to update the security related ones (SSL being a recent example). Which non-pure Python ones in particular are you interested in? The more feedback we get the better we can understand the demand.

On Sun, Nov 6, 2016, 2:59 PM Я Юь <mikie....@gmail.com> wrote:

What about other libraries that cannot be vendored in the Standard Environment? See the second post above for the list.

(I am aware of Flex Env / Comp Engine)

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/emgUEr2_nLc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.

To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/ed172348-2d26-4469-b799-1542b7336786%40googlegroups.com.

[Я Юь]

I am glad to know that someone is working on updating 3rd party libraries, which answers my question, though I understand there are no time-frames when/if updates may actually happen.

You'll find the list of 3rd party libraries in the 1st post, of which those with binaries were identified in the 2nd post. Here I also present you with a list of requests for updates from the issues tracker, all acknowledged and still unresolved. This should help you to understand the interest, and get you started.

crypto: https://code.google.com/p/googleappengine/issues/detail?id=13293

django: https://code.google.com/p/googleappengine/issues/detail?id=12144

jinja: https://code.google.com/p/googleappengine/issues/detail?id=12143

mysqldb: https://code.google.com/p/googleappengine/issues/detail?id=11742

lxml: https://code.google.com/p/googleappengine/issues/detail?id=11554

pillow: https://code.google.com/p/googleappengine/issues/detail?id=10079

numpy: https://code.google.com/p/googleappengine/issues/detail?id=9499

[Jon Wayne Parrott]

Thank you for your feedback! I've responded for each library you've mentioned. In general, the more activity on the bugs in the issue tracker the better we can understand the desire - for example, the numpy issue only has two stars and no comments.

> crypto

We're investigating the level of effort to support cryptography as a bundled library. Which cryptography features in particular are you interested in? There are several pure-python libraries (like pyrsa and pyasn1) that can be used.

> django

Django is a pure python library. We recommend vendoring it. Our Django starter app vendors the latest release: https://github.com/GoogleCloudPlatform/appengine-django-skeletonhttps://github.com/GoogleCloudPlatform/appengine-django-skeleton

> jinja2

Jinja2 is also a pure python library. We recommend vendoring it like our Flask starter app does: https://github.com/GoogleCloudPlatform/appengine-flask-skeleton

> mysqldb

We have the latest version, 1.2.5.

> lxml

Are you actively using this? Can you comment on the issue so we can know that users still want this?

> pillow

Hey, I ran into that issue too! Is it still occurring for you? If so, please leave a comment on the issue.

> numpy

Please star and comment on the issue.

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/emgUEr2_nLc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/c9fcb3bf-26f8-403f-a011-23938dadc400%40googlegroups.com.

[Я Юь]

Honestly, I don't know how can someone be passed as an engineer who needs a continuing guidance on what should be a simple and straightforward task. I can find you a university student who will update all these libraries on a Linux system in one day. If it takes years to do the same on your system, then something must be broken on your side. I don't see a point here in explaining why these libraries are useful, you should be capable of finding that on your own. There are good reasons that these libraries were included. A motto of App Engine was, that Google sets up the system, so that developers can focus on their apps.