grpcio-1.21.1-cp36-cp36m-linux_armv7l.whl does not contain checks for python version

[Michaela Ervin]

Hi,
When getting google-cloud-pubsub==0.39.1|0.41 and google-api-python-client==1.7.8 using a requirements file during a dpkg build using a line like pip3 install -r requirements.txt --ignore-installed --target debian/<redacted> I see the following...

Collecting futures>=2.2.0 (from grpcio>=1.8.2; extra == "grpc"->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-pubsub==0.41->-r requirements.txt (line 20))
Downloading https://files.pythonhosted.org/packages/cc/26/b61e3a4eb50653e8a7339d84eeaa46d1e93b92951978873c220ae64d0733/futures-3.1.1.tar.gz
Collecting enum34>=1.0.4 (from grpcio>=1.8.2; extra == "grpc"->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-pubsub==0.41->-r requirements.txt (line 20))

enum34 and futures is being collected and installed even though we are using python3.6

This is on an armhf device running ubuntu 18.04.
pip3 -V
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)

When I build the dpkg on an x64 vm using the same repo we have, it does not collect these.

I downloaded the wheels from https://pypi.org/project/grpcio/#files
Inside grpcio-1.21.1-cp36-cp36m-linux_armv7l.whl is:

Metadata-Version: 2.0
Name: grpcio
Version: 1.21.1
Summary: HTTP/2-based RPC framework
Home-page: https://grpc.io
Author: The gRPC Authors
Author-email: grp...@googlegroups.com
License: Apache License 2.0
Platform: UNKNOWN
Classifier: Development Status :: 5 - Production/Stable
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: License :: OSI Approved :: Apache Software License
Requires-Dist: enum34 (>=1.0.4)
Requires-Dist: futures (>=2.2.0)
Requires-Dist: six (>=1.5.2)

And inside grpcio-1.21.1-cp36-cp36m-manylinux1_x86_64.whl is:

Metadata-Version: 2.1
Name: grpcio
Version: 1.21.1
Summary: HTTP/2-based RPC framework
Home-page: https://grpc.io
Author: The gRPC Authors
Author-email: grp...@googlegroups.com
License: Apache License 2.0
Platform: UNKNOWN
Classifier: Development Status :: 5 - Production/Stable
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: License :: OSI Approved :: Apache Software License
Requires-Dist: six (>=1.5.2)
Requires-Dist: futures (>=2.2.0); python_version < "3.2"
Requires-Dist: enum34 (>=1.0.4); python_version < "3.4"

So it appears that the arm package does not have the python_version checks.

The arm .whl also contains a metadata.json file and the x64 version does not.

Any chance a new wheel can get uploaded to pypi to fix the issue?

[rbel...@google.com]

This problem was first reported here and has been resolved on master. How critical is this issue from your perspective? As I understand it, this should just result in an unused package being installed. Is this actually breaking anything for you?

[michae...@lavorotechnologies.com]

Yes, I reported it on github and on here.  I did notice that the issue is resolved in the codebase but as gnossen stated, the wheel build system is outdated.

It is quite critical for us as we have armhf versions of our software package and since implementing this we cannot distribute the armhf version atm.

After building and installing the dpkg, the following error occurs...

"""

Jun 05 13:29:49 X: Error in sys.excepthook:
Jun 05 13:29:49 X: Traceback (most recent call last):
Jun 05 13:29:49 X:   File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 53, in apport_excepthook
Jun 05 13:29:49 X:     if not enabled():
Jun 05 13:29:49 X:   File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 24, in enabled
Jun 05 13:29:49 X:     import re
Jun 05 13:29:49 X:   File "/usr/lib/python3.6/re.py", line 142, in <module>
Jun 05 13:29:49 X:     class RegexFlag(enum.IntFlag):
Jun 05 13:29:49 X: AttributeError: module 'enum' has noattribute 'IntFlag'

"""

Looks like it imports enum from enum34 instead of the standard library.

I tried just removing that from the dpkg, but then there was an error with the other piece, 'futures' that is also installed because of the lack of check in the setup and I have not yet tried excluding that one as well.

I will be trying that next and sometime soon.

[rbel...@google.com]

I can't reproduce that with some very lightweight experiments.

"""

rbellevi@rbell:/tmp/tmp.kwfqNimGpm$ virtualenv venv -p python3.6

Running virtualenv with interpreter /usr/bin/python3.6

Using base prefix '/usr'

New python executable in /tmp/tmp.kwfqNimGpm/venv/bin/python3.6

Also creating executable in /tmp/tmp.kwfqNimGpm/venv/bin/python

Installing setuptools, pkg_resources, pip, wheel...done.

rbellevi@rbell:/tmp/tmp.kwfqNimGpm$ source venv/bin/activate

(venv) rbellevi@rbell:/tmp/tmp.kwfqNimGpm$ python --version

Python 3.6.5

(venv) rbellevi@rbell:/tmp/tmp.kwfqNimGpm$ python -m pip install futures enum34

Collecting futures

Collecting enum34

  Downloading https://files.pythonhosted.org/packages/af/42/cb9355df32c69b553e72a2e28daee25d1611d2c0d9c272aa1d34204205b2/enum34-1.1.6-py3-none-any.whl

Installing collected packages: futures, enum34

Successfully installed enum34-1.1.6 futures-3.1.1

(venv) rbellevi@rbell:/tmp/tmp.kwfqNimGpm$ pythoin

bash: pythoin: command not found

(venv) rbellevi@rbell:/tmp/tmp.kwfqNimGpm$ python

Python 3.6.5 (default, Mar 31 2018, 05:34:57) 

[GCC 7.3.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import enum

>>> enum.IntFlag

<enum 'IntFlag'>

>>> enum.__spec__

ModuleSpec(name='enum', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7f2562098d30>, origin='/tmp/tmp.kwfqNimGpm/venv/lib/python3.6/enum.py')

[michae...@lavorotechnologies.com]

I think I found a workaround for now

In the debian/rules I explicitly remove the enum34 and concurrent/futures after it installs the pip packages.

rm -rf debian/<product>/opt/<company>/<product>/lib/python3.6/site-packages/enum*

rm -rf debian/<product>/opt/<company>/<product>/lib/python3.6/site-packages/concurrent*

This appears to work. I no longer see any of the enum or futures errors.