gc3pie fresh install fails on python 2.7.14 and pip 18.0

[David Dreher]

I have a machine running Linux Mint with a fresh python 2.7.14 with pip 18.0
Importing gc3pie library fails with the following traceback:

>> py.importlib.import_module('MatlabSession')
Error using weakref><module> (line 14)
Python Error: ImportError: cannot import name _remove_dead_weakref
Error in __init__><module> (line 26)
import sys, os, time, cStringIO, traceback, warnings, weakref, collections
Error in __init__><module> (line 49)
import logging
Error in MatlabSession><module> (line 9)
from gc3libs import Application, create_engine, ANY_OUTPUT
Error in __init__>import_module (line 37)
__import__(name)


I have found a selection of references to this error, but no clear solution for me.
Interestingly I get the same error when I tried to run gc3pie on macOS earlier, but I thought this was due to the unsupported nature of macOS.
One of the more direct recommended solutions are: to recreate the virtualenv, or reinstall python. I tried the second and I don’t think I can do the second with gc3pie right?

Refs:
https://github.com/pypa/pip/issues/5240
https://github.com/pyenv/pyenv/issues/1141
https://github.com/pypa/pip/issues/5080
https://github.com/pypa/pip/pull/5149

[David Dreher]

So I also tried installing gc3pie with an anaconda python built but this wouldn’t even install:

==========================
GC3Pie installation script
==========================

This script installs *development version* of GC3Pie in '/home/ttecadm/gc3pie'.

If you encounter any problem running this script, please contact
the GC3Pie team by sending an email to gc3...@googlegroups.com.

Remember to attach the full output of the script, in order to help us
to identify the problem.

Installation info:

Destination directory: /home/ttecadm/gc3pie
Overwrite existing dir: ask
Python executable: /home/ttecadm/anaconda2/bin/python
Ask for confirmation: yes
Install unreleased code: yes
Install gc3apps: yes
Optional features: ['openstack']
Installer script version: 2.0.3


Are you ready to continue? [Y/n]
<GC3Pie install> INFO: Applying workaround for Anaconda Python's incompatibility with virtualenv ...
<GC3Pie install> INFO: /home/ttecadm/anaconda2/lib/libpython2.7.so -> /home/ttecadm/gc3pie/lib/libpython2.7.so
<GC3Pie install> INFO: /home/ttecadm/anaconda2/lib/libpython2.7.so.1.0 -> /home/ttecadm/gc3pie/lib/libpython2.7.so.1.0

WARNING
=======


Anaconda Python detected! I'm creating the virtual environment with a
'--system-site-packages' option, to let GC3Pie programs use all the
libraries that come bundled with Anaconda. If this leads to errors
related to 'setuptools', 'distribute' or 'pip', then please report a
bug to the GC3Pie mailing list gc3...@googlegroups.com



Do you still want to proceed? [Y/n]
Already using interpreter /home/ttecadm/anaconda2/bin/python
New python executable in /home/ttecadm/gc3pie/bin/python
Installing pip, wheel...done.
<GC3Pie install> INFO: Created Python virtual environment in '/home/ttecadm/gc3pie'
mode of \u2018/home/ttecadm/gc3pie/bin/pip\u2019 retained as 0755 (rwxr-xr-x)
<GC3Pie install> INFO: Installed Anaconda's version of `pip` into the virtual env
<GC3Pie install> INFO: Trying to install or upgrade setuptools ...
<GC3Pie install> INFO: Downloaded setuptools-40.0.0.zip from PyPI; now installing it ...
tar: This does not look like a tar archive
tar: Skipping to next header
gzip: stdin has more than one entry--rest ignored
tar: Child returned status 2
tar: Error is not recoverable: exiting now

====================================================
GC3Pie install: ERROR: Failed to install the latest version of Python 'setuptools'
====================================================


The required Python package 'setuptools' could not be installed.

Please get in touch with the GC3Pie developers at the email address
gc3...@googlegroups.com to get help with this error.


If the above does not help you solve the issue, please contact the
GC3Pie team by sending an email to gc3...@googlegroups.com
(mailing-list subscription required). Include the full output of this
script to help us identifying the problem.

Aborting installation!

> --
> You received this message because you are subscribed to the Google Groups "gc3pie" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to gc3pie+un...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[David Dreher]

So it seems to be related to the default python installation 2.7.6.
I removed my updated version and now I cant even install gc3pie :

python install.py -D -a openstack
Traceback (most recent call last):
File "install.py", line 61, in <module>
import logging
File "/usr/local/lib/python2.7/logging/__init__.py", line 26, in <module>

import sys, os, time, cStringIO, traceback, warnings, weakref, collections

File "/usr/local/lib/python2.7/weakref.py", line 14, in <module>
from _weakref import (

ImportError: cannot import name _remove_dead_weakref

ttecadm@IMLS-55K45-p01:~/Repos/gc3pie$ python -V

[Riccardo Murri]

Hello David,

> <GC3Pie install> INFO: Trying to install or upgrade setuptools ...
> <GC3Pie install> INFO: Downloaded setuptools-40.0.0.zip from PyPI; now installing it ...
> tar: This does not look like a tar archive

This definitely looks like a bug: the `install.py` script should be
downloading a `.tgz` file while it's getting `.zip` instead.

Ciao,
R

--
Riccardo Murri / Email: riccard...@gmail.com / Tel.: +41 77 458 98 32

[Riccardo Murri]

Hello David,

> I have a machine running Linux Mint with a fresh python 2.7.14 with pip 18.0
> Importing gc3pie library fails with the following traceback:
>
> >> py.importlib.import_module('MatlabSession')
> Error using weakref><module> (line 14)
> Python Error: ImportError: cannot import name _remove_dead_weakref

Symbol `_remove_dead_weakref` is imported from machine code file
`_weakref.so` (see import list at the beginning of `weakref.py`) so this
means that the Python interpreter that gave this error is very confused
about where to find its files and modules, and ends actually reading
object files that belong to a different Python interpreter. (I think
you mentioned having multiple versions of Python installed?)

If you need to install multiple versions of Python side-by-side, `pyenv`
is a good tool to manage them.