What could cause 'system-site-packages' to not work as expected?

[Xtian Simon]

My coding environment uses the Archlinux repo versions of Python and Virtual Environments (installed in the usual Archlinux way). To enable my virtual environment to use these system packages, I've created my virtual environment using the optional argument, '--system-site-packages'. 

The problem occurs when I turn on my virtual environment ($workon venv). Without the virtual environment activated, I've had no trouble with my Jupyter Notebooks.

But now I get an error that dependencies are missing:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-1c788c7f1d94> in <module>()
----> 1 import pandas as pd
      2 import os
      3 import magic

/usr/lib/python3.6/site-packages/pandas/__init__.py in <module>()
     16 
     17 if missing_dependencies:
---> 18     raise ImportError("Missing required dependencies {0}".format(missing_dependencies))
     19 del hard_dependencies, dependency, missing_dependencies
  

if I use the command `pip freeze`, I see the system packages as I expect and the one pypi package I installed:

$ pip freeze 
appdirs==1.4.3
cycler==0.10.0
decorator==4.0.11
gmpy2==2.0.8
ipykernel==4.5.2
ipython==5.3.0
ipython-genutils==0.1.0
ipywidgets==6.0.0
isc==2.0
Jinja2==2.9.6
jsonschema==2.6.0
jupyter-client==5.0.1
jupyter-console==5.1.0
jupyter-core==4.3.0
MarkupSafe==1.0
matplotlib==2.0.0
mistune==0.7.4
mpmath==0.19
nbformat==4.3.0
networkx==1.11
nose==1.3.7
notebook==5.0.0
numpy==1.12.1
packaging==16.8
pandas==0.19.2
pexpect==4.2.1
pickleshare==0.7.4
prompt-toolkit==1.0.14
ptyprocess==0.5.1
Pygments==2.2.0
pyparsing==2.2.0
python-dateutil==2.6.0
python-magic==0.4.13
pytz==2017.2
pyzmq==16.0.2
ranger==1.8.1
scikit-learn==0.18.1
scipy==0.19.0
simplegeneric==0.8.1
six==1.10.0
stevedore==1.21.0
sympy==1.0
terminado==0.6
tornado==4.4.3
traitlets==4.3.2
virtualenv==15.1.0
virtualenv-clone==0.2.6
virtualenvwrapper==4.7.2
wcwidth==0.1.7
widgetsnbextension==2.0.0


What could cause 'system-site-packages' to not work as expected when I load a Jupyter notebook that otherwise works fine?

[Jason Myers]

Can you share a console log leading up to the starting the Jupiter notebook? Also can you use a regular python shell to test if you can use those libraries in there?

Cheers,

Jason

--
You received this message because you are subscribed to the Go

[Xtian Simon]

Thanks for the reply Jason.

Can you share a console log leading up to the starting the Jupiter notebook?

Here's a Pastebin of  `journalctl` output today for just the commands '$workon venv' and `$jupyter notebook`. (actually my venv is actually called `jupyter`).
I also did an strace on the Jupyter call, and it's ~19k lines.

 Also can you use a regular python shell to test if you can use those libraries in there?

No luck from a bash launched iPython interpreter either.

I think the problem is just with iPython. Running a similar script from the regular python CLI interpreter, I don't have the import errors.

[Xtian Simon]

Can you share a console log leading up to the starting the Jupiter notebook?
Here's a Pastebin of  `journalctl` output today for just the commands '$workon venv' and `$jupyter notebook`.

I think I misunderstood your request. The first link is the system log, and there's not much of anything there.

So, here's the output for starting jupyter notebook.
And for good measure, here's the output from running the command `mkvirtualenv` and arguments.
 

[Xtian Simon]

I really don't understand virtual environments!

Here is a diff of `pip freeze`
    base Arch install <---> new virtual environment

When I looked at the requirements output inside of the venv, I just assumed this was the same output as the non-venv (with the exception of the one pip package I installed, `python-magic`). The diff file shows there are a lot more differences than I realized.

That said, Numpy is installed, so I'm not sure about that problem.
But this is more complicated than I first imagined--and maybe easier to fix, too!