conda (4.0.5) create -p <env-dir> not setting python path as expected
0 views
Skip to first unread message
Tim Butler
Jun 29, 2016, 12:49:03 PM6/29/16
to conda - Public
I'm encountering problems with the python path setup for a python binary installed in a new environment.
This is starting with at least conda 4.0.5 supplied with miniconda2-4.0.5.
The specific environment's lib directories are not added to the python path,
and so most imports fail (e.g. for pip).
Instead the python path refers to what appears to be a cached environment for python 2.7 build 5.
For example when I create an enviroment /apps/testenv1
my python path ends up without any mention of /apps/testenv1 :
>>> import sys
>>> sys.path
['', '/home/nmsperf/.conda/envs/.pkgs/python-2.7.11-5/lib/python27.zip', '/home/nmsperf/.conda/envs/.pkgs/python-2.7.11-5/lib/python2.7', '/home/nmsperf/.conda/envs/.pkgs/python-2.7.11-5/lib/python2.7/plat-linux2', '/home/nmsperf/.conda/envs/.pkgs/python-2.7.11-5/lib/python2.7/lib-tk', '/home/nmsperf/.conda/envs/.pkgs/python-2.7.11-5/lib/python2.7/lib-old', '/home/nmsperf/.conda/envs/.pkgs/python-2.7.11-5/lib/python2.7/lib-dynload', '/home/nmsperf/.conda/envs/.pkgs/python-2.7.11-5/lib/python2.7/site-packages']
>>>
nmsperf@ossstage01:/apps$ conda create -p /apps/testenv1 python
Fetching package metadata: ......
Solving package specifications: .........
Package plan for installation in environment /apps/testenv1:
The following packages will be downloaded:
package | build
---------------------------|-----------------
openssl-1.0.2h | 1 3.2 MB
sqlite-3.13.0 | 0 4.0 MB
zlib-1.2.8 | 3 101 KB
python-2.7.11 | 5 12.0 MB
setuptools-23.0.0 | py27_0 455 KB
pip-8.1.2 | py27_0 1.5 MB
------------------------------------------------------------
Total: 21.2 MB
The following NEW packages will be INSTALLED:
openssl: 1.0.2h-1 (soft-link)
pip: 8.1.2-py27_0 (soft-link)
python: 2.7.11-5 (soft-link)
readline: 6.2-2 (soft-link)
setuptools: 23.0.0-py27_0 (soft-link)
sqlite: 3.13.0-0 (soft-link)
tk: 8.5.18-0 (soft-link)
wheel: 0.29.0-py27_0 (soft-link)
zlib: 1.2.8-3 (soft-link)
Proceed ([y]/n)? y
Fetching packages ...
openssl-1.0.2h 100% |################################################################################################################################| Time: 0:00:00 7.47 MB/s
sqlite-3.13.0- 100% |################################################################################################################################| Time: 0:00:00 8.75 MB/s
zlib-1.2.8-3.t 100% |################################################################################################################################| Time: 0:00:00 1.29 MB/s
python-2.7.11- 100% |################################################################################################################################| Time: 0:00:01 8.00 MB/s
setuptools-23. 100% |################################################################################################################################| Time: 0:00:00 2.17 MB/s
pip-8.1.2-py27 100% |################################################################################################################################| Time: 0:00:00 3.91 MB/s
Extracting packages ...
[ COMPLETE ]|###################################################################################################################################################| 100%
Linking packages ...
[ COMPLETE ]|###################################################################################################################################################| 100%
#
# To activate this environment, use:
# $ source activate /apps/testenv1
#
# To deactivate this environment, use:
# $ source deactivate
#
nmsperf@ossstage01:/apps$
nmsperf@ossstage01:/apps$ cd testenv1
nmsperf@ossstage01:/apps/testenv1$ ls
bin conda-meta include lib share ssl
nmsperf@ossstage01:/apps/testenv1$ ./bin/python
Python 2.7.11 |Continuum Analytics, Inc.| (default, Jun 15 2016, 15:21:30)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import sys
>>> sys.path
>>> sys.path
['',
'/home/nmsperf/.conda/envs/.pkgs/python-2.7.11-5/lib/python27.zip',
'/home/nmsperf/.conda/envs/.pkgs/python-2.7.11-5/lib/python2.7',
'/home/nmsperf/.conda/envs/.pkgs/python-2.7.11-5/lib/python2.7/plat-linux2',
'/home/nmsperf/.conda/envs/.pkgs/python-2.7.11-5/lib/python2.7/lib-tk',
'/home/nmsperf/.conda/envs/.pkgs/python-2.7.11-5/lib/python2.7/lib-old',
'/home/nmsperf/.conda/envs/.pkgs/python-2.7.11-5/lib/python2.7/lib-dynload',
'/home/nmsperf/.conda/envs/.pkgs/python-2.7.11-5/lib/python2.7/site-packages']
nmsperf@ossstage01:/apps/testenv1$ . bin/activate `pwd`
discarding /opt/anaconda_test2/bin from PATH
prepending /apps/testenv1/bin to PATH
(/apps/testenv1)nmsperf@ossstage01:/apps/testenv1$ conda info
Current conda install:
platform : linux-64
conda version : 4.0.5
conda-build version : not installed
python version : 2.7.11.final.0
requests version : 2.9.1
root environment : /opt/anaconda_test2 (read only)
default environment : /apps/testenv1
envs directories : /home/nmsperf/.conda/envs
/home/nmsperf/envs
/opt/anaconda_test2/envs
package cache : /home/nmsperf/.conda/envs/.pkgs
/home/nmsperf/envs/.pkgs
/opt/anaconda_test2/pkgs
channel URLs : http://osslab01.chi5.prlss.int/conda_pkgs/peer_db7/linux-64/
http://osslab01.chi5.prlss.int/conda_pkgs/peer_db7/noarch/
https://repo.continuum.io/pkgs/free/linux-64/
https://repo.continuum.io/pkgs/free/noarch/
https://repo.continuum.io/pkgs/pro/linux-64/
https://repo.continuum.io/pkgs/pro/noarch/
config file : /home/nmsperf/.condarc
is foreign system : False
rf/.conda/envs/.pkgs/python-2.7.11-5/lib/python2.7/site-packages']
>>>When I update conda to 4.1.4, then I encounter the conda_lock perfmission denied problem
described at https://github.com/conda/conda/issues/2778.
-tim
Tim Butler
Jun 29, 2016, 5:10:27 PM6/29/16
to conda - Public
I've changed permissions on the system-wide /opt/anaconda/pkgs directory to
get around the conda_lock problem so that I could experiment with conda 4.1.4.
Using conda 4.1.4 I see that the python-2.7.11 build 0 works, but 2.7.11 build 5 fails
to override the python prefix.
2.7.12 build 0 also fails.
I'll speculate that 2.7.11 build 5 has used a different build mechanism
and avoided prefix replacement on the python binary.
Here is a listing of the meta directories from build 0 and build 5.
I see that build 5 has a meta.yaml whereas build 0 does not.
Maybe build 5 was built from a recipe.
python-2.7.11-0/info$ ls
files has_prefix index.json meta no_softlink requires
python-2.7.11-5/info$ ls
about.json files has_prefix index.json LICENSE.txt meta.yaml no_link platform
-tim
get around the conda_lock problem so that I could experiment with conda 4.1.4.
Using conda 4.1.4 I see that the python-2.7.11 build 0 works, but 2.7.11 build 5 fails
to override the python prefix.
2.7.12 build 0 also fails.
I'll speculate that 2.7.11 build 5 has used a different build mechanism
and avoided prefix replacement on the python binary.
Here is a listing of the meta directories from build 0 and build 5.
I see that build 5 has a meta.yaml whereas build 0 does not.
Maybe build 5 was built from a recipe.
python-2.7.11-0/info$ ls
files has_prefix index.json meta no_softlink requires
python-2.7.11-5/info$ ls
about.json files has_prefix index.json LICENSE.txt meta.yaml no_link platform
-tim
Tim Butler
Jun 29, 2016, 6:07:55 PM6/29/16
to conda - Public
Okay, I believe the problem is that anaconda python builds from python-2.7.11 build 5 onward
use the recipe at https://github.com/ContinuumIO/anaconda-recipes/blob/master/python-2.7/meta.yaml
which specifies a
build:
nolink:
bin/python
which should probably refer to bin/python2.7 instead.
bin/python is itself a symbolic link to python2.7.
Because python2.7 is linked to the package directory on install
python -> python2.7
python2 -> python2.7
python2.7 -> /home/nmsperf/.conda/envs/.pkgs/python-2.7.11-5/bin/python2.7
python itself thinks it is running out of the package directory
and sets sys.prefix accordingly.
If python2.7 is installed as a copy,
then python sets sys.prefix to the conda environment path.
It should look like this:
python -> python2.7
python2 -> python2.7
python2.7 (a copy)
Versions 2.7.11 build 0 and before contain bin/python2.7 in the file that excludes links.
python-2.7.11-0/info$ cat no_softlink
bin/python
bin/python-config
bin/python2
bin/python2.7
I'll post a git issue to the anaconda-recipes repo.
-tim
use the recipe at https://github.com/ContinuumIO/anaconda-recipes/blob/master/python-2.7/meta.yaml
which specifies a
build:
nolink:
bin/python
which should probably refer to bin/python2.7 instead.
bin/python is itself a symbolic link to python2.7.
Because python2.7 is linked to the package directory on install
python -> python2.7
python2 -> python2.7
python2.7 -> /home/nmsperf/.conda/envs/.pkgs/python-2.7.11-5/bin/python2.7
python itself thinks it is running out of the package directory
and sets sys.prefix accordingly.
If python2.7 is installed as a copy,
then python sets sys.prefix to the conda environment path.
It should look like this:
python -> python2.7
python2 -> python2.7
python2.7 (a copy)
Versions 2.7.11 build 0 and before contain bin/python2.7 in the file that excludes links.
python-2.7.11-0/info$ cat no_softlink
bin/python
bin/python-config
bin/python2
bin/python2.7
I'll post a git issue to the anaconda-recipes repo.
-tim
Tim Butler
Jun 29, 2016, 6:26:17 PM6/29/16
to conda - Public
Rolf Erickson
Jun 29, 2016, 6:42:50 PM6/29/16
to Tim Butler, conda - Public
it ol
On Wed, Jun 29, 2016, 6:26 PM Tim Butler <tim.it...@gmail.com> wrote:
--
You received this message because you are subscribed to the Google Groups "conda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to conda+un...@continuum.io.
To post to this group, send email to co...@continuum.io.
Visit this group at https://groups.google.com/a/continuum.io/group/conda/.
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/conda/a9a622d9-ec9a-4dfd-92fe-f5d794d2ccd3%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.
Reply all
Reply to author
Forward
0 new messages