failed to install at mac os 10.11 with anaconda python

168 views
Skip to first unread message

Hyeuk Ryu

unread,
Nov 1, 2016, 12:32:23 AM11/1/16
to OpenQuake Users
Hi,

I failed to install the OQ at my mac which has os 10.11 and anaconda python 2.7.
Could you provide me any solution?
Thanks.

Regards,

Hyeuk

~/Downloads$ ./openquake-setup-macos-2.1.0-1.run 
Verifying archive integrity... All good.
Uncompressing installer for the OpenQuake Engine  100%  
Type the path where you want to install OpenQuake, followed by [ENTER]. Otherwise leave blank, it will be installed in /Users/hyeuk/openquake: 
Creating a new python environment in /Users/hyeuk/openquake. Please wait.
Traceback (most recent call last):
  File "virtualenv/virtualenv.py", line 2332, in <module>
    main()
  File "virtualenv/virtualenv.py", line 711, in main
    symlink=options.symlink)
  File "virtualenv/virtualenv.py", line 944, in create_environment
    download=download,
  File "virtualenv/virtualenv.py", line 900, in install_wheel
    call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)
  File "virtualenv/virtualenv.py", line 795, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /Users/hyeuk/openquake/bin/python - setuptools pip wheel failed with error code 1
~/Downloads$ which python
/Users/hyeuk/anaconda2/bin/python


Daniele Viganò

unread,
Nov 1, 2016, 6:31:20 AM11/1/16
to openqua...@googlegroups.com

Dear Hyeuk,

the OpenQuake macOS self-installer is not designed for Anaconda, but for the standard Python provided by Apple: https://github.com/gem/oq-engine/blob/engine-2.1/doc/installing/macos.md#installing-the-openquake-engine-on-macos

We'll add a check to abort the installation if the wrong python is used (another user just reported the same issue).

You can either:

  • Temporary disable Anaconda removing its path (/Users/hyeuk/anaconda2/bin) from $PATH (it should be set in your .profile or .bashrc)
  • Install the OpenQuake Engine manually in Anaconda as you do with any other python project. First you need to install all the dependencies in your environment:

conda install Cython mock h5py nose numpy scipy psutil shapely decorator futures django==1.8 requests

but there's one more dependency that is not available in anaconda an must be installed via pip

pip install pyshp

Then you can install oq-hazardlib and oq-engine

pip install https://github.com/gem/oq-hazardlib/archive/master.zip
pip install https://github.com/gem/oq-engine/archive/master.zip

Please note that this installation is not officially supported nor tested by our Continuous Integration system: it should work, but some calculations may fail and numbers may be slight different.

Best regards,
Daniele

--
You received this message because you are subscribed to the Google Groups "OpenQuake Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openquake-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
DANIELE VIGANÒ | System Administrator | Skype dennyv85 | +39 0382 5169882
GLOBAL EARTHQUAKE MODEL | working together to assess risk
Message has been deleted

Hyeuk Ryu

unread,
Nov 1, 2016, 11:02:02 PM11/1/16
to OpenQuake Users
Hi Daniele,

Thanks for your suggestions. 
I've created an environmental file (as attached), and managed to set up the libraries. But I've got the following error message when I run oq-engine.
Any suggestions?
Thanks.

Regards,

Hyeuk

(openquake) ~/openquake/src$ oq-engine
The command 'oq-engine' is deprecated and will be removed. Please use 'oq engine' instead
Could not import openquake.calculators.base: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.classical: ImportError: cannot import name pmap_from_grp
Could not import openquake.calculators.classical_bcr: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.classical_damage: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.classical_risk: ImportError: No module named stats
Could not import openquake.calculators.disaggregation: ImportError: cannot import name SourceSitesFilter
Could not import openquake.calculators.event_based: ImportError: cannot import name ContextMaker
Could not import openquake.calculators.event_based_risk: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.reportwriter: ImportError: cannot import name zero_curves
Could not import openquake.calculators.scenario: ImportError: cannot import name GmfComputer
Could not import openquake.calculators.scenario_damage: ImportError: No module named stats
Could not import openquake.calculators.scenario_risk: ImportError: cannot import name from_string
Could not import openquake.calculators.ucerf_classical: ImportError: No module named probability_map
Could not import openquake.calculators.ucerf_event_based: ImportError: No module named probability_map
Could not import openquake.calculators.ucerf_risk: ImportError: No module named stats
Could not import openquake.calculators.views: ImportError: cannot import name ContextMaker
Could not import openquake.calculators.tests.__init__: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.base_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.calc_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.classical_bcr_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.classical_damage_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.classical_risk_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.classical_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.classical_tiling_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.disagg_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.event_based_risk_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.event_based_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.scenario_damage_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.scenario_risk_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.scenario_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.ucerf_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.base: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.classical: ImportError: cannot import name pmap_from_grp
Could not import openquake.calculators.classical_bcr: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.classical_damage: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.classical_risk: ImportError: No module named stats
Could not import openquake.calculators.disaggregation: ImportError: cannot import name SourceSitesFilter
Could not import openquake.calculators.event_based: ImportError: cannot import name ContextMaker
Could not import openquake.calculators.event_based_risk: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.reportwriter: ImportError: cannot import name zero_curves
Could not import openquake.calculators.scenario: ImportError: cannot import name GmfComputer
Could not import openquake.calculators.scenario_damage: ImportError: No module named stats
Could not import openquake.calculators.scenario_risk: ImportError: cannot import name from_string
Could not import openquake.calculators.ucerf_classical: ImportError: No module named probability_map
Could not import openquake.calculators.ucerf_event_based: ImportError: No module named probability_map
Could not import openquake.calculators.ucerf_risk: ImportError: No module named stats
Could not import openquake.calculators.views: ImportError: cannot import name ContextMaker
Could not import openquake.calculators.tests.__init__: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.base_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.calc_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.classical_bcr_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.classical_damage_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.classical_risk_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.classical_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.classical_tiling_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.disagg_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.event_based_risk_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.event_based_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.scenario_damage_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.scenario_risk_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.scenario_test: ImportError: cannot import name RtreeFilter
Could not import openquake.calculators.tests.ucerf_test: ImportError: cannot import name RtreeFilter
Traceback (most recent call last):
  File "/Users/hyeuk/anaconda2/envs/openquake/bin/oq", line 9, in <module>
    load_entry_point('openquake.engine', 'console_scripts', 'oq')()
  File "/Users/hyeuk/openquake/src/oq-engine/openquake/commands/__main__.py", line 47, in oq
    importlib.import_module(modname)
  File "/Users/hyeuk/anaconda2/envs/openquake/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/Users/hyeuk/openquake/src/oq-engine/openquake/commands/db.py", line 23, in <module>
    from openquake.calculators.views import rst_table
  File "/Users/hyeuk/openquake/src/oq-engine/openquake/calculators/views.py", line 35, in <module>
    from openquake.hazardlib.gsim.base import ContextMaker
ImportError: cannot import name ContextMaker

(openquake) ~/openquake/src$ conda list
# packages in environment at /Users/hyeuk/anaconda2/envs/openquake:
#
cython                    0.25.1                   py27_0  
decorator                 4.0.10                   py27_0  
django                    1.8.4                    py27_0  
funcsigs                  1.0.2                    py27_0  
futures                   3.0.5                    py27_0  
geos                      3.3.3                         0  
h5py                      2.5.0               np110py27_3  
hdf5                      1.8.15.1                      2  
mkl                       11.3.1                        0  
mock                      2.0.0                    py27_0  
nose                      1.3.7                    py27_1  
numpy                     1.10.2                   py27_0  
openquake.hazardlib       0.11.0                    <pip>
openssl                   1.0.2h                        1  
pbr                       1.10.0                   py27_0  
pip                       8.1.2                    py27_0  
psutil                    4.4.2                    py27_0  
pyshp                     1.2.3                     <pip>
python                    2.7.12                        1  
readline                  6.2                           2  
requests                  2.11.1                   py27_0  
scipy                     0.16.1              np110py27_1  
setuptools                25.1.6                   py27_0  
shapely                   1.5.11                   py27_0  
six                       1.10.0                   py27_0  
sqlite                    3.13.0                        0  
tk                        8.5.18                        0  
wheel                     0.29.0                   py27_0  
zlib                      1.2.8                         3  
environment.yml

Daniele Viganò

unread,
Nov 2, 2016, 2:57:38 AM11/2/16
to openqua...@googlegroups.com

Dear Hyeuk,

it seems to me that you have installed oq-hazardlib (using pip I guess) but you are using oq-engine from sources. This is is causing troubles with the imports. If you need to work with sources you should do this instead:

Remove the installed oq-hazardlib first

- pip uninstall openquake.hazardlib

Make a development installation of both oq-hazardlib and oq-engine

- cd ~/openquake/src
- pip install -e ~/openquake/src/oq-hazardlib
- pip install -e ~/openquake/src/oq-engine

Do not forget '-e' in the pip command, is what makes the difference. '-e' does not make a real installation, but it links the sources to the python path.

Cheers,
Daniele

Hyeuk Ryu

unread,
Nov 6, 2016, 9:04:22 PM11/6/16
to OpenQuake Users
Hi Daniele,

Thanks for your kind reply. I managed to install the engine through the conda installation procedure, but it failed during the nosetest as below. 
I can provide the full log if you'd like to look into the cause. 


----------------------------------------------------------------------
Ran 2062 tests in 1277.667s

FAILED (SKIP=7, errors=31, failures=7)


Also here is the list of packages installed through the conda. As you see there are two of the hazardlib packages with different version installed, which seems strange.
Thanks.

Regards,

Hyeuk

(openquake) ~/anaconda2/envs/openquake/lib/python2.7/site-packages/openquake$ conda list

# packages in environment at /Users/hyeuk/anaconda2/envs/openquake:

#

cython                    0.25.1                   py27_0  

decorator                 4.0.10                   py27_0  

django                    1.8.4                    py27_0  

docutils                  0.12                     py27_2  

funcsigs                  1.0.2                    py27_0  

futures                   3.0.5                    py27_0  

geos                      3.3.3                         0  

h5py                      2.5.0               np110py27_3  

hdf5                      1.8.15.1                      2  

mkl                       11.3.1                        0  

mock                      2.0.0                    py27_0  

nose                      1.3.7                    py27_1  

numpy                     1.10.2                   py27_0  

openquake.engine          2.2.0                     <pip>

openquake.hazardlib       0.11.0                    <pip>

openquake.hazardlib       0.22.0                    <pip>

openssl                   1.0.2h                        1  

pbr                       1.10.0                   py27_0  

pip                       8.1.2                    py27_0  

psutil                    4.4.2                    py27_0  

pyshp                     1.2.3                     <pip>

python                    2.7.12                        1  

readline                  6.2                           2  

requests                  2.11.1                   py27_0  

scipy                     0.16.1              np110py27_1  

setuptools                25.1.6                   py27_0  

shapely                   1.5.11                   py27_0  

six                       1.10.0                   py27_0  

sqlite                    3.13.0                        0  

tk                        8.5.18                        0  

wheel                     0.29.0                   py27_0  

zlib                      1.2.8                         3  

Hyeuk Ryu

unread,
Nov 6, 2016, 11:07:10 PM11/6/16
to OpenQuake Users
Hi Daniele,

I've tried installation using the binary file by disabling the anaconda as you suggested. Again, I ran the nose tests, and the result is not the same as the previous case, still I've got errors and failures. (FAILED (SKIP=3, errors=32, failures=7). Any thoughts?
Thanks.

Regards,

Hyeuk 

Daniele Viganò

unread,
Nov 7, 2016, 4:09:16 AM11/7/16
to openqua...@googlegroups.com

Dear Hyeuk,

on Mac, with Engine 2.1, you could expect a bunch of failed tests (less than 5, which are platform-specific), but not as many as 32.

Could you please send us the output from the nose command?

nosetests -v -a '!slow' openquake 2>&1 | tee nose-out.txt

This command will print the log on screen and save it to nose-out.txt

Current master should pass all the tests since the platform-specific ones have been fixed/skipped in a pull requested that entered master just this morning.

Best regards,
Daniele

Hyeuk Ryu

unread,
Nov 7, 2016, 4:16:20 AM11/7/16
to OpenQuake Users
Hi Daniele,

Here I send the output for your reference.
Thanks.

Hyeuk
nose-out.txt

Michele Simionato

unread,
Nov 7, 2016, 8:39:22 AM11/7/16
to OpenQuake Users
Hi Hyeuk.

I am pretty much convinced that you are seeing false errors, i.e. nosetests does not play well with the installation from packages, but in reality the engine is working correctly.
We are working at fixing nose, for the moment you can bypass it with this command: python -m openquake.baselib.runtests openquake.calculators
Can you check if this works?

Hyeuk Ryu

unread,
Nov 7, 2016, 3:02:21 PM11/7/16
to OpenQuake Users
Here is the output of the test.
Thanks.

Regards,

Hyeuk
test_output.txt

Michele Simionato

unread,
Nov 7, 2016, 11:57:42 PM11/7/16
to OpenQuake Users


On Monday, November 7, 2016 at 9:02:21 PM UTC+1, Hyeuk Ryu wrote:
Here is the output of the test.

That's right. Everything green except case_13 which has been already fixed in master. You can feel confident with your installation. You numbers can be slightly different than if you run the computation on Linux (because the Mac uses different libraries/libraries compiled with different flags) but the difference is insignificant, well below other kind of uncertainties. Anyway we thank you for raising the issue, now we have fixed the problem with nose too. If you install the nightly buillds all tests will be green (at least, they are on the Mac we have in the office).
Message has been deleted
Message has been deleted

Hyeuk Ryu

unread,
Nov 8, 2016, 8:12:59 PM11/8/16
to OpenQuake Users
Hi Michele and Daniele,

I just want to share installation steps and environmental.yml for whoever wants to use the anaconda python environment.

1. copy the environment.yml and openquake.cfg to a directory (~/openquake).

2. go to the directory where environment.yml is coped and run the command to create the python environment 
conda env create -f ./environment.yml

3. activate the environment
source activate openquake

4. copy open quake.cfg file  
cp ./openquake.cfg ~/anaconda2/envs/openquake/lib/python2.7/site-packages/.

5. run a nose test
cd ~/anaconda2/envs/openquake/lib/python2.7/site-packages/openquake
nosetests -v -a '!slow'

Hope it helps.
Thanks.

Regards,

Hyeuk
environment.yml
openquake.cfg

Daniele Viganò

unread,
Nov 10, 2016, 11:15:11 AM11/10/16
to openqua...@googlegroups.com

Dear Hyeuk,

many thanks for your contribution! I see you've been also able to install rtree.

We'll add your work to our community contributed documentation on GitHub.

Maybe in the future Anaconda will be part of the official supported way of installing the OpenQuake Engine: for now we are not going do that because we need to make sure that numbers are ok across all the releases/platforms where Anaconda is supported and unfortunately today we do not have enough resources to add new officially supported platforms.

Best,
Daniele

--
You received this message because you are subscribed to the Google Groups "OpenQuake Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openquake-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages