HMTKBaseMap kills kernel in jupyter notebook

77 views
Skip to first unread message

Phil Cummins

unread,
Jul 8, 2018, 9:31:47 PM7/8/18
to OpenQuake Users

Hi,

I just installed Openquake version 3.1.0 on MacOS 10.13.4.
I would like to use some of the catalog tools in the hmtk. However, I found that when I run in a jupyter notebook run under the openquake environment, HMTKBaseMap kills the kernel. In fact, even a call to the normal basemap kills the kernel:

import numpy as np   # Numpy - Python's numerical library
import matplotlib.pyplot as plt  # Matplotlib - Python's plotting library
from openquake.hmtk.plotting.mapping import HMTKBaseMap
from mpl_toolkits.basemap import Basemap as Basemap
# Map configuration
llon, ulon, llat, ulat = (94.0000,142.0000, -11.986, 8.0000)
m = Basemap(llon,llat,ulon,ulat,projection='merc')

There is no error message other than that the kernel died.
If I just run the above at the python prompt, the interpreter doesn't die but it doesn't plot anything, even if I use plt.show().
Does anyone else have this problem and know how to get around it?

Thanks,

- Phil

Daniele Viganò

unread,
Jul 9, 2018, 3:46:39 AM7/9/18
to openqua...@googlegroups.com

Hi Phil,

unfortunately we recently discovered that Basemap on macOS is really broken when used in it's binary version because of a known bug of Shapely.

You can read the full story and links to upstream bug reports here: https://github.com/gem/oq-builders/issues/99

The only known workarounds are: 1) switch to another OS (Windows or Linux are fine) or 2) uninstall binary versions of Shapely and Basemap and install them via source code:

pip uninstall basempa shapely

pip install --no-binary :all: basemap shapely

The installation from sources isn't a trivial task: you have to compiled a compatible version of libgeos on your system (see https://github.com/Toblerity/Shapely/#source-distributions)

We are going to temprarily disable Basemap on macOS for the Engine 3.2 until the fix is fixed upstream (hopefully).

Daniele

--
You received this message because you are subscribed to the Google Groups "OpenQuake Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openquake-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
DANIELE VIGANÒ | System Administrator | Skype dennyv85 | +39 0382 5169882
GLOBAL EARTHQUAKE MODEL | working together to assess risk

Phil Cummins

unread,
Jul 11, 2018, 12:15:42 AM7/11/18
to OpenQuake Users
Hi Daniele,

Thanks fr the reply, but what a mess! And, judging form the link you sent, it sounds like we will also have to switch to Cartopy soon?

- Phil

Daniele Viganò

unread,
Jul 11, 2018, 10:33:32 AM7/11/18
to openqua...@googlegroups.com

Hi Phil,


On 07/11/2018 06:15 AM, Phil Cummins wrote:
Hi Daniele,

Thanks fr the reply, but what a mess!

unfortunately yes, it's a mess, at least on macOS.

And, judging form the link you sent, it sounds like we will also have to switch to Cartopy soon?

Basemap has not been really maintained since long ago and we are already in the process of switching to Cartopy. Unfortunately, for what concern the specific issue of this thread, it suffers from the same problems as Basemap because the bad guy in this story is Shapely which, like Basemap and Cartopy, depends on libgeos.

Long story short (I will use Basemap, but for Cartopy is the same story):

Basemap uses libgeos like Shapely does, but Shapely also extends libgeos 'C' routines 'on the fly'. Basemap depends (import) Shapely. What happens here is that: Basemap loads it's own copy of libgeos; Shapely does the same but with a slightly modified version of libgeos. At this time in memory there are two copies (one altered) of libgeos that are causing troubles to the OS linker.

This does not happen when:
  • Using Linux and Windows because how the OS loader/linker works
  • Not using Python wheels (.whl files). Wheels allow each python library to be distributed with a binary copy of each C/C++/Fortran dependency they have (if any), saving the user from a lot of troubles (having a compiler, even more dependencies, sometime a powerful machine is needed to compile stuff...), but in this case is producing side effects because of the two copies of libgeos loaded in memory (with one changed on the fly by Shapely)

That's why the only (known) solution on macOS is to compile libgeos, Shapely and Basemap from sources and let Shapely and Basemap share the single copy of libgeos.

Cheers,
Daniele

Reply all
Reply to author
Forward
0 new messages