Partially solved: installing holoviews

[Edward K. Ream]

As far as conda environments go, I don't want to be the clueless newbie who disses something I barely understand.  Suffice it to say that I haven't been able to create conda environments for Python 3.6 or 3.8 that work with Qt.  It's likely a setting somewhere, perhaps a setting that should have been deleted, and wasn't...

For now, I'm happy to use just Python 3.7.  That being so, I did the following:

1. Create py37, based on my base env: `conda create --name py37 --clone base`

2. Create holo37, based on py37: `conda create --name holo37 --clone py37`

Both of these envs run Leo properly.

3. Within the holo37 env, I used pip to install packages, in the order that Thomas used (without installing Leo):

- matplotlib

- bokeh

- holoviews

Note: matplotlib and bokeh had already been installed. My guess is that these two packages are part of the full Anaconda distro, but maybe I had installed them somehow previously.

Finally, I pip installed holoviews. Success. This uninstalled and then reinstalled bokeh, and also holoviews.

Still to do

The following still crashes:

import holoviews as hv
hv.extension('matplotlib')

Presumably I need to install something else.

Summary

Using pip to install matplotlib, bokeh and holoviews works on a Python 3.7 conda environment.

Still to do: allow holoviews to use matplotlib.

I still don't understand how to use conda to create Python 3.6 or 3.8 environments, but I don't care much at present :-)

Edward

[Thomas Passin]

On Thursday, March 26, 2020 at 10:09:37 AM UTC-4, Edward K. Ream wrote:

[snip]

Summary

Using pip to install matplotlib, bokeh and holoviews works on a Python 3.7 conda environment.

Still to do: allow holoviews to use matplotlib.

I still don't understand how to use conda to create Python 3.6 or 3.8 environments, but I don't care much at present :-)

This all makes me glad I've never worked with Anaconda.  OTOH, it seems that a lot of scientific computing is being done with Anaconda installs, so it's working for *some* folks. Maybe it's just all too complicated for its own good.

[Edward K. Ream]

On Thursday, March 26, 2020 at 9:29:35 AM UTC-5, Thomas Passin wrote:

This all makes me glad I've never worked with Anaconda. 

:-) I can relate, sometimes.

Thomas, what versions of bokeh and holoviews are you using? Are they the same on Windows and Linux?

I plan to step through the code to determine just why holoviews isn't finding the bokeh renderers.

Edward

[Thomas Passin]

Windows

    Python:      3.8.0

    Holoviews: 1.13.0

    Bokeh:       2.0.0

    Matplotlib:  3.1.2

also, in  a virtual env:

    Holoviews: 1.13.1

    Bokeh:       2.0.0

    MatPlotLib: 3.2.1  (not a typo, it really is a different version)

Linux

    Python      3.6.9

    Holoviews: 1.13.1

    Bokeh:       2.0.0

    Matplotlib:  3.1.13

Everything was installed using pip.

Note that when I try to use Holoviews with Bokeh, I get this error:

RuntimeError: Neither firefox and geckodriver nor a variant of chromium browser and chromedriver are available on system PATH. You can install the former with 'conda install -c conda-forge firefox geckodriver'.

HV uses a headless browser to render the Bokeh output and (by taking a screenshot) render it to an image.  pip couldn't find these drivers.  I thought they were on Anaconda, but I don't use Anaconda.  It's ironic because hv actually wants to use the same QT chrome widget we have on Leo, but it needs to import its own packaging.  This is why I used matplotlib for rendering, and didn't output a live interactive graph, with holoviews.  I thought that the geckodriver package  was available via Anaconda.  But using it would leave out all those Leo users that don't run Anaconda.

[Thomas Passin]

Note that when I try to use Holoviews with Bokeh, I get this error:

RuntimeError: Neither firefox and geckodriver nor a variant of chromium browser and chromedriver are available on system PATH. You can install the former with 'conda install -c conda-forge firefox geckodriver'.

HV uses a headless browser to render the Bokeh output and (by taking a screenshot) render it to an image.  pip couldn't find these drivers.  I thought they were on Anaconda, but I don't use Anaconda.  It's ironic because hv actually wants to use the same QT chrome widget we have on Leo, but it needs to import its own packaging.  This is why I used matplotlib for rendering, and didn't output a live interactive graph, with holoviews.  I thought that the geckodriver package  was available via Anaconda.  But using it would leave out all those Leo users that don't run Anaconda.

I discovered how to use Bokeh in Holoviews without needing that geckodriver.  If you write the actual interactive output - instead of a png file - you don't even need a renderer.  You can just embed the interactive html output directly.

Here is the revised code for RsT rendering in VR3:

@language python

import holoviews as hv
hv.extension('bokeh')

xs = range(-10,11)
ys = [100-el**2 for el in xs]

curve = hv.Curve((xs, ys))
hv.save(curve, 'curve.html',  backend='bokeh')

@language rest
Here is the resulting interactive graph:

.. raw:: html
   :file: curve.html

If you change the extension and backend to 'matplotlib', you get a static graph instead of an interactive plot.

[Edward K. Ream]

On Fri, Mar 27, 2020 at 7:26 AM Thomas Passin <tbp10...@gmail.com> wrote:

Windows

    Python:      3.8.0

    Holoviews: 1.13.0

    Bokeh:       2.0.0

    Matplotlib:  3.1.2

also, in  a virtual env:

    Holoviews: 1.13.1

    Bokeh:       2.0.0

    MatPlotLib: 3.2.1  (not a typo, it really is a different version)

Linux

    Python      3.6.9

    Holoviews: 1.13.1

    Bokeh:       2.0.0

    Matplotlib:  3.1.13

Everything was installed using pip.

Note that when I try to use Holoviews with Bokeh, I get this error:

RuntimeError: Neither firefox and geckodriver nor a variant of chromium browser and chromedriver are available on system PATH. You can install the former with 'conda install -c conda-forge firefox geckodriver'.

I got the same error. I also installed selenium.

Here is what conda list shows for the relevant packages:

Previously installed, using conda

bokeh                     2.0.0                    pypi_0    pypi
markdown                  3.2.1                    pypi_0    pypi
matplotlib                3.1.3                    py37_0
matplotlib-base           3.1.3            py37h64f37c6_0

Installed today, in holo37 env:

selenium                  3.141.0                  pypi_0    pypi
firefox                   74.0                 h6538335_0    conda-forge
geckodriver               0.26.0               he025d50_0    conda-forge

The failing code is at line 712 in ...Anaconda3\envs\holo37\Lib\site-packages\holoviews\util\__init__.py

__import__(backend)

where backend is 'matplotlib'.

I would have expected a matplotlib directory in the ...Anaconda3\envs\holo37\Lib\site-packages\holoviews\plotting folder, because matplotlib is the default.

I can import matplotlib without problem in the holo37 env, so there is a mystery.

I am far from being stuck, however. More tomorrow.

Edward

[Edward K. Ream]

On Fri, Mar 27, 2020 at 8:02 AM Thomas Passin <tbp1...@gmail.com> wrote:

I discovered how to use Bokeh in Holoviews without needing that geckodriver.  If you write the actual interactive output - instead of a png file - you don't even need a renderer. 

Excellent.

Thomas, I've been thinking we two could hash this out more easily if we were cubical mates. And then, Doh!, what about using Zoom?

I have a Zoom pro account for at least this month. Would you be interested in talking on Zoom? It won't cost you anything.

Edward

[Thomas Passin]

All right , I've used Zoom before ... as long as you are running it, since I never have.  Speaking of Zoom, when I got my new desktop computer and retired my old laptop, I didn't think about how I would need a camera, a mike, and speakers.  They are all built into my laptop, of course.  Even if you get them, they are all separate pieces with their separate cables.  Rats! So I'm using the old laptop for my Zoom calls.

How do you want to work it?  If want to send me meeting information, by sending it to me at my Google-linked account from here, please make sure to let me know you've done it because I would never think to look at it otherwise.

I'm on US East Coast time, BTW.  I can usually be available most any time, as long as I get some lead time to make sure I'm not out somewhere.

Tom

[Thomas Passin]

On Friday, March 27, 2020 at 7:44:33 PM UTC-4, Edward K. Ream wrote:

RuntimeError: Neither firefox and geckodriver nor a variant of chromium browser and chromedriver are available on system PATH. You can install the former with 'conda install -c conda-forge firefox geckodriver'.

I got the same error. I also installed selenium.

Selenium would only be needed for generating a bitmap image.  So I'd say to stick with my revised code and just use the interactive HTML graphs.

[Thomas Passin]

Here is a Leo outline containing

1 - Holoview examples using Bokeh, with and without using VR3;

2 - Holoviews examples using matplotlib with and without using VR3;

3 - Bokeh examples with and without using VR3.

All of these work on my Windows machine as of tonight.  I cleaned them up, corrected for what seem to have been updates to Bokeh (or maybe my misunderstanding), so they are, I think, minimal examples that illustrate how to write the code.  Edward, would you try them out and see what happens with your setup?

Tom

[Edward K. Ream]

On Fri, Mar 27, 2020 at 11:02 PM Thomas Passin <tbp1...@gmail.com> wrote:

Here is a Leo outline containing

1 - Holoview examples using Bokeh, with and without using VR3;

2 - Holoviews examples using matplotlib with and without using VR3;

3 - Bokeh examples with and without using VR3.

All of these work on my Windows machine as of tonight.  I cleaned them up, corrected for what seem to have been updates to Bokeh (or maybe my misunderstanding), so they are, I think, minimal examples that illustrate how to write the code.  Edward, would you try them out and see what happens with your setup?

Thanks for this. Very helpful.

I'm seeing something strange. I have a conda holo37 env from which I can import holoviews and matplotlib from the python interp itself.  But the same imports fail when executed from hv_bokeh_examples.leo.

Let me do some researches of various kinds before setting up a zoom meeting.

Edward

[Edward K. Ream]

On Fri, Mar 27, 2020 at 9:06 PM Thomas Passin <tbp1...@gmail.com> wrote:

How do you want to work it? 

I'll send a zoom invite to tbp1...@gmail.com. You check that regularly, right?

As I said in another reply, I'd like to do some homework before setting this up. It might be a day or so.

Edward

[Edward K. Ream]

On Fri, Mar 27, 2020 at 9:09 PM Thomas Passin <tbp1...@gmail.com> wrote:

Selenium would only be needed for generating a bitmap image.  So I'd say to stick with my revised code and just use the interactive HTML graphs.

Will do. This clears up one mystery. The static bitmap image is way less interesting than the interactive graphs.

Edward

[Edward K. Ream]

On Fri, Mar 27, 2020 at 8:02 AM Thomas Passin <tbp1...@gmail.com> wrote:

I discovered how to use Bokeh in Holoviews without needing that geckodriver.  If you write the actual interactive output - instead of a png file - you don't even need a renderer.  You can just embed the interactive html output directly.

Good to know. Thanks.

Edward

[Thomas Passin]

On Saturday, March 28, 2020 at 4:47:31 AM UTC-4, Edward K. Ream wrote:

On Fri, Mar 27, 2020 at 9:06 PM Thomas Passin <tbp1...@gmail.com> wrote:

How do you want to work it? 

I'll send a zoom invite to tbp1...@gmail.com. You check that regularly, right?

 

Not usually, but I will for the next few days anyway, until I hear from you..  The email account I normally use is not a gmail account at all.

As I said in another reply, I'd like to do some homework before setting this up. It might be a day or so.

OK.

[Thomas Passin]

That is strange.  Oh, I forgot to write that I ran the non-VR3 examples using CTRL-B.  I imagine that's how you tried them, too.

[Edward K. Ream]

On Sat, Mar 28, 2020 at 6:15 AM Thomas Passin <tbp1...@gmail.com> wrote:

That is strange.  Oh, I forgot to write that I ran the non-VR3 examples using CTRL-B.  I imagine that's how you tried them, too.

Yes, iirc.

Edward

[Thomas Passin]

If you want to see something else really strange, the same matplotlib-without-VR3 example that runs successfully with CTRL-B fails to run from a (non-Leo) terminal.  The error says that this line invokes parameters that don't exist.  I had to remove the line and then the plot was produced:

curve1.opts(height=350, width=300)

[Later] After I upgraded Leo from Pypi, the same example no longer runs within Leo until I remove that same line.  I don't see how any caching of a package could survive across multiple restarts of Python, but how else could this be explained?  I wonder if there is a problem with the (relatively) new mechanism that Python uses for deciding whether to re-compile .pyc files in the pycache directories.

I also wonder if this weirdness has something to do with the difference between my results and yours.

[Thomas Passin]

I think I see the answer.  When you change something in a Leo node, you do not induce Python to re-compile any .pyc files in the various pycaches. If Leo had already loaded matplotlib earlier, then it would reuse that compiled code when you did another matplotlib import from within Leo (e.g., when trying out those examples).   So if something changed in, say, a new updated matplotlib package, you wouldn't see it until you deleted all the pycache files - or until you ran from a terminal instead of Leo.

Not exactly a Python bug, then, but say an unexpected feature?

[Edward K. Ream]

On Sat, Mar 28, 2020 at 6:57 AM Thomas Passin <tbp1...@gmail.com> wrote:

I think I see the answer.  When you change something in a Leo node, you do not induce Python to re-compile any .pyc files in the various pycaches.

Updating a lib might invalidate cached calls to the lib if the lib's api changed.

If Leo had already loaded matplotlib earlier, then it would reuse that compiled code when you did another matplotlib import from within Leo (e.g., when trying out those examples).

Updating a lib had better invalidate the lib's caches. Otherwise python is broken.

Edward

[Thomas Passin]

I have updated this Leo file to add Seaborn plots, and to make a few minor adjustments in the other examples.  Note that Seaborn uses matplotlib behind the scenes.