fenics under singularity?

[William Wilson]

Does anyone have a concise way of running fenics under singularity on a cluster?

[Jack Hale]

https://bitbucket.org/fenics-project/docker

See: Experimental Singularity Support

On Thu, 8 Mar 2018 at 17:34 William Wilson <gwy...@gmail.com> wrote:

Does anyone have a concise way of running fenics under singularity on a cluster?

--
You received this message because you are subscribed to the Google Groups "fenics-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fenics-suppor...@googlegroups.com.
To post to this group, send email to fenics-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fenics-support/fa51898a-e39b-4903-a0e2-1b38289a8db5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jack Hale]

P.S. Feedback on how it went on a cluster would be appreciated. I've tried it here at University of Luxembourg and it seemed like rebuilding with OpenMPI was necessary, but I didn't have time to explore further. I have OpenMPI builds on a branch here:

https://bitbucket.org/fenics-project/docker/src/61c4bc99075f2a41105396e3d269195fab3d0403/?at=jackhale%2Fopenmpi

Hopefully Singularity will support MPICH ABI-compatible implementations soon, allowing us to perform the same trick as described in this paper:

https://arxiv.org/abs/1608.07573

[William Wilson]

One other question. I'm testing things out on an RHEL 7 system as well with docker. I launched it as follows:
docker run -ti -v $(pwd):/home/fenics/shared quay.io/fenicsproject/stable:current

We are not enforcing selinux right now. When in the container we get permission denied when we try and access the shared folder. Any idea what is up with that?

I'll look into the conversion to singularity.

[William Wilson]

On the singularity support page it shows the following:

Experimental: Singularity support

This repository contains a script to build dev-env and stable images that are compatible with the Singularity container runtime http://singularity.lbl.gov/:

cd dockerfiles
./build-singularity-images.sh
cd stable
singularity run -e stable.img

However, it does not say how to get the docker files that you cd into to build the singularity container.

[Jack Hale]

Clone the bitbucket repository using git.

To view this discussion on the web visit https://groups.google.com/d/msgid/fenics-support/427e082d-53e8-4488-aad3-33e60b426471%40googlegroups.com.

[William Wilson]

Having issues with that.

git clone https://quay.io/repository/fenicsproject/dev-env
Initialized empty Git repository in /common/contrib/containers/fenics/docker/dev-env/.git/
warning: remote HEAD refers to nonexistent ref, unable to checkout.

[Jan Blechta]

git clone https://bitbucket.org/fenics-project/docker

or

git clone g...@bitbucket.org:fenics-project/docker

Jan

> >>>> <https://groups.google.com/d/msgid/fenics-support/fa51898a-e39b-4903-a0e2-1b38289a8db5%40googlegroups.com?utm_medium=email&utm_source=footer>
> >>>> .

> >>>> For more options, visit https://groups.google.com/d/optout.
> >>>>
> >>> --
> >> You received this message because you are subscribed to the Google
> >> Groups "fenics-support" group.
> >> To unsubscribe from this group and stop receiving emails from it,

> >> send an email to fenics-suppor...@googlegroups.com <javascript:>.

> >> To post to this group, send email to fenics-...@googlegroups.com

> >> <javascript:>.

> >> To view this discussion on the web visit

> >> https://groups.google.com/d/msgid/fenics-support/427e082d-53e8-4488-aad3-33e60b426471%40googlegroups.com
> >> <https://groups.google.com/d/msgid/fenics-support/427e082d-53e8-4488-aad3-33e60b426471%40googlegroups.com?utm_medium=email&utm_source=footer>
> >> .

[William Wilson]

The singularity build script specifically looks for the dev-env directory tree which appears to be missing. So, makes it a little difficult to build the singularity container.

[Johannes Ring]

What do you mean? This works fine for me:

git clone https://bitbucket.org/fenics-project/docker
cd docker/dockerfiles
./build-singularity-images.sh

Johannes

> https://groups.google.com/d/msgid/fenics-support/80a3b829-7bcd-44b6-a1a3-cc60ff193d4f%40googlegroups.com.

[William Wilson]

I'm going to post notes here on our work for getting fenics up and running under singularity.

We successfully did the build and now are doing some testing. On the stable.img release that is created we get the following error when running the poisson demo:
wew@wind:~/demo/documented/poisson/python$ python3 demo_poisson.py
Solving linear variational problem.
Traceback (most recent call last):
  File "/usr/lib/python3.5/tkinter/__init__.py", line 36, in <module>
    import _tkinter
ImportError: No module named '_tkinter'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "demo_poisson.py", line 188, in <module>
    import matplotlib.pyplot as plt
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/pyplot.py", line 116, in <module>
    _backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/backends/__init__.py", line 60, in pylab_setup
    [backend_name], 0)
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/backends/backend_tkagg.py", line 6, in <module>
    from six.moves import tkinter as Tk
  File "/usr/lib/python3/dist-packages/six.py", line 92, in __get__
    result = self._resolve()
  File "/usr/lib/python3/dist-packages/six.py", line 115, in _resolve
    return _import_module(self.mod)
  File "/usr/lib/python3/dist-packages/six.py", line 82, in _import_module
    __import__(name)
  File "/usr/lib/python3.5/tkinter/__init__.py", line 38, in <module>
    raise ImportError(str(msg) + ', please install the python3-tk package')
ImportError: No module named '_tkinter', please install the python3-tk package
Aborted

[William Wilson]

Now seeing a new issue after fiddling with things. So, am running singularity shell with the fenics singularity container that I created. Running as my user and when I try and run the demo_poisson.py program under python3 I get the following error. My initial ssh session is using the -X option to allow for X11.

Singularity fenics-sandbox-orig:~> export DISPLAY=:0.0
Singularity fenics-sandbox-orig:~> cd demo/documented/poisson/python/
Singularity fenics-sandbox-orig:~/demo/documented/poisson/python> python3 demo_poisson.py

Solving linear variational problem.
Traceback (most recent call last):

  File "demo_poisson.py", line 189, in <module>
    plot(u)
  File "/usr/lib/python3/dist-packages/dolfin/common/plotting.py", line 448, in plot
    return _plot_matplotlib(object, mesh, kwargs)
  File "/usr/lib/python3/dist-packages/dolfin/common/plotting.py", line 290, in _plot_matplotlib
    ax = plt.gca()
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/pyplot.py", line 962, in gca
    return gcf().gca(**kwargs)
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/pyplot.py", line 592, in gcf
    return figure()
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/pyplot.py", line 539, in figure
    **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/backend_bases.py", line 171, in new_figure_manager
    return cls.new_figure_manager_given_figure(num, fig)
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/backends/backend_tkagg.py", line 1049, in new_figure_manager_given_figure
    window = Tk.Tk(className="matplotlib")
  File "/usr/lib/python3.5/tkinter/__init__.py", line 1871, in __init__
    self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: couldn't connect to display ":0.0"

Aborted



On Thursday, March 8, 2018 at 9:34:42 AM UTC-7, William Wilson wrote:

[Johannes Ring]

Plotting in docker is not straightforward, but there are workarounds:
https://fenics-containers.readthedocs.io/en/latest/work_flows.html#use-graphical-applications-on-linux-hosts.
I don't know about singularity.

If you don't need plotting you can use `export DOLFIN_NOPLOT=1`.

Johannes

> --
> You received this message because you are subscribed to the Google Groups
> "fenics-support" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to fenics-suppor...@googlegroups.com.
> To post to this group, send email to fenics-...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/fenics-support/728b5fe1-45a7-4391-b336-2a19d1b743f9%40googlegroups.com.

[Jack Hale]

To visualise on HPC you should write out to XDMF and then use Paraview.

The matplotlib plotting is a feature intended for basic feedback when running with one process.

To view this discussion on the web visit https://groups.google.com/d/msgid/fenics-support/CALjQY_HPfxDiUUsM2tExD%3Ddjc4vhGCbmSrm-rvuH4%3Dd2zyVA1A%40mail.gmail.com.

[William Wilson]

I now have a reasonable singularity image that I can use interactively. I will be working with one of our researchers to make sure that we can use the exec function and then run jobs under slurm. The demos that I have tried work fine in a shell.

[Douglas Arnold]

I have just started using the singularity container built from the docker containers on github.  I ran into the same problem as William Wilson, the message "ImportError: No module named '_tkinter'" and then no plotting.  It turns out there is a simple fix.  I just added one line to the singularity recipe in the file docker/dockerfiles/stable/Singularity.  In the "%post" section, after "ldconfig", I added "apt-get install -y python3-tk".  I then built as normal (either with

"singularity build stable.img Singularity" or the script "build-singularity-images.sh").  The resulting container supports matplotlib and FEniCS graphics.

I made one other small correction to the Singularity recipe: I added "chmod a+r /usr/local/share/WELCOME" to the %post section.  This gets rids of an error message that otherwise occurs when the container is run.

[Jack Hale]

I've baked python3-tk into the Docker dev-env image. We'll be releasing 2019.1.0 shortly and all of these changes should be baked into the hosted images then.