Intel oneAPI fails to work with Singularity

[Samy]

Hello,

I upgraded my containers to run with intel onAPI but I'm having issues. it looks like the oneApi script breaks my singularity container  environment and runscripts.

How to set  up shell in %environment and %app (scfi) in the definition files for this scenario to work please?

Here is an example def file:

Bootstrap: localimage
From: /tmp/base.sif
%environment
source /opt/intel/oneapi/setvars.sh --force

%apprun lammps
mpirun --version
mpiexec.hydra --version
~

Here is the error when sourcing the script. I tried other ways to source the script (bash, /bin/bash) but they all failed):

Thank you,

[Samy]

I meant Intel oneApi compiler 

https://www.intel.com/content/www/us/en/developer/tools/oneapi/toolkits.html

[Fatih Ertinaz]

Hi

Is there a file called mpivars.sh you can source?

Fatih

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.
To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/e52d47a5-76b9-4397-91bf-4b29209b10a6n%40lbl.gov.

[smahane douyeb]

I need many other libraries like mil, tbb, ICC ..

[v]

It’s hard to help you debug this without having the base image to work with.

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/CAMK%3DTbEGLczKVzyKO3ChNNFGJ8BEaniHFn9HyDC%3D71AG1OD_%3DQ%40mail.gmail.com.

[Samy]

Hello Vanessa,

This is the definition file for the base image:

BootStrap: docker  
From: intel/oneapi-runtime
Include: apt-get

%post
#####
export DEBIAN_FRONTEND=noninteractive
apt-get update && apt-get install apt-file -y && apt-file update
apt install -y build-essential
apt-get install -y sudo git wget #vim which tree
apt-get install -y hostname #lscpu uptime numactl
apt-get install bc -y
apt-get -y install software-properties-common

apt-get install -y libnl*

Thank you,

[v]

Some quick feedback - there is no "setvars.sh" in the folder you are trying to source.

Singularity> ls /opt/intel/oneapi/
lib

Just a folder named lib.

[Glenn (Gedaliah) Wolosh]

Hello,

If this is an ubuntu image it is quite possible that dash rather than bash is being used to execute setvars.sh.  You can configure the container to use bash.

In your def file for the bas image:

    echo "dash dash/sh boolean false" | debconf-set-selections

    DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash

Glenn (Gedaliah) Wolosh, Ph.D. Ass't Director Research Software and Cloud Computing Acad & Research Computing Systems gwo...@njit.edu • (973) 596-5437 A Top 100 National University U.S. News & World Report

<oneapi.PNG>

Thank you,

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/27184ba7-04db-4257-9045-e1259d75f4a8n%40lbl.gov.
<oneapi.PNG>

[Glenn (Gedaliah) Wolosh]

This is because Samy is binding /opt/intel to the container.  I assume /opt/intel has the full installation.

Glenn (Gedaliah) Wolosh, Ph.D. Ass't Director Research Software and Cloud Computing Acad & Research Computing Systems gwo...@njit.edu • (973) 596-5437 A Top 100 National University U.S. News & World Report

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/CAM%3Dpu%2B%2BAWGpSEgFVceBzp%3Dc%3DHWEjJhTMEypbv5d0BBJuAd19gw%40mail.gmail.com.

[v]

Ah, so I'll say again... you have to provide enough for me to reproduce if I am able to help! How could I otherwise?

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/1FF259AD-2536-4A5A-8738-ABD897D8081E%40njit.edu.

[Glenn (Gedaliah) Wolosh]

v,

I know what you mean. I happened to have caught it in the screenshot Samy provided.

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/CAM%3Dpu%2BJjSxYMr3ugY-XOWTsknF4h9y%3DDZtHW5k0HB5S6DJMefg%40mail.gmail.com.

[Fatih Ertinaz]

Looks like Samy pulls the wrong image.

$ docker run -it intel/oneapi-runtime bash
root@9f51d9ca772d:/# find . -iname "*vars*" -type f
./lib/init/vars.sh
./opt/intel/oneapi/lib/env/compiler_rt_vars.sh
./usr/share/perl5/Dpkg/Vars.pm
./usr/share/perl5/Dpkg/Substvars.pm
./usr/share/perl/5.26.1/vars.pm
./usr/lib/x86_64-linux-gnu/perl/5.26.1/CORE/perlvars.h
./usr/lib/x86_64-linux-gnu/perl-base/vars.pm

Apparently, there is no setvars.sh file in this image. However, if you pull the hpc version;

$ docker run -it intel/oneapi-hpckit bash

You can see that there are a lot of source files for compilers, mpi, mkl etc. Then, sourcing setvars.sh gives access to these binaries;

root@955d496c4ecd:/# source ./opt/intel/oneapi/setvars.sh

root@955d496c4ecd:/# which mpirun
/opt/intel/oneapi/mpi/2021.4.0//bin/mpirun
root@955d496c4ecd:/# which icc
/opt/intel/oneapi/compiler/2021.4.0/linux/bin/intel64/icc

Fatih

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/781287B8-D05A-44E2-8A61-21132CBE1322%40njit.edu.

[smahane douyeb]

Hello @vanessa sochat and @Fatih Ertinaz ,

Sorry for the lack of details. 

I'm not using the intel one API that is installed inside the container because I couldn't get it to work.

I installed it on the host as :

wget https://registrationcenter-download.intel.com/akdlm/irc_nas/18236/l_BaseKit_p_2021.4.0.3422_offline.sh

sudo bash l_BaseKit_p_2021.4.0.3422_offline.sh

Built the container. This recipe should reproduce my issue:

$ sudo singularity build lammps.sif lammps.def

$ cat lammps.def

BootStrap: docker  

From: ubuntu:20.04

%environment

source /opt/intel/oneapi/setvars.sh --force

%apprun lammps

mpirun --version

mpiexec.hydra --version

~

Run the container as:

$ singularity run --writable-tmpfs -B /opt/intel --app lammps lammps.sif

Output:

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/CAJk3%2BYVbxOFWv%2BftHQWVbDiO%3DmoXPBnNuD6%3DPn-qsPKDMAbhmg%40mail.gmail.com.