Singularity exec error when running the container

[upendra kumar Devisetty]

Hi,

This is my first post to Singularity forum. I have recently started woking with Singularity and I find it awesome way of running Docker containers on HPC. As part of that i have today converted Docker container to Singularity container using `singularityware/docker2singularity` Docker image. Then I copied that image to HPC (XSEDE's stampede) and when I tried to execute it I am getting this error

singularity exec evolinc_evolinc-i_1.2-2017-05-03-44de0c39396b.img -c sample_data_evolinc-I/Sample_cuffcompare_out.gtf -g sample_data_evolinc-I/TAIR10_chr1.fasta -r sample_data_evolinc-I/TAIR10_chr1_genes.gff -n 4 -o test_out
/.exec: 3: exec: -c: not found

This is how I run the Docker container and i was just trying to replicate the same with Singularity containers. The docker container absolutely works great but for some reason not the Singularity container.

docker run --rm -v $(pwd):/working-dir -w /working-dir evolinc/evolinc-i:1.2 -c Sample_cuffcompare_out.gtf -g TAIR10_chr1.fasta -r TAIR10_chr1_genes.gff -o test_out -n 4

This is what the Singularity file inside the container looks like. Do I need to change anything in this file to make the container work. 

Singularity.evolinc_evolinc-i_1.2-2017-05-03-44de0c39396b.img> $ cat singularity
#!/bin/sh
evolinc-part-I.sh $@

Thanks,

Upendra

[CARLOS EDUARDO ARANGO GUTIERREZ]

Hi upendra kumar Devisetty , are you using the development branch?

On development branch this are my outputs

$ singularity run issue.img -c Sample_cuffcompare_out.gtf -g TAIR10_chr1.fasta -r TAIR10_chr1_genes.gff -o test_out -n 4

BEGIN!

Wed May 31 19:37:36 UTC 2017

grep: TAIR10_chr1.fasta: No such file or directory 

grep: TAIR10_chr1_genes.gff: No such file or directory

Remember that if you want to use the singularity file you must call it with run, or as an executable ./evolinc_evolinc-i_1.2-2017-05-03-44de0c39396b.img

hope it helps!

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

--

------------------------------------------------------------------------------------------------------------------------

CARLOS EDUARDO ARANGO GUTIERREZ.  

Cel : +57 3182545671

Computer Science PhD Student

UNIVERSIDAD DEL VALLE

Laboratorio de redes y sistemas distribuidos - LASCILAB

Tecnología, Información y Complejidad Cuántica - QuanTIC (GrupLAC) 

Centro de Investigación e Innovación en Bioinformática y Fotónica (CIBioFi)

                                                              

[upendra kumar Devisetty]

Hi Carlos,

Yay. It worked. Thanks for explaining the difference between run and exec commands. 

I am using the module that is installed on Stampede and I am not sure if is is development or a stable version. I will find out soon.

Regardless, it worked. Thanks a lot.

Upendra

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

[vanessa s]

Hi Upendra,

The docker2singularity is a bit different than the singularity software proper, and I would suggest you create your image using singularity and try again:

git clone -b development https://www.github.com/singularityware/singularity.git

cd singularity

./autogen.sh

./configure --prefix=/usr/local

make

sudo make install

Then for your docker image, you could do any of the following

1. Create and import

singularity create --size 6000 evolinc.img

singularity import docker://ubuntu:latest   # (replace your Docker image uri here)

2. Pull

singularity pull docker://ubuntu:latest

3. Bootstrap

bootstrap is the full, proper container creation, and using the development branch you can use different sections to add files, labels, etc. You would make a file called Singularity that has something like:

Bootstrap: docker
From: ubuntu:latest
%post
apt-get update && apt-get install -y git
# put other setup and install here
%files
/path/on/host /path/in/container
%runscript
# This is what your container does when you execute it
 
exec echo Hello "$@"

Check out the docs to see all the options! Please give those a try then we can look more closely at the bug.

Best,

Vanessa

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

--

------------------------------------------------------------------------------------------------------------------------

CARLOS EDUARDO ARANGO GUTIERREZ.  

Cel : +57 3182545671

Computer Science PhD Student

UNIVERSIDAD DEL VALLE

Laboratorio de redes y sistemas distribuidos - LASCILAB

Tecnología, Información y Complejidad Cuántica - QuanTIC (GrupLAC) 

Centro de Investigación e Innovación en Bioinformática y Fotónica (CIBioFi)

                                                              

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

--

Vanessa Villamia Sochat
Stanford University '16

(603) 321-0676

[upendra kumar Devisetty]

Hi Venessa,

I used the development branch and used the method 2 for making creating singularity image but I got the error. This is what I did..

vagrant up --provider virtualbox

vagrant ssh -c /bin/sh <<EOF
    sudo apt-get update
    sudo apt-get -y install build-essential curl git sudo man vim autoconf libtool
    git clone -b development https://github.com/singularityware/singularity.git

    cd singularity
    ./autogen.sh
    ./configure --prefix=/usr/local

    make
    sudo make install
EOF

vagrant ssh

vagrant@vagrant-ubuntu-trusty-64:~$ singularity pull docker://evolinc/evolinc-ii:1.2

image="evolinc-ii-1.2.img"
gtf="sample_data_evolinc-I/Sample_cuffcompare_out.gtf"
genome="sample_data_evolinc-I/TAIR10_chr1.fasta"
annot="sample_data_evolinc-I/TAIR10_chr1_genes.gff"
threads=12
output="test_out_2"
singularity run ${image} -c ${gtf} -g ${genome} -r ${annot} -n ${threads} -o ${output}

WARNING: Container bind point does not exist: '/home1' (homedir_base)
WARNING: Non existant 'bind point' in container: '/work'
WARNING: Non existant 'bind point' in container: '/scratch'
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Invalid option: -c

Thanks,

Upendra

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

[vanessa s]

What is your runscript set to do? Can you try using `exec` to pass a specific command to the container instead of run? And then please do that with `--debug` so we can see what is going on. If you set this up then overlayfs should be enabled so those directories should be created in the container (if they are not) but it would be good to test that as well!

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

[upendra kumar Devisetty]

Hi Venessa,

Firstly apologies because i was using the wrong image. I have two docker images which have almost identical names and I didn't realize that until you asked me to run it using exec. Anyway, I used the correct image but eventhough the run finished it is throwing some bunch of warnings and the output files are incomplete. I did not see these errors when I used the docker2singularity based method. Here are few errors

perl: warning: Setting locale failed.

perl: warning: Please check that your locale settings:

        LANGUAGE = (unset),

        LC_ALL = "en_US.UTF-8",

        LANG = "en_US.UTF-8"

    are supported and installed on your system.

perl: warning: Falling back to the standard locale ("C").

The second warning is interesting because it says it cannot find the input file but the input file exists 

/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

sed: can't read sample_data_evolinc-I/TAIR10_chr1_genes.gff: No such file or directory

sed: can't read sample_data_evolinc-I/TAIR10_chr1_genes.gff: No such file or directory

/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

Error: Unable to open file sample_data_evolinc-I/TAIR10_chr1_genes.gff. Exiting.

Error: Unable to open file sample_data_evolinc-I/TAIR10_chr1_genes.gff. Exiting.

login3.stampede(30)$ ls sample_data_evolinc-I/TAIR10_chr1_genes.gff 

sample_data_evolinc-I/TAIR10_chr1_genes.gff

Let me know if there is anything you want me to do.

Thanks,

Upendra

[David Godlove]

Try using ENV in your Dockerfile to set LANG=C.  Then everything should be groovy.  

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

[vanessa s]

For the path errors, they are telling you exactly what is the case, that the path `sample_data_evolinc-I/TAIR10_chr1_genes.gff` does not exist in the container. I would advise you to provide a full path, and not a relative one. To figure out where things are, you might try simple things to sniff around where things are, eg:

singularity exec ls container.img /

will list what is in the root. If you bind a path, you can confirm that the files are there. For example:

singularity exec -B /path/to/data:/path/in/container/ container.img ls /path/in/container

to verify that you see the files. Once you do, then when you run the script, you want to give it the full path to the files in the container.

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

[upendra kumar Devisetty]

Hi Venessa,

I have fixed the issues with the missing file error. However I have a new error which I don't see when i run the docker image using docker

IOError: [Errno 13] Permission denied: '/usr/lib/python2.7/dist-packages/.wh.six-1.5.2.egg-info'

Would you know what this indicates.

Thanks,

Upendra

[vanessa s]

Oh I've seen errors with the wheel files before - try adding a line to your bootstrap file in %post to change permissions for that file, eg like:

chmod u+x /usr/lib/python2.7/dist-packages/.wh.six-1.5.2.egg-info

I think (@satra) hit this bug too - there are likely many different of these files, and if it's a reasonably small number you could change permissions manually, or change for a wildcard.

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

[upendra kumar Devisetty]

Hi Venessa,

Unfortunately bootstrapping with the command you suggested did changed the permission of the file. What else can I try? Thanks..

singularity create --size 6000 test.img
sudo singularity bootstrap test.img file.def

Bootstrap: docker
From: docker://evolinc/evolinc-i:1.2
%post
    chmod u+x /usr/lib/python2.7/dist-packages/.wh.six-1.5.2.egg-info
%runscript
     exec evolinc-part-I.sh "$@"

vagrant@vagrant:~$ singularity shell test.img
Singularity test.img:~> ls -lh /usr/lib/python2.7/dist-packages/.wh.six-1.5.2.egg-info
---x------ 1 root root 0 May  3 00:40 /usr/lib/python2.7/dist-packages/.wh.six-1.5.2.egg-info

[vanessa s]

What permission do you want it to have, and what are you trying to do? Wheels are (I think) Python's new way to package installations (a replacement for egg maybe?) I think you have a few options - either change the permission of the file to what you would want (chmod 0755? something else?), modify how/where python is storing wheels (do a google search for python / caching/ wheels), or use a virtual environment (without sudo) so your pip installs don't need sudo (and thus aren't owned by root).

If you are able to share your Singularity build file in entirely I can try to reproduce the error and help  further.

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

[upendra kumar Devisetty]

Venessa, finally it worked. There are two python wheel files and changing the permission for those two with chmod 0755 fixed the problem.. I wonder why it worked when i used docker2singularity method rather than creating,pulling and bootstrapping singularity image from the scratch

[vanessa s]

Great news! The two methods handle permissions differently, I'll just leave it at that.

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

[upendra kumar Devisetty]

Thanks for letting me know that..I will keep that in mind..Thanks again for your help troubleshooting..