squash-fs error on build

[Ketan Maheshwari]

Hello,

When I try to build an example Singularity image, I get this error:

# singularity build lolcow.simg shub://GodloveD/lolcow

ERROR: You must install squashfs-tools to build images

ABORT: Aborting with RETVAL=255

I tested this with both root and mortal users--the same error shows up.

The Singularity version is 2.4-dist. I remember this used to work in the past.

Why does it ask for installation of squashfs-tools? I looked up the documents a bit but could not find solution.

Any help is appreciated.

Thanks,

--

Ketan

[David Godlove]

Hi Ketan,

Sorry for the trouble.  In 2.4 we are moving to a new image format.  Instead of ext3 we are using squashfs.  So singularity now requires the squashfs-tools suite to be installed in it's default system location to build images.  

Dave

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

[Ketan Maheshwari]

Thank you Dave for the quick response. Does this mean that my host system should have the squashfs--the filesystem installed? Could I just install squashfs-tools without installing the filesystem? If yes, wondering if the squashfs-tools are available to install from source somewhere?

Thank you and much appreciated.

--

Ketan

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

[vanessa s]

You should be able to do apt-get install -y squashfs-tools

Having the tools to make squashies doesn't mean your filesystem has to be a squashie :)

--

Vanessa Villamia Sochat
Stanford University '16

(603) 321-0676

[David Godlove]

You don't need to have a squashfs file system on your host.  You just need the tools.  Basically you just need the mksquashfs command to be available.  And it needs to be available from one of the following locations:

/bin:/usr/bin:/usr/local/bin:/sbin:/usr/sbin:/usr/local/sbin

You can install it using apt or yum.  As for getting it from source, I _think_ this is the official source.  But you should double check before installing.  

[Ketan Maheshwari]

Thank you Vanessa and Dave for your prompt responses, much appreciated. The problem is solved. -- Ketan

[Igor Yakushin]

On Tuesday, October 24, 2017 at 1:11:42 PM UTC-5, David Godlove wrote:

You don't need to have a squashfs file system on your host.  You just need the tools.  Basically you just need the mksquashfs command to be available.  And it needs to be available from one of the following locations:

/bin:/usr/bin:/usr/local/bin:/sbin:/usr/sbin:/usr/local/sbin

Hi David,

Why does it have to be from those directories? I am having hard time convincing my sysadmin to install it from rpm. I tried to compile it myself and put in a non-standard location, set PATH, but it did not work. Is the path to mksquashfs hardcoded somewhere?

Thank you,

Igor

 

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

[David Godlove]

Sorry Igor,

It's a security feature.  Singularity sanitizes the path when it starts so that a user cannot slip something silly into the $PATH and then execute it with elevated privs.  In the past it wasn't a big deal because all dependancies were already in those directories on your average RHEL, CentOS, Ubuntu or what-have-you.  But now we have the added dep of mksquashfs.

I'm thinking of writing a PR to make the location of mksquashfs configurable in singularity.conf.  But even if Greg likes the idea and it is merged it won't be in an official release for a little while.  

Dave

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

[Igor Yakushin]

Hi David,

I'm thinking of writing a PR to make the location of mksquashfs configurable in singularity.conf.  But even if Greg likes the idea and it is merged it won't be in an official release for a little while.  

What would work for me as well, since I am managing Singularity installation, is to install mksquashfs into the same bin directory where singularity is. I tried that but it still does not work. Is it easy to modify code for that?

Thank you,

Igor

--

Igor Yakushin, Ph.D.

Computational Scientist

Kavli Institute for Cosmological Physics, ERC #413

Research Computing Center, room #2

The University of Chicago

773-834-5313

https://rcc.uchicago.edu

[Gregory M. Kurtzer]

I would concur with both of these ideas and it shouldn't be difficult to do as it is all happening in shell scripts at this point of the process... But let us know if you have any questions.

Thanks!

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

--

Gregory M. Kurtzer

CEO, SingularityWare, LLC.

Senior Architect, RStor

[Igor Yakushin]

On Tue, Oct 24, 2017 at 5:06 PM, Gregory M. Kurtzer <gmku...@gmail.com> wrote:

I would concur with both of these ideas and it shouldn't be difficult to do as it is all happening in shell scripts at this point of the process... But let us know if you have any questions.

So, all I have to do is to modify 

PATH="/bin:/usr/bin:/usr/local/bin:/sbin:/usr/sbin:/usr/local/sbin"

in the main singularity script?

At first glance it is working.

[Igor Yakushin]

Correction, it is working on our Scientific Linux 7 cluster but seems to have problems on Scientific Linux 6 cluster:

==========

singularity build d.img  docker://tensorflow/tensorflow:latest-gpu

...

Building Singularity image...
/software/singularity-2.4-el6-x86_64/libexec/singularity/cli/build.exec: line 379: 28780 Bus error               mksquashfs "$SINGULARITY_ROOTFS/" "$SINGULARITY_CONTAINER_OUTPUT" -noappend $
OPTS > /dev/null
ERROR: Failed squashing image, left template directory at: /tmp/.singularity-build.XfW0mO
Cleaning up...
==========

The resulting d.img seems to be unusable. BTW, is there a way to use a different temporary directory? Perhaps, it is running out of space in /tmp

Also, I noticed that mksquashfs seems to be using all the cores it can find. Is it expected?

[David Godlove]

That would be one way to do it.  But in general you shouldn't, and you shouldn't have to modify the source code to get things working on your setup.  

Have a look at this PR.  It should address your issue.  

https://github.com/singularityware/singularity/pull/1094

[Igor Yakushin]

On Thu, Oct 26, 2017 at 1:13 PM, David Godlove <davidg...@gmail.com> wrote:

That would be one way to do it.  But in general you shouldn't, and you shouldn't have to modify the source code to get things working on your setup.  

Have a look at this PR.  It should address your issue.  

https://github.com/singularityware/singularity/pull/1094

David,

So this feature is not available in 2.4 but I have to get the latest git version?

Thank you,

Igor

[David Godlove]

Hi Igor,

Yes you are right, it is not available in the 2.4 distribution.  It's also not available in the latest version of the master branch on github.  I've submitted a pull request to the development branch.  So as soon as a few members of the Singularity team review it and decide that it is good to go, it will be merged into development.  Then when the next release comes out, it will be included there.  But right now, the only place you can get it is from my fork.  So if you are interested in testing it to make sure it does what you need, you can get it with the following commands:

git clone https://github.com/GodloveD/singularity.git

cd singularity

git fetch

git checkout specify-paths

Then you can do the ./autogen, ./confgure, make, sudo make install procedure as usual.  Let me know if that works for you.  Thank you.

Dave

[varun narisetty]

Hi Dave,

I have similar issue but after installing the squashfs-tools fixed that issue.

But now when I try to run a container I get the following error.

singularity run d.img

ERROR  : Failed to mount squashfs image in (read only): No such device

ABORT  : Retval = 255

This is the error when I run in debug mode

VERBOSE [U=0,P=34698]      singularity_image_squashfs_mount()        Mounting squashfs image: /dev/loop0 -> /usr/local/var/singularity/mnt/container

ERROR   [U=0,P=34698]      singularity_image_squashfs_mount()        Failed to mount squashfs image in (read only): No such device

ABORT   [U=0,P=34698]      singularity_image_squashfs_mount()        Retval = 255

I was not able to understand what this error is?

It would be really helpful If any one can through some light on this.

Thanks,

Varun

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.