Help/Pointers on using docker with travis-ci

[Simon King]

Hi!

Last week at Sage Days 100 I learned how to put the code for my group
cohomology spkg on github and got an introduction on using travis-ci.
However, it seems that I need more help/pointers, in particular on docker.

The things that I want travis-ci to do are:
- Install dependencies of my code into a docker container. I guess that
goes to the "before_install" section of my .travis.yml file. So, it
should be pulling the latest sagemath-dev docker image, creating a
container and sending the commands "sage -i meataxe" and "sage -b" to
the container.
- Install the package itself. I guess that goes to the "install"
section. It involves opening a sage shell, installing some C library
with configure/make/make install, and install some Python/cython code
with "pip install .".
- Run some tests; I guess that goes to the "script" section.

My problems:
How to keep a docker container running? Currently, after executing a
command, the docker container simply exits. It was suggested to me at
the workshop to put all commands used to install and test my software
into a single shell script and execute that (which means to send only
a single command to the docker container), but it doesn't seem clean
to me (better separate before_install, install and script).

I tried to create and start a container and send commands using "docker
container exec". It did work sometimes, i.e., after some commands the
container stays up and running, but, e.g., it happened that after
"docker container exec <name> sage -i meataxe" the container exited.

What is the recommendet way to proceed?

PS: I just tried some things again, getting a different error:
docker container create --name bla sagemath/sagemath-dev:latest
docker container start bla
docker container exec bla ls
docker container exec bla sage -i meataxe
results in an error in the last command, namely

make build/make/Makefile --stop
make[1]: Entering directory '/home/sage/sage'
rm -f config.log
mkdir -p logs/pkgs
ln -s logs/pkgs/config.log config.log
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether to enable maintainer-specific portions of Makefiles... yes
checking for root user... no
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking for a sed that does not truncate output... /bin/sed
checking for ar... yes
checking for m4... yes
checking for pkg-config... no
checking for ranlib... yes
checking for strip... yes
checking for GNU or BSD tar... /bin/tar
checking for GNU make... /usr/bin/make
checking for latex... no
configure: You do not have 'latex', which is recommended, but not
configure: required. Latex is only really used for building pdf
configure: documents and for %latex mode in the Sage notebook.
checking for perl... /usr/bin/perl
checking for Perl version 5.8.0 or later... yes
checking for gcc... gcc
sed: can't read conftest.err: No such file or directory
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... configure: error: in `/home/sage/sage':
configure: error: cannot compute suffix of executables: cannot compile and link
See `config.log' for more details
If you would like to try to build Sage anyway (to help porting),
export the variable 'SAGE_PORT' to something non-empty.
make[1]: *** [build/make/Makefile] Error 1
Makefile:39: recipe for target 'build/make/Makefile' failed
make[1]: Leaving directory '/home/sage/sage'
Makefile:31: recipe for target 'all-toolchain' failed
make: *** [all-toolchain] Error 2

Why is that? "sage -i meataxe" did work before (at least with
"cocker container run ...")

Best regards,
Simon

[Nathan Dunfield]

Simon,

You can start a container and open a shell on it via:

docker run -it image_name /bin/bash

The container will keep running until you exit the shell, if not longer.  You can open a shell on any running container via

docker exec -it container_name /bin/bash

Best,

Nathan

[Simon King]

Nathan,

It works to some extent: When I do
docker run -it --name bla sagemath/sagemath-dev:latest /bin/bash
then (after waiting for some time because apparently the docker image
does some compilation of Sage before doing anything else) I am indeed in
a bash in docker - and stay there, interactively, which is not what I
want.

I.e., I need to leave with ctrl-p and ctrl-q, and then I can execute
commands in that container (such as

docker container exec bla sage -i meataxe

)

How can I start/run the container so that I don't need ctrl-p and
ctrl-q? After all, I want to use the container with a script, not
interactively. I tried the option "-dit", but then I don't know (and the
script doesn't know) how long I need to wait until the preparatory steps
of the container are done. Right after
docker run -dit --name bla sagemath/sagemath-dev:latest /bin/bash
the command
docker container exec bla sage -c "print('Hello')"
fails, because "sage.all" is not there yet:
Traceback (most recent call last):
File "/home/sage/sage/src/bin/sage-eval", line 4, in <module>
from sage.all import *
ImportError: No module named sage.all

The same command works after waiting long enough.

Best regards,
Simon

[Simon King]

Hi!

It seems that the following works:

- Start the container and wait until everything is functional, using
docker's health check:
docker run --name=test -dit --health-cmd='sage -c "print(1)"' --health-interval='20s' --health-timeout='20s' --health-start-period='50s' sagemath/sagemath-dev:latest /bin/bash && until docker inspect --format "{{json .State.Health.Status }}" test | grep -m 1 -w "healthy"; do sleep 5; done

- Execute commands:
docker exec test command1
docker exec test command2
...

- Stop and remove the docker container.

Would you recommend a different solution?

Best regards,
Simon