Jenkins running docker container slaves that run builds using docker

[Yakobe]

Hello everyone,

Here is my desired scenario:

A master jenkins server that can spin up containerized jenkins slaves using docker as required.
These slaves can use docker / docker-compose to build projects and run tests.
I would preferably like to use swarm / ECS to run all these containers (both the slaves and the builds & tests).

Is this even possible? If so, could someone point me in the right direction? I'm willing to do a lot of investigation etc. I have been trying to find documentation / guides but have not come up with anything concrete for this double-docker solution yet. Any help would be much appreciated.

Jake

[Yakobe]

The main aim behind this post is to find out if this is even possible before losing too much time. So if anyone could any insight at all it would be greatly appreciated.

[nicolas de loof]

You have two options :

1. use a Docker in Docker image (play-with-docker.com for sample is using this approach)

2. bind mount /var/run/docker.sock into your build container so you can interact with the host's docker daemon

Both of them involve some security consideration, so you might need to double check your infrastructure requirements.

2017-07-10 15:11 GMT+02:00 Yakobe <jakew...@gmail.com>:

The main aim behind this post is to find out if this is even possible before losing too much time. So if anyone could any insight at all it would be greatly appreciated.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/fa88594b-1780-48d5-aad6-c4a960e8dae1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Yakobe]

Then it sounds like this i achievable. Thanks so much for the response. At least i know that this is something worth looking in to.

I read somewhere about this Docker in Docker approach (i think it is sometimes referred to as DinD?). Some people where saying it is not a good idea.
Am i right in understanding that it is better to bind mount /var/run/docker.sock?

[nicolas de loof]

Not so simple

bind mounting docker.sock means you give access to anything on host, so you just give up with security and isolation between builds.

Docker in Docker require some adequate combination for the docker daemon setup to be nested, especially on how it stores containers layers on disk (and then, nested layers for second level docker daemon). play-with-docker uses device mapper + overlay2 with success.

--

You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/a65338f8-a2da-4a3a-93dd-224d8b3fb74b%40googlegroups.com.

[Yakobe]

The security of these jenkins builds are not critical since they are just for testing. Therefore it sounds perhaps a simpler approach.
However, i'm interested in what the issues are with build isolation. Does that mean that build could fail, and under what sort of circumstance?

[nicolas de loof]

it means a build with some "malicious code" (sic) could access all other builds, secrets injected into a build environment by jenkins, all host ressources, etc. It could as well create by intent or because of some buggy script create thousands containers and exhaust host resources. 

consider docker.sock as some super-root access to your build node.

2017-07-10 17:45 GMT+02:00 Yakobe <jakew...@gmail.com>:

The security of these jenkins builds are not critical since they are just for testing. Therefore it sounds perhaps a simpler approach.
However, i'm interested in what the issues are with build isolation. Does that mean that build could fail, and under what sort of circumstance?

--

You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/6fadf55e-ccff-4e9a-951b-8e9731e8c01a%40googlegroups.com.

[Jake Bishop]

Yes, that makes sense. I think this is acceptable in our scenario.

Thanks for the pointer. 

If you (or anyone else here) has any links to examples it would be greatly appreciated 👍

You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/GgLlUPWRLcQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJzkm%2BJFqFKROs-0%2BgNM9ovo-ThT0zSF%3DrtRxe9PS5D51uQ%40mail.gmail.com.