Securely insert ssh keys into Jenkins Docker image

[Barry Laffoy]

Hi

I'm trying to set-up our Jenkins master in a reproducible way using the Docker image here: https://github.com/jenkinsci/docker

One problem I am having is how to make ssh private keys (e.g. for polling git repos) available within the container in a secure manner. Ideally, I would like to be able set the private keys at runtime for the container, as a way to test the configuration in a safe environment before it is promoted to production.

Copying the keys at build time fails this second requirement (and may have security implications if it would make a user's private key visible to anybody with permission to run the container?).

Mounting the key location as volume with `docker run -v /path/to/keys:/var/jenkins_home/.ssh` does not work, as the mount point in the container inherits the uid/gid of the host directory, which does not match the uid/gid of the "jenkins" user within the container. There are solutions to this available, using gosu, but that seems to be strongly discouraged by the community.

What is the advised/best practise way of injecting ssh private keys into the Jenkins master container?

Thanks

[Stephen Connolly]

there are two other options:

1. Bake the secret key used to encrypt Jenkins secrets into your image (downside, anyone with access to the image can decrypt they keys)

2. Seed the JENKINS_HOME volume with unencrypted credentials.xml and have an init.groovy.d script trigger a save (which will encrypt them on first start)

Outside of that, you could look into implementing a plugin that extends from CredentialsProvider and exposes your ssh credentials how you see fit.

one I think would be nice is one that basically takes an SSH_AGENT from the Jenkins user and exposes its keys as SSH keys (so good an idea I may implement it myself... if I get spare time or if my employer agrees it is a good idea)

That way you could mount the SSH_AGENT socket into docker and away you go... of course none of that is written yet... so fit now just hack it working 

--


You received this message because you are subscribed to the Google Groups "Jenkins Users" group.


To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.


To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/8d8d6fd2-11c2-4946-93b3-07981944af23%40googlegroups.com.


For more options, visit https://groups.google.com/d/optout.

--

Sent from my phone

[Dirk Heinrichs]

Am 03.01.2017 um 15:21 schrieb Barry Laffoy:

Mounting the key location as volume with `docker run -v /path/to/keys:/var/jenkins_home/.ssh` does not work, as the mount point in the container inherits the uid/gid of the host directory, which does not match the uid/gid of the "jenkins" user within the container.

Well, then let it match. Or do you need it on the host?

HTH...

    Dirk

--
Dirk Heinrichs
Senior Systems Engineer, Delivery Pipeline
OpenText^TM Discovery | Recommind
Email: dirk.he...@recommind.com
Website: www.recommind.de

Recommind GmbH, Von-Liebig-Straße 1, 53359 Rheinbach

Vertretungsberechtigte Geschäftsführer John Marshall Doolittle, Gordon Davies, Roger Illing, Registergericht Amtsgericht Bonn, Registernummer HRB 10646

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden

Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail sind nicht gestattet.

www.recommind.com

[Barry Laffoy]

Well, then let it match. Or do you need it on the host?

HTH...

    Dirk

Thanks for the response, but I think I must be missing a trick because I don't know what you're getting at.

If say, the uid of the host user is 1005, and the uid of the container's jenkins user is 1000, these clearly don't match. I would need to set the uid of the user in the container to 1005 in order to let it use the mounted keys, as in this PR https://github.com/jenkinsci/docker/pull/238/files

ENTRYPOINT usermod -u $(stat -c "%u" /var/jenkins_home) jenkins && \+ gosu jenkins /bin/tini -- /usr/local/bin/jenkins.sh

But this change was rejected for reasons that running a container as root is against best practises.

Or am I misunderstanding and you're suggesting something else? Would it be possible to simply alter the uid of the Jenkins user without root & gosu? Or do you mean to only run the container as a user on host with uid 1000?

[Barry Laffoy]

one I think would be nice is one that basically takes an SSH_AGENT from the Jenkins user and exposes its keys as SSH keys (so good an idea I may implement it myself... if I get spare time or if my employer agrees it is a good idea)

In my reading I came across some references to mounting $SSH_AUTH_SOCK as a volume in the container and using that for ssh-agent forwarding. I didn't really understand what I was doing, I'll need to read more about ssh-agent forwarding, but I think that it will fail at least for the same reason as mounting ~/.ssh/ as a volume fails: the uid will inherit from the host, and not match the container uid.

2. Seed the JENKINS_HOME volume with unencrypted credentials.xml and have an init.groovy.d script trigger a save (which will encrypt them on first start)

This sounds like a workable way forward. 

[Dirk Heinrichs]

Am 04.01.2017 um 09:35 schrieb Barry Laffoy:

Thanks for the response, but I think I must be missing a trick because I don't know what you're getting at.

If say, the uid of the host user is 1005, and the uid of the container's jenkins user is 1000, these clearly don't match. I would need to set the uid of the user in the container to 1005 in order to let it use the mounted keys,

You just need to make sure that the host dir is owned by the correct uid/gid pair (the one that matches your jenkins user inside the container). There's no need to also have that user on the host, too (although you can, but then their uid/gid should simply match). I'd just make sure there's no user on the host with the same uid/gid pair. This way, only root on the host and the jenkins user inside the container will have access to the keys.

[Barry Laffoy]

Aha, I understand what you're saying now. There is such a user on both the development and the production hosts I'm using, unfortunately. I'll investigate what can be done to change that.

[Barry Laffoy]

Dirk Heinrichs' comments have made me realise that this problem goes both ways. Since my host has a user with uid 1000, the data volume on the host at, say, /var/lib/docker/volumes/6a70b50dc1c6d201bd772e8d4bf8cb17d9668fbed0661e27b5fcf7d6b7847070/_data is actually owned by that host user.

What I need to be doing is building the jenkinsci/docker image with "--build-arg uid=$(id -u) --build-arg gid=$(id -g)" to match the container to the host user.

[Barry Laffoy]

If this is of use to anybody else, I was able to workaround this implementation problem in the jenkinsci/docker with the following:

FROM jenkins:2.19.4

ARG user=jenkins

ARG group=jenkins

ARG olduid=1000

ARG oldgid=1000

ARG uid

ARG gid

ENV JENKINS_HOME /var/lib/jenkins

ENV COPY_REFERENCE_FILE_LOG $JENKINS_HOME/copy_reference_file.log

USER root

RUN usermod -u $uid $user

RUN usermod -d /var/lib/jenkins $user

RUN groupmod -g $gid $group

RUN usermod -g $gid $group

RUN find / -path /proc -prune -o -group $oldgid -print | xargs chgrp -hv $group

RUN find / -path /proc -prune -o -user $olduid -print | xargs chown -hv $user

RUN cp -R /var/jenkins_home /var/lib/jenkins

RUN chown -R $user:$group /var/lib/jenkins

VOLUME /var/lib/jenkins

USER $user

And building with `--build-args uid=$(id -u) --build-args gid=$(id -g)`

This isn't pretty, and I'm not entirely sure that it's the way I want to do it. Cloning the entire jenkinsci/docker repo and setting these build args directly seems like a better option.

[Dan Tran]

I would prefer to clone the repo and build the container, this way I do not have maintain my own jenkins image

Maybe Jenkins dev can allow options to run the container using user prefer Ids?

Thanks

-Dan

[Barry Laffoy]

I have my own Dockerfile for some further configuration anyway, so this version suits me.

It would be nice to be able to set the uid at build or run time, but I can't see how one might achieve that, given the fundamentals of how Jenkins and Docker work.