Docker images as Jenkins agents failing, "/agent.log Permission Denied"

[Zach LaCelle]

Hello,

I'm trying to set up a Jenkins master to use another server as a Jenkins agent host, running Docker containers on the host for different environments.

My master is running jenkinsci/blueocean. The agent host server is running Docker with tcp://0.0.0.0:2376 open.

TL;DR Question: Why isn't my ubuntu:16.04-based Docker image launching correctly, and how should I set up this environment?

I can run my test pipeline on the host with the default jenkinsci/slave image, which is based on the openjdk-8:jre image.

I've created my own image which is based on ubuntu:16.04, in order to create a Ubuntu build environment. I've copied in most of the jenkinsci/slave sections, to do things like get slave.jar in the right place, set up users/groups, etc. My Dockerfile is located at the bottom of this post.

The issue is that, when the Master tries to provision an agent to build, I get the following error:

INFO: Trying to run container for node ubuntu-1604-0001undhvwt76 from image: test:0.1

Oct 19, 2018 6:10:16 PM com.nirima.jenkins.plugins.docker.DockerTemplate doProvisionNode

INFO: Started container ID dab1ac24bbad9ca00855f97306ebbcf938e7c6ce028a9c35a137989cf784d3f6 for node ubuntu-1604-0001undhvwt76 from image: test:0.1

Exception in thread "main"

java.io.FileNotFoundException: /agent.log (Permission denied)

        at java.io.FileOutputStream.open0(Native Method)

        at java.io.FileOutputStream.open(FileOutputStream.java:270)

        at java.io.FileOutputStream.<init>(FileOutputStream.java:213)

        at java.io.FileOutputStream.<init>(FileOutputStream.java:162)

        at org.jenkinsci.remoting.engine.WorkDirManager.legacyCreateTeeStream(WorkDirManager.java:319)

        at org.jenkinsci.remoting.engine.WorkDirManager.setupLogging(WorkDirManager.java:288)

        at hudson.remoting.Launcher.run(Launcher.java:304)

        at hudson.remoting.Launcher.main(Launcher.java:283)

When jenkinsci/slave runs, it gets the following output:

INFO: Trying to run container for node test-0001uqx3pd659 from image: jenkinsci/slave

Oct 19, 2018 6:14:54 PM com.nirima.jenkins.plugins.docker.DockerTemplate doProvisionNode

INFO: Started container ID 4b9d599a4a516ab0f4583bce79e6c6c681b370882ea5307fd5b8a867b773cec3 for node test-0001uqx3pd659 from image: jenkinsci/slave

Oct 19, 2018 6:14:56 PM hudson.slaves.NodeProvisioner$2 run

INFO: Image of jenkinsci/slave provisioning successfully completed. We have now 6 computer(s)

channel started

Clearly my Dockerfile is missing something, but I have no idea what. I can run it with the "docker run -i --rm --name agent --init test:0.1 java -jar /usr/share/jenkins/slave.jar" and get the correct REMOTE line.

Dockerfile:

# Start with Docker Hub Ubuntu image

FROM ubuntu:16.04

## Install ubuntu-server

USER root

# These configurations are copied so that dpkg-reconfigure doesn't hang

COPY --chown=root:root etc_default_keyboard /etc/default/keyboard

COPY --chown=root:root etc_default_console-setup /etc/default/console-setup

#

RUN apt-get update

RUN DEBIAN_FRONTEND=noninteractive apt-get install -y ubuntu-server

## Install wget

RUN DEBIAN_FRONTEND=noninteractive apt-get install -y wget

## Install Java 8 and sshd

USER root

RUN DEBIAN_FRONTEND=noninteractive apt-get install -y openjdk-8-jre-headless ssh

## Setup Jenkins environment

ARG user=jenkins

ARG group=jenkins

ARG uid=10000

ARG gid=10000

ENV HOME /home/${user}

RUN groupadd -g ${gid} ${group}

RUN useradd -c "Jenkins user" -d $HOME -u ${uid} -g ${gid} -m ${user}

ARG VERSION=3.26

ARG AGENT_WORKDIR=/home/${user}/agent

RUN curl --create-dirs -sSLo /usr/share/jenkins/slave.jar https://repo.jenkins-ci.org/public/org/jenkins-ci/main/remoting/${VERSION}/remoting-${VERSION}.jar \

  && chmod 755 /usr/share/jenkins \

  && chmod 644 /usr/share/jenkins/slave.jar

USER ${user}

ENV AGENT_WORKDIR=${AGENT_WORKDIR}

RUN mkdir /home/${user}/.jenkins && mkdir -p ${AGENT_WORKDIR}

VOLUME /home/${user}/.jenkins

VOLUME ${AGENT_WORKDIR}

WORKDIR /home/${user}

## Start the Jenkins slave process

#CMD java -jar /usr/share/jenkins/slave.jar -workDir ${AGENT_WORKDIR}

[Zach LaCelle]

OK, I'm one step closer, but still hope to get some direction.

On both types of images (my test image and the jenkinsci/slave image), /usr/share/jenkins/slave.jar is present with identical permissions.

On the correct image (jenkinsci/slave), the files "remoting-3.25.jar" and then "agent.log" get created in /home/jenkins.

However, on my test image, "remoting-3.25.jar" gets created in the root directory, and then agent.log fails to be created (I assume because of permissions errors, per the previous post).

I'm unable to figure out which environment variable I've failed to set. The environments look identical. HOME and AGENT_WORKDIR are both set (although it looks like HOME is where these files are getting created). The commands for running the agent are not changing: I'm simply running it from the Jenkins master and switching which image the Docker Cloud is using.

1. What creates remoting-3.25.jar? I assume it's copied from /usr/share/jenkins/slave.jar, but I'm not sure.

2. What variable, setting, etc is controlling where this file is saved?

[Zach LaCelle]

Solved. In case anybody makes the same mistake I did:

The docker-plugin does the remoting-3.25.jar copy, with the injectRemotingJar() function in DockerComputerAttachConnector.java. This uses the remoteFs variable which gets set way back in DockerTemplate.java's provisionNode() function. The value is gotten directly from the Docker Image using getContainerConfig().getWorkingDir().

I had made a silly mistake where, for the Jenkins Docker Cloud Template I was testing with, setting "Remote Root Filesystem" to "/". Must have been a fat finger.

But for those following this in the future, make sure you set WORKDIR in your custom Docker image to /home/jenkins (or wherever your Jenkins home is).