Amazon ECR Plugin not providing credentials for Docker Build & Push Plugin

[A. Best]

Versions used in testing:

Jenkins: 1.642.3

Amazon ECR Plugin: 1.0

Cloudbees Docker Build & Publish: 1.2.1

ECS Container Instance

Docker Version: 1.9.1

API Version: 1.21

I have a couple of quick question about the amazon-ecr-plugin.

In the About section of the plugin, the Cloudbees Docker Build and Publish is referenced as an example of how the ECR plugin can be used.

For my specific use case, I have the Jenkins master connecting to a Jenkins JNLP slave running in an ECS cluster. I’m using a container based on the jenkinsci/jnlp-slave to perform the build.  With the right permissions and mounted volumes, I am able to use the docker host(which is the ECS container instance) to build docker images.

The Docker Build and Publish plugin does use the dockerfile at the root of the project and build it as expected. However, I’m running into an issue when the plugin attempts to push the image to ECR.

The push refers to a repository [<my-user-id>.dkr.ecr.us-east-1.amazonaws.com/test-repository] (len: 1)
7a8e1872c5e2: Preparing
Post https://<my-user-id>.dkr.ecr.us-east-1.amazonaws.com/v2/test-repository/blobs/uploads/: no basic auth credentials
Build step 'Docker Build and Publish' marked build as failure
Finished: FAILURE

It seems that the build is attempting to push to the registry with no credentials. I was assuming that the ECR plugin would provide docker with the correct AWS credentials to login to the registry so that the newly built image could be pushed.  

Do I need any additional packages installed on the Jenkins slave to get this to work?

Am I missing something?

Are my expectations for the plugin wrong?

Thanks, 

Adam

[nicolas de loof]

Did you well selected amazon credentials in build and publish build step configuration ?

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/eaf6f71a-9758-4838-bb05-fd4fa43ee021%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[A. Best]

Nicolas,

Thanks for the response.

Yes, I did select amazon credentials, as those are the only credentials I have setup on this instance.  In this particular case I was doing a variation of the a Jenkins-Amazon build pipeline.  I was using a simple prebuilt source repo to test the process. 

Here's a screenshot of the Docker Build and Publish build step.

Thanks,

[A. Best]

Just so that I can bring this thread to a proper conclusion, I worked out the issue.

TL;DR:

The Jenkins slave container derived from the jnlp-slave image (with the docker client installed) was using the root user instead of the jenkins user. This causes the ECR credentials to be stored in the wrong place. So, when the docker-build-publish plugin pushes to the registry, docker push xxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/<repo>:latest, there is no docker config file with the proper credentials. This causes the no basic auth credentials error.

Recap:

I am using a Jenkins master to trigger builds in an ECS cloud. The ECS slave task template uses an image derived from the jnlp-slave image with the docker client added. Additionally, /var/run/docker.sock is mounted from the source to the container to give the slave container access to the docker host’s docker server.

The goal of this configuration is to provide a simple way for jnlp-worker containers to build & push docker images to a registry. The physical configuration is as follows:

So, what was the source of the problem?

Well, initially I was having problems with the jenkins user accessing /var/run/docker.sock. The socket belongs to the docker group on the host and is assigned an random? GID. The docker group, however, was 1) not a group in the container and 2) the jenkins user was not a member of the group. So, I copped out and had the container run as root–laziness invites issues.

The jenkins worker will hum along properly until it’s time to docker push to the registry and it cannot authenticate. The Docker Build and Publish plugin does correctly utilize the ECR plugin to retrieve a token to access the ECR registry. But, because this is all happening as the root user, the Docker Commons plugin stores the resultant login info at /root/.dockercfg. When docker push is invoked by the plugin, it can’t find credentials…booo.

A resolution

The resolution is simple, ensure that the jnlp worker if running as the jenkins user and ensure that the docker group from the host is replicated in the worker. A bit of searching led me to this post on the docker forums and this script by Sven Dowideit. With a few modifications to use this script as the ENTRYPOINT of the jnlp worker image, everything now works.

--

a