AWS ECS Plugin

[Brandon Wagner]

I'm trying to use the Jenkins AWS EC2 Container Service (ECS) plugin to deploy Jenkins Slave Containers for builds. I have an ECS cluster setup, and I have it configured in my Jenkins Configuration. Everything appears to be good until I try to build a job, restricting to the ecs cloud label I setup, and it just comes back with "(pending—Jenkins-Container-Cloud-456...164is offline)" .  

Does anyone know how to fix this, or have suggestions on debugging steps? 

[nicolas de loof]

First look into jenkins logs.

Also check on ECS a task definition has been created for jenkins-slaves. 

Also double check ECS nodes can ping your jenkins master URL.

2016-01-21 22:13 GMT+01:00 Brandon Wagner <bmwag...@gmail.com>:

I'm trying to use the Jenkins AWS EC2 Container Service (ECS) plugin to deploy Jenkins Slave Containers for builds. I have an ECS cluster setup, and I have it configured in my Jenkins Configuration. Everything appears to be good until I try to build a job, restricting to the ecs cloud label I setup, and it just comes back with "(pending—Jenkins-Container-Cloud-456...164is offline)" .  

Does anyone know how to fix this, or have suggestions on debugging steps? 

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/f9bf24d9-a9e3-46f6-8fba-041b3344bdc7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Brandon Wagner]

So I ended up figuring my issue out. I think there were a couple of issues. My JNLP port was set to 50,000 instead of 5,000 which the Jenkins Docker Container I was using was mapping to the host port 5000. And my Load Balancer in front of Jenkins was not forwarding port 5000 to the host (only 443). Anyways, all of that is fixed and I can now run builds on slaves in ECS.

My next problem: I want to use docker-in-docker to run docker builds on my docker jenkins slaves. I'm trying to use https://github.com/tehranian/dind-jenkins-slave which looks good, but I don't see an option on the Jenkins ECS plugin to run the slave as privileged which is necessary for docker-in-docker.

-Brandon Wagner

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/kSBWy3gBQbg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJzks3yceTvUxrFd5cGo8tYwSFh%3D00V%3DzL%2BxJVB2kCmi%3DhQ%40mail.gmail.com.

[nicolas de loof]

privileged flag has been added to development build (https://jenkins.ci.cloudbees.com/job/plugins/job/amazon-ecs-plugin/)

anyway, DinD is probably not a good idea (there's really few docker usages to actually require it). What's your actual need ?

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAKgdMg-%2Bs99MCNpwJz99QBs0NTke7EV2Tt%3DWPEj5NZBzjkJaHg%40mail.gmail.com.

[Brandon Wagner]

I would like to run Docker containers for all of my software projects and dynamically allocate build slaves via ECS (so that I can have a core cluster that can do builds quickly; in contrast to spinning up an EC2 instance).

 For example, I have a Java application which is completely contained in a Docker container (I can build it, run tests, and run the actual application with Tomcat all within the container). I want to use this for a variety of different purposes (dev, testing, qa, and production). In order to make it reusable like that, I don't want to include Jenkins Slave packages and expose ports. I'd rather deploy out a Jenkins-Slave container to my ECS cluster, and then have that slave handle running my application's docker container which also performs tests. 

I also support other team applications which are already running docker container builds in Jenkins (locally on the Jenkins Master). I want it to be transparent to them that I'm "outsourcing" build slaves to ECS. It shouldn't matter to them that I'm changing the way Jenkins is performing builds.

Let me know if you (or anyone else) have suggestions based on my goals.

-Brandon Wagner

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJzmdKoB9PZfYNcA6tGMyC2pT5Mg13LP9UMMW%2Bt3FAC1qTA%40mail.gmail.com.

[nicolas de loof]

You could use docker-custom-build-environment-plugin for this exact scenario, with bind mounted /var/run/docker.sock so you can run containers side by side (vs "in-docker"). Would need some tests on my side to ensure this scenario is supported, I have this on my TOD for a while but never took time to setup a test environment for it...

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAKgdMg_jXunkKPoQHertFKDtu9Sikmm4X8vLvD2oOAm2n-OF7A%40mail.gmail.com.

[Brandon Wagner]

If I'm understanding correctly, I would mount the docker.sock to the slave container? I created an image extended from the jenkinsci/jnlp-slave that does a wget for the docker CLI. 

However, I don't see a place to mount the docker.sock through the ECS configuration options. 

-Brandon Wagner

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJz%3DQhj1%2Bs2bKikkhbrn-8ws6ducuRr4c2o80i3X8XA_neQ%40mail.gmail.com.

[nicolas de loof]

Latest development build for ECS plugin do let you define bind mounts, see https://jenkins.ci.cloudbees.com/job/plugins/job/amazon-ecs-plugin/

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAKgdMg9RGOr3yEuAN8CutBT9x3RcAGUya4fomKhE%2BsyHK77_cg%40mail.gmail.com.

[Brandon Wagner]

Ok, I've made progress. I'm able to run a docker container along side the slave container on ECS. However, I just used the amazon-ecs-plugin. When I tried using the CloudBees Docker Custom build Environment Plugin, I kept getting "Cannot start container ........ System error: no such file or directory" 

If I just do a shell script "docker pull java && docker run java java -version" it works.

Is there any advantage in using the CloudBees Docker Custom build Environment Plugin rather than shell scripts? I'm hesitant in using shell scripts since people may see shell commands and start plugging in other commands rather than just using the Docker CLI commands.

-Brandon Wagner

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJznQ%2BZFRDK9Y8X1HjtozxmB6uYU1vC1CAeaG6dkefiUaeA%40mail.gmail.com.

[Mulloy Morrow]

Has anyone successfully been able to mount the Docker UNIX socket on the slave containers? I've attempted to mount this socket using the Jenkins ECS plugin (v1.2) by configuring "container mount points".  (see jenkins config screenshot below)

[Mulloy Morrow]

Was able to get this working. Mounted the docker socket using the mount point configs in the jenkins plugin. However, I was getting a permission denied when trying to nc or curl the socket for info. I had to run the slaves as user root rather than user jenkins. Has either of you come across this issue? 

[nicolas de loof]

docker socket is only accessible to users in docker group or to root. running from a container doesn't bypass such permission check.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/b99c9efe-c1a8-49fb-a60f-029b551f0051%40googlegroups.com.

[Mulloy Morrow]

My Jenkins user is part of the docker group. That's why I was surprised by this error.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJzkwsGswL-x83YBkBepRgVfHLg0yMwVY3q%3DJ4W8htv1FTg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Best,
Mulloy Morrow
858.598.3059

[Rajasekaran R]

Yes you are correct I am also running my slave jenkins user under docker group but I cant run docker commands using Jenkins user.

My Dockerfile end of the lines look likes..

RUN usermod -aG docker jenkins

RUN gpasswd -a jenkins docker

#RUN systemctl restart docker

USER jenkins

ENTRYPOINT ["/usr/local/bin/jenkins-slave"]

Kidly suggest how to fix this issue.