Build Docker Image from a Dockerized Jenkins

[Jeeva Chelladhurai]

Hello All,

I would like to build Docker images from a Dockerized Jenkins. There are multiple solution out there but what is the recommended solution...

I have been using ssh node as a work around, but I was told to use https://wiki.jenkins.io/display/JENKINS/Docker+Slaves+Plugin . In order for this plugin to work, I have to install docker inside the Jenkins (my docker host is RHEL) and also mount the /var/run/docker.sock or enable TLS. Isn't installing docker blow up the size of the Jenkins image? 

Any way to install just the docker client in side the Jenkins image? 

Thanks,

Jeeva

--

Jeeva K S Chelladhurai

[nicolas de loof]

There's no "recommended" solution, each plugin comes with it's own vision, benefits and drawbacks.

docker-plugin can be used to run dockerized agents, it can run from a dockerized jenkins master with /var/run/docker.sock without docker CLI installed, as it uses a java client library for docker API.

docker-slaves-plugin do rely on running docker CLI executed from master, I planned to use a java API at some point but still need to work on this. In the meantime I don't think there's a simple way to just install docker CLI. Installing full docker will make your image a bit bigger with additional files on filesystem but there won't be any extra process running, so not a major issue (considering jenkins docker image is already a big one)

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZV3Sc-fgYS_34PgEwRzsF20%3Dp6mbaagbR2Ehtz01op2%3Dw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Jacob Larsen]

I wouldn't use Docker Slaves for this. Just install the docker client in your Jenkins image and bind mount /var/run/docker.sock. Then a simple sh step with docker build + docker push should do the trick.

If you don't want to use your jenkins master for this, you could make a slave container with these tools and bind mount.

The Docker Slaves plugin is more about running your builds in containers than it is about building images.

/Jacob

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

[Jeeva Chelladhurai]

Hi Nicolas,

Thanks for your input. Using Java API is an excellent idea. There is hackathon planned for Jenkins this week https://www.meetup.com/jenkinsBLR/events/240212894/

Perhaps, this requirement could be prioritized at the hackathon if you could give a helping hand..

Thanks,

Jeeva

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJzk7GJDmKUKQNvt4PzLpXHtTp%2B7BnikYfeysX7_kdpxcUA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Ramanathan Muthaiah]

Hi Jeeva,

 

I would like to build Docker images from a Dockerized Jenkins. There are multiple solution out there but what is the recommended solution...

I have been using ssh node as a work around, but I was told to use https://wiki.jenkins.io/display/JENKINS/Docker+Slaves+Plugin . In order for this plugin to work, I have to install docker inside the Jenkins (my docker host is RHEL) and also mount the /var/run/docker.sock or enable TLS. Isn't installing docker blow up the size of the Jenkins image? 

Any way to install just the docker client in side the Jenkins image? 

I have not tried this, so, cannot vouch for this. Please be cautious about the approach presented in this blog.

http://niels.nu/blog/2017/continuous-blog-delivery-p2.html : you might be interested in Docker-in-Docker.

/Ram 

[Jeeva Chelladhurai]

I have not tried this, so, cannot vouch for this. Please be cautious about the approach presented in this blog.

http://niels.nu/blog/2017/continuous-blog-delivery-p2.html : you might be interested in Docker-in-Docker.

Thanks for the suggestion.. Docker-in-Docker is been discouraged here is good read on it https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/

- Jeeva

[Jeeva Chelladhurai]

Any  thoughts on https://wiki.jenkins.io/display/JENKINS/CloudBees+Docker+Custom+Build+Environment+Plugin

[nicolas de loof]

(another) plugin of mines.

Same design as docker-pipeline's "docker.inside"

Can you please describe your use-case so we give better advise ?

if you just want to "build Docker images" as your initial message told, then just run "docker build" within your job, what else ?

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZVSDwjOAUe_uB_u89GRufSrbk2MyeUzSHz9w88K6af0QA%40mail.gmail.com.

[Jeeva Chelladhurai]

I would like to run my builds inside docker, so that I can manage different run times like node, python, go thru docker images.

- Jeeva

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJznKg%3DXKpeaT62PVkmEvLh4kruCyc1HV%2BQGFNtnjLhX%3DLQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[nicolas de loof]

ok, so for this purpose you can rely on docker-plugin to create agents on-demand from configured docker images, or docker-slaves-plugin if you want this configured directly within your jobs, and docker-pipeline "inside" if you prefer Jenkinsfiles to setup CI/CD in jenkins.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZVoig-s%3DvfVi5SEcrLVjBwZ5-0danHP-X54_CogRT9n0Q%40mail.gmail.com.

[Jacob Larsen]

Here is a list of the options I know, but one important aspect to this is if you use pipeline or not in your builds as that affects your choices:

Docker Custom Build Environment Plugin. This will use an existing executor to launch a container with the build. The images used need not be aware of Jenkins or dependencies. When I looked at it, it seemed abandoned, but it looks like someone is trying to pick it up again. It will probably take some time before it is up to date with new docker and Jenkins features. E.g. last I checked, pipeline was not supported. Also, it only works from physical Jenkins nodes.

Docker Slaves Plugin. This will provision slave containers on the fly from a single Docker host. Images need not be aware of Jenkins or dependencies. Limit on containers launched did not work for pipeline jobs last I checked. Also, this only works on a single Docker host.

Docker Pipeline Plugin https://plugins.jenkins.io/docker-workflow. I just learned about this one from the previous message, but looking at the adoption rate, this must be the mainstream option. Odds are this would be your best bet to start, provided you are using pipeline.

Using a build wrapper like SCUBA: https://github.com/JonathonReinhart/scuba in a shell step. This has the benefit of enabling manual/development builds to use the same steps as the automated Jenkins builds. For that particular wrapper, it does not support running from inside a container and creating sibling containers. I am considering a PR for that, but I think there are other wrappers that might support something like this.

There are plugins that dynamically provision slaves from images and run builds on those. The common drawback for these would be that the images themselves must support running a jenkins Slave, e.g. have a Java runtime of a certain version installed.

/Jacob

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZVoig-s%3DvfVi5SEcrLVjBwZ5-0danHP-X54_CogRT9n0Q%40mail.gmail.com.

[nicolas de loof]

2017-12-04 21:38 GMT+01:00 Jacob Larsen <ja...@larsen.net>:

Here is a list of the options I know, but one important aspect to this is if you use pipeline or not in your builds as that affects your choices:

Docker Custom Build Environment Plugin. This will use an existing executor to launch a container with the build. The images used need not be aware of Jenkins or dependencies. When I looked at it, it seemed abandoned, but it looks like someone is trying to pick it up again. It will probably take some time before it is up to date with new docker and Jenkins features. E.g. last I checked, pipeline was not supported. Also, it only works from physical Jenkins nodes.

pipeline is not supported by this plugin as the exact equivalent (even code is a re-implementation) is docker-pipeline's "inside".

 

Docker Slaves Plugin. This will provision slave containers on the fly from a single Docker host. Images need not be aware of Jenkins or dependencies. Limit on containers launched did not work for pipeline jobs last I checked. Also, this only works on a single Docker host.

docker-slaves-plugin do NOT require image to be aware of Jenkins or dependencies, you can use arbitrary docker image, even without a JDK. This was one initial goal of this plugin.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZVoig-s%3DvfVi5SEcrLVjBwZ5-0danHP-X54_CogRT9n0Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/189e04a2-be8d-1052-bb05-cebd482ffc72%40larsen.net.

[Jacob Larsen]

On 2017-12-05 09:03, nicolas de loof wrote:

2017-12-04 21:38 GMT+01:00 Jacob Larsen <ja...@larsen.net>:

Here is a list of the options I know, but one important aspect to this is if you use pipeline or not in your builds as that affects your choices:

Docker Custom Build Environment Plugin. This will use an existing executor to launch a container with the build. The images used need not be aware of Jenkins or dependencies. When I looked at it, it seemed abandoned, but it looks like someone is trying to pick it up again. It will probably take some time before it is up to date with new docker and Jenkins features. E.g. last I checked, pipeline was not supported. Also, it only works from physical Jenkins nodes.

pipeline is not supported by this plugin as the exact equivalent (even code is a re-implementation) is docker-pipeline's "inside".

Yes I just discovered this plugin myself. Not sure how I missed it.

 

Docker Slaves Plugin. This will provision slave containers on the fly from a single Docker host. Images need not be aware of Jenkins or dependencies. Limit on containers launched did not work for pipeline jobs last I checked. Also, this only works on a single Docker host.

docker-slaves-plugin do NOT require image to be aware of Jenkins or dependencies, you can use arbitrary docker image, even without a JDK. This was one initial goal of this plugin.

I hope that was what I wrote too. An important feature IMO.

For myself I am moving towards a setup where the docker run is handled by the build logic in the workspace and not by the automation tool/jenkins plugins. I hope this can let development builds get closer to the automated ones.
This involves a one-shot docker run wrapper of some sorts instead of Jenkins plugins. Still investigating though.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJzn7U2NvWGsYrWggNK_-Kc_zM9hE6AYh56BT-c3SHb5Wzw%40mail.gmail.com.