build image based on PullRequest and test it

[Piotr Bracha]

I would like to do building image based on PullRequest, run tests on it and destroy container. I attach Jenkinsfile where is whole pipeline which works on Jenkins master, which is installed on physical machine. There is also installed Docker. I have used image from  https://hub.docker.com/r/jenkinsci/jnlp-slave/  with some extras added by me - ruby, rails, postgres to create jenkins-slave. As next step I attached it to Jenkins master. Of course it works perfect. Each step from pipeline execute properly. But it's not what I want to reach. I would like to build docker image based on PullRequest, then run step with tests on it. After finish this part with success I would like to move next steps in Jenkins master and destroy/delete container and image build from PR. 

--

Pozdrawiam / Kind regards, 
Piotr Bracha
Administrator Systemów

Vasco Electronics sp. z o.o. S.K.A.
al. Pokoju 1, CTA/350 
31-548 Kraków
NIP 677 236 91 51

 

E-mail:  p.br...@vasco-electronics.com

  

Vasco Electronics Spółka z ograniczoną odpowiedzialnością Spółka Komandytowo-Akcyjna, Al. Pokoju 1, CTA/350, 31-548 Kraków, Polska, NIP: 6772369151, REGON: 122581850, zarejestrowana w Sądzie Rejonowym dla Krakowa Śródmieścia XI Wydział KRS pod nr KRS: 0000421705, Kapitał zakładowy 50 000 zł (słownie: pięćdziesiąt tysięcy złotych) w całości wpłacony. Klauzula Bezpieczeństwa: treść tej wiadomości wraz z załącznikami stanowią informacje chronione przed ujawnieniem. Jeśli wiadomość ta nie jest przeznaczona dla Ciebie uprzedzamy, że ujawnianie, kopiowanie, rozpowszechnianie lub korzystanie z niej lub z załączników jest zabronione. Jeśli otrzymałeś tę wiadomość przez pomyłkę, uprzejmie prosimy o niezwłoczne zawiadomienie nadawcy i odesłanie jej z powrotem wraz z załącznikami a także usunięcie ze swoich systemów.

[Quang Truong]

We implemented a similar Use Case: 

• Create an isolate environment for each team
• Whenever they have commit/PR triggers a build and create a container to build that project.
• If the Dockerfile in that project has changed, refresh the docker image then create container based on the new image.

The solution is (refer and tailor on your case):

• Use multibranch project so Jenkins will trigger with git commit/PR
• Put Dockerfile into the project folder (let the team maintains their environment), use the Jenkins ChangeSet to verify if the Dockerfile has changed so you will refresh (remove the existing image then build a new one with the updated Dockerfile) the docker image on your Jenkins executor
• Build the docker image/container as a Jenkins slave (use swarm-client, accept the security issue with this plugin)
• Run the build/test the project on your new slave container (consider about concurrent pipeline with the new container)
• If the build is green, back to the slave/host then push/update the image into the docker registry

Hope this can help

[Piotr Bracha]

Sorry for late answer from me. Thank you for your solution. Really appreciate. It's really hard subject for me (not long time ago I have only heard about something like Jenkins and Docker). Currently I have:

1. Multibranch Pipeline.

2. Dockerfile is inside project folder. What is "Jenkins ChangeSet" - do you mean "currentBuild.changeSets"? Why and how should I change dockerfile during pipeline execution? :)

3. Currently I made own image of jenkins slave based on https://hub.docker.com/r/jenkinsci/jnlp-slave/ + installed postgres, rail, ruby. Whole multibranch pipeline works perfect (build with success) on this slave.

4. Do you run build/test using some plugin or by adding something to Jenkinsfile or maybe something another?

5. Honestly, also no idea how... :/

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/5ca8978f-78ed-468a-8050-f7193df7f5c1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Quang Truong]

Hi Polak,

Please find my update inline below

On Sunday, May 26, 2019 at 10:51:29 PM UTC-7, Polak wrote:

Sorry for late answer from me. Thank you for your solution. Really appreciate. It's really hard subject for me (not long time ago I have only heard about something like Jenkins and Docker). Currently I have:

1. Multibranch Pipeline.

2. Dockerfile is inside project folder. What is "Jenkins ChangeSet" - do you mean "currentBuild.changeSets"? Why and how should I change dockerfile during pipeline execution? :)

[Quang] Because I don't want to rebuild the docker image every build (it takes me around 15 mins to build the image) so I build it whenever the Dockerfile has changed. So when there is a Git commit triggers our pipeline, I will check the changeSets (you're correct, I use the currentBuild.changeSets) to list all the committed files, if the Dockerfile has changed means we have to rebuild the existing image. If you don't have this need, ignore it, rebuild the docker image on every build to make sure you have the latest image.

3. Currently I made own image of jenkins slave based on https://hub.docker.com/r/jenkinsci/jnlp-slave/ + installed postgres, rail, ruby. Whole multibranch pipeline works perfect (build with success) on this slave.

[Quang] I'd suggest using the swarm client: https://plugins.jenkins.io/swarm because you don't have to create a node entry from Jenkins master, just start the slave client with the java command. So in your Jenkinsfile, after create the docker image, start a container with the java command that point to your master, something like: 

java -jar /home/jenkins/swarm-client-3.9.jar -master ${env.JENKINS_URL} -username ${jenkins_user} -password ${jenkins_pwd} -labels ${docker_label} -name ${docker_build} -executors 1 -fsroot /home/jenkins

4. Do you run build/test using some plugin or by adding something to Jenkinsfile or maybe something another?

[Quang] As long as your docker container is a Jenkins slave, you can do whatever you like from Jenkins. 

5. Honestly, also no idea how... :/

Here is a brief sample:

Dockerfile

FROM ubuntu:16.04

RUN apt-get -y update && apt-get install -y \

  sudo \

  openssh-server \

  openssl \

  openjdk-8-jre \

  git

RUN adduser --disabled-password --gecos "" --uid 1000 jenkins \

  && adduser jenkins sudo \

  && mkdir /home/jenkins/slave

    && echo "jenkins ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers

RUN curl -L https://repo.jenkins-ci.org/releases/org/jenkins-ci/plugins/swarm-client/3.9/swarm-client-3.9.jar \

  -o /home/jenkins/slave/swarm-client-3.9.jar

USER jenkins

====================
Jenkinsfile

node('docker_host') {
  stage('Prepare Env') {
    def imgID = sh(script: "docker images -q ${imgName}", returnStdout: true)
    if ((imgID == null) || (imgID == '')) {
      sh(script: "docker build -t ${imgName} -q -f ${dockerfile} ./")
    }
    def cmd = "java -jar /home/jenkins/swarm-client-3.9.jar -master ${env.JENKINS_URL} -username ${jenkins_user} -password ${jenkins_pwd} -labels ${docker_label} -name ${docker_build} -executors 1 -fsroot /home/jenkins"
    sh """#!/bin/sh -e
          docker run -td --name ${containerName} --volume ${map_volume} ${imgName} bash -c ${cmd}"""
  }
}
node(docker_build) {
  stage('Build') {
    // Do your build and test here
  }
}
node('docker_host') {
  // Cleanup your docker container and image
}

Please make sure you have try catch in case your pipeline has failed then you will cleanup the docker garbage. I haven't had time to explore the docker plugin to have a better approach yet, you can take a look at this: https://go.cloudbees.com/docs/plugins/docker-workflow/

Good luck!

 

pt., 24 maj 2019 o 03:17 Quang Truong <truongdi...@gmail.com> napisał(a):

We implemented a similar Use Case: 

• Create an isolate environment for each team
• Whenever they have commit/PR triggers a build and create a container to build that project.
• If the Dockerfile in that project has changed, refresh the docker image then create container based on the new image.

The solution is (refer and tailor on your case):

• Use multibranch project so Jenkins will trigger with git commit/PR
• Put Dockerfile into the project folder (let the team maintains their environment), use the Jenkins ChangeSet to verify if the Dockerfile has changed so you will refresh (remove the existing image then build a new one with the updated Dockerfile) the docker image on your Jenkins executor
• Build the docker image/container as a Jenkins slave (use swarm-client, accept the security issue with this plugin)
• Run the build/test the project on your new slave container (consider about concurrent pipeline with the new container)
• If the build is green, back to the slave/host then push/update the image into the docker registry

Hope this can help

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkins...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/5ca8978f-78ed-468a-8050-f7193df7f5c1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Pozdrawiam / Kind regards, 
Piotr Bracha
Administrator Systemów

Vasco Electronics sp. z o.o. S.K.A.
al. Pokoju 1, CTA/350 
31-548 Kraków
NIP 677 236 91 51

 

E-mail:  p.b...@vasco-electronics.com

[Piotr Bracha]

Thank you for answer. I am going to check it. I see you use scripting pipeline, not declarative. ;) I found out that I don't need to do slave but only one step where I will build rails app using docker compose, then run it (with docker compose also) with command which run tests.  :) But honestly it still hardcore, because I need to authorize database user somehow. :D

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/752ad51e-a9e0-45d7-a892-2c2407920c45%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Pozdrawiam / Kind regards, 
Piotr Bracha
Administrator Systemów

Vasco Electronics sp. z o.o. S.K.A.
al. Pokoju 1, CTA/350 
31-548 Kraków
NIP 677 236 91 51

 

E-mail:  p.br...@vasco-electronics.com