parallel stages using multiple pod (with Kubernetes plugin)

[이호광]

Hello.

In the master branch, my code has feature flags (a.k.a feature toggles).

So, I want to make a pipeline like

start - sync - build - test - end

        - sync - build - test -

        - sync - build - test -

(if there are 3 feature sets)

sync is like git clone, and code is same or can be little bit different.

and build command also can be slightly different for using feature flags.

and when compiling, I use make -j (multi-processing) command.

So I think creating one pod and multiple containers in Jenkins is not good idea due to heavy CPU and memory needed.

I think Jenkins should be able to create multiple pods. Am I right?

Is it possible ?

In scripted pipeline, I create two container in a one pod successfully. But I don't know create two pods.

podTemplate(

label: 'mypod',

containers: [

containerTemplate(name: 'm1', image: 'maven', ttyEnabled: true, command: 'cat'),

containerTemplate(name: 'm2', image: 'maven', ttyEnabled: true, command: 'cat')

],

volumes: [

hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock'),

]

) {

node('mypod') {

          stage('Sync') {

parallel Syncfortype1: {

container(name: 'm1', shell: '/bin/bash') {

sync

}

},

Syncfortype2: {

container(name: 'm2', shell: '/bin/bash') {

sync

}

}

}

}

}

I think there may be the way like, but I can't find.

// pre-define

podTemplate(label: 'm1')

podTemplate(label: 'm2')

// use it for node

{

  stage('sync') {

    parallel A: {

      node('m1') {

        sync

      }

    },

    B: {

      node('m2') {

        sync

      }

    }

  }

}

In Declarative pipeline, samely, I can't find out pre-define multiple agents.

Guide document says, define agent in stage.

pipeline {
    agent none 
    stages {
        stage('Example Build') {
            agent { docker 'maven:3-alpine' } 
            steps {
                echo 'Hello, Maven'
                sh 'mvn --version'
            }
        }
        stage('Example Test') {
            agent { docker 'openjdk:8-jre' } 
            steps {
                echo 'Hello, JDK'
                sh 'java -version'
            }
        }
    }
}

But I don't know how to use that agent at next stage (one of parallel) ?

[Carlos Sanchez]

if you want to use the same template this should work

podTemplate(label: 'm1')

{

  stage('sync') {

    parallel A: {

      node('m1') {

        sync

      }

    },

    B: {

      node('m1') {

        sync

      }

    }

  }

}

If they are different, move the podTemplate to the parallel section

{

  stage('sync') {

    parallel A: {

      podTemplate(label: 'm1')

      node('m1') {

        sync

      }}

    },

    B: {

      podTemplate(label: 'm2') {

      node('m2') {

        sync

      }}

    }

  }

}

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/e33ae964-ee84-4dc5-9f85-60335c265f6a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Hokwang]

Hello. Carlos,

Anyway, What do you think about 

all of agent definition write upper side in Jenkinsfile.

Here in this case,

I need below pipeline. what is the proper location of agent syntax.

pipeline {

  stages{

    parallel {

      stages {

        stage('A - sync') {

          agent('1st pod') {}

        }

        stage('A - build') {

          agent('1st pod') {}

        }

        stage('A - test') {

          agent('1st pod') {}

        }

      }

      stages {

        stage('B - sync') {

          agent('2nd pod') {}

        }

        stage('B - build') {

          agent('2nd pod') {}

        }

        stage('B - test') {

          agent('2nd pod') {}

        }

      }

    }

  }

}

2018년 5월 24일 목요일 오후 10시 24분 2초 UTC+9, Carlos Sanchez 님의 말:

[Carlos Sanchez]

you can do that I think, you just need to repeat the agent { kubernetes ... } in each stage

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/c88427d7-2668-4975-b133-a49bd894ca64%40googlegroups.com.