Using the same label multiple times in pipeline

53 views
Skip to first unread message

Daniel Becroft

unread,
Oct 29, 2017, 10:51:02 PM10/29/17
to jenkins...@googlegroups.com
Hi,
I'm experimenting with the scripted pipeline, and have a question about the use of node('') with a label. If I have the following scenario:

node('A') { }
node('B') { }
node('A') { // Which node will be used here? }

Is there any guarantee that the second node() step for "A" will hit the same slave as the first one, or will it use whichever slave is now available? I'd like for the first node() to allocate one from the pool of slaves with the label of "A", but then the second one to somehow reuse the first one (there might be some cleanup tasks, etc that need to happen on the original node).

Cheers,
Daniel B.

Robert Hales

unread,
Oct 29, 2017, 11:03:53 PM10/29/17
to Jenkins Users
If you have multiple nodes with a label of A, you will need to declare a variable to hold the name of the specific node you ended up on in the first "node" declaration, and then use that node in the third one: 

def allocatedNode

node('A') { allocatedNode = $NODE_NAME } //NODE_NAME is an environment variable provided by Jenkins
node('B') {}
node($NODE_NAME) {}


HOWEVER, be aware that this isn't always a safe thing to do, depending on what your goal is. There is no guarantee that the workspace that was used in the first node block will still be there when the third block gets there. Even if it is, there is no guarantee that the third block will use the original workspace. It probably will most of the time, but not guaranteed. Depending on what you are doing, you may need to allocate a custom workspace (still not a guaranteed solution), move to a separate directory, stash/unstash the things you need between nodes, or some other solution. 

Daniel Butler

unread,
Oct 30, 2017, 5:05:03 AM10/30/17
to jenkins...@googlegroups.com

Couple of approaches:

You could use the External Workspace Manager plugin: https://jenkins.io/doc/pipeline/steps/external-workspace-manager/

 

Or as Robert suggested use the ws(){} block, this should give a guarantee on the workspace but you’ll need to confirm that you’re not about to fill up your node’s disks with old workspaces:

Something along the lines of:
def sharedWorkspace = “/shared-workspace/${JOB_NAME}/${BUILD_NUMBER}”

node(‘A’){

    ws(sharedWorkspace){

        //do stuff

    }

}

node(‘B’){}

node(‘A’){

    ws(sharedWorkspace){

        //do more stuff

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/f43a48c1-15bd-4feb-9a91-3e818a3cf268%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

 

Robert Hales

unread,
Oct 30, 2017, 10:51:08 AM10/30/17
to Jenkins Users
The ws{} block will not give a guarantee that you will get the same workspace. If, for example, 2 jobs run at the same time, they cannot use the workspace concurrently. The second one to request the workspace will get one with a @2 on the end. You would have to make sure multiple jobs can't run at the same time, or use resource locking to prevent concurrent use of the workspace.  If you don't care about concurrent use, then skip the ws{} and use dir{} instead. That will always use the same directory and doesn't care about multiple jobs accessing it. 

Daniel Becroft

unread,
Oct 31, 2017, 2:38:57 AM10/31/17
to jenkins...@googlegroups.com
Thanks for the reply, Robert and Daniel. The actual workspace that I get is not an issue - I just need to get back to that agent in order to do some cleanup. 

My scenario is:
node('A') => create and initialize a database, and start it on the port. Database is outside of the workspace, in a build# folder
node('B') => run the integration tests against that database
node('A') => stop and remove the database

This currently works as I only have a single node with a label of "a", but we might be adding more.

Thanks again.

Andreas Tscharner

unread,
Nov 2, 2017, 11:15:22 AM11/2/17
to jenkins...@googlegroups.com
I would try something like this:

node('A') {
...
...
node('B') {
...
...
}
...
...
}

HTH and best regards
Andreas
--
Andreas Tscharner sterne...@gmail.com

Gordon's Law:
If you think you have the solution, the question was poorly phrased.

Lee Meador

unread,
Nov 3, 2017, 10:35:08 AM11/3/17
to jenkins...@googlegroups.com
You can use 'stash' to solve this problem since you can't count on the same workspace in part 3 as in part 1 nor, if it is the same workspace can you count on it being unchanged in the interim. You can get confused by trying it and maybe you do get it this time but not next time. Further, depending on whether one build agent has the both labels, A and B, all three parts could happen in the same workspace.

node('A') {
    ... some stuff that creates the files you want to use in part 3 (perhaps load stuff from source code repo and build everything)
     stash 'stuff'
 }
node('B') { }
node('A') { 
    unstash 'stuff'
     .... use those files
 }

You need to understand the implications of what stash does. Basically it takes all files in the workspace (unless you specify specific ones) and stores them compressed on the master. the unstash retrieves them and puts them in the workspace. Perhaps you want to delete the workspace content before unstashing or maybe move to a subfolder.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/2ca95dc7-20f3-385e-bf87-c88dd8888ef5%40gmail.com.

For more options, visit https://groups.google.com/d/optout.



--
-- Lee Meador
Sent from gmail. My real email address is lee AT leemeador.com
Reply all
Reply to author
Forward
0 new messages