Which agents are used when a pipeline calls multiple downstream freestyle projects

[Tony P]

Hi,

We have multiple pipelines for which each will run 3+ downstream freestyle projects.

This process is defined by an automation tool that creates and initiates Jenkins builds, so using other methodologies is out :-). The problem I have is that I need all the freestyle projects to run on the same agent, because the first couple of projects clone repos and other projects then do the build or deploy - so obviously it all needs to happen in exactly the same place.

I also want to initiate this process using agent/slave labels rather than agent/slave names as we will have a group of agents to handle the multiple sometimes duplicate pipeline builds.

I am beginning to think that even though I am using labels Jenkins will always run all the downstream projects on the same agent. Trouble is I am having trouble finding any conclusive documentation that says I am right or I am wrong - I have spent an awful lot of time in the company of Mr Google trying to figure this out. If each of the downstream projects could potentially run on any available agent using the same label then I have a big problem and my next task is to figure out how to get them to all use the same agent.

Can someone tell me whether I am correct that Jenkins will run all downstream projects on the same agent and/or where do I look for this level of documentation ?

Thanks in advance

[Stephen Connolly]

I think you would need to write a plugin that enforces your "same node" constraint.

It would not be general utility as eg enough people are doing builds on ephemeral agents and you'd end up with stuck jobs waiting for a free executor on the same "ephemeral" node that will never return.

But I can see it being of utility for some people. Probably a JobProperty that does the enforcing and some other coordination token to identify which jobs you need to be on the same node as (plus a timeout if you are stuck for too long waiting on that node)

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/e806f3a6-02c8-4fbd-b574-d29fb8dee547%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Sent from my phone

[Tony P]

Does this mean I am correct that the default behavior is for downstream projects to run on any agents that have the same label ? This was perhaps the big thing I was trying to find out.

Also our builds are done via "build job: " but I am struggling to find out how to force it to use a particular agent.

My plan has been kick off the first downstream project, find out which agent it used and then force the remaining downstream projects to use that agent. Try as I may and I tried so many ideas, I can't find out how to determine what agent that downstream agent used. It seems next to impossible ?

Thanks again

[Stephen Connolly]

Downstream jobs will run on any agent matching the label they are configured for, with preference given to running on the agent that they most recently ran on *if idle*.

Upstream jobs have no affect - unless they are still running at the time - in which case they may force a *different* agent if they are occupying all available slots.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/52b89bf8-4615-40cb-affd-51ab29792f00%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Tony P]

Found the solution wrap the whole thing in a node() block and everything within the node block will run on the same node

Cheers

[Stephen Connolly]

I thought you were launching freestyle jobs from a pipeline (where this will not work)?

If I'd known toyou were doing all in the same pipeline I could have told you this sooner...

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/a6aedb7f-a48b-4359-9319-13aa7a4b893d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.