How to disable branch scan while running job created by multi branch pipeline

[TechnoLost]

Hi all,

         I've been using Jenkins for about 2 years but I've never used multi branch pipeline. I'm starting using them now.

There's a behavior I don't really understand, I tried to look for any explanation or documentation about it but I couldn't find anything. I do really hope someone could explain me what's happening and why!

My setup is made of a master Jenkins on windows and an agent node on macOS.

I've just configured a new Multibranch pipeline job to scan periodically a given repo, actually just for testing purpose I used an "aggressive" 1 min polling interval.

Then I added a Jenkinsfile to a dedicated branch of this given repo just to test the pipeline.

The first time the multi job pipeline was triggered it started scanning for branches, it found the only branch containing a Jenkinsfile and it created a dedicated job for that branch.

As well as every one minute the multi branch pipeline job gets triggered and scan for branches as well and if it finds that the test branch has some changes (pushed files) it does start the job it created for this branch.

Everything as expected for now.

What I really DON'T UNDERSTAND and I would like not to happen is that every time the job created by the multi branch pipeline starts, it starts on the master node (windows), IT DOES START SCANNING BRANCHES even if this is not the multibranch pipeline job, after branch scanning it performs a light clone to get the Jenkinsfile of its branch, it does analyze and then it does run the branch on the macOS agent given the node label found into the Jenkinsfile.

What I don't understand and I would like not to happen is that it does perform a branches scanning even into the created job, the one created on the discovered branch having the Jenkinsfile. What's the reason for this behavior? Considering we have more than 3k branches and they are bound to grow (it's quite an old project), every time a build start I'm wasting time just to perform an unneeded branches scanning!!!!

Can anyone explain me please and possibly giving me a solution to disable this unneeded scanning?

Thanks in advance

Marco

[James Nord]

Hi Marco,

Are you sure the build of the branch job is actually scanning all the branches?

What's normally happening is the scan is only checking for new "interesting” branches, or changes to existing branches for which a job has already been created.

this can be time consuming and so you generally want to rely on a webhook to tell Jenkins something specific has changed, then it will scan a specific branch only rather than a whole repo (or a whole organisation if you are using an organisation folder).

The branch job won't be scanning, but it can perform a clone of the repo on your controller.  As the pipeline is always run on the controller the Jenkinsfile is needed here, so a clone to obtain it will be performed (unless your SCM supports a lightweight method for obtaining just a single file from a single branch - GitHub can do this but a generic got repo can not).

Onelce you have the Jenkinsfile this is parsed and run on the controller.  The agent will only be used for specific steps within a `node` block (like `sh`), all the groovy code is always run on the controller.

HTH

/James

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/b7857e6e-e160-45cd-8e58-53d7a31cd3d1n%40googlegroups.com.

[TechnoLost]

Hi James, thanks for your answer first of all :)

Yes, I can confirm the branch job is looking for branches, or at least this is what it seems to me. Just to be clear, this is what the branch job write to log whenever it starts:

11:47:34 Setting origin to https://bitbucket.org/XXXX/YYYY.git 11:47:34 > git.exe config remote.origin.url https://bitbucket.org/XXXX/YYYY.git # timeout=10 11:47:34 Fetching origin... 11:47:34 Fetching upstream changes from origin 11:47:34 > git.exe --version # timeout=10 11:47:34 > git --version # 'git version 2.38.1.windows.1' 11:47:34 > git.exe config --get remote.origin.url # timeout=10 11:47:34 using GIT_ASKPASS to set credentials Jenkins 11:47:34 > git.exe fetch --tags --force --progress -- origin +refs/heads/*:refs/remotes/origin/* # timeout=10 11:47:41 Seen branch in repository origin/16.06.XX 11:47:41 Seen branch in repository origin/19.02.07

and it goes on for other over 3K branches (yes, we do have quite a lot of branches :P)

After this, it does start finally doing branch clone:

11:47:49 The recommended git tool is: /usr/bin/git 11:47:49 using credential 3cea3a8b-6a-40-a6-4abbfb046598 11:47:49 Cloning the remote Git repository 11:47:49 Cloning repository https://bitbucket.org/XXXX/YYYY.git

As you can see it takes around 7 seconds now to scan all those 3k branches, but they are bound to grow! That's why I'd like to avoid the branch job to perform such a scan.

Any idea?

Thanks in advance

PS: I thought that once performed the light fetch to get the Jenkinfile such a file and its groovy code as well was run on node...I mean, the node itself is cloning the branch code right? So isn't it the node who's running groovy code?

Marco