"Scan Multibranch Pipeline Now" triggers immediate build

[Dallas Clement]

If I click on the "Scan Multibranch Pipeline Now" link in the Jenkins dashboard, it will kick off a build immediately even when there were no changes.  I have my declarative Jenkinsfile configured to poll for SCM changes.  I only want builds to be triggered from SCM changes.  Any idea how I can suppress this behavior?

pipeline {

  agent any

  options {

    disableConcurrentBuilds()

    timestamps()

    buildDiscarder(logRotator(numToKeepStr: '8'))

  }

  triggers {

    // Poll every 5 minutes for new changes

    pollSCM 'H/5 * * * *'

  }

[Stephen Connolly]

You do know in multibranch that this poll is ignored?

  }

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/4dff0257-8170-4976-8998-13f20023e528%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Sent from my phone

[Stephen Connolly]

On Tue 29 Aug 2017 at 23:21, Stephen Connolly <stephen.al...@gmail.com> wrote:

On Tue 29 Aug 2017 at 22:35, Dallas Clement <dallas.a...@gmail.com> wrote:

If I click on the "Scan Multibranch Pipeline Now" link in the Jenkins dashboard, it will kick off a build immediately even when there were no changes.  I have my declarative Jenkinsfile configured to poll for SCM changes.  I only want builds to be triggered from SCM changes.  Any idea how I can suppress this behavior?

pipeline {

  agent any

  options {

    disableConcurrentBuilds()

    timestamps()

    buildDiscarder(logRotator(numToKeepStr: '8'))

  }

  triggers {

    // Poll every 5 minutes for new changes

    pollSCM 'H/5 * * * *'

You do know in multibranch that this poll is ignored?

To explain why the design is this way:

1. Polling is evil

2. In most SCM implementations the poll for each branch is effectively a redundant check, so if you have 10 branches all polling then Jenkins will end up doing the *exact same* operation in a parallel batch of 5 (default polling pool size) and follow up with another batch of 5... hammering the SCM where one request would provide the answer for all 10 branches.

3. Branch API needs to maintain its own state as to the last revision built. Polling will not update that correctly.

4. Polling for Git can be a mess due to how git can be configured to built multiple branches in one job.

So the decision is that polling is disabled on multibranch leaf jobs (if you have a case where it is not disabled... congratulations you have found a bug... likely not the one you thought, but a bug... polling should not happen for leaf nodes of a multibranch project)

You want to have indexing at a frequency such that if an event was lost, you cannot go more than this long without the build (typically somewhere between 4-24h for most people)

You want event support to trigger the builds, not polling

  }

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/4dff0257-8170-4976-8998-13f20023e528%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Sent from my phone

[Dallas Clement]

Thanks for the info Stephen.  I was not aware that the pollSCM trigger was ignored in a multibranch pipeline.  Something is polling for changes.  When someone pushes a change, a build gets triggered shortly thereafter.  I do agree with you that polling is evil and not scalable with many branches.  I would like to make things event driven.  Will need to investigate how with git.

[Stephen Connolly]

Sounds like you have a post commit hook pushing the commit details to Jenkins, in which case events are working... to confirm, check the cause of a build, if it says "Branch Event" then you are solid gold... 

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/c4d93f77-80ec-4f7c-a783-0947fdb3af5c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Dallas Clement]

No I don't have any post commit hook defined.  Starting with a green field here.  Sounds like that's what I need to setup though.

I also don't have the "Scan Multibranch Pipeline Triggers" checkbox enabled on my job configuration.  I really don't know how it could be triggering unless that PollSCM in my Jenkinsfile is somehow working.

[Stephen Connolly]

On 29 August 2017 at 18:38, Dallas Clement <dallas.a...@gmail.com> wrote:

No I don't have any post commit hook defined.  Starting with a green field here.  Sounds like that's what I need to setup though.

I also don't have the "Scan Multibranch Pipeline Triggers" checkbox enabled on my job configuration.  I really don't know how it could be triggering unless that PollSCM in my Jenkinsfile is somehow working.

Ok, well that sounds like a bug.

I've filed https://issues.jenkins-ci.org/browse/JENKINS-46536 to track that.

 

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/1f85a58d-70a2-4e74-9aba-244baa19f2ef%40googlegroups.com.

[Dallas Clement]

I removed the "PollSCM" from the "triggers" section of my declarative Jenkinsfile.  And then I enabled "Scan multibranch pipeline triggers" and the "Periodically if not otherwise run" option.  This method of polling seems to work fine so far.  I'm afraid I will need to rely on polling because our git repo is on another machine in a different facility.  Our jenkins is deployed on an EC2 host and the company doesn't want to open a port to the world for the trigger notification.

[Stephen Connolly]

On 31 August 2017 at 07:06, Dallas Clement <dallas.a...@gmail.com> wrote:

I removed the "PollSCM" from the "triggers" section of my declarative Jenkinsfile.  And then I enabled "Scan multibranch pipeline triggers" and the "Periodically if not otherwise run" option.  This method of polling seems to work fine so far.  

It also is "constant time" for the number of branches that you have, whereas the triggers route is proportional to the number of branches you have... so if you need to use polling, this is the better way to do it

 

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/1236fa8d-de0a-4256-8b14-66a192c9e3a8%40googlegroups.com.

[Steven Foster]

About scan repository triggers, I generated multibranch jobs with a jobDSL and replaced "periodic" with "cron" to avoid a huge amount of indexing happening at once. Is that supposed to even be an option? Since it's not in the configuration UI.

[Stephen Connolly]

On Thu 31 Aug 2017 at 17:00, Steven Foster <steven...@gmail.com> wrote:

About scan repository triggers, I generated multibranch jobs with a jobDSL and replaced "periodic" with "cron" to avoid a huge amount of indexing happening at once. Is that supposed to even be an option? Since it's not in the configuration UI.

Periodic randomly levels all jobs

Cron is not supposed to be used

You are using jobdsl ensure that the SCMSource id is preserved on update or you will lose all branches and have them recreated periodically 

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/d4537fb1-4bd3-4bac-ac79-b95fbcb6a2fb%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Steven Foster]

On Thursday, August 31, 2017 at 5:55:27 PM UTC+1, Stephen Connolly wrote:

Periodic randomly levels all jobs

Ah, my mistake. What I was noticing was a symptom of something else (user error)

[Steven Foster]

On Thursday, August 31, 2017 at 5:55:27 PM UTC+1, Stephen Connolly wrote:

Periodic randomly levels all jobs

 The specs on my 30m interval jobs are all H/2 * * * *, doesn't this mean every 2 minutes rather than every 30 minutes which would be H/30 * * * *? Or is the internal system different from the user-facing one?

[Stephen Connolly]

https://github.com/jenkinsci/cloudbees-folder-plugin/blob/master/src/main/java/com/cloudbees/hudson/plugins/folder/computed/PeriodicFolderTrigger.java#L91-L109

The crontab is just controlling how often it checks... it will only schedule an index on one of those checks if the interval since the last run is longer than configured

--

You received this message because you are subscribed to the Google Groups "Jenkins Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/a7c933c2-e457-4a23-9794-bb88c788e9be%40googlegroups.com.

[Robert Hales]

I agree that polling is not the best use of resources, but I don't think this option should be killed just because it may not be a good idea. 

I use the PollSCM option in several multibranch jobs (Jenkins 2.32.2) and it works just fine. Sometimes red tape and bureaucracy in a large enterprise cause problems with what should be simple--I don't have the option to configure hooks from our subversion repository. The option for "Periodically if not otherwise run" is very inflexible and didn't suit our needs, and configuring the polling worked just fine for us. 

I feel like it shouldn't be up to the software to stop me from doing stupid things if that is what I want or need to do. If there is a technical reason why it can't be done, or it is a bad idea so was never implemented, that might be something different. But it has been developed, and it works. I don't see a benefit to breaking it.