[JIRA] (JENKINS-39682) Generated Multibranch Pipeline Job does not index branches

[mail@daniel-spilker.com (JIRA)]

Daniel Spilker updated an issue

Jenkins / JENKINS-39682 Generated Multibranch Pipeline Job does not index branches Change By: Daniel Spilker Summary: Generated Mutlibranch Multibranch Pipeline Job does not index branches Add Comment

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

[bitwiseman@gmail.com (JIRA)]

Liam Newman assigned an issue to Daniel Spilker

Jenkins / JENKINS-39682 Generated Multibranch Pipeline Job does not index branches

Change By: Liam Newman Assignee: Stephen Connolly Daniel Spilker

Add Comment

[aaron.franssell@gmail.com (JIRA)]

Aaron Franssell commented on JENKINS-39682

Re: Generated Multibranch Pipeline Job does not index branches John F did that resolve your issue? I got hit by the exact same thing on my project. I tried greg oire's suggestion but that didn't work either...

Add Comment

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)

[michelzanini@gmail.com (JIRA)]

Michel Zanini commented on JENKINS-39682

Re: Generated Multibranch Pipeline Job does not index branches

PR  https://github.com/jenkinsci/branch-api-plugin/pull/158 was merged to fix this issue but has caused a worse issue in my opinion. My seed job runs every 4 hours, it scans repositories in Bitbucket and create and/or update hundreds of jobs as a result. After updating the Branch API plugin to 2.5.5 with this PR included, now every 4 hours, ALL my ~ 250 jobs are triggering a branch scanning at the same time after seed job completes. This causes several API calls to Bitbucket all at the same time and then they hang, as Bitbucket has a limit on number of API calls. That means I have to revert the plugin to 2.5.4 as its impossible to continue with this behaviour.   I understand what you are trying to fix here - to run it initially when the job gets created for the first time. But it should not happen when the job gets updated. Also, using a scan trigger fixes your problem and it is recommend to use one anyway (as sometimes Webhooks will be missed). It has never being a problem for us.   Anyway, I think this has made things worse at the point of being a blocker. Thanks.

Add Comment

[michelzanini@gmail.com (JIRA)]

Michel Zanini edited a comment on JENKINS-39682

Re: Generated Multibranch Pipeline Job does not index branches

PR  [https://github.com/jenkinsci/branch-api-plugin/pull/158] was merged to fix this issue but has caused a worse issue in my opinion. My seed job runs every 4 hours, it scans repositories in Bitbucket and create and/or update hundreds of jobs as a result. After updating the Branch API plugin to 2.5.5 with this PR included, now every 4 hours, ALL my ~ 250 jobs are triggering a branch scanning at the same time after seed job completes. This causes several API calls to Bitbucket all at the same time and then they hang, as Bitbucket has a limit on number of API calls. That means I have to revert the plugin to 2.5.4 as its impossible to continue with this behaviour.   I understand what you are trying to fix here - to run it initially when the job gets created for the first time. But it should not happen when the job gets updated.

Also, using a scan trigger fixes your the problem of initial branch scanning and it is recommend to use one anyway (as sometimes Webhooks will be missed). It has never being a problem for us.

Anyway, I think this has made things worse at the point of being a blocker. Thanks.

Add Comment

[gregoire.waymel@decathlon.com (JIRA)]

greg oire commented on JENKINS-39682

Re: Generated Multibranch Pipeline Job does not index branches

Triggering scan is not always possible. For instance you speak about 250 repos let say around 15 branches per repos that will quickly kill your api quota. On large orgs, you do that exceptionally and more per repos than at full scale

Add Comment

[michelzanini@gmail.com (JIRA)]

Michel Zanini commented on JENKINS-39682

Re: Generated Multibranch Pipeline Job does not index branches

I am not sure I understood well what you mean greg oire. The issue here is that my seed job creates/updates many jobs every few hours. That was working ok before and now after this change it started re-scanning all of the jobs at once. Do you agree this is a problem or what are you suggesting?

Add Comment

[gregoire.waymel@decathlon.com (JIRA)]

greg oire commented on JENKINS-39682

Re: Generated Multibranch Pipeline Job does not index branches

You were suggesting to enable the index scan trigger to prevent the initial scan. I was just pointing out that it is not always possible as it would result jenkins to make a lot of api call from time to time, that would result in the exact same issue.

Add Comment

[michelzanini@gmail.com (JIRA)]

Michel Zanini commented on JENKINS-39682

Re: Generated Multibranch Pipeline Job does not index branches

But you can configure the the trigger to be every day or every week etc. They go randomically and then the API calls are spread and don't cause issues.

Add Comment

[michelzanini@gmail.com (JIRA)]

Michel Zanini commented on JENKINS-39682

Re: Generated Multibranch Pipeline Job does not index branches

Also, another suggestion is do what has being done here but only if the job has just being created for the first time, then you trigger the scan, but not on a job update.

Add Comment

[michelzanini@gmail.com (JIRA)]

Michel Zanini edited a comment on JENKINS-39682

Re: Generated Multibranch Pipeline Job does not index branches

But you can configure the the trigger to be every day or every week etc. They go randomically and then the API calls are spread and don't cause issues.

This is what I am using now: ``` triggers {   periodic(24 * 60) } ```

Add Comment

[michelzanini@gmail.com (JIRA)]

Michel Zanini edited a comment on JENKINS-39682

Re: Generated Multibranch Pipeline Job does not index branches

But you can configure the the trigger to be every day or every week etc. They go randomically and then the API calls are spread and don't cause issues. This is what I am using now:

{code:java} ``` triggers {   periodic(24 * 60) } ``` {code}

Add Comment

[michelzanini@gmail.com (JIRA)]

Michel Zanini edited a comment on JENKINS-39682

Re: Generated Multibranch Pipeline Job does not index branches

But you can configure the the trigger to be every day or every week etc. They go randomically and then the API calls are spread and don't cause issues. This is what I am using now: {code:java} triggers {     periodic(24 * 60)

//every 24 hours } {code}

Add Comment

[michelzanini@gmail.com (JIRA)]

Michel Zanini edited a comment on JENKINS-39682

Re: Generated Multibranch Pipeline Job does not index branches

But you can configure the the trigger to be every day or every week etc. They go randomically and then the API calls are spread and don't cause issues. This is what I am using now: {code:java} triggers {     periodic(24 * 60) //every 24 hours } {code}

This does not mean that all jobs will trigger the scan every at the same time. Unless I am wrong they are spread randomically and happen separately.

Add Comment

[michelzanini@gmail.com (JIRA)]

Michel Zanini edited a comment on JENKINS-39682

Re: Generated Multibranch Pipeline Job does not index branches

But you can configure the the trigger to be every day or every week etc. They go randomically and then the API calls are spread and don't cause issues. This is what I am using now: {code:java} triggers {     periodic(24 * 60) //every 24 hours }{code}

This does not mean that all jobs will trigger the scan every day at the same time. Unless I am wrong they are spread randomically and happen separately.

Add Comment

[rene.scheibe@gmail.com (JIRA)]

René Scheibe commented on JENKINS-39682

Re: Generated Multibranch Pipeline Job does not index branches

Michel Zanini triggering a scan is required on job update too as different things might have changed, e.g. the repository, different behaviours (looking for branches and/or pull request and/or tags, filtering of heads and so on...).

Add Comment

[michelzanini@gmail.com (JIRA)]

Michel Zanini commented on JENKINS-39682

Re: Generated Multibranch Pipeline Job does not index branches

René Scheibe, they might have changed but might NOT as well. Scanning it on every update is expensive because Bitbucket has limits on API calls. At least, at minimum, maybe one thing could be done is to triggered the scan in the future at a random time. For example, choose a random time from in a space of 1 hour from now and run at that time. That way the calls will be spread out. Maybe this period could even be configured. That's how periodic triggers work. Actually, again, maybe it is not needed to have it on update. Because if you have it on create, and use a periodic trigger, it should be fine as eventually the job will be corrected. At the moment, with the change that has being made, its impossible to update several jobs at the same time. There is no flag, nothing I can do to disable this. So its unusable.

Add Comment

[michelzanini@gmail.com (JIRA)]

Michel Zanini edited a comment on JENKINS-39682

Re: Generated Multibranch Pipeline Job does not index branches

[~renescheibe], they might have changed but might NOT as well. Scanning it on every update is expensive because Bitbucket has limits on API calls. At least, at minimum, maybe one thing what could be done is to triggered trigger the scan in the future at a random time. For example, choose a random time from in a space of 1 hour from now and run at that time. That way the calls will be spread out. Maybe this period could even be configured. That's how periodic triggers work. Actually, again, maybe it is not needed to have it on update. Because if you have it on create, and use a periodic trigger, it should be fine as eventually the job will be corrected.

At the moment, with the change that has being made, its impossible to update several jobs at the same time. There is no flag, nothing I can do to disable this. So its unusable.

Add Comment

[ingwar@ingwar.eu.org (JIRA)]

Karol Lassak commented on JENKINS-39682

Re: Generated Multibranch Pipeline Job does not index branches

Michel Zanini we also found that issue that all our ~ 800 repos was reindexed every night (ofc one day is not enough to reindex all of them with GH API limits) But I found way how to fix it.. See https://github.com/jenkinsci/job-dsl-plugin/blob/master/job-dsl-plugin/src/main/groovy/javaposse/jobdsl/plugin/JenkinsJobManagement.java#L456 If your configuration is not changed then updateByXml is not called, so no reindexing is done..   In our case it was even harder to catch cause jobs itself did not changed but folders did (we have some portlets there that have random ID by default) We changed them to static IDs and problem is gone..

Add Comment