| There are two periodic scans "configured". One is user configurable, namely the one at the org folder level. The other is not currently configurable, namely the one at the branch folder level. Now the way this is designed is on the basis that you will set up webhooks because "polling must die" to quote a blog post from KK ages back. So the periodic scanning is more to cover two cases: 1. To allow controlled pruning of orphaned items. For technical reasons, the code was actually written originally to allow multiple sources in the same org folder. So you could have one source be a github organization and another be a bitbucket team. Then if you moved a repository from the bitbucket team to the github organization the multibranch project would still stay there. This was to mirror the fact that a multibranch project can have multiple sources. However this was too confusing, so we locked the org folders to one source only. In any case, because there originally could be (and for a multibranch project there still can be) multiple things that can claim ownership of a child name, we cannot decide if an event that a name has been deleted by one source will actually result in the deletion of the child. We have to ask all the candidate sources in the priority order: "is this yours?" until we get someone answering "yes"... So the orphaned item strategy only runs with a full scan. This also allows the orphaned item strategy to correctly decide the order in which orphans should be removed. 2. Events may be dropped or lost. We should find those changes eventually. The scan fills that gap. If you have webhooks set up correctly then you only need scanning say once per day or less frequently. All the cron triggers store a random offset as a transient field. This random offset is used to decide when in the `H` portion of a trigger rule to actually pick. The periodic folder trigger converts the times into rules like H * * * * (for hourly) or H H * * * (for daily) so this transient field is important that it remain consistent. What I think is happening is that the org folder scan is resetting the transient field. So if your org folder is scanning every 5 minutes, you will have a 5 in 60 chance that the transient gets reset to a value in the next 5 minutes. Of course we have two H's for the daily scan that the multibranch has. So basically if my theory that the transient is getting reset is correct then there is a 1 in 288 chance that the transient will have a value resulting in a scan during the next 5 minutes... and then the transient gets reset again. Each 5 minute interval gas a 287 in 288 chance of not scanning... so we are naïvely looking at the probability of a multibranch project not scanning being (287/288)^288 = 0.36... it's actually worse, because that is actually the probability of it checking whether a scan is due... and if the transient field for H is being cleared then so too will this one: https://github.com/jenkinsci/cloudbees-folder-plugin/blob/23367538c5c6b405a3ba63d8033ca459aa2f65a0/src/main/java/com/cloudbees/hudson/plugins/folder/computed/PeriodicFolderTrigger.java#L71 and so if we win the 63% lottery then one of those will run through to initialize lastTriggered to a random value https://github.com/jenkinsci/cloudbees-folder-plugin/blob/23367538c5c6b405a3ba63d8033ca459aa2f65a0/src/main/java/com/cloudbees/hudson/plugins/folder/computed/PeriodicFolderTrigger.java#L211-L217 So if my theory is correct then you would expect:
- When the org folder is set to scan at a period much shorter than 24h the multibranch will rarely scan.
- When the org folder is set to scan at a period much longer than 24h the multibranch will scan mostly daily except for the 24h after the org folder scans.
NOTE: the org folder scan is NOT supposed to trigger the child multibranch scans as that would be sympathetic harmonisation and basically result in everything scanning at the same time. The aim is to level out all scanning across the entire period. NOTE: I would like the child multibranch scanning period to be configurable, but the UI for that was not agreed so I couldn't get approval to work on that in my paid time... and my wife vetoed me working on it in my spare time :shrug: |
