| It looks like it's a combination of two things that's causing this bug. There's a bug that's causing Jenkins to see changes to it's dependencies even though there aren't any. This seems random for us. And I'm not sure how or why it thinks there's a dependency on itself? https://issues.jenkins-ci.org/browse/JENKINS-46995 i.e. Job A #300 has dependency changes in Job A #290-299. Job A #290 has dependency changes in Job A #275-289.... The DefaultIssueSelector looks at all of these dependencies, which can take a really long time to finish. This recursive call doesn't have any limit on how far down the dependencies it's looking. This can end up being hundreds of builds down the chain. https://github.com/jenkinsci/jira-plugin/blob/master/src/main/java/hudson/plugins/jira/selector/DefaultIssueSelector.java#L110 https://github.com/jenkinsci/jira-plugin/blob/master/src/main/java/hudson/plugins/jira/selector/DefaultIssueSelector.java#L135 It looks like it's a bug in the refactoring of the original issue selector. https://github.com/jenkinsci/jira-plugin/commit/2eace064c77979cab041e61d82dea47eb44cc886#diff-e381acd1474876247543cc9edc4ea09e As far as I can tell, the original code (roughly) looks like this:
findIssueIdsRecursive()
{
findIssues()
for each in dependency
findIssues()
// findIssueIdsRecursive is not called again
}
The refactoring changed it to roughly this:
addIssuesRecursive()
{
addIssuesCarriedOverFromPreviousBuild()
addIssuesFromCurrentBuild()
addIssuesFromDependentBuilds()
}
addIssuesFromDependentBuilds()
{
for each in dependency
// Looking at dependent builds of this build, along with all of its dependent builds
addIssuesRecursive()
}
When we're running into this problem, it looks like a deadlock, but it's actually just taking hours instead of minutes like it normally does. Those spikes are probably when no one noticed on our team and it finished. The smaller spikes are probably when someone noticed and we had to manually abort the thread, thinking the plugin deadlocked again. 
|
