[JIRA] (JENKINS-36666) Jenkins startup fails due to cyclic dependency miscalculation

12 views
Skip to first unread message

jglick@cloudbees.com (JIRA)

unread,
Jul 13, 2016, 1:27:02 PM7/13/16
to jenkinsc...@googlegroups.com
Jesse Glick created an issue
 
Jenkins / Bug JENKINS-36666
Jenkins startup fails due to cyclic dependency miscalculation
Issue Type: Bug Bug
Assignee: Jesse Glick
Components: core
Created: 2016/Jul/13 5:26 PM
Labels: plugin-manager robustness
Priority: Critical Critical
Reporter: Jesse Glick

To reproduce in 2.14-SNAPSHOT:

  • start on a new home
  • swat aside the setup wizard
  • install github-branch-source including all dependencies
  • manually upload workflow-scm-step-2.0.hpi (downgrading from the 2.2 installed from the UC)
  • restart

You will see a warning about cyclic dependencies:

... hudson.PluginManager$1$3$2$3 reactOnCycle
SEVERE: found cycle in plugin dependencies: (root=Plugin:workflow-scm-step, deactivating all involved) Plugin:workflow-scm-step -> Plugin:git -> Plugin:workflow-scm-step

Which looks fine (you need to update workflow-scm-step to solve the conflict) until you realize it is disabling git but not its dependencies. So then you see a lot of extension loading errors, which would normally be just noise:

... hudson.ExtensionFinder$GuiceFinder$FaultTolerantScope$1 error
WARNING: Failed to instantiate Key[type=org.jenkinsci.plugins.github_branch_source.GitHubSCMSource$DescriptorImpl, annotation=[none]]; skipping this component
com.google.inject.ProvisionException: Guice provision errors:

1) Error injecting constructor, java.lang.NoClassDefFoundError: jenkins/plugins/git/AbstractGitSCMSource
  at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource$DescriptorImpl.<init>(GitHubSCMSource.java:722)

1 error
	at ...
	at hudson.ExtensionFinder$GuiceFinder._find(ExtensionFinder.java:386)
	at hudson.ExtensionFinder$GuiceFinder.find(ExtensionFinder.java:377)
	at hudson.ClassicPluginStrategy.findComponents(ClassicPluginStrategy.java:471)
	at hudson.ExtensionList.load(ExtensionList.java:349)
	at hudson.ExtensionList.ensureLoaded(ExtensionList.java:287)
	at hudson.ExtensionList.getComponents(ExtensionList.java:167)
	at hudson.DescriptorExtensionList.load(DescriptorExtensionList.java:185)
	at hudson.ExtensionList.ensureLoaded(ExtensionList.java:287)
	at hudson.ExtensionList.iterator(ExtensionList.java:156)
	at hudson.ExtensionList.get(ExtensionList.java:147)
	at com.cloudbees.jenkins.GitHubPushTrigger$DescriptorImpl.get(GitHubPushTrigger.java:359)
	at org.jenkinsci.plugins.github.migration.Migrator.migrate(Migrator.java:40)
	at org.jenkinsci.plugins.github.GitHubPlugin.postInitialize(GitHubPlugin.java:39)
	at hudson.PluginManager$2$4$5.run(PluginManager.java:537)
	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282)
	at jenkins.model.Jenkins$7.runTask(Jenkins.java:998)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoClassDefFoundError: jenkins/plugins/git/AbstractGitSCMSource
	at ...
	at java.lang.Class.getEnclosingClass(Class.java:1277)
	at hudson.model.Descriptor.<init>(Descriptor.java:270)
	at jenkins.scm.api.SCMSourceDescriptor.<init>(SCMSourceDescriptor.java:42)
	at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource$DescriptorImpl.<init>(GitHubSCMSource.java:722)
	at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource$DescriptorImpl$$FastClassByGuice$$fee958e6.newInstance(<generated>)
	at ...
	... 28 more
Caused by: java.lang.ClassNotFoundException: jenkins.plugins.git.AbstractGitSCMSource
	at jenkins.util.AntClassLoader.findClassInComponents(AntClassLoader.java:1376)
	at ...
... hudson.ExtensionFinder$GuiceFinder$FaultTolerantScope$1 error
WARNING: Failed to instantiate Key[type=org.jenkinsci.plugins.github_branch_source.GitHubSCMNavigator$DescriptorImpl, annotation=[none]]; skipping this component
com.google.inject.ProvisionException: Guice provision errors:

1) Error injecting constructor, java.lang.NoClassDefFoundError: jenkins/plugins/git/AbstractGitSCMSource
  at org.jenkinsci.plugins.github_branch_source.GitHubSCMNavigator$DescriptorImpl.<init>(GitHubSCMNavigator.java:326)

1 error
	at ...

However after all this you get

... hudson.init.impl.InstallUncaughtExceptionHandler$DefaultUncaughtExceptionHandler uncaughtException
SEVERE: A thread (pool-6-thread-5/40) died unexpectedly due to an uncaught exception, this may leave your Jenkins in a bad way and is usually indicative of a bug in the code.
java.lang.NoClassDefFoundError: org/jenkinsci/plugins/github_branch_source/GitHubSCMSource
	at java.lang.Class.getDeclaringClass0(Native Method)
	at java.lang.Class.getDeclaringClass(Class.java:1235)
	at java.lang.Class.getEnclosingClass(Class.java:1277)
	at java.lang.Class.getSimpleBinaryName(Class.java:1443)
	at java.lang.Class.getSimpleName(Class.java:1309)
	at hudson.init.TaskMethodFinder.getDisplayNameOf(TaskMethodFinder.java:80)
	at hudson.init.TaskMethodFinder$TaskImpl.getDisplayName(TaskMethodFinder.java:167)
	at jenkins.InitReactorRunner$1.onTaskStarted(InitReactorRunner.java:64)
	at org.jvnet.hudson.reactor.ReactorListener$Aggregator.onTaskStarted(ReactorListener.java:89)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:208)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

and at this point Jenkins startup ends; the UI just says Please wait while Jenkins is restarting... forever.

These steps to reproduce are somewhat artificial. The original report comes from CloudBees Jenkins Enterprise: after starting 1.642.4.1 and installing subversion from the update center, Jenkins will not start (though in that case the fatal error comes a little later and lacks the "in a bad way" message.

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
Atlassian logo

jglick@cloudbees.com (JIRA)

unread,
Jul 13, 2016, 1:31:01 PM7/13/16
to jenkinsc...@googlegroups.com

jglick@cloudbees.com (JIRA)

unread,
Jul 13, 2016, 1:38:01 PM7/13/16
to jenkinsc...@googlegroups.com
Jesse Glick started work on Bug JENKINS-36666
 
Change By: Jesse Glick
Status: Open In Progress

jglick@cloudbees.com (JIRA)

unread,
Jul 13, 2016, 1:51:01 PM7/13/16
to jenkinsc...@googlegroups.com

First line of defense is to prevent failed calls to Task.getDisplayName from being fatal. That yields

WARNING: failed to find displayName of public static void org.jenkinsci.plugins.github_branch_source.GitHubSCMSource$DescriptorImpl.addAliases()
java.lang.NoClassDefFoundError: org/jenkinsci/plugins/github_branch_source/GitHubSCMSource
	at java.lang.Class.getDeclaringClass0(Native Method)
	at java.lang.Class.getDeclaringClass(Class.java:1235)
	at java.lang.Class.getEnclosingClass(Class.java:1277)
	at java.lang.Class.getSimpleBinaryName(Class.java:1443)
	at java.lang.Class.getSimpleName(Class.java:1309)
	at hudson.init.TaskMethodFinder.getDisplayNameOf(TaskMethodFinder.java:80)
	at hudson.init.TaskMethodFinder$TaskImpl.getDisplayName(TaskMethodFinder.java:167)
	at jenkins.InitReactorRunner$1.getDisplayName(InitReactorRunner.java:77)
	at jenkins.InitReactorRunner$1.onTaskStarted(InitReactorRunner.java:64)
	at org.jvnet.hudson.reactor.ReactorListener$Aggregator.onTaskStarted(ReactorListener.java:89)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:208)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
	at …

coming from this code. That still leaves us with

…hudson.util.BootFailure publish
SEVERE: Failed to initialize Jenkins
hudson.util.HudsonFailedToLoad: org.jvnet.hudson.reactor.ReactorException: java.lang.Error: java.lang.reflect.InvocationTargetException
	at hudson.WebAppMain$3.run(WebAppMain.java:244)
Caused by: org.jvnet.hudson.reactor.ReactorException: java.lang.Error: java.lang.reflect.InvocationTargetException
	at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:269)
	at jenkins.InitReactorRunner.run(InitReactorRunner.java:47)
	at jenkins.model.Jenkins.executeReactor(Jenkins.java:1019)
	at jenkins.model.Jenkins.<init>(Jenkins.java:877)
	at hudson.model.Hudson.<init>(Hudson.java:85)
	at hudson.model.Hudson.<init>(Hudson.java:81)
	at hudson.WebAppMain$3.run(WebAppMain.java:227)
Caused by: java.lang.Error: java.lang.reflect.InvocationTargetException
	at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:110)
	at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282)
	at jenkins.model.Jenkins$7.runTask(Jenkins.java:998)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104)
	... 8 more
Caused by: java.lang.NoClassDefFoundError: org/jenkinsci/plugins/github_branch_source/GitHubSCMSource
	at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource$DescriptorImpl.addAliases(GitHubSCMSource.java:738)
	... 13 more

jglick@cloudbees.com (JIRA)

unread,
Jul 13, 2016, 1:55:01 PM7/13/16
to jenkinsc...@googlegroups.com

Now this @Initializer could be marked fatal=false, since if we decline to add XStream aliases it is no big deal. But that just seems to be kicking the ball down the road—there are a lot of initializers out there, which rarely claim to be dispensable, and anyway the plugin author should reasonably expect that if his plugin is loaded then it has satisfied all its dependencies.

jglick@cloudbees.com (JIRA)

unread,
Jul 13, 2016, 2:35:01 PM7/13/16
to jenkinsc...@googlegroups.com

Confirmed that the main problem is simply JENKINS-21486; the rest is just trying to recover from problems loading plugins we should not have been loading to begin with.

dbeck@cloudbees.com (JIRA)

unread,
Jul 25, 2016, 5:45:02 AM7/25/16
to jenkinsc...@googlegroups.com

jglick@cloudbees.com (JIRA)

unread,
Jul 25, 2016, 1:03:01 PM7/25/16
to jenkinsc...@googlegroups.com
Jesse Glick resolved as Fixed
 

Daniel Beck yes, though the JIRA link daemon seems dead.

Change By: Jesse Glick
Status: In Progress Resolved
Resolution: Fixed

dbeck@cloudbees.com (JIRA)

unread,
Jul 25, 2016, 1:19:01 PM7/25/16
to jenkinsc...@googlegroups.com
 
Re: Jenkins startup fails due to cyclic dependency miscalculation

Jesse Glick Known issue, caused by the JIRA upgrade removing the SOAP API – Kohsuke Kawaguchi is looking into it, as that process still (*sigh*) runs in his basement.

No 'FIXED JENKINS-whatever' commit message anyway though, so I wasn't sure.

li.xiaoyun1983@gmail.com (JIRA)

unread,
Aug 3, 2016, 11:27:05 PM8/3/16
to jenkinsc...@googlegroups.com

I met the same issue too. And found the reason of your dependency cycle was that, in the commit of https://github.com/jenkinsci/git-plugin/commit/470a4ca0837e1af444423a156e20115f58ae9124 of Git Plugin 2.9, they added a dependency "workflow-scm-step" for supporting "pipeline-plugin". This will cause dependency cycle in Jenkins server, while loading plugins.

I my case, this dependency cycle happened while loading pluings, "Plugin:workflow-scm-step -> Plugin:git -> Plugin:parameterized-trigger -> Plugin:subversion -> Plugin:workflow-scm-step", due to pom.xml of "subversion-plugin", it calls workflow-scm-step from version 2.6. So I rolled back to 2.5.8 to workaround this.
I think this should be fixed by Git-Plugin and Subversin-Plugin side?

dbeck@cloudbees.com (JIRA)

unread,
Aug 4, 2016, 8:41:01 AM8/4/16
to jenkinsc...@googlegroups.com

Jennie Li What version of Jenkins are you on?

Also, updating workflow-scm-step should resolve this.

Reply all
Reply to author
Forward
0 new messages