[JIRA] (JENKINS-36896) Stack Trace on startup

[michaeldkfowler@gmail.com (JIRA)]

Michael Fowler created an issue

Jenkins / JENKINS-36896 Stack Trace on startup Issue Type: Bug Assignee: Matthew DeTullio Components: multi-branch-project-plugin Created: 2016/Jul/23 6:00 AM Priority: Critical Reporter: Michael Fowler Multi-branch plugin causes jenkins to crash on startup when it searches for config files in symbolic links: hudson.util.HudsonFailedToLoad: org.jvnet.hudson.reactor.ReactorException: java.lang.Error: java.lang.reflect.InvocationTargetException at hudson.WebAppMain$3.run(WebAppMain.java:237) Caused by: org.jvnet.hudson.reactor.ReactorException: java.lang.Error: java.lang.reflect.InvocationTargetException at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:269) at jenkins.InitReactorRunner.run(InitReactorRunner.java:44) at jenkins.model.Jenkins.executeReactor(Jenkins.java:916) at jenkins.model.Jenkins.<init>(Jenkins.java:815) at hudson.model.Hudson.<init>(Hudson.java:85) at hudson.model.Hudson.<init>(Hudson.java:81) at hudson.WebAppMain$3.run(WebAppMain.java:225) Caused by: java.lang.Error: java.lang.reflect.InvocationTargetException at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:110) at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:176) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282) at jenkins.model.Jenkins$7.runTask(Jenkins.java:905) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:106) ... 8 more Caused by: java.io.IOException: Tried to treat 'D:\Jenkins_Home\jobs\RaidenDevelopment\promotions\Upload packages to NuGet share\lastStable' as a directory, but could not get a listing at com.github.mjdetullio.jenkins.plugins.multibranch.TemplateDrivenMultiBranchProject.getConfigFiles(TemplateDrivenMultiBranchProject.java:717) at com.github.mjdetullio.jenkins.plugins.multibranch.TemplateDrivenMultiBranchProject.getConfigFiles(TemplateDrivenMultiBranchProject.java:732) at com.github.mjdetullio.jenkins.plugins.multibranch.TemplateDrivenMultiBranchProject.getConfigFiles(TemplateDrivenMultiBranchProject.java:732) at com.github.mjdetullio.jenkins.plugins.multibranch.TemplateDrivenMultiBranchProject.getConfigFiles(TemplateDrivenMultiBranchProject.java:732) at com.github.mjdetullio.jenkins.plugins.multibranch.TemplateDrivenMultiBranchProject.getConfigFiles(TemplateDrivenMultiBranchProject.java:732) at com.github.mjdetullio.jenkins.plugins.multibranch.TemplateDrivenMultiBranchProject.migrate(TemplateDrivenMultiBranchProject.java:758) ... 13 more Add Comment

This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)

[michaeldkfowler@gmail.com (JIRA)]

Michael Fowler updated an issue

Jenkins / JENKINS-36896 Stack Trace on startup

Change By: Michael Fowler Environment: Windows

Add Comment

[janarend@gmail.com (JIRA)]

Jan Arend Jansen commented on JENKINS-36896

Re: Stack Trace on startup Same for me trying to upgrade existing 1.6 to 2.x on Windows.

Add Comment

[sckoobs@gmail.com (JIRA)]

Andrew Murray commented on JENKINS-36896

Re: Stack Trace on startup

I see a similar error after trying to upgrade the multi branch project plugin: hudson.util.HudsonFailedToLoad: org.jvnet.hudson.reactor.ReactorException: java.lang.Error: java.lang.reflect.InvocationTargetException at hudson.WebAppMain$3.run(WebAppMain.java:244)

Caused by: org.jvnet.hudson.reactor.ReactorException: java.lang.Error: java.lang.reflect.InvocationTargetException at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:269)

at jenkins.InitReactorRunner.run(InitReactorRunner.java:45) at jenkins.model.Jenkins.executeReactor(Jenkins.java:1010) at jenkins.model.Jenkins.<init>(Jenkins.java:878)

at hudson.model.Hudson.<init>(Hudson.java:85) at hudson.model.Hudson.<init>(Hudson.java:81)

at hudson.WebAppMain$3.run(WebAppMain.java:227)

Caused by: java.lang.Error: java.lang.reflect.InvocationTargetException at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:110)

at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282) at jenkins.model.Jenkins$7.runTask(Jenkins.java:999)

at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)

at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)

Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104) ... 8 more Caused by: java.io.IOException: Tried to treat 'E:\Program Files (x86)\Jenkins\jobs\TestCoreDependency\lastStable' as a directory, but could not get a listing

at com.github.mjdetullio.jenkins.plugins.multibranch.TemplateDrivenMultiBranchProject.getConfigFiles(TemplateDrivenMultiBranchProject.java:717) at com.github.mjdetullio.jenkins.plugins.multibranch.TemplateDrivenMultiBranchProject.getConfigFiles(TemplateDrivenMultiBranchProject.java:732) at com.github.mjdetullio.jenkins.plugins.multibranch.TemplateDrivenMultiBranchProject.getConfigFiles(TemplateDrivenMultiBranchProject.java:732)

at com.github.mjdetullio.jenkins.plugins.multibranch.TemplateDrivenMultiBranchProject.migrate(TemplateDrivenMultiBranchProject.java:758) ... 13 more

Add Comment

[frank.racis@cerner.com (JIRA)]

Frank Racis commented on JENKINS-36896

Re: Stack Trace on startup

Here's a diff that will just log a warning instead of completely failing the startup: diff --git a/src/main/java/com/github/mjdetullio/jenkins/plugins/multibranch/TemplateDrivenMultiBranchProject.java b/src/main/java/com/github/mjdetullio/jenkins/plugins/multibranch/TemplateDrivenMultiBranchProject.java index c52dceb..8cb30d4 100644 --- a/src/main/java/com/github/mjdetullio/jenkins/plugins/multibranch/TemplateDrivenMultiBranchProject.java +++ b/src/main/java/com/github/mjdetullio/jenkins/plugins/multibranch/TemplateDrivenMultiBranchProject.java @@ -714,7 +714,8 @@ public abstract class TemplateDrivenMultiBranchProject<P extends AbstractProject File[] contents = dir.listFiles(); if (null == contents) { - throw new IOException("Tried to treat '" + dir + "' as a directory, but could not get a listing"); + LOGGER.log(Level.WARNING, "Tried to treat '" + dir + "' as a directory, but could not get a listing"); + return files; } for (final File file : contents) {

Add Comment

[Melven.Roehrig-Zoellner@DLR.de (JIRA)]

Melven Roehrig-Zoellner updated an issue

Jenkins / JENKINS-36896 Stack Trace on startup

Change By: Melven Roehrig-Zoellner Priority: Critical Blocker

Add Comment

[cobexer@gmail.com (JIRA)]

Christoph Obexer commented on JENKINS-36896

Re: Stack Trace on startup

On my instance I could observe this: The lastStable link starts by pointing to "-1", the windows dir command shows it as type <SYMLINK>. Later when the build finished successfully the link is updated to point to "builds\lastSuccessful" (or something like that). The type is still <SYMLINK> tough and Explorer shows it as a file with type .symlink. If I correct the type of the symlink by recreating it using mklink /D the type changes to <SYMLINKD> Explorer shows it as a Folder link and Jenkins is happy with that. So I guess the problem is where that symlink is created instead of in this plugin?

Add Comment

[michaeldkfowler@gmail.com (JIRA)]

Michael Fowler commented on JENKINS-36896

Re: Stack Trace on startup

Christoph Obexer There are tow issues: 1) the plugin dies when it hits the invalid symlink 2) Jenkins creates invalid symlinks. This has been an issue with Jenkins in windows since symlinks were added a few years ago. Cloudbees does not test on Windows. Half the time Jenkins creates a file instead of symlink, and when it does create an actual symlink, the link is broken anyway. The fast solution is to have a try catch so that the plugin doesn't die on the invalid links. Getting cloud bees to fix the symlink issue seems unlike given how long it has existed.

Add Comment

[michaeldkfowler@gmail.com (JIRA)]

Michael Fowler updated an issue

Jenkins / JENKINS-36896 Stack Trace on startup

Change By: Michael Fowler Component/s: core

Add Comment

[aburdajewicz@cloudbees.com (JIRA)]

Allan BURDAJEWICZ commented on JENKINS-36896

Re: Stack Trace on startup

proposed a fix in https://github.com/jenkinsci/multi-branch-project-plugin/pull/144

Add Comment

[ryan.campbell@gmail.com (JIRA)]

recampbell commented on JENKINS-36896

Re: Stack Trace on startup

Oleg Nenashev says this may be related to a recently fixed issue in core: JENKINS-29956

Add Comment

[jamesdengel@googlemail.com (JIRA)]

james Dengel commented on JENKINS-36896

Re: Stack Trace on startup

This also causes as issue for me

Caused by: java.io.IOException: Tried to treat 'C:\Program Files (x86)\Jenkins\jobs\z_old_jobs\jobs\Deploy_scm_installer\lastSuccessful' as a directory, but could not get a listing

at com.github.mjdetullio.jenkins.plugins.multibranch.TemplateDrivenMultiBranchProject.getConfigFiles(TemplateDrivenMultiBranchProject.java:717) at com.github.mjdetullio.jenkins.plugins.multibranch.TemplateDrivenMultiBranchProject.getConfigFiles(TemplateDrivenMultiBranchProject.java:732) at com.github.mjdetullio.jenkins.plugins.multibranch.TemplateDrivenMultiBranchProject.getConfigFiles(TemplateDrivenMultiBranchProject.java:732) at com.github.mjdetullio.jenkins.plugins.multibranch.TemplateDrivenMultiBranchProject.getConfigFiles(TemplateDrivenMultiBranchProject.java:732) at com.github.mjdetullio.jenkins.plugins.multibranch.TemplateDrivenMultiBranchProject.getConfigFiles(TemplateDrivenMultiBranchProject.java:732) at com.github.mjdetullio.jenkins.plugins.multibranch.TemplateDrivenMultiBranchProject.migrate(TemplateDrivenMultiBranchProject.java:758) ... 13 more

removing the file did solve my issue, however i doubt very much this is a root cause fix.

Add Comment

[nebsta@gmail.com (JIRA)]

Ben Stewart commented on JENKINS-36896

Re: Stack Trace on startup

I am also getting this issue? Does anyone have a workaround for this? E.g. creating new symlinks or installing a previous version of Multi-Branch plugin?

Add Comment

[jesper.klit.jensen@intelligrated.com (JIRA)]

Jesper Jensen commented on JENKINS-36896

Re: Stack Trace on startup

My workaround was to remove this plugin.

Add Comment

[nebsta@gmail.com (JIRA)]

Ben Stewart commented on JENKINS-36896

Re: Stack Trace on startup

Yes I've done the same, I was hoping to start using the Multi-Branch plugin though

Add Comment

[idigwww@yahoo.com (JIRA)]

D S commented on JENKINS-36896

Re: Stack Trace on startup

Please help. This plugin is crashing my production server. It will not let me remove or disable it because there is a dependent called "environment script plugin" I use the environment script plugin so I am afraid to remove that too. I could remove "environment script plugin" then remove "Multi-Branch plugin" and add back in the "environment script plugin" but I am worried it will break my heavily used produciton jenkins jobs. What is the best way to remove Multi-Branch plugin without breaking jobs or jenkins. Please advise. Thanks!

Add Comment

[idigwww@yahoo.com (JIRA)]

D S edited a comment on JENKINS-36896

Re: Stack Trace on startup

Please help.  This plugin is crashing my production jenkins windows server.  It will not let me remove or disable it because there is a dependent called "environment script plugin"

I use the environment script plugin so I am afraid to remove that too. I could remove "environment script plugin"  then remove "Multi-Branch plugin"  and add back in

the  "environment script plugin" but I am worried it will break my heavily used produciton production jenkins jobs.  What is the best way to remove Multi-Branch plugin without breaking jobs or jenkins.  Please advise. What is the name of the .jpi file?   Thanks!

Add Comment

[idigwww@yahoo.com (JIRA)]

D S edited a comment on JENKINS-36896

Re: Stack Trace on startup

Please help.  This plugin is crashing my production jenkins windows server.  It will not let me remove or disable it because there is a dependent called "environment script plugin" I use the environment script plugin so I am afraid to remove that too. I could remove "environment script plugin"  then remove "Multi-Branch plugin"  and add back in

the  "environment script plugin" but I am worried it will break my heavily used production jenkins jobs.  What is the best way to remove Multi-Branch plugin without breaking jobs or jenkins.  Please advise. What is the name of the Can I just remove multi-branch-project-plugin .jpi file and it's folder structure ?  Thanks!

Add Comment

[jesper.klit.jensen@intelligrated.com (JIRA)]

Jesper Jensen commented on JENKINS-36896

Re: Stack Trace on startup

The plugin is multi-branch-project-plugin.jpi create a file next to it called multi-branch-project-plugin.jpi.disabled (just empty) restart Jenkins and the plugin is now disabled and can now uninstall it using Jenkins normal tools.

Add Comment

[jesper.klit.jensen@intelligrated.com (JIRA)]

Jesper Jensen edited a comment on JENKINS-36896

Re: Stack Trace on startup

The plugin is multi-branch-project-plugin.jpi create a file next to it called multi-branch-project-plugin.jpi.disabled (just empty) restart Jenkins and the plugin is now disabled and can now uninstall it using Jenkins normal tools.

See https://wiki.jenkins-ci.org/display/JENKINS/Removing+and+disabling+plugins

Add Comment

[idigwww@yahoo.com (JIRA)]

D S updated an issue

Jenkins / JENKINS-36896 Stack Trace on startup

Change By: D S Attachment: crash.jpg

Add Comment

[idigwww@yahoo.com (JIRA)]

D S commented on JENKINS-36896

Re: Stack Trace on startup

Thanks for your help. I did that but when I click on configure for any job even a Freestyle project I get at crash. Please see attached trace. It would not let me copy and paste the error . So I had to screenshot it. Basically I got a jellyTagExpection on the plugin even though I had a multi-branch-project-plugin.jpi.disabled file. I don't have maven projects but maybe maven plugins depend on it? I am not sure what is going on. Wish we never installed this plugin. Thanks again for your help! We are non profit doing good things so it is very appreciated.

Add Comment

[jesper.klit.jensen@intelligrated.com (JIRA)]

Jesper Jensen commented on JENKINS-36896

Re: Stack Trace on startup

You might want to try to find another env plugin and install this one Environment Injector Plugin, that might solve some of your problems.

Add Comment

[vishnu.kumar7@gmail.com (JIRA)]

Vishnu Kumar commented on JENKINS-36896

Re: Stack Trace on startup

It is indeed a blocker. the latest version of environment script plugin has multi-branch as a dependency thus it will impact all the instances of Jenkins with Environment script plugin installed. I had first disabled the mult-branch plugin and then uninstalled Environment script plugin + multi-branch plugin. Later i downloaded version 1.2.2 of environment script and installed it manually. i hope this work around will work.

Add Comment

[idigwww@yahoo.com (JIRA)]

D S commented on JENKINS-36896

Re: Stack Trace on startup

Thanks. That worked!! What a life saver. The last step was to remove all the bad links for things like lastStable in all my jobs. You can delete the symlink for lastStable and it will just get created again the next time you run the job. No more crashes.

Add Comment

[dirk.heinrichs@recommind.com (JIRA)]

Dirk Heinrichs commented on JENKINS-36896

Re: Stack Trace on startup

"No more crashes" is not true. It will crash again upon next Jenkins restart. Any chance to get this fixed?

Add Comment

[petr.lefner@aponia.com (JIRA)]

Pe Le commented on JENKINS-36896

Re: Stack Trace on startup

Updating Jenkins can be real pain. @Dirk, can you confirm that you experienced the crash after restart without previous update of Jenkins or its plugins? I have restarted my Jenkins just recently, maybe more times since applying the workaround suggested by @Vishnu Kuzmar and the problem did not occur again. Of course, I would eventually welcome fix to this in order to get rid of the workaround.

Add Comment

[petr.lefner@aponia.com (JIRA)]

Pe Le edited a comment on JENKINS-36896

Re: Stack Trace on startup

Updating Jenkins can be real pain. @Dirk [~dhs] , can you confirm that you experienced the crash after restart without previous update of Jenkins or its plugins? I have restarted my Jenkins just recently, maybe more times since applying the workaround suggested by @Vishnu Kuzmar [~vishnukumar7]mar and the problem did not occur again.

Of course, I would eventually welcome fix to this in order to get rid of the workaround.

Add Comment

[petr.lefner@aponia.com (JIRA)]

Pe Le edited a comment on JENKINS-36896

Re: Stack Trace on startup

Updating Jenkins can be real pain. [~dhs], can you confirm that you experienced the crash after restart without previous update of Jenkins or its plugins? I have restarted my Jenkins just recently, maybe more times since applying the workaround suggested by [~vishnukumar7] mar and the problem did not occur again.

Of course, I would eventually welcome fix to this in order to get rid of the workaround.

Add Comment

[dirk.heinrichs@recommind.com (JIRA)]

Dirk Heinrichs commented on JENKINS-36896

Re: Stack Trace on startup

I just deleted the symlinks as D S proposed. The fix to this was already given above, by Frank Racis, wasn't it?

Add Comment

[dirk.heinrichs@recommind.com (JIRA)]

Dirk Heinrichs edited a comment on JENKINS-36896

Re: Stack Trace on startup

Yes, but I just deleted the symlinks as [~heropurpose] proposed. The fix to this was already given above, by [~fwr], wasn't it?

Add Comment

[dirk.heinrichs@recommind.com (JIRA)]

Dirk Heinrichs commented on JENKINS-36896

Re: Stack Trace on startup

Did another restart and this time it came up fine.

Add Comment

[dirk.heinrichs@recommind.com (JIRA)]

Dirk Heinrichs edited a comment on JENKINS-36896

Re: Stack Trace on startup

Did another restart and this time it came up fine. But in the meantime I also did a Jenkins update from 2.19.4 to 2.32.1 (this is a playground environment anyway).

Add Comment

[petr.lefner@aponia.com (JIRA)]

Pe Le commented on JENKINS-36896

Re: Stack Trace on startup

Well I can confirm that only removing the invalid links does not help, the crash can occur again. So I think one has to either apply the patch provided above or do the workaround (and make sure to remove the invalid links to clean the mess).

Add Comment

[mjdetullio@gmail.com (JIRA)]

Matthew DeTullio resolved as Fixed

Fixed in version 0.6

Jenkins / JENKINS-36896 Stack Trace on startup

Change By: Matthew DeTullio Status: Open Resolved Resolution: Fixed

Add Comment

[petr.lefner@aponia.com (JIRA)]

Pe Le commented on JENKINS-36896

Re: Stack Trace on startup

Just updated the Multi Branch Project plugin to 0.6, the Environment Script plugin to 1.2.5; I had to get rid of one (possibly forgotten) invalid symlink from one of the folders to fix the crash which occured again. Now Jenkins runs, will see if any ongoing builds will keep it.

Add Comment

[petr.lefner@aponia.com (JIRA)]

Pe Le edited a comment on JENKINS-36896

Re: Stack Trace on startup

Just updated the Multi Branch Project plugin to 0.6, the Environment Script plugin to 1.2.5; I had to get rid of one (possibly forgotten) invalid symlink from one of the folders to fix the crash which occured again. Now Jenkins runs fine , will see if any ongoing builds will keep it.

Add Comment

[idigwww@yahoo.com (JIRA)]

D S commented on JENKINS-36896

Re: Stack Trace on startup

Hi, Just to be clear I followed the steps above to remove the plugin clearing the symlinks is the step after removing the plugin

Add Comment

This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)