Jenkins Job hanging

[Bryan Adams]

I am having an issue with a Job hanging.

The job has two build steps to Execute shell script on remote host using ssh. It connects to the first server, executes all the commands and disconnects and connects to the next build step which is the next Execute shell script on remote host using ssh. It executes all the commands but instead of disconnecting and finishing the build it is hanging. 

I cannot figure out why it is hanging. I tried changing the order of these build steps, putting the one that was hanging before the one that wasn't and it still hangs so it doesn't seem to be the order of the build steps.

Has anyone else experienced this?

Thanks in advance for your time.

-Bryan

[Ben Slote]

Were you ever able to resolve this issue? My jenkins install started behaving in a similar manner a few days ago. Git checkouts/clones/fetches all seem to hang, as do any shell script executions.

[Victor Martinez]

Do you mean you run an ssh command within your shell build action? If so, can you add -v flags?

There is already a good plugin to debug those java memory stacktraces and You can see those java processes, threads: https://wiki.jenkins-ci.org/display/JENKINS/Monitoring

[Ben Slote]

No, I'm not running an ssh command. In some of my jobs I execute a python script that modifies some files and pushes the modifications to a git repo. Upon upgrading some git plugins and bumping Jenkins to 1.626, jenkins never continues with the job's build steps after executing the python script. The same thing happens when jobs are kicked off and they pull in changes from git.. it just hangs even after it's finished fetching or cloning, etc. It's like Jenkins is unaware the git or the script has exited so it never moves on to other build steps.

There is nothing interesting showing up in the jenkins system logs, no stack traces or anything, but I'll take a look at that plugin. Thanks.

On Thu, Aug 27, 2015 at 2:47 PM, Victor Martinez <victormar...@gmail.com> wrote:

Do you mean you run an ssh command within your shell build action? If so, can you add -v flags?

There is already a good plugin to debug those java memory stacktraces and You can see those java processes, threads: https://wiki.jenkins-ci.org/display/JENKINS/Monitoring

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/cE7DjySXdtg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/8f46d342-a2d3-491f-aeaf-ae1ee8be37be%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Daniel Beck]

On 27.08.2015, at 23:47, Victor Martinez <victormar...@gmail.com> wrote:

> There is already a good plugin to debug those java memory stacktraces and You can see those java processes, threads: https://wiki.jenkins-ci.org/display/JENKINS/Monitoring

Thread dumps are available without any plugin in Jenkins on the /threadDump URL.

[Ben Slote]

Thread dumps look like this when it hangs when executing a shell build step:

Executor #0 for master : executing <my job> #23" Id=40 Group=main WAITING on java.lang.UNIXProcess@71cad1e6
	at java.lang.Object.wait(Native Method)
	-  waiting on java.lang.UNIXProcess@71cad1e6
	at java.lang.Object.wait(Object.java:503)
	at java.lang.UNIXProcess.waitFor(UNIXProcess.java:261)
	at hudson.Proc$LocalProc.join(Proc.java:319)
	at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:137)
	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97)
	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
	at hudson.model.Build$BuildExecution.build(Build.java:205)
	at hudson.model.Build$BuildExecution.doRun(Build.java:162)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:537)
	at hudson.model.Run.execute(Run.java:1741)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:381)

And here's another case where it hangs trying to clone a repo:

"Executor #3 for master : executing <my other job> #1" Id=43 Group=main WAITING on java.lang.UNIXProcess@67150794
	at java.lang.Object.wait(Native Method)
	-  waiting on java.lang.UNIXProcess@67150794
	at java.lang.Object.wait(Object.java:503)
	at java.lang.UNIXProcess.waitFor(UNIXProcess.java:261)
	at hudson.Proc$LocalProc.join(Proc.java:319)
	at hudson.Proc.joinWithTimeout(Proc.java:168)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1587)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1379)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:86)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:324)
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:733)
	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:983)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1016)
	at hudson.scm.SCM.checkout(SCM.java:485)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1284)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:610)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:532)
	at hudson.model.Run.execute(Run.java:1741)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:381)

I've tried downgrading Jenkins (now at 1.622) and all git and github related plugins to the versions I have on another jenkins instance that is working properly and I'm still getting these hangs. Both installs are on mac minis running osx 10.10.5. Java 1.7.

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/cE7DjySXdtg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/69028AF1-B8F1-456D-BAB1-E1ECF0C76743%40beckweb.net.