Multiple Git.exe processes running on my build server even when there are no builds going

[S Day]

I was wondering if you had a few minutes to check out this problem I seem to be experiencing with the Jenkings Github plugin..

 As you can see I have many instances of git.exe running however…

 I have nothing building, nothing pulling from git, or anything

 It appears nothing is closing gracefully, is this a known issue or how the software is supposed to run or something wrong with my Jenkins configuration.

 Thanks for taking the time to read this 

[Mark Waite]

How long have the processes been running?  

If they are short-lived processes, they might be the result of many jobs polling concurrently with small polling intervals (like once every minute).  If you are polling that frequently, you should consider the "polling must die" post from Kohsuke Kawaguchi for a faster and lighter technique to accomplish the same result.

If they are long-lived processes, they might be the result of git.exe blocking while prompting for authentication.  Be sure you're running the latest version of the git-client-plugin and the git-plugin to reduce the chances of blocking for authentication.

Since other jobs are not running, and since a checkout will not complete until the git process exits, I assume they are not the "remnants" of a git checkout.  You might be able to better guess the purpose of those processes if you can list the arguments to the command.  Refer to http://windowsxp.mvps.org/listproc.htm for a suggestion of a way you can list the arguments passed to a Windows process.

Mark Waite

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Thanks!

Mark Waite

[S Day]

Hey Mark, thanks so much for your reply, I will read this blogpost:

http://kohsuke.org/2011/12/01/polling-must-die-triggering-jenkins-builds-from-a-git-hook/

and check the time on the processes. We have the latest version of gitplugin and we're on 19 looks 191 came out 2 weeks ago so i'll update that as well.

I will also follow up thanks again!

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/38yQEn4izNM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.

[S Day]

it sounds like this is on the right track...

the problem we're having is when we have too many ssh,git.exe,andconhosts running we're unable to fetch from git

Building on master in workspace J:\jobs\Jobname\workspace

Fetching changes from the remote Git repository

Fetching upstream changes from ssh://g...@github.com/Name/Jobname.git

ERROR: Timeout after 10 minutes

FATAL: Failed to fetch from ssh://g...@github.com/Name/jobname.git

hudson.plugins.git.GitException: Failed to fetch from ssh://g...@github.com/Name/JobName.git

at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:621)

at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:853)

at hudson.plugins.git.GitSCM.checkout(GitSCM.java:878)

at hudson.model.AbstractProject.checkout(AbstractProject.java:1411)

at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:652)

at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)

at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:561)

at hudson.model.Run.execute(Run.java:1665)

at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)

at hudson.model.ResourceController.execute(ResourceController.java:88)

at hudson.model.Executor.run(Executor.java:246)

Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress ssh://g...@github.com/Name/JobName.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:

stdout: 

stderr: Could not create directory 'c/Users/GithubUserName/.ssh'.

fatal: Could not read from remote repository.

Please make sure you have the correct access rights

and the repository exists.

at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1192)

at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1062)

at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$200(CliGitAPIImpl.java:89)

at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:222)

at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:619)

... 10 more

however, that article is not very clear, where do I use that curl command? 

[Mark Waite]

I think the core problem is the "Could not create directory c/Users/GitHubUserName/.ssh".  Somehow, there is a permissions problem attempting to create that directory.

As one guess, maybe you're running the Jenkins server on Windows as a service, and the service user does not have permission to create that directory?

As another guess, maybe there is something incorrect in your git configuration within Jenkins.  I expected "/c/Users/GitHubUserName" rather than "c/Users/GitHubUserName/.ssh", since I assumed the path would be absolute rather than relative.

Mark Waite

On Mon, Mar 31, 2014 at 10:22 AM, S Day <zilla...@gmail.com> wrote:

it sounds like this is on the right track...

the problem we're having is when we have too many ssh,git.exe,andconhosts running we're unable to fetch from git

Building on master in workspace J:\jobs\Jobname\workspace

Fetching changes from the remote Git repository

Fetching upstream changes from ssh://g...@github.com/Name/Jobname.git

ERROR: Timeout after 10 minutes

FATAL: Failed to fetch from ssh://g...@github.com/Name/SiriusDecisions.git

--

You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Thanks!

Mark Waite

[S Day]

currently everything is building fine now after I shut down the multiple git, ssh and conhost .exe's.  however once it balloons up again I know i'm going to have that issue. 

The service account has full write access to that folder currently and that folder already exists so the reason it can't create the directory is because it's already there.

I did lower the number of build executors so hopefully that will slow the mutiple exe's spawning too quickly. 

For my git configuration in jenkins I only have 2 parameters. 

NAME: (git)

Path to git executable (C:\Program Files (x86)\Git\cmd\git.exe)

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/38yQEn4izNM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.

[Mark Waite]

You should use the git.exe from the bin directory, not from the cmd directory.

[S Day]

Hey!!! Just a follow up, I've decreased my polling time significantly and used a h/ format to stagger them.

I've also implemented the post commit jenkins service webhook built into github's servers and this significantly dropped down the number of git.exe and ssh.exe processes spawning. 

I may revisit Kohsuke Kawaguchi article and suggest some updates for it since it appears git had made some improvements since he wrote it back!

thanks very much for you help Mark, I think we can consider this closed. 

On Monday, March 31, 2014 1:10:38 PM UTC-4, Mark Waite wrote:

You should use the git.exe from the bin directory, not from the cmd directory.

On Mar 31, 2014 11:08 AM, "S Day" <zilla...@gmail.com> wrote:

currently everything is building fine now after I shut down the multiple git, ssh and conhost .exe's.  however once it balloons up again I know i'm going to have that issue. 

The service account has full write access to that folder currently and that folder already exists so the reason it can't create the directory is because it's already there.

I did lower the number of build executors so hopefully that will slow the mutiple exe's spawning too quickly. 

For my git configuration in jenkins I only have 2 parameters. 

NAME: (git)

Path to git executable (C:\Program Files (x86)\Git\cmd\git.exe)

On Mon, Mar 31, 2014 at 12:53 PM, Mark Waite <mark.ea...@gmail.com> wrote:

I think the core problem is the "Could not create directory c/Users/GitHubUserName/.ssh".  Somehow, there is a permissions problem attempting to create that directory.

As one guess, maybe you're running the Jenkins server on Windows as a service, and the service user does not have permission to create that directory?

As another guess, maybe there is something incorrect in your git configuration within Jenkins.  I expected "/c/Users/GitHubUserName" rather than "c/Users/GitHubUserName/.ssh", since I assumed the path would be absolute rather than relative.

Mark Waite

On Mon, Mar 31, 2014 at 10:22 AM, S Day <zilla...@gmail.com> wrote:

it sounds like this is on the right track...

the problem we're having is when we have too many ssh,git.exe,andconhosts running we're unable to fetch from git

Building on master in workspace J:\jobs\Jobname\workspace

Fetching changes from the remote Git repository

Fetching upstream changes from ssh://g...@github.com/Name/Jobname.git

ERROR: Timeout after 10 minutes

FATAL: Failed to fetch from ssh://g...@github.com/Name/jobname.git

--

Thanks!

Mark Waite