[JIRA] [git-plugin] (JENKINS-24454) Windows GIT SCM fetch code hung

maximindsl@gmail.com (JIRA)

unread,

May 26, 2015, 2:31:03 AM5/26/15

to jenkinsc...@googlegroups.com

Maximin Das S L commented on

JENKINS-24454

Re: Windows GIT SCM fetch code hung

Getting the same error frequently.

Jenkins - 1.574
Git Plugin - 2.2.7
git version 1.9.5.msysgit.1

While the fetch is stuck, from the Process Explorer it could be seen that the ssh.exe is stuck on the command ssh g...@github.faked.com "git-upload-pack 'XYZ/Faked.git'"

Below is from Process Explorer.

 
                                                    jenkins.exe
  java.exe
    git.exe
      git.exe
        ssh.exe // this one is stuck
 
                                                

While the process is stuck, executing the command ssh g...@github.faked.com "git-upload-pack 'XYZ/Faked.git'" from command line gives the response which ends with

 
                                                    .........
005467bd6f492ad36325aea516dfc2f423b1bc5e8dfe refs/tags/branch1
0057747b9750f2389c6ca630480674a85e1decad2387 refs/tags/branch1^{}
0000
Connection to github.faked.com closed by remote host.
 
                                                

From the dump which generated while the process was hung,

 
                                                    STACK_TEXT:  
0028d53c 74ee15f7 00000002 0028d58c 00000001 ntdll!NtWaitForMultipleObjects+0x15
0028d5d8 76741a0c 0028d58c 0028d600 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x100
0028d620 767441f0 00000002 7efde000 00000000 kernel32!WaitForMultipleObjectsExImplementation+0xe0
0028d63c 68015424 00000002 0028d694 00000000 kernel32!WaitForMultipleObjects+0x18
 
                                                

The last control flow was to ntdll!NtWaitForMultipleObjects. From the name of the thread it seems like it is waiting for some resources, which is not known at this point.

Any ideas on how to fix this or workarounds which is working?

Add Comment

This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)

mark.earl.waite@gmail.com (JIRA)

unread,

May 26, 2015, 8:22:01 AM5/26/15

to jenkinsc...@googlegroups.com

Mark Waite commented on

JENKINS-24454

Re: Windows GIT SCM fetch code hung

Maximin I'm afraid that I have no ideas to offer. You're running a recent version of msysgit (1.9.5), which contains (as far as I know) a recent version of ssh.

You could try switching to JGit instead of using command line git. There are some use cases which the JGit implementation in the plugin does not support (submodules, pushing tags, and several others), but for simple use cases the JGit implementation is sufficient. You may find that the age of your Jenkins installation (Jenkins 1.574 is now about 2 years old) and the version of the git plugin (2.2.x has been replaced by the 2.3.x series) may be too old to have the most recent JGit implementation fixes, but you could try JGit to see if it resolves your issue.

Add Comment

jake.cobb@gmail.com (JIRA)

unread,

Jul 7, 2015, 10:45:01 AM7/7/15

to jenkinsc...@googlegroups.com

Jake Cobb commented on

JENKINS-24454

Re: Windows GIT SCM fetch code hung

We're seeing this same problem with:

Jenkins - 1.617
Git Plugin - 2.3.5
git 1.9.0.msysgit.0

We started getting this problem after upgrading from a quite old version of the Git Plugin - 1.4.0, which we were using with the same version of git on the windows slave (1.9.0.mysysgit.0).

We see mostly the same behavior Maximin described. We do not get the error about the .ssh directory mentioned in the original description here.

Running the git command spawned by the Jenkins slave manually in a git bash shell in the workspace works every time without delay, regardless of whether or not other jobs are hung on it in the same slave. I did this by copying the command line from process explorer on the hung git command and just pasting it in, so it's exactly the same.

Running just the ssh command gives a response but hangs, the remote end does not close the connection:

 
                                                    003c29363ef2df43efb9d3e517e6f78fc7bda2f46f7e refs/tags/help
0000

However, this behavior should be fine by the Git protocol, the 0000 indicates the end of message.

I wonder if there could be a change in input/output buffering when git is run by another process and this is causing some communication deadlock. We can't reproduce this behavior with git alone (version unchanged) and never saw it with the old Git Plugin version.

Add Comment

mark.earl.waite@gmail.com (JIRA)

unread,

Jul 7, 2015, 2:40:02 PM7/7/15

to jenkinsc...@googlegroups.com

Mark Waite commented on

JENKINS-24454

Re: Windows GIT SCM fetch code hung

Jake Cobb I doubt there is a change of input/output buffering when git is run by another process, but you'd need to investigate the git source code to decide that for sure.

If you're running your Windows slave as a Windows service, then you'll have real difficulty interactively duplicating the environment where the git process runs. You could try running that process from inside a Jenkins job (using a Windows Batch job step, for instance) to see if the same good behavior exists when the git program is run inside a Jenkins job.

You might also consider updating from msysgit 1.9.0 to the most recent 1.9.5 version. I don't know that it will fix your problem, but there are several useful fixes in the intervening releases between what you're running and the latest version. Among other things, the version of OpenSSH was upgraded between those two versions so that "git clone" using an ssh protocol URL is no longer limited to 1 MB / second download.