[JIRA] (JENKINS-52935) Git LFS failing clone

Issue Type:	Bug
Assignee:	Mark Waite
Components:	git-client-plugin, git-plugin
Created:	2018-08-08 14:20
Environment:	Jenkins 2.121.2 Git client plugin 2.7.3 Git plugin 3.9.1
Priority:	Major
Reporter:	julien staub

git clone fails when there are LFS files although the clone is working on repos without LFS file.
The clone is working fine when using command line manually. I am unable to reproduce the issue outside of Jenkins.

 
                                                                [EnvInject] - Loading node environment variables.[EnvInject] - Loading node environment variables.Building in workspace C:\Program Files (x86)\Jenkins\sharedspace > git.exe rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git.exe config remote.origin.url g...@github.xxxxxxxxxxxxxx.com:projects/FW_Toolchain.git # timeout=10Fetching upstream changes from g...@github.xxxxxxxxxxxxxx.com:projects/FW_Toolchain.git > git.exe --version # timeout=10using GIT_SSH to set credentials ssh key for github > git.exe fetch --tags --progress g...@github.xxxxxxxxxxxxxx.com:projects/FW_Toolchain.git +refs/heads/*:refs/remotes/origin/* > git.exe rev-parse "refs/remotes/origin/master^{commit}" # timeout=10 > git.exe rev-parse "refs/remotes/origin/origin/master^{commit}" # timeout=10Checking out Revision 1bdf338d926888805c171ed19fc3088bc110b6cb (refs/remotes/origin/master) > git.exe config core.sparsecheckout # timeout=10 > git.exe checkout -f 1bdf338d926888805c171ed19fc3088bc110b6cbFATAL: Could not checkout 1bdf338d926888805c171ed19fc3088bc110b6cbhudson.plugins.git.GitException: Command "git.exe checkout -f 1bdf338d926888805c171ed19fc3088bc110b6cb" returned status code 128:stdout: stderr: warning: unable to rmdir 'src/ImagerAlgos/externals': Directory not emptyDownloading eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack (127 MB)Error downloading object: eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack (d283580): Smudge error: Error downloading eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack (d2835802804ebf0401a41c2676eb7c119cbf8af7550c7fc2f291ea317bfc7dee): batch request: g...@github.xxxxxxxxxxxxxx.com: Permission denied (publickey).: exit status 255
Errors logged to C:\Program Files (x86)\Jenkins\sharedspace\.git\lfs\logs\20180807T084736.9187151.logUse `git lfs logs last` to view the log.error: external filter 'git-lfs filter-process' failedfatal: eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack: smudge filter lfs failed
 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2016) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$800(CliGitAPIImpl.java:72) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:2315)Caused: hudson.plugins.git.GitException: Could not checkout 1bdf338d926888805c171ed19fc3088bc110b6cb at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:2339) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1236) at hudson.scm.SCM.checkout(SCM.java:504) at hudson.model.AbstractProject.checkout(AbstractProject.java:1208) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1798) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429)Finished: FAILURE 
                                                            

This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396)

mark.earl.waite@gmail.com (JIRA)

unread,

Aug 8, 2018, 10:36:01 AM8/8/18

to jenkinsc...@googlegroups.com

Mark Waite updated an issue

Jenkins /

Change By:	Mark Waite

git clone fails when there are LFS files although the clone is working on repos without LFS file.
The clone is working fine when using command line manually. I am unable to reproduce the issue outside of Jenkins.

{ code:java noformat }

[EnvInject] - Loading node environment variables.[EnvInject] - Loading node environment variables.Building in workspace C:\Program Files (x86)\Jenkins\sharedspace

> git.exe rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository

> git.exe config remote.origin.url g...@github.xxxxxxxxxxxxxx.com:projects/FW_Toolchain.git # timeout= 10Fetching 10
Fetching upstream changes from g...@github.xxxxxxxxxxxxxx.com:projects/FW_Toolchain.git
> git.exe --version # timeout= 10using 10
using GIT_SSH to set credentials ssh key for github

> git.exe fetch --tags --progress g...@github.xxxxxxxxxxxxxx.com:projects/FW_Toolchain.git +refs/heads/*:refs/remotes/origin/*
> git.exe rev-parse "refs/remotes/origin/master^{commit}" # timeout=10

> git.exe rev-parse "refs/remotes/origin/origin/master^{commit}" # timeout= 10Checking 10
Checking out Revision 1bdf338d926888805c171ed19fc3088bc110b6cb (refs/remotes/origin/master)

> git.exe config core.sparsecheckout # timeout=10

> git.exe checkout -f 1bdf338d926888805c171ed19fc3088bc110b6cbFATAL 1bdf338d926888805c171ed19fc3088bc110b6cb
FATAL : Could not checkout 1bdf338d926888805c171ed19fc3088bc110b6cbhudson 1bdf338d926888805c171ed19fc3088bc110b6cb
hudson .plugins.git.GitException: Command "git.exe checkout -f 1bdf338d926888805c171ed19fc3088bc110b6cb" returned status code 128:

stdout:
stderr: warning: unable to rmdir 'src/ImagerAlgos/externals': Directory not emptyDownloading eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack (127 MB)Error downloading object: eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack (d283580): Smudge error: Error downloading eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack (d2835802804ebf0401a41c2676eb7c119cbf8af7550c7fc2f291ea317bfc7dee):

batch request: g...@github.xxxxxxxxxxxxxx.com: Permission denied (publickey).: exit status 255

Errors logged to C:\Program Files (x86)\Jenkins\sharedspace\.git\lfs\logs\20180807T084736.9187151. logUse log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failedfatal failed
fatal : eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack: smudge filter lfs failed
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2016)

at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$800(CliGitAPIImpl.java:72)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:2315)
Caused: hudson.plugins.git.GitException: Could not checkout 1bdf338d926888805c171ed19fc3088bc110b6cb
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:2339)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1236) at hudson.scm.SCM.checkout(SCM.java:504)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1798)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)Finished: FAILURE

{ code noformat }

mark.earl.waite@gmail.com (JIRA)

unread,

Aug 8, 2018, 10:37:02 AM8/8/18

to jenkinsc...@googlegroups.com

Mark Waite assigned an issue to Unassigned

Jenkins /

Change By:	Mark Waite
Assignee:	Mark Waite

mark.earl.waite@gmail.com (JIRA)

unread,

Aug 8, 2018, 10:37:02 AM8/8/18

to jenkinsc...@googlegroups.com

Mark Waite updated an issue

Jenkins /

Change By:	Mark Waite

git clone fails when there are LFS files although the clone is working on repos without LFS file.
The clone is working fine when using command line manually. I am unable to reproduce the issue outside of Jenkins.

{noformat}

[EnvInject] - Loading node environment variables.

Building in workspace C:\Program Files (x86)\Jenkins\sharedspace
> git.exe rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository
> git.exe config remote.origin.url g...@github.xxxxxxxxxxxxxx.com:projects/FW_Toolchain.git # timeout=10

Fetching upstream changes from g...@github.xxxxxxxxxxxxxx.com:projects/FW_Toolchain.git
> git.exe --version # timeout=10

using GIT_SSH to set credentials ssh key for github
> git.exe fetch --tags --progress g...@github.xxxxxxxxxxxxxx.com:projects/FW_Toolchain.git +refs/heads/*:refs/remotes/origin/*
> git.exe rev-parse "refs/remotes/origin/master^{commit}" # timeout=10
> git.exe rev-parse "refs/remotes/origin/origin/master^{commit}" # timeout=10

Checking out Revision 1bdf338d926888805c171ed19fc3088bc110b6cb (refs/remotes/origin/master)
> git.exe config core.sparsecheckout # timeout=10

> git.exe checkout -f 1bdf338d926888805c171ed19fc3088bc110b6cb

FATAL: Could not checkout 1bdf338d926888805c171ed19fc3088bc110b6cb

hudson.plugins.git.GitException: Command "git.exe checkout -f 1bdf338d926888805c171ed19fc3088bc110b6cb" returned status code 128:

stdout:
stderr: warning: unable to rmdir 'src/ImagerAlgos/externals': Directory not emptyDownloading eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack (127 MB)Error downloading object: eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack (d283580): Smudge error: Error downloading eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack (d2835802804ebf0401a41c2676eb7c119cbf8af7550c7fc2f291ea317bfc7dee):
batch request: g...@github.xxxxxxxxxxxxxx.com: Permission denied (publickey).: exit status 255
Errors logged to C:\Program Files (x86)\Jenkins\sharedspace\.git\lfs\logs\20180807T084736.9187151.log

Use `git lfs logs last` to view the log.

error: external filter 'git-lfs filter-process' failed

fatal: eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack: smudge filter lfs failed
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2016)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$800(CliGitAPIImpl.java:72)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:2315)
Caused: hudson.plugins.git.GitException: Could not checkout 1bdf338d926888805c171ed19fc3088bc110b6cb
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:2339)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1236) at hudson.scm.SCM.checkout(SCM.java:504)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1798)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)Finished: FAILURE

{noformat}

mark.earl.waite@gmail.com (JIRA)

unread,

Aug 8, 2018, 10:38:03 AM8/8/18

to jenkinsc...@googlegroups.com

stderr: warning: unable to rmdir 'src/ImagerAlgos/externals': Directory not emptyDownloading empty
Downloading eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack (127 MB)

Error downloading object: eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack (d283580):
Smudge error:
Error downloading eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack (d2835802804ebf0401a41c2676eb7c119cbf8af7550c7fc2f291ea317bfc7dee):
batch request: g...@github.xxxxxxxxxxxxxx.com: Permission denied (publickey).: exit status 255
Errors logged to C:\Program Files (x86)\Jenkins\sharedspace\.git\lfs\logs\20180807T084736.9187151.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack: smudge filter lfs failed
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2016)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$800(CliGitAPIImpl.java:72)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:2315)
Caused: hudson.plugins.git.GitException: Could not checkout 1bdf338d926888805c171ed19fc3088bc110b6cb
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:2339)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1236) at hudson.scm.SCM.checkout(SCM.java:504)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1798)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)Finished: FAILURE
{noformat}

mark.earl.waite@gmail.com (JIRA)

unread,

Aug 8, 2018, 10:53:02 AM8/8/18

to jenkinsc...@googlegroups.com

stderr: warning: unable to rmdir 'src/ImagerAlgos/externals': Directory not empty

mark.earl.waite@gmail.com (JIRA)

unread,

Aug 8, 2018, 10:55:01 AM8/8/18

to jenkinsc...@googlegroups.com

Mark Waite updated an issue

Jenkins /

Change By:	Mark Waite

git clone fails when there are LFS files although the clone is working on repos without LFS file.
The clone is working fine when using command line manually. I am unable to reproduce the issue outside of Jenkins.

{noformat}
[EnvInject] - Loading node environment variables.
[EnvInject] - Loading node environment variables.
Building in workspace C:\Program Files (x86)\Jenkins\sharedspace

> git.exe rev-parse --is-inside-work-tree # timeout= 10Fetching 10
Fetching changes from the remote Git repository

mark.earl.waite@gmail.com (JIRA)

unread,

Aug 8, 2018, 10:58:01 AM8/8/18

to jenkinsc...@googlegroups.com

Mark Waite commented on

Sorry you've encountered a problem. I suspect that this is a configuration problem rather than a bug in the plugin. Many users successfully clone with git lfs to Windows machines very often.

Some things you might try in order to diagnose the configuration issue include:

Use a workspace defined by Jenkins rather than "C:\Program Files (x86)\Jenkins\sharedspace". Jenkins does not create workspaces with that type of name. I assume that means you've specifically defined the workspace instead of allowing Jenkins to manage the workspace. Let Jenkins manage the workspace so that there is less chance of some other program keeping files busy in the workspace. Since you're using Windows, there is a high risk that open files will block other programs from modifying the content of those files. For example, the message "unable to rmdir ... Directory not empty" is a likely hint that some other program has that directory busy or a file in that directory busy
Use a workspace that does not include a space in the path to the workspace. Command line git uses command line ssh for ssh based authentication and there are cases where a space in the path to the workspace can break command line git authentication. Authentication can also be broken by spaces in the path for https based authentication
Move the system default workspace to a location which is not in the "Program Files" tree. Windows intends that "Program Files" contains programs, not the working data of programs
Use a system temporary directory (%TEMP%) which does not include spaces in its path
Run git lfs logs last from inside the workspace directory and include the output in the bug report in case there is something happening there

mark.earl.waite@gmail.com (JIRA)

unread,

Aug 8, 2018, 11:56:02 AM8/8/18

to jenkinsc...@googlegroups.com

Mark Waite updated an issue

Jenkins /

Change By:	Mark Waite
Component/s:	git-client-plugin

julien.staub@schneider-electric.com (JIRA)

unread,

Aug 9, 2018, 10:52:02 AM8/9/18

to jenkinsc...@googlegroups.com

julien staub commented on

Use a workspace defined by Jenkins rather than "C:\Program Files (x86)\Jenkins\sharedspace". Jenkins does not create workspaces with that type of name. I assume that means you've specifically defined the workspace instead of allowing Jenkins to manage the workspace. Let Jenkins manage the workspace so that there is less chance of some other program keeping files busy in the workspace. Since you're using Windows, there is a high risk that open files will block other programs from modifying the content of those files. For example, the message "unable to rmdir ... Directory not empty" is a likely hint that some other program has that directory busy or a file in that directory busy

I use Shared workspace plugin. Cleaning up I do not get the rmdir error anymore. But I confirm, default location is in form(C:\Program Files (x86)\Jenkins\workspace\githubToolchainLFScheckout)

This not solves the problem

Use a workspace that does not include a space in the path to the workspace. Command line git uses command line ssh for ssh based authentication and there are cases where a space in the path to the workspace can break command line git authentication. Authentication can also be broken by spaces in the path for https based authentication

As You suggesst I switch to a path without spaces (C:\githubToolchainLFScheckout)

Same error

Move the system default workspace to a location which is not in the "Program Files" tree. Windows intends that "Program Files" contains programs, not the working data of programs
Use a system temporary directory (%TEMP%) which does not include spaces in its path

I use Item specific workspace like (C:\githubToolchainLFScheckout)

Run git lfs logs last from inside the workspace directory and include the output in the bug report in case there is something happening there

content of the log

 
                                                                git-lfs/2.4.2 (GitHub; windows amd64; go 1.8.3; git 6f4b2e98)
git version 2.17.1.windows.2

$ git-lfs.exe filter-process
Error downloading object: eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack (d283580): Smudge error: Error downloading eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack (d2835802804ebf0401a41c2676eb7c119cbf8af7550c7fc2f291ea317bfc7dee): batch request: g...@github.schneider-electric.com: Permission denied (publickey).: exit status 255

exit status 255
g...@github.schneider-electric.com: Permission denied (publickey).
github.com/git-lfs/git-lfs/errors.newWrappedError
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/errors/types.go:170
github.com/git-lfs/git-lfs/errors.Wrap
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/errors/errors.go:74
github.com/git-lfs/git-lfs/lfsapi.(*Client).NewRequest
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/lfsapi/client.go:39
github.com/git-lfs/git-lfs/tq.(*tqClient).Batch
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/tq/api.go:60
github.com/git-lfs/git-lfs/tq.Batch
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/tq/api.go:44
github.com/git-lfs/git-lfs/tq.(*TransferQueue).enqueueAndCollectRetriesFor
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/tq/transfer_queue.go:432
github.com/git-lfs/git-lfs/tq.(*TransferQueue).collectBatches.func1
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/tq/transfer_queue.go:354
runtime.goexit
	C:/Go/src/runtime/asm_amd64.s:2197
batch request
github.com/git-lfs/git-lfs/errors.newWrappedError
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/errors/types.go:170
github.com/git-lfs/git-lfs/errors.Wrap
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/errors/errors.go:74
github.com/git-lfs/git-lfs/tq.(*tqClient).Batch
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/tq/api.go:62
github.com/git-lfs/git-lfs/tq.Batch
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/tq/api.go:44
github.com/git-lfs/git-lfs/tq.(*TransferQueue).enqueueAndCollectRetriesFor
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/tq/transfer_queue.go:432
github.com/git-lfs/git-lfs/tq.(*TransferQueue).collectBatches.func1
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/tq/transfer_queue.go:354
runtime.goexit
	C:/Go/src/runtime/asm_amd64.s:2197
Error downloading eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack (d2835802804ebf0401a41c2676eb7c119cbf8af7550c7fc2f291ea317bfc7dee)
github.com/git-lfs/git-lfs/errors.newWrappedError
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/errors/types.go:170
github.com/git-lfs/git-lfs/errors.Wrapf
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/errors/errors.go:85
github.com/git-lfs/git-lfs/lfs.(*GitFilter).downloadFile
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:98
github.com/git-lfs/git-lfs/lfs.(*GitFilter).Smudge
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:59
github.com/git-lfs/git-lfs/commands.smudge
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/commands/command_smudge.go:127
github.com/git-lfs/git-lfs/commands.filterCommand
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/commands/command_filter_process.go:118
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).execute
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:477
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).Execute
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:551
github.com/git-lfs/git-lfs/commands.Run
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/commands/run.go:77
main.main
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/git-lfs.go:35
runtime.main
	C:/Go/src/runtime/proc.go:185
runtime.goexit
	C:/Go/src/runtime/asm_amd64.s:2197
Smudge error
github.com/git-lfs/git-lfs/errors.newWrappedError
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/errors/types.go:170
github.com/git-lfs/git-lfs/errors.NewSmudgeError
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/errors/types.go:256
github.com/git-lfs/git-lfs/lfs.(*GitFilter).Smudge
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:68
github.com/git-lfs/git-lfs/commands.smudge
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/commands/command_smudge.go:127
github.com/git-lfs/git-lfs/commands.filterCommand
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/commands/command_filter_process.go:118
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).execute
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:477
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).Execute
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:551
github.com/git-lfs/git-lfs/commands.Run
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/commands/run.go:77
main.main
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/git-lfs.go:35
runtime.main
	C:/Go/src/runtime/proc.go:185
runtime.goexit
	C:/Go/src/runtime/asm_amd64.s:2197

Current time in UTC: 
2018-08-09 14:40:27

ENV:
LocalWorkingDir=C:\githubToolchainLFScheckout
LocalGitDir=C:\githubToolchainLFScheckout\.git
LocalGitStorageDir=C:\githubToolchainLFScheckout\.git
LocalMediaDir=C:\githubToolchainLFScheckout\.git\lfs\objects
LocalReferenceDir=
TempDir=C:\githubToolchainLFScheckout\.git\lfs\tmp
ConcurrentTransfers=3
TusTransfers=false
BasicTransfersOnly=false
SkipDownloadErrors=false
FetchRecentAlways=false
FetchRecentRefsDays=7
FetchRecentCommitsDays=0
FetchRecentRefsIncludeRemotes=true
PruneOffsetDays=3
PruneVerifyRemoteAlways=false
PruneRemoteName=origin
LfsStorageDir=C:\githubToolchainLFScheckout\.git\lfs
AccessDownload=none
AccessUpload=none
DownloadTransfers=basic
UploadTransfers=basic
GIT_ASKPASS=echo
GIT_BRANCH=origin/master
GIT_COMMIT=1bdf338d926888805c171ed19fc3088bc110b6cb
GIT_DIR=.git
GIT_EXEC_PATH=C:/Program Files/Git/mingw64/libexec/git-core
GIT_PREFIX=
GIT_PREVIOUS_COMMIT=1bdf338d926888805c171ed19fc3088bc110b6cb
GIT_URL=g...@github.schneider-electric.com:IoTSECprojects/FW_Toolchain.git

Client IP addresses:
fe80::388e:8370:63cf:9127 10.195.116.94

  
                                                            

mark.earl.waite@gmail.com (JIRA)

unread,

Aug 9, 2018, 11:04:02 AM8/9/18

to jenkinsc...@googlegroups.com

Mark Waite commented on

You might try cloning from an https URL instead of using an ssh URL. The error message from git LFS indicates that it is trying to download large files through the ssh protocol and is failing to authenticate with the ssh protocol.

The initial implementation of git LFS required https for download. It has been extended to operate with ssh URL's as well, though I don't know the details of how that works. For example, I am able to clone LFS repositories from github URL's which use SSH protocol. There may be some additional configuration required to allow your GitHub Enterprise installation to clone LFS repos over SSH.

You might try cloning from a GitHub repository which has LFS enabled, rather than cloning from your GitHub Enterprise server. That might help confirm if the issue is related to the GItHub Enterprise configuration or something else. The lts-with-plugins branch of my docker-lfs repository requires LFS if you want to use that as a test repository.

julien.staub@schneider-electric.com (JIRA)

unread,

Aug 10, 2018, 2:43:01 AM8/10/18

to jenkinsc...@googlegroups.com

julien staub commented on

I tried your github (not entreprise) repo with LFS.

https

 
                                                                [EnvInject] - Loading node environment variables.
Building in workspace C:\githubToolchainLFScheckout
 > git.exe rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git.exe config remote.origin.url https://github.com/MarkEWaite/docker-lfs.git # timeout=10
Fetching upstream changes from https://github.com/MarkEWaite/docker-lfs.git
 > git.exe --version # timeout=10
 > git.exe fetch --tags --progress https://github.com/MarkEWaite/docker-lfs.git +refs/heads/*:refs/remotes/origin/*
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from https://github.com/MarkEWaite/docker-lfs.git
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
	at hudson.scm.SCM.checkout(SCM.java:504)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
	at hudson.model.Run.execute(Run.java:1798)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.plugins.git.GitException: Command "git.exe fetch --tags --progress https://github.com/MarkEWaite/docker-lfs.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: fatal: unable to access 'https://github.com/MarkEWaite/docker-lfs.git/': Failed to connect to github.com port 443: Timed out

	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2016)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1735)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:420)
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
	... 11 more
ERROR: Error fetching remote repo 'origin'
Finished: FAILURE
 
                                                            

and ssh

 
                                                                [EnvInject] - Loading node environment variables.
Building in workspace C:\githubToolchainLFScheckout
 > git.exe rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git.exe config remote.origin.url g...@github.com:MarkEWaite/docker-lfs.git # timeout=10
Fetching upstream changes from g...@github.com:MarkEWaite/docker-lfs.git
 > git.exe --version # timeout=10
using GIT_SSH to set credentials ssh key for github
 > git.exe fetch --tags --progress g...@github.com:MarkEWaite/docker-lfs.git +refs/heads/*:refs/remotes/origin/*
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from g...@github.com:MarkEWaite/docker-lfs.git
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
	at hudson.scm.SCM.checkout(SCM.java:504)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
	at hudson.model.Run.execute(Run.java:1798)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.plugins.git.GitException: Command "git.exe fetch --tags --progress g...@github.com:MarkEWaite/docker-lfs.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: g...@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2016)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1735)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:420)
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
	... 11 more
ERROR: Error fetching remote repo 'origin'
Finished: FAILURE
 
                                                            

julien.staub@schneider-electric.com (JIRA)

unread,

Aug 10, 2018, 3:00:04 AM8/10/18

to jenkinsc...@googlegroups.com

julien staub commented on

Moreover, on my github entreprise, user+credential identification is not enabled, I need to use token, but was not able to configure this with Jenkins. If https usage is needed I will need some help to configure it.

mark.earl.waite@gmail.com (JIRA)

unread,

Aug 10, 2018, 8:43:02 AM8/10/18

to jenkinsc...@googlegroups.com

Mark Waite commented on

The https based failure to clone from github.com likely indicates that your company has a proxy configured and the proxy configuration was not used for the git clone command. A timeout trying to reach port 443 on the public internet most often means that the command line git program is unable to connect to the https port of the remote server and some network infrastructure (proxy, etc.) is intentionally blocking that connection.

The ssh based failure might indicate that you didn't configure the job to use an ssh credential, or the ssh credential you configured is not registered with github.com, or the same network configuration that is blocking access to https on the public internet is also blocking ssh access to the public internet.

The Jenkins git plugin supports two forms of credentials:

username/password for https and http connections
private key for ssh connections (g...@example.com:dir/repo.git and ssh://example.com/dir/repo.git)

I believe that I've used GitHub keys in the username/password case by creating a credential with MarkEWaite as the username and the deploy key as the password. You might try the same thing.

Using the git plugin with a typical secured git server without using credentials is quite difficult. It requires that every agent be configured to login to the secured git server by default. It is possible to do that, but it tends to make the management of Jenkins agents much more difficult. I assumed you are using Jenkins credentials rather than attempting to configure each agent so that it can login to the secured git server without credentials. Are you using Jenkins credentials?

mark.earl.waite@gmail.com (JIRA)

unread,

Aug 10, 2018, 10:06:02 AM8/10/18

to jenkinsc...@googlegroups.com

Mark Waite edited a comment on

The https based failure to clone from github.com likely indicates that your company has a proxy configured and the proxy configuration was not used for the git clone command. A timeout trying to reach port 443 on the public internet most often means that the command line git program is unable to connect to the https port of the remote server and some network infrastructure (proxy, etc.) is intentionally blocking that connection.

The ssh based failure might indicate that you didn't configure the job to use an ssh credential, or the ssh credential you configured is not registered with github.com, or the same network configuration that is blocking access to https on the public internet is also blocking ssh access to the public internet.

The Jenkins git plugin supports two forms of credentials:

# username/password for https and http connections
# private key for ssh connections (g...@example.com:dir/repo.git and ssh://example.com/dir/repo.git)

I assume when you say that you're required to use a token, that you mean you are using a private key. At least that is the format of the URL that is mentioned in the logs. If you're not using a private key and are instead using a GitHub deploy key or some other form of key, then the ssh-based URL like g...@example.com:organization/repo.git is not what you want.

I believe that I've used GitHub keys in the username/password case by creating a credential with MarkEWaite as the username and the deploy key as the password. You might try the same thing.

Using the git plugin with a typical secured git server without using credentials is quite difficult. It requires that every agent be configured to login to the secured git server by default. It is possible to do that, but it tends to make the management of Jenkins agents much more difficult. I assumed you are using Jenkins credentials rather than attempting to configure each agent so that it can login to the secured git server without credentials. Are you using Jenkins credentials?

julien.staub@schneider-electric.com (JIRA)

unread,

Aug 20, 2018, 6:57:04 AM8/20/18

to jenkinsc...@googlegroups.com

julien staub commented on

Sorry, I was not clear enough about the logging procedure I was using (in my head also).

GitHub Pull Request Builder is using OAuth token as "secret text" Jenkins credential and for git cloning I use Jenkins credentials with SSH private key.
I retried the clone of your repository because I forgot to add the SSH key on my github (not enterprise) account and result is SUCCESS.
I also retried SSH clone on my enterprise repository, but get an error as descried in 2nd comment.

mark.earl.waite@gmail.com (JIRA)

unread,

Aug 20, 2018, 9:32:02 AM8/20/18

to jenkinsc...@googlegroups.com

Mark Waite commented on

julien staub since it works for github.com, that seems to indicate that you can perform an LFS clone from the public internet to your agent. That's a good step.

Since it doesn't work for your GitHub enterprise, I assume that the GitHub enterprise administrators or the network administrators at your company will need to be involved in diagnosing the problem.

julien.staub@schneider-electric.com (JIRA)

unread,

Aug 20, 2018, 9:57:02 AM8/20/18

to jenkinsc...@googlegroups.com

julien staub commented on

Thank you. I will try this but I'm unable to reproduce the problem outside jenkins. In a console I am perfectly capable to clone the repo using SSH.

How can I reproduce the problem on a simple git bash command line interface ?

It seems the console output is not totally complete with what jenkins is doing. Here is the output when I try to execute the same commands as jenkins in a console:

 
                                                                me@computer MINGW64 /c/GIT/test
$ git --bare init
Initialized empty Git repository in C:/GIT/test/

me@computer MINGW64 /c/GIT/test (BARE:master)
$ git.exe rev-parse --is-inside-work-tree
false

me@computer MINGW64 /c/GIT/test (BARE:master)
$ git.exe config remote.origin.url g...@github.schneider-electric.com:IoTSECprojects/FW_Toolchain.git

me@computer MINGW64 /c/GIT/test (BARE:master)
$ git --version
git version 2.17.0.windows.1

me@computer MINGW64 /c/GIT/test (BARE:master)
$ git.exe fetch --tags --progress g...@github.schneider-electric.com:IoTSECprojects/FW_Toolchain.git +refs/heads/*:refs/remotes/origin/*
remote: Counting objects: 22811, done.
remote: Total 22811 (delta 0), reused 0 (delta 0), pack-reused 22811
Receiving objects: 100% (22811/22811), 861.32 MiB | 3.46 MiB/s, done.
Resolving deltas: 100% (14695/14695), done.
From github.schneider-electric.com:IoTSECprojects/FW_Toolchain
 * [new branch]        master     -> origin/master

me@computer MINGW64 /c/GIT/test (BARE:master)
$ git.exe rev-parse "refs/remotes/origin/master^{commit}"
1bdf338d926888805c171ed19fc3088bc110b6cb

me@computer MINGW64 /c/GIT/test (BARE:master)
$ git.exe rev-parse "refs/remotes/origin/origin/master^{commit}"
fatal: ambiguous argument 'refs/remotes/origin/origin/master^{commit}': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
refs/remotes/origin/origin/master^{commit}

me@computer MINGW64 /c/GIT/test (BARE:master)
$ git.exe config core.sparsecheckout

me@computer MINGW64 /c/GIT/test (BARE:master)
$ git.exe checkout -f 1bdf338d926888805c171ed19fc3088bc110b6cb
fatal: this operation must be run in a work tree


 
                                                            

mark.earl.waite@gmail.com (JIRA)

unread,

Aug 20, 2018, 10:26:01 AM8/20/18

to jenkinsc...@googlegroups.com

Mark Waite commented on

In a workspace, the plugin never uses:

> git --bare init

That creates a repository which has no work tree. Checkout fails if a repository does not include a work tree.

Use

> git init

mark.earl.waite@gmail.com (JIRA)

unread,

Aug 20, 2018, 10:27:01 AM8/20/18

to jenkinsc...@googlegroups.com

Mark Waite edited a comment on

In a workspace, the plugin never uses:

> {noformat}
git --bare init
{noformat}

That creates a repository which has no work tree. Checkout fails if a repository does not include a work tree.

Use

> {noformat}
git init
{noformat}

julien.staub@schneider-electric.com (JIRA)

unread,

Aug 20, 2018, 10:42:04 AM8/20/18

to jenkinsc...@googlegroups.com

julien staub commented on

Thank you again, this time I reproduced and may be able to contact our support but I have few hope on theyr side.

The error comes from "bad gateway" although a normal clone will just work fine

 
                                                                $ git lfs logs last
git-lfs/2.4.0 (GitHub; windows amd64; go 1.8.3; git 1a98fa5c)
git version 2.17.0.windows.1

$ git-lfs.exe filter-process
Error downloading object: eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack (d283580): Smudge error: Error downloading eclipse-cpp-mars-2-win32-x86_64/Packages/.cache/Keil.Kinetis_K20_DFP.1.3.0.pack (d2835802804ebf0401a41c2676eb7c119cbf8af7550c7fc2f291ea317bfc7dee): batch response: Post https://github.myEnterprise.com/_lfs/our_projects/FW_Toolchain/objects/batch: Bad Gateway

Post https://github.myEnterprise.com/_lfs/IoTSECprojects/FW_Toolchain/objects/batch: Bad Gateway
batch response

github.com/git-lfs/git-lfs/errors.newWrappedError
        C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/errors/types.go:170
github.com/git-lfs/git-lfs/errors.Wrap
        C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/errors/errors.go:74
github.com/git-lfs/git-lfs/tq.(*tqClient).Batch

        C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/tq/api.go:71

        C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/commands/run.go:71

main.main
        C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/git-lfs.go:35
runtime.main
        C:/Go/src/runtime/proc.go:185
runtime.goexit
        C:/Go/src/runtime/asm_amd64.s:2197
Smudge error
github.com/git-lfs/git-lfs/errors.newWrappedError
        C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/errors/types.go:170
github.com/git-lfs/git-lfs/errors.NewSmudgeError
        C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/errors/types.go:256
github.com/git-lfs/git-lfs/lfs.(*GitFilter).Smudge
        C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:68
github.com/git-lfs/git-lfs/commands.smudge
        C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/commands/command_smudge.go:127
github.com/git-lfs/git-lfs/commands.filterCommand
        C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/commands/command_filter_process.go:118
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).execute
        C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:477
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).Execute
        C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:551
github.com/git-lfs/git-lfs/commands.Run

        C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/commands/run.go:71
main.main
        C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/git-lfs.go:35
runtime.main
        C:/Go/src/runtime/proc.go:185
runtime.goexit
        C:/Go/src/runtime/asm_amd64.s:2197

Current time in UTC:
2018-08-20 14:33:52

ENV:
LocalWorkingDir=C:\GIT\test
LocalGitDir=C:\GIT\test\.git
LocalGitStorageDir=C:\GIT\test\.git
LocalMediaDir=C:\GIT\test\.git\lfs\objects
LocalReferenceDir=
TempDir=C:\GIT\test\.git\lfs\tmp
ConcurrentTransfers=3
TusTransfers=false
BasicTransfersOnly=false
SkipDownloadErrors=false
FetchRecentAlways=false
FetchRecentRefsDays=7
FetchRecentCommitsDays=0
FetchRecentRefsIncludeRemotes=true
PruneOffsetDays=3
PruneVerifyRemoteAlways=false
PruneRemoteName=origin
LfsStorageDir=C:\GIT\test\.git\lfs
AccessDownload=none
AccessUpload=none
DownloadTransfers=basic
UploadTransfers=basic
GIT_DIR=.git
GIT_PREFIX=

Client IP addresses:

 
                                                            

julien.staub@schneider-electric.com (JIRA)

unread,

Aug 21, 2018, 8:59:01 AM8/21/18

to jenkinsc...@googlegroups.com

julien staub updated an issue

Jenkins /

Change By:	julien staub
Comment:

Thank you again, this time I reproduced and may be able to contact our support but I have few hope on theyr side.

The error comes from "bad gateway" although a normal clone will just work fine

{code:none}

{code}

julien.staub@schneider-electric.com (JIRA)

unread,

Aug 21, 2018, 9:05:02 AM8/21/18

to jenkinsc...@googlegroups.com

julien staub commented on

Multiple interleaved problems make me have wrong conclusions. On the computer with Jenkins, I'm able to reproduce the problem with Jnekins but not on commandline (error is still Permission denied (publickey).: exit status 255) making me think the ssh identification works for some part of process but not with LFS part in my situation. To get in touch with my IT team I will need to be able to reproduce the connection issue on command line interface.

How to know exactly the list of commands executed by Jenkins ?

In particular :

 
                                                                using GIT_SSH to set credentials ssh key for github

mark.earl.waite@gmail.com (JIRA)

unread,

Aug 21, 2018, 11:03:01 AM8/21/18

to jenkinsc...@googlegroups.com

Mark Waite commented on

That specific command is not logged because that would risk leaking credentials into the log output.

The command line git implementation in the git client plugin performs the equivalent of the following steps (assuming Windows environment):

 
                                                                C:\Jenkins> REM Write the private key to a temporary file %PRIVATE_KEY%
C:\Jenkins> REM Create a batch file with the line ssh -i %PRIVATE_KEY% -l username -o StrictHostKeyChecking=no %*
C:\Jenkins> REM Set environment variable GIT_SSH to absolute path of that batch file
C:\Jenkins> REM Set environment variable GIT_SSH_VARIANT=ssh
C:\Jenkins> REM Create a batch file with the line echo "<private-key-passphrase-if-it-exists>""
C:\Jenkins> REM Set environment variable SSH_ASKPASS to absolute path of that batch file
C:\Jenkins> REM Set environment variable DISPLAY=:
 
                                                            

Those are the steps embedded in "using GIT_SSH to set credentials ssh key". Immediately after those steps, the code performs a

 
                                                                git fetch

yves.schumann@ti8m.ch (JIRA)

unread,

Nov 22, 2018, 5:07:02 PM11/22/18

to jenkinsc...@googlegroups.com

Yves Schumann commented on

Any news on this issue?

I've stumbled into the same problem but cannot switch to https. The failing job is part of a GitHub orga scanner and other jobs require ssh as they perform git push to publish changes/releases. So how can I fix git lfs clones using ssh?

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

mark.earl.waite@gmail.com (JIRA)

unread,

Nov 22, 2018, 10:18:02 PM11/22/18

to jenkinsc...@googlegroups.com

Mark Waite commented on

Yves Schumann I can't duplicate the problem. If you're seeing the same problem, then I assume you have the same conditions. Can you describe the conditions precisely enough that others can duplicate your conditions?

My understanding of the conditions described by julien staub include:

Jenkins server 2.121.2 running on Windows
GitHub Enterprise (not github.com)
Git for Windows installed with LFS support enabled - version unknown
Git LFS version unknown

I don't understand your specific description, or I can't dupli. You say that you're using a GitHub organization folder and other jobs require ssh. As far as I can tell from my experiment with a GitHub Organization project, I must provide an https URL and https credentials. I can then add the "checkout over ssh" trait and provide an ssh private key. I provided an ed25519 private key without passphrase and confirmed that my github.com LFS repositories (docker-lfs and docker-private-lfs) are both able to clone successfully.

I'm sure there are differences between my environment and yours. Differences that I can identify include:

I'm using github.com rather than GitHub Enterprise
I'm running the master as a Docker image on Linux, not as a Windows master
I'm running Jenkins 2.138.3 with a pre-release version of git plugin 4.0 and a pre-release version of git client plugin 3.0

mark.earl.waite@gmail.com (JIRA)

unread,

Nov 22, 2018, 11:57:03 PM11/22/18

to jenkinsc...@googlegroups.com

Mark Waite edited a comment on

[~swf] I can 't duplicate the problem . If you described if I don ' re seeing t enable the same problem, then I assume you have "Git LFS pull after checkout" trait on the same conditions GitHub Organization folder definition . Can you describe Once I enable "Git LFS pull after checkout" in the conditions precisely enough that others can duplicate your conditions? GitHub organization folder definition, the {{git lfs pull}} operation runs successfully.

My understanding of the conditions described by [~atsju2] include:

* Jenkins server 2.121.2 running on Windows
* GitHub Enterprise (not github.com)
* Git for Windows installed with LFS support enabled - version unknown
* Git LFS version unknown

I don't understand your specific description, or I can't dupli. You say that you're using a GitHub organization folder and other jobs require ssh. As far as I can tell from my experiment with a GitHub Organization project, I must provide an https URL and https credentials. I can then add the "checkout over ssh" trait and provide an ssh private key. I provided an ed25519 private key without passphrase and confirmed that my github.com LFS repositories (docker-lfs and docker-private-lfs) are both able to clone successfully.

I'm sure there are differences between my environment and yours. Differences that I can identify include:

* I'm using github.com rather than GitHub Enterprise
* I'm running the master as a Docker image on Linux, not as a Windows master
* I'm running Jenkins 2.138.3 with a pre-release version of git plugin 4.0 and a pre-release version of git client plugin 3.0

When I was misconfigured (without the {{Git LFS pull before checkout}} trait, the stack trace I saw was:

{noformat}
GitHub has been notified of this commit’s build result

hudson.plugins.git.GitException: Command "git checkout -f 103402b4417738fd6a7dee6386e11710c938950d" returned status code 128:
stdout:
stderr: Downloading ref/plugins/ace-editor.jpi (4.3 MB)
Error downloading object: ref/plugins/ace-editor.jpi (abc9702): Smudge error: Error downloading ref/plugins/ace-editor.jpi (abc97028893c8a71581a5f559ea48e8e1f1a65164faee96dabfed9e95e9abad2): batch request: g...@github.com: Permission denied (publickey).: exit status 255

Errors logged to /home/mwaite/mark-pc2.markwaite.net-agent/workspace/e-lfs_cjp-with-plugins-debian9-a/.git/lfs/logs/20181122T211435.33423253.log

Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed

fatal: ref/plugins/ace-editor.jpi: smudge filter lfs failed

at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2305)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$800(CliGitAPIImpl.java:83)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:2614)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to testing-a-mwaite
  at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
  at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
  at hudson.remoting.Channel.call(Channel.java:955)
  at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
  at sun.reflect.GeneratedMethodAccessor156.invoke(Unknown Source)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
  at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
  at com.sun.proxy.$Proxy92.execute(Unknown Source)
  at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1217)
  at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:120)
  at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:90)
  at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:77)
  at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$1$1.call(SynchronousNonBlockingStepExecution.java:50)
  at hudson.security.ACL.impersonate(ACL.java:290)
  at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$1.run(SynchronousNonBlockingStepExecution.java:47)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
Caused: hudson.plugins.git.GitException: Could not checkout 103402b4417738fd6a7dee6386e11710c938950d
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:2638)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{noformat}

mark.earl.waite@gmail.com (JIRA)

unread,

Nov 23, 2018, 12:13:04 AM11/23/18

to jenkinsc...@googlegroups.com

Mark Waite edited a comment on

[~swf] I can duplicate the problem you described if I don't enable the "Git LFS pull after checkout" trait on the GitHub Organization folder definition. Once I enable "Git LFS pull after checkout" in the GitHub organization folder definition, the {{git lfs pull}} operation runs successfully.

My understanding of the conditions described by [~atsju2] include:

* Jenkins server 2.121.2 running on Windows
* GitHub Enterprise (not github.com)
* Git for Windows installed with LFS support enabled - version unknown
* Git LFS version unknown

I don't understand your specific description, or I can't dupli. You say that you're using a GitHub organization folder and other jobs require ssh. As far as I can tell from my experiment with a GitHub Organization project, I must provide an https URL and https credentials. I can then add the "checkout over ssh" trait and provide an ssh private key. I provided an ed25519 private key without passphrase and confirmed that my github.com LFS repositories (docker-lfs and docker-private-lfs) are both able to clone successfully , so long as I add the {{Git LDS pull after checkout}} trait .

mark.earl.waite@gmail.com (JIRA)

unread,

Nov 23, 2018, 12:14:03 AM11/23/18

to jenkinsc...@googlegroups.com

Mark Waite edited a comment on

[~swf] I can duplicate the problem you described if I don't enable the "Git LFS pull after checkout" trait on the GitHub Organization folder definition. Once I enable "Git LFS pull after checkout" in the GitHub organization folder definition, the {{git lfs pull}} operation runs successfully.

My understanding of the conditions described by [~atsju2] include:

* Jenkins server 2.121.2 running on Windows
* GitHub Enterprise (not github.com)
* Git for Windows installed with LFS support enabled - version unknown
* Git LFS version unknown

As far as I can tell from my experiment with a GitHub Organization project, I must provide an https URL and https credentials. I can then add the "checkout over ssh" trait and provide an ssh private key. I provided an ed25519 private key without passphrase and confirmed that my github.com LFS repositories (docker-lfs and docker-private-lfs) are both able to clone successfully, so long as I add the {{Git LDS LFS pull after checkout}} trait.

julien.staub@schneider-electric.com (JIRA)

unread,

Nov 23, 2018, 3:56:02 AM11/23/18

to jenkinsc...@googlegroups.com

julien staub commented on

Hi

I had no time to continue investigation. However, thank you for this update.

Git LFS pull before checkout seems to have solved the problem.

mark.earl.waite@gmail.com (JIRA)

unread,

Nov 23, 2018, 7:16:02 AM11/23/18

to jenkinsc...@googlegroups.com

Mark Waite edited a comment on

[~swf] I can duplicate the problem you described if I don't enable the "Git LFS pull after checkout" trait on the GitHub Organization folder definition. Once I enable "Git LFS pull after checkout" in the GitHub organization folder definition, the {{git lfs pull}} operation runs successfully.

My understanding of the conditions described by [~atsju2] include:

* Jenkins server 2.121.2 running on Windows
* GitHub Enterprise (not github.com)
* Git for Windows installed with LFS support enabled - version unknown
* Git LFS version unknown

As far as I can tell from my experiment with a GitHub Organization project, I must provide an https URL and https credentials. I can then add the "checkout over ssh" trait and provide an ssh private key. I provided an ed25519 private key without passphrase and confirmed that my github.com LFS repositories (docker-lfs and docker-private-lfs) are both able to clone successfully, so long as I add the {{Git LFS pull after checkout}} trait.

I'm sure there are differences between my environment and yours. Differences that I can identify include:

* I'm using github.com rather than GitHub Enterprise
* I'm running the master as a Docker image on Linux, not as a Windows master
* I'm running Jenkins 2.138.3 with a pre-release version of git plugin 4.0 and a pre-release version of git client plugin 3.0

When I was misconfigured (without the {{Git LFS pull before checkout}} trait ) , the stack trace I saw was:

naveen.akkapeddi@anki.com (JIRA)

unread,

Dec 20, 2018, 6:40:02 PM12/20/18

to jenkinsc...@googlegroups.com

Naveen Akkapeddi commented on

Hello,

I am also running into this issue. I have a static build agent running OSX. My Linux based agents do not run into this issue.

I am using pipeline and run into this problem:
Enabling Git LFS pull
> /usr/local/bin/git config core.sparsecheckout # timeout=10
> /usr/local/bin/git checkout -f 92270cd6f36f9136e82661792c1c7f11a21e7fe8
[Pipeline] }
[Pipeline] // stage[Pipeline] }
[Pipeline] // withEnv[Pipeline] echo
/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
[Pipeline] echo
hudson.plugins.git.GitException: Command "/usr/local/bin/git checkout -f 92270cd6f36f9136e82661792c1c7f11a21e7fe8" returned status code 128:
stdout:
stderr: git-lfs filter-process: git-lfs: command not found
fatal: the remote end hung up unexpectedly

I enabled the trait

 
                                                                Git LFS pull after checkout  
                                                            

I do not see anything called "Git LFS pull before checkout". Please advise what I should do.

Thanks.

naveen.akkapeddi@anki.com (JIRA)

unread,

Dec 20, 2018, 6:41:02 PM12/20/18

to jenkinsc...@googlegroups.com

Naveen Akkapeddi edited a comment on

Hello,

I am also running into this issue. I have a static build agent running OSX. My Linux based agents do not run into this issue.

I am using pipeline and run into this problem:

{code:java}

Enabling Git LFS pull
> /usr/local/bin/git config core.sparsecheckout # timeout=10
> /usr/local/bin/git checkout -f 92270cd6f36f9136e82661792c1c7f11a21e7fe8
[Pipeline] }
[Pipeline] // stage[Pipeline] }
[Pipeline] // withEnv[Pipeline] echo
/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
[Pipeline] echo
hudson.plugins.git.GitException: Command "/usr/local/bin/git checkout -f 92270cd6f36f9136e82661792c1c7f11a21e7fe8" returned status code 128:
stdout:
stderr: git-lfs filter-process: git-lfs: command not found
fatal: the remote end hung up unexpectedly

{code}

I enabled the trait
{code:java}
Git LFS pull after checkout {code}

I do not see anything called "Git LFS pull *before* checkout". Please advise what I should do.

Thanks.

naveen.akkapeddi@anki.com (JIRA)

unread,

Dec 20, 2018, 6:58:02 PM12/20/18

to jenkinsc...@googlegroups.com

Naveen Akkapeddi edited a comment on

Hello,

I am also If you are running into this issue. I have a static build agent running OSX. My Linux based agents do not run into this issue.

I am using pipeline and run into this problem:
{code:java}
Enabling Git LFS pull

> /usr/local/bin/ , you need to make sure that git config core.sparsecheckout # timeout=10
> /usr/local/bin/git checkout - f 92270cd6f36f9136e82661792c1c7f11a21e7fe8

[Pipeline] }
[Pipeline] // stage[Pipeline] }
[Pipeline] // withEnv[Pipeline] echo
/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
[Pipeline] echo
hudson.plugins.git.GitException: Command "/usr/local/bin/git checkout -f 92270cd6f36f9136e82661792c1c7f11a21e7fe8" returned status code 128:
stdout:

stderr: git- lfs filter-process: git-lfs: command not found
fatal: is both installed on your master and slave, at least for the remote end hung up unexpectedly
{code}

I enabled latest version of Jenkins and the trait

{code:java}
Git LFS pull after checkout {code}

I do not see anything called "Git LFS pull *before* checkout" git-client plugin . Please advise what I should do.

Thanks.

naveen.akkapeddi@anki.com (JIRA)

unread,

Dec 20, 2018, 6:58:03 PM12/20/18

to jenkinsc...@googlegroups.com

Naveen Akkapeddi edited a comment on