[JIRA] [git-client-plugin] (JENKINS-30318) Git plugin breaks usage of Git LFS due to lack of Git Clone use

11 views
Skip to first unread message

s.t.richmond@gmail.com (JIRA)

unread,
Sep 5, 2015, 12:35:01 PM9/5/15
to jenkinsc...@googlegroups.com
Scott Richmond created an issue
 
Jenkins / Bug JENKINS-30318
Git plugin breaks usage of Git LFS due to lack of Git Clone use
Issue Type: Bug Bug
Assignee: Nicolas De Loof
Components: git-client-plugin, git-plugin
Created: 05/Sep/15 4:33 PM
Priority: Major Major
Reporter: Scott Richmond

I'm currently involved in setting up a Jenkins-based build for our project and I've found that the popular Git plugin for Jenkins takes the slightly weird approach of git init+fetch instead of git clone when setting up the repo on agent machines.

This appears to break Git LFS as all the binary files are still only the reference pointer files.

The plugin does not currently appear to have any advanced options to force git cloning and there are a number of aged active tickets for the plugin to support it.

I have raised an issue ticket over at Git LFS to discuss options with regards to this problem, which can be viewed here. It is not an issue for them to resolve, however they may be helpful in providing advice on how to best solve the problem for the plugin that provides minimal impact.
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)
Atlassian logo

s.t.richmond@gmail.com (JIRA)

unread,
Sep 5, 2015, 1:57:01 PM9/5/15
to jenkinsc...@googlegroups.com
Scott Richmond updated an issue
 
Jenkins / Improvement JENKINS-30318
Change By: Scott Richmond
Issue Type: Bug Improvement

jacob.keller@gmail.com (JIRA)

unread,
Feb 19, 2016, 6:16:02 PM2/19/16
to jenkinsc...@googlegroups.com
Jacob Keller commented on Improvement JENKINS-30318
 
Re: Git plugin breaks usage of Git LFS due to lack of Git Clone use

The original reasoning for this hack was due to credentials issues, which should be resolved after the 1.20.0-beta3 release, if I understand correctly. However, the decision to remove the init+fetch was not accepted at the time I suggested it.

The reasoning for init+fetch was due to needing abillity to use the git credential store inside .git/config which does not exist when you perform a clone. This can be fixed using the GIT_SSH and GIT_ASKPASS programs as I did for fixing submodule credentials issues among other related problems.

I would promote removal of the git init+fetch combination due to this, as the original reason for this workaround should no longer be valid.

alex@gonebusy.com (JIRA)

unread,
Mar 23, 2016, 10:00:02 PM3/23/16
to jenkinsc...@googlegroups.com

Can we raise the priority of this issue? It is currently not possible to add jobs with Git repos making use of git-lfs strictly via Jenkins UI.

At best, the current workaround I've found is the following:

  • Via Jenkins Credentials, add a "Username with Password" credential
  • Create new job, making sure to use HTTPS url for repo and not GIT, along with credentials from previous step
  • Attempt first build, it will fail on git-lfs. Make note of `git checkout -f <tag>` that failed.
  • Manually ssh into Jenkins box, cd to job workspace, and re-attempt `git checkout -f <tag>`. When prompted, enter U/P
  • Subsequent builds for this job will run unattended

alexk@Motu.com (JIRA)

unread,
May 2, 2016, 6:17:02 PM5/2/16
to jenkinsc...@googlegroups.com

I can reproduce the same state Alex Agranov mentioned. But I really don't want to manually initialize a workspace any time I'm working with an LFS repo. I'd really prefer to have an appropriate way to work with LFS in Jenkins.

jacob.keller@gmail.com (JIRA)

unread,
May 3, 2016, 2:01:02 PM5/3/16
to jenkinsc...@googlegroups.com

We can fix Git plugin, but there is an issue. The init+fetch workaround would allow us to avoid an issue with checking out into a non-empty directory (which currently we do a "wipe" of the workspace which breaks multiple-SCM plugin if the order of repositories is bad).

However, I would propose going ahead with this fix, assuming the maintainer agrees.

nicolas.deloof@gmail.com (JIRA)

unread,
Jun 3, 2016, 9:54:01 AM6/3/16
to jenkinsc...@googlegroups.com

I'm +1 to use git clone if we have a correct workaround for credentials support.
That being said, I don't get why git-lfs would not support init + fetch. Is there any special hook during a clone operation that git-lfs rely on to get large file downloaded ?

nicolas.deloof@gmail.com (JIRA)

unread,
Jun 3, 2016, 10:13:02 AM6/3/16
to jenkinsc...@googlegroups.com
Nicolas De Loof assigned an issue to Unassigned
 
Change By: Nicolas De Loof
Assignee: Nicolas De Loof

alex@gonebusy.com (JIRA)

unread,
Jun 3, 2016, 3:23:01 PM6/3/16
to jenkinsc...@googlegroups.com
Alex Agranov commented on Improvement JENKINS-30318
 
Re: Git plugin breaks usage of Git LFS due to lack of Git Clone use

I haven't tracked down the exact code that limits to clone/pull but their own man-page refers to their supported functionality: https://github.com/github/git-lfs/blob/master/docs/man/git-lfs-install.1.ronn#L27-L30

s.t.richmond@gmail.com (JIRA)

unread,
Jun 4, 2016, 6:31:03 AM6/4/16
to jenkinsc...@googlegroups.com

Hey guys. So. The reason this is an issue is because Git LFS uses various Git hooks to run its binary download processes. There hooks don't fire if one uses the granular Git commands that make up a `git clone` or `git pull` etc. Its not really a bug, its just Git design.

If it really is a difficulty in using `git clone`, that is fine. The other option in this case is to simply explicitly support Git LFS by adding an option in the settings of this plugin such as 'Enable/Use Git LFS' and to run `git lfs pull` when appropriate.

s.t.richmond@gmail.com (JIRA)

unread,
Jun 4, 2016, 6:32:03 AM6/4/16
to jenkinsc...@googlegroups.com
Scott Richmond edited a comment on Improvement JENKINS-30318

s.t.richmond@gmail.com (JIRA)

unread,
Jun 4, 2016, 6:34:02 AM6/4/16
to jenkinsc...@googlegroups.com
Scott Richmond edited a comment on Improvement JENKINS-30318
Hey guys. So. The reason this is an issue is because Git LFS uses various Git hooks to run its binary download processes. There hooks don't fire if one uses the granular Git commands that make up a {{git clone}} or {{git pull}} etc. Its not really a bug, its just Git design.

If it really is a difficulty in using {{git clone}}, that is fine. The other option in this case is to simply explicitly support Git LFS by adding an option in the settings of this plugin such as {{Enable/Use Git LFS}} and to run {{git lfs pull}} when appropriate.


Feel free to drop into the (Gitter chat room)[https://gitter.im/github/git-lfs] for Git LFS, where we have a fairly active community including the key maintainers of the project.

s.t.richmond@gmail.com (JIRA)

unread,
Jun 4, 2016, 6:35:02 AM6/4/16
to jenkinsc...@googlegroups.com
Scott Richmond edited a comment on Improvement JENKINS-30318
Hey guys. So. The reason this is an issue is because Git LFS uses various Git hooks to run its binary download processes. There hooks don't fire if one uses the granular Git commands that make up a {{git clone}} or {{git pull}} etc. Its not really a bug, its just Git design.

If it really is a difficulty in using {{git clone}}, that is fine. The other option in this case is to simply explicitly support Git LFS by adding an option in the settings of this plugin such as {{Enable/Use Git LFS}} and to run {{git lfs pull}} when appropriate.

Feel free to drop into the  ( Gitter chat room )  ( [https://gitter.im/github/git-lfs] )  for Git LFS, where we have a fairly active community including the key maintainers of the project.

ankitmaru@gmail.com (JIRA)

unread,
Jun 13, 2016, 2:30:02 AM6/13/16
to jenkinsc...@googlegroups.com

HI, I am also getting stuck with this issue. Hope to see resolution soon. Thanks

paskas@gmail.com (JIRA)

unread,
Jun 16, 2016, 7:17:02 AM6/16/16
to jenkinsc...@googlegroups.com

Is there any update or workaround?
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
Atlassian logo

jacob.keller@gmail.com (JIRA)

unread,
Jun 17, 2016, 2:30:01 AM6/17/16
to jenkinsc...@googlegroups.com

I can take a look tomorrow and see what it would take to use clone instead of fetch+init like we are now. I think with recent changes it might be possible now.

s.t.richmond@gmail.com (JIRA)

unread,
Jun 17, 2016, 4:57:04 AM6/17/16
to jenkinsc...@googlegroups.com

Actually if you can implement it without clone it would be even better - Git LFS cannot download files in batches when using Git Clone and can be very slow, starting a new http request for each binary file. The more efficient way to implement Git LFS for repo cloning is to run git init/fetch as this plugin does right now, but to also include a `git lfs pull` afterwards. This will allow LFS to batch download binary files and increase the speed of the download by 10.

jacob.keller@gmail.com (JIRA)

unread,
Jun 17, 2016, 5:14:05 PM6/17/16
to jenkinsc...@googlegroups.com

The problem is we can't depend on git-lfs, and I'd rather not have a checkbox for it. Also as I am unfamiliar with git-lfs I am not aware of how and what needs to be added everywhere. It would be better if it worked seamless.

Switching to use git-clone is easier, and likely fixes other possible issues with the init+fetch combo we have now.

s.t.richmond@gmail.com (JIRA)

unread,
Jun 18, 2016, 6:35:04 AM6/18/16
to jenkinsc...@googlegroups.com

I understand where you're coming from, but I would strongly suggest otherwise - Without some explicit support for LFS you will be forcing users to run the first clone with no parallelism. I wasn't kidding with how slow it is - A repo with 10k binary files can take around an hour using the default Git clone and under 10 minutes with some explicit LFS command/s.

There is now a fairly new command in LFS - `git lfs clone` - It will perform the initial clone with parallelized file downloads. It is the recommended way to clone an LFS repo.

Without some explicit support for LFS I fear that you'll only later get hammered by users to fix the above issue. That said, by moving to `git clone` I suppose at least LFS repos will work in the first place. Though you may want to consider adding a checkbox along the lines of 'Perform Git LFS Clone' that is optional and will simply swap all `git clone` commands with `git lfs clone` and thusly speed up LFS repos.

jacob.keller@gmail.com (JIRA)

unread,
Jun 18, 2016, 5:03:02 PM6/18/16
to jenkinsc...@googlegroups.com

I would rather focus on just getting things working, and someone who is actively interested in using LFS is free to submit patches that fix the performance issues and they can be reviewed. Unfortunately I do not have the time to do that myself.

jacob.keller@gmail.com (JIRA)

unread,
Jun 20, 2016, 1:23:02 PM6/20/16
to jenkinsc...@googlegroups.com

Ok so I did some digging, and it is not feasible to return to a clone from an init+fetch, because of the various enhancements added to the clone command which are not feasible to remove (as that would be removing a feature that people may depend on).

If one of the people here wishes to take the effort to add explicit git-lfs support, I will be happy to review it, but I don't have time myself to work on that patch.

michelene.chon@am.sony.com (JIRA)

unread,
Jul 14, 2016, 1:02:05 PM7/14/16
to jenkinsc...@googlegroups.com
M Chon commented on Improvement JENKINS-30318

I set up a Mac Mini slave, and installed git-lfs into /usr/local/bin/git
Then I configured the slave to use /usr/local/bin/git instead of /usr/bin/git
It seems to work just fine - LFS files are fetched seamlessly.

mark.earl.waite@gmail.com (JIRA)

unread,
Jan 20, 2017, 7:03:08 PM1/20/17
to jenkinsc...@googlegroups.com
Mark Waite started work on Improvement JENKINS-30318
 
Change By: Mark Waite
Status: Open In Progress

mark.earl.waite@gmail.com (JIRA)

unread,
Jan 20, 2017, 7:03:08 PM1/20/17
to jenkinsc...@googlegroups.com
Mark Waite assigned an issue to Mark Waite
 
Change By: Mark Waite
Assignee: Mark Waite

mark.earl.waite@gmail.com (JIRA)

unread,
Feb 1, 2017, 8:01:08 PM2/1/17
to jenkinsc...@googlegroups.com
Mark Waite commented on Improvement JENKINS-30318
 
Re: Git plugin breaks usage of Git LFS due to lack of Git Clone use

A pre-release of large file support is being tested now. You can download the git client plugin and the git plugin from the lts-with-plugins branch of my docker instance.

I deeply appreciate comments on the pull request which say "I've installed the pre-release of this plugin and am using it successfully". I also deeply appreciate comments which indicate when a problem has been encountered.

mark.earl.waite@gmail.com (JIRA)

unread,
Feb 1, 2017, 9:58:07 PM2/1/17
to jenkinsc...@googlegroups.com
Mark Waite edited a comment on Improvement JENKINS-30318
A pre-release of large file support is being tested now. You can download the [git client plugin|https://github.com/MarkEWaite/docker/blob/50c2c1c67232462cb1f7a5e75bd73d3611e6be9b/ref/plugins/git-client.jpi] and the [git plugin|https://github.com/MarkEWaite/docker/blob/ 50c2c1c67232462cb1f7a5e75bd73d3611e6be9b c9d4beeec4c861bdcbfe692f8060ee78c98da4b3 /ref/plugins/git.jpi] from the lts-with-plugins branch of my docker instance.

I deeply appreciate comments on the [pull request|https://github.com/jenkinsci/git-plugin/pull/463] which say "I've installed the pre-release of this plugin and am using it successfully". I also deeply appreciate comments which indicate when a problem has been encountered.

mark.earl.waite@gmail.com (JIRA)

unread,
Mar 4, 2017, 12:22:08 PM3/4/17
to jenkinsc...@googlegroups.com
Mark Waite resolved as Fixed
 

Git plugin 3.1.0 released on 4 Mar 2017 now includes support for command line git large file support.
Change By: Mark Waite
Status: In Progress Resolved
Resolution: Fixed
This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
Atlassian logo

alex@gonebusy.com (JIRA)

unread,
Mar 4, 2017, 7:25:02 PM3/4/17
to jenkinsc...@googlegroups.com
Alex Agranov commented on Improvement JENKINS-30318
 
Re: Git plugin breaks usage of Git LFS due to lack of Git Clone use

Just tested, after adding "Git LFS Pull After Checkout" as an Additional Behavior, and works like a charm!

You. Da. Man.

mark.earl.waite@gmail.com (JIRA)

unread,
Mar 4, 2017, 7:29:02 PM3/4/17
to jenkinsc...@googlegroups.com

 Special thanks to Matt Hauck for the initial implementation and to creste for the final implementation and adaptations as needed.

matthauck (JIRA)

unread,
Mar 6, 2017, 7:06:03 PM3/6/17
to jenkinsc...@googlegroups.com

Woot! :sparkles:

jakvlk@seznam.cz (JIRA)

unread,
Mar 11, 2017, 3:43:02 PM3/11/17
to jenkinsc...@googlegroups.com

I am still getting error "stderr: git: 'lfs' is not a git command. See 'git --help'." but just on OS X system. On the Windows I am fine. I am using ssh btw... Any advice?

Thx, Jakub

mark.earl.waite@gmail.com (JIRA)

unread,
Mar 11, 2017, 4:25:03 PM3/11/17
to jenkinsc...@googlegroups.com

Jakub Vlk, you probably need to check your PATH settings to assure that git-lfs is installed as a program in the expected location on your MacOS machine.

jakvlk@seznam.cz (JIRA)

unread,
Mar 11, 2017, 4:46:05 PM3/11/17
to jenkinsc...@googlegroups.com

Yeah I was thinking in the same way. But I can use git lfs from terminal by just writing git lfs. So it should be OK...

I used echo for my $PATH and I can see this: 

/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin

and which git-lfs returns /usr/local/bin/git-lfs so it looks OK for me...

Maybe there is a problem that git is installed in /usr/bin and git-lfs in /usr/local/bin/git-lfs? No sure though...

mark.earl.waite@gmail.com (JIRA)

unread,
Mar 11, 2017, 4:56:04 PM3/11/17
to jenkinsc...@googlegroups.com

Installing git-lfs in /usr/local/bin worked for me on several different Linux computers.

You say that you used echo for your $PATH, but did you do that from within a Jenkins job? If so, then I have no idea what could be wrong.

jakvlk@seznam.cz (JIRA)

unread,
Mar 11, 2017, 5:30:06 PM3/11/17
to jenkinsc...@googlegroups.com

Oooh god, you're right! It was echo from terminal, where I am logged... Thank you very much! I added lfs path to env variable in Jenkins settings and it's working!

mark.earl.waite@gmail.com (JIRA)

unread,
Oct 22, 2019, 9:33:18 PM10/22/19
to jenkinsc...@googlegroups.com
Mark Waite closed an issue as Fixed
 
Change By: Mark Waite
Status: Resolved Closed
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo
Reply all
Reply to author
Forward
0 new messages