[JIRA] (JENKINS-35687) Add explicit support for git lfs

Issue Type:	Bug
Assignee:	Mark Waite
Components:	git-client-plugin, git-plugin
Created:	2016/Jun/14 6:10 AM
Priority:	Minor
Reporter:	Matt Hauck

I would like to use git lfs and I would like to jenkins, and I would like for jenkins to use its same credentials already configured for git to fetch the git lfs resources. Rather than trying to change the init+fetch pattern, I would like to add a custom behavior to the git plugin that will execute `git lfs pull` after checkout using the same credentials as the fetch command.

It seems that git lfs is the way forward for git repos with large files and I don't see a workaround right now for using git lfs with jenkins, short of sprinkling credentials everywhere so that `git lfs` can be run from the build context which doesn't seem like a good idea.

I am looking into what it would take to add this. Would the maintainers be interested in taking in such a feature?

This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)

matthauck (JIRA)

unread,

Jun 14, 2016, 2:54:01 AM6/14/16

to jenkinsc...@googlegroups.com

Matt Hauck commented on

Added some simple pull requests to get this going:

mark.earl.waite@gmail.com (JIRA)

unread,

Jun 14, 2016, 8:56:01 PM6/14/16

to jenkinsc...@googlegroups.com

Mark Waite assigned an issue to Unassigned

Jenkins /

Change By:	Mark Waite
Assignee:	Mark Waite

michelene.chon@am.sony.com (JIRA)

unread,

Jul 14, 2016, 1:02:03 PM7/14/16

to jenkinsc...@googlegroups.com

M Chon commented on

I set up a Mac Mini slave, and installed git-lfs into /usr/local/bin/git
Then I configured the slave to use /usr/local/bin/git instead of /usr/bin/git
It seems to work just fine - LFS files are fetched seamlessly.

swashburn@cloudbees.com (JIRA)

unread,

Jul 26, 2016, 2:45:02 PM7/26/16

to jenkinsc...@googlegroups.com

Spike Washburn updated an issue

Jenkins /

Change By:	Spike Washburn
Issue Type:	Bug Improvement

swashburn@cloudbees.com (JIRA)

unread,

Jul 26, 2016, 2:46:01 PM7/26/16

to jenkinsc...@googlegroups.com

Spike Washburn commented on

See the related discussion in the Git LFS issue tracker https://github.com/github/git-lfs/issues/640

bitsys@gmail.com (JIRA)

unread,

Dec 29, 2016, 12:30:01 PM12/29/16

to jenkinsc...@googlegroups.com

John Smith commented on

I created some pull requests that fixed a few issues with Matt Hauck's PRs:

mark.earl.waite@gmail.com (JIRA)

unread,

Jan 20, 2017, 7:02:04 PM1/20/17

to jenkinsc...@googlegroups.com

Mark Waite started work on

Change By:	Mark Waite
Status:	Open In Progress

mark.earl.waite@gmail.com (JIRA)

unread,

Jan 20, 2017, 7:02:05 PM1/20/17

to jenkinsc...@googlegroups.com

Mark Waite assigned an issue to Mark Waite

Jenkins /

Change By:	Mark Waite
Assignee:	Mark Waite

mark.earl.waite@gmail.com (JIRA)

unread,

Jan 21, 2017, 8:58:02 AM1/21/17

to jenkinsc...@googlegroups.com

Mark Waite commented on

I've added an unauthenticated verification job to my lts-with-plugins docker repository. That job passes with git client plugin PR232 and git plugin PR463.

mark.earl.waite@gmail.com (JIRA)

unread,

Jan 21, 2017, 10:01:03 AM1/21/17

to jenkinsc...@googlegroups.com

Mark Waite edited a comment on

I've added an unauthenticated [verification job|https://github.com/MarkEWaite/docker/commit/ea220761a2ad8134428d54f59d0a69c3853b5837] to my [lts-with-plugins|https://github.com/MarkEWaite/docker/tree/lts-with-plugins] docker repository.  That job passes with [git client plugin PR232|https://github.com/jenkinsci/git-client-plugin/pull/232] and [git plugin PR463|https://github.com/jenkinsci/git-plugin/pull/463].

I've added an authenticated verification job to my private docker repo.  That job also passes with  [git client plugin PR232|https://github.com/jenkinsci/git-client-plugin/pull/232] and [git plugin PR463|https://github.com/jenkinsci/git-plugin/pull/463].

I've added an authenticated verification job with Jenkins pipeline and it fails to checkout with an error message report.  I'll need to investigate further to understand why the failure.

mark.earl.waite@gmail.com (JIRA)

unread,

Jan 21, 2017, 10:01:04 AM1/21/17

to jenkinsc...@googlegroups.com

Mark Waite edited a comment on

I've added an unauthenticated [verification job|https://github.com/MarkEWaite/docker/commit/ea220761a2ad8134428d54f59d0a69c3853b5837] to my [lts-with-plugins|https://github.com/MarkEWaite/docker/tree/lts-with-plugins] docker repository. That job passes with [git client plugin PR232|https://github.com/jenkinsci/git-client-plugin/pull/232] and [git plugin PR463|https://github.com/jenkinsci/git-plugin/pull/463].

I've added an authenticated verification job to my private docker repo. That job also passes with [git client plugin PR232|https://github.com/jenkinsci/git-client-plugin/pull/232] and [git plugin PR463|https://github.com/jenkinsci/git-plugin/pull/463].

I've added an authenticated verification job with Jenkins pipeline and it fails to checkout with an error message report . I'll need to investigate further to understand why the failure.

mark.earl.waite@gmail.com (JIRA)

unread,

Jan 21, 2017, 10:01:04 AM1/21/17

to jenkinsc...@googlegroups.com

Mark Waite edited a comment on

I've added an unauthenticated [verification job|https://github.com/MarkEWaite/docker/commit/ea220761a2ad8134428d54f59d0a69c3853b5837] to my [lts-with-plugins|https://github.com/MarkEWaite/docker/tree/lts-with-plugins] docker repository. That job passes with [git client plugin PR232|https://github.com/jenkinsci/git-client-plugin/pull/232] and [git plugin PR463|https://github.com/jenkinsci/git-plugin/pull/463].

I've added an authenticated verification job to my private docker repo. That job also passes with [git client plugin PR232|https://github.com/jenkinsci/git-client-plugin/pull/232] and [git plugin PR463|https://github.com/jenkinsci/git-plugin/pull/463].

I've added an authenticated verification job with Jenkins pipeline and it fails to checkout with an error. I'll need to investigate further to understand why the failure.

mark.earl.waite@gmail.com (JIRA)

unread,

Jan 21, 2017, 5:53:02 PM1/21/17

to jenkinsc...@googlegroups.com

Mark Waite edited a comment on

I've added an unauthenticated [verification job|https://github.com/MarkEWaite/docker/commit/ea220761a2ad8134428d54f59d0a69c3853b5837] to my [lts-with-plugins|https://github.com/MarkEWaite/docker/tree/lts-with-plugins] docker repository. That job passes with [git client plugin PR232|https://github.com/jenkinsci/git-client-plugin/pull/232] and [git plugin PR463|https://github.com/jenkinsci/git-plugin/pull/463].

I've added an authenticated verification job to my private docker repo. That job also passes with [git client plugin PR232|https://github.com/jenkinsci/git-client-plugin/pull/232] and [git plugin PR463|https://github.com/jenkinsci/git-plugin/pull/463].

I've added an authenticated verification job with Jenkins pipeline and it fails to checkout with an error. I'll need Refer to investigate further to understand the failure comments on [git plugin PR463|https://github . com/jenkinsci/git-plugin/pull/463] for the latest status.

scm_issue_link@java.net (JIRA)

unread,

Jan 31, 2017, 10:54:03 AM1/31/17

to jenkinsc...@googlegroups.com

SCM/JIRA link daemon commented on

Code changed in jenkins
User: Matt Hauck
Path:
src/main/java/org/jenkinsci/plugins/gitclient/CheckoutCommand.java
src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java
src/main/java/org/jenkinsci/plugins/gitclient/JGitAPIImpl.java
http://jenkins-ci.org/commit/git-client-plugin/e7ddf98e290de8c38b05807c579359dd1782f7de
Log:
JENKINS-35687 Add simple git lfs support

mark.earl.waite@gmail.com (JIRA)

unread,

Feb 1, 2017, 8:01:10 PM2/1/17

to jenkinsc...@googlegroups.com

Mark Waite commented on

A pre-release of large file support is being tested now. You can download the git client plugin and the git plugin from the lts-with-plugins branch of my docker instance.

I deeply appreciate comments on the pull request which say "I've installed the pre-release of this plugin and am using it successfully". I also deeply appreciate comments which indicate when a problem has been encountered.

mark.earl.waite@gmail.com (JIRA)

unread,

Feb 1, 2017, 9:58:10 PM2/1/17

to jenkinsc...@googlegroups.com

Mark Waite edited a comment on

A pre-release of large file support is being tested now. You can download the [git client plugin|https://github.com/MarkEWaite/docker/blob/50c2c1c67232462cb1f7a5e75bd73d3611e6be9b/ref/plugins/git-client.jpi] and the [git plugin|https://github.com/MarkEWaite/docker/blob/ 50c2c1c67232462cb1f7a5e75bd73d3611e6be9b c9d4beeec4c861bdcbfe692f8060ee78c98da4b3 /ref/plugins/git.jpi] from the lts-with-plugins branch of my docker instance.

I deeply appreciate comments on the [pull request|https://github.com/jenkinsci/git-plugin/pull/463] which say "I've installed the pre-release of this plugin and am using it successfully". I also deeply appreciate comments which indicate when a problem has been encountered.

monitorjbl@gmail.com (JIRA)

unread,

Feb 8, 2017, 9:50:02 AM2/8/17

to jenkinsc...@googlegroups.com

Taylor Jones commented on

Would it be possible to add support for remotes that are not the default LFS remote URL? It would make it a lot easier for teams that are using a separate LFS repo like Artifactory to create Jenkins jobs.

mark.earl.waite@gmail.com (JIRA)

unread,

Feb 8, 2017, 10:56:03 AM2/8/17

to jenkinsc...@googlegroups.com

Mark Waite commented on

Adding support for remotes that are not the default LFS remote URL will not be in the initial release.

I don't plan to implement that support, even after the initial release of git LFS because I need to focus on submodule bugs and on authentication bugs once the basic LFS capability has been added.

mark.earl.waite@gmail.com (JIRA)

unread,

Feb 8, 2017, 11:00:01 AM2/8/17

to jenkinsc...@googlegroups.com

Mark Waite edited a comment on

Adding support for remotes that are not the default LFS remote URL will not be in the initial release.

I don't plan to implement that support, even after the initial release of git LFS because I need to focus on submodule bugs and on authentication bugs once the basic LFS capability has been added.

I'm willing to evaluate pull requests from others that propose to add such support, but I don't see it high enough on my list to do that myself. There are too many open pull requests with good ideas for me to spend much time writing improvements myself.

itait@us.ibm.com (JIRA)

unread,

Feb 9, 2017, 5:23:01 PM2/9/17

to jenkinsc...@googlegroups.com

Ian Tait commented on

Mark Waite Is there a way to change the timeout time for the LFS pull? Currently it is set at 10 minutes, but we need it to be longer for the first pull to finish successfully.

mark.earl.waite@gmail.com (JIRA)

unread,

Feb 9, 2017, 5:42:02 PM2/9/17

to jenkinsc...@googlegroups.com

Mark Waite commented on

Since the LFS pull is performed by "git checkout", the timeout should be modified by adding the additional behavior "Advanced checkout behaviours" and set the timeout to the value you'd like.

itait@us.ibm.com (JIRA)

unread,

Feb 9, 2017, 9:57:02 PM2/9/17

to jenkinsc...@googlegroups.com

Ian Tait commented on

Ah there it is. Thank you for the help!

scm_issue_link@java.net (JIRA)

unread,

Feb 26, 2017, 11:53:05 PM2/26/17

to jenkinsc...@googlegroups.com

SCM/JIRA link daemon commented on

Compare: https://github.com/jenkinsci/git-client-plugin/compare/9c12a007c5a0...470f1257482e

Code changed in jenkins
User: Mark Waite
Path:
src/main/java/org/jenkinsci/plugins/gitclient/CheckoutCommand.java
src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java
src/main/java/org/jenkinsci/plugins/gitclient/JGitAPIImpl.java
src/test/java/org/jenkinsci/plugins/gitclient/GitClientTest.java
http://jenkins-ci.org/commit/git-client-plugin/470f1257482e0b04c67fcadcbe66d9b4462d2f84
Log:
Merge pull request #232 from creste/JENKINS-35687

JENKINS-35687 Add simple git lfs support

scm_issue_link@java.net (JIRA)

unread,

Feb 27, 2017, 12:10:06 AM2/27/17

to jenkinsc...@googlegroups.com

SCM/JIRA link daemon commented on

Code changed in jenkins
User: Mark Waite
Path:

src/main/java/hudson/plugins/git/extensions/impl/GitLFSPull.java
src/test/java/hudson/plugins/git/extensions/impl/CheckoutOptionTest.java
http://jenkins-ci.org/commit/git-plugin/ae6883b49bb3973e6139b666b7cea81464317274
Log:
Merge pull request #463 from creste/JENKINS-35687

JENKINS-35687 Add simple git lfs support

Compare: https://github.com/jenkinsci/git-plugin/compare/7fb16cbe8ff5...ae6883b49bb3

scm_issue_link@java.net (JIRA)

unread,

Feb 27, 2017, 12:10:06 AM2/27/17

to jenkinsc...@googlegroups.com

SCM/JIRA link daemon commented on

Code changed in jenkins
User: Matt Hauck
Path:
src/main/java/hudson/plugins/git/extensions/impl/GitLFSPull.java
src/test/java/hudson/plugins/git/extensions/impl/CheckoutOptionTest.java
http://jenkins-ci.org/commit/git-plugin/a809052a9b8d59975477e52f106383d63b869830
Log:

JENKINS-35687 Add simple git lfs support

matthauck (JIRA)

unread,

Feb 28, 2017, 8:04:03 PM2/28/17

to jenkinsc...@googlegroups.com

Matt Hauck commented on

This can probably be marked as Resolved fixed now?

mark.earl.waite@gmail.com (JIRA)

unread,

Feb 28, 2017, 8:12:01 PM2/28/17

to jenkinsc...@googlegroups.com

Mark Waite commented on

Not yet. I still need to release the git-plugin that includes the fix. I hope to do that within the next 24 hours.

matthauck (JIRA)

unread,

Mar 3, 2017, 9:13:02 PM3/3/17

to jenkinsc...@googlegroups.com

Matt Hauck commented on

Ah, roger. Just upgraded and realized it wasn't there yet, haha.

mark.earl.waite@gmail.com (JIRA)

unread,

Mar 4, 2017, 12:22:08 PM3/4/17

to jenkinsc...@googlegroups.com

Mark Waite resolved as Fixed

Git plugin 3.1.0 released on 4 Mar 2017 now includes support for command line git large file support.

Jenkins /

Change By:	Mark Waite
Status:	In Progress Resolved
Resolution:	Fixed

This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)

carlos.ruiz.lantero@gmail.com (JIRA)

unread,

Mar 9, 2017, 6:20:03 AM3/9/17

to jenkinsc...@googlegroups.com

Carlos Ruiz Lantero commented on

Mark Waite that's a wonderful job for supporting git LFS, thanks a lot!

Is there a way I can use it with pipelines? I've been trying to find examples and/or documentation with no luck.

mark.earl.waite@gmail.com (JIRA)

unread,

Mar 9, 2017, 10:11:05 AM3/9/17

to jenkinsc...@googlegroups.com

Mark Waite edited a comment on

The "pipeline syntax" link on most pipeline job pages will present a dropdown of pipeline steps. One of those is "checkout". Inside that "checkout" step, you can select git as the source control system, then add the "Additional Behaviour" for Git LFS pull.

If you prefer to read a script example, refer to the [Jenkinsfile in the - JENKINS-35687 - branch in my jenkins-bugs repo|[https://github.com/MarkEWaite/jenkins-bugs/blob/JENKINS-35687/Jenkinsfile] . ] That shows an example using GitLFSPull in a pipeline definition.

mark.earl.waite@gmail.com (JIRA)

unread,

Mar 9, 2017, 10:11:06 AM3/9/17

to jenkinsc...@googlegroups.com

Mark Waite commented on

The "pipeline syntax" link on most pipeline job pages will present a dropdown of pipeline steps. One of those is "checkout". Inside that "checkout" step, you can select git as the source control system, then add the "Additional Behaviour" for Git LFS pull.

If you prefer to read a script example, refer to the [Jenkinsfile in the ~~JENKINS-35687~~ branch in my jenkins-bugs repo|https://github.com/MarkEWaite/jenkins-bugs/blob/JENKINS-35687/Jenkinsfile.] That shows an example using GitLFSPull in a pipeline definition.

mark.earl.waite@gmail.com (JIRA)

unread,

Mar 9, 2017, 10:12:06 AM3/9/17

to jenkinsc...@googlegroups.com

Mark Waite edited a comment on

The "pipeline syntax" link on most pipeline job pages will present a dropdown of pipeline steps. One of those is "checkout". Inside that "checkout" step, you can select git as the source control system, then add the "Additional Behaviour" for Git LFS pull.

If you prefer to read a script example, refer to the [Jenkinsfile in the - JENKINS-35687 - branch in my jenkins-bugs repo|[https://github.com/MarkEWaite/jenkins-bugs/blob/JENKINS-35687/Jenkinsfile]] That shows an example using GitLFSPull in a pipeline definition.

mark.earl.waite@gmail.com (JIRA)

unread,

Mar 9, 2017, 10:12:07 AM3/9/17

to jenkinsc...@googlegroups.com

Mark Waite edited a comment on

The "pipeline syntax" link on most pipeline job pages will present a dropdown of pipeline steps. One of those is "checkout". Inside that "checkout" step, you can select git as the source control system, then add the "Additional Behaviour" for Git LFS pull.

If you prefer to read a script example, refer to the [Jenkinsfile in the JENKINS-35687 branch in my jenkins-bugs repo|[https://github.com/MarkEWaite/jenkins-bugs/blob/JENKINS-35687/Jenkinsfile]] That shows an example using GitLFSPull in a pipeline definition.

mark.earl.waite@gmail.com (JIRA)

unread,

Mar 9, 2017, 10:13:05 AM3/9/17

to jenkinsc...@googlegroups.com

Mark Waite edited a comment on

The "pipeline syntax" link on most pipeline job pages will present a dropdown of pipeline steps. One of those is "checkout". Inside that "checkout" step, you can select git as the source control system, then add the "Additional Behaviour" for Git LFS pull.

If you prefer to read a script example, refer to the [Jenkinsfile in my jenkins-bugs repo |[https://github.com/MarkEWaite/jenkins-bugs/blob/JENKINS-35687/Jenkinsfile]] in my jenkins-bugs repo. That shows an example using GitLFSPull in a pipeline definition.

mark.earl.waite@gmail.com (JIRA)

unread,

Mar 9, 2017, 10:16:21 AM3/9/17

to jenkinsc...@googlegroups.com

Mark Waite edited a comment on