[JIRA] (JENKINS-59139) don't set GIT_LFS_SKIP_SMUDGE=1 anymore while cloning

flokli@flokli.de (JIRA)

unread,

Aug 29, 2019, 7:03:02 AM8/29/19

to jenkinsc...@googlegroups.com

Florian Klink created an issue

Jenkins /

JENKINS-59139

don't set GIT_LFS_SKIP_SMUDGE=1 anymore while cloning

Issue Type:	Bug
Assignee:	Mark Waite
Components:	git-client-plugin
Created:	2019-08-29 11:02
Priority:	Minor
Reporter:	Florian Klink

Since https://github.com/jenkinsci/git-client-plugin/commit/853603cccd4434b116ef9b8e094c3f5b815aa75a, we set GIT_LFS_SKIP_SMUDGE=1 to improve performance, and require users to add the "Git LFS pull after checkout" "Additional behaviour" to resolve pointers.

This was mostly done because git clone performance was pretty bad with the smudge filter enabled, and in some configurations, (ssh clone URLs), git lfs pull didn't work.

Since then, a lot has improved:

Clone Performance through the smudge filter has improved since https://github.com/git-lfs/git-lfs/pull/2511.

Backends should support the `git-lfs-authenticate` dance as described in https://github.com/git-lfs/git-lfs/blob/master/docs/api/authentication.md#ssh, which retrieves a temporary authorization token to do the clone over https.

I propose to just do a git clone without setting `GIT_LFS_SKIP_SMUDGE=1`. Most systems should have done a `git lfs install` before, so they will resolve the lfs pointers during a clone. We can keep the "Git LFS pull after checkout" "Additional behaviour", for systems that don't have setup global smudge filters via `git lfs install`.

Add Comment

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

mark.earl.waite@gmail.com (JIRA)

unread,

Aug 29, 2019, 9:20:02 AM8/29/19

to jenkinsc...@googlegroups.com

Mark Waite assigned an issue to Unassigned

Jenkins /

JENKINS-59139

don't set GIT_LFS_SKIP_SMUDGE=1 anymore while cloning

Change By:	Mark Waite
Assignee:	Mark Waite

Add Comment

mark.earl.waite@gmail.com (JIRA)

unread,

Aug 29, 2019, 9:21:03 AM8/29/19

to jenkinsc...@googlegroups.com

Mark Waite commented on

JENKINS-59139

Re: don't set GIT_LFS_SKIP_SMUDGE=1 anymore while cloning

Your description seems reasonable to me. I want to see the results of evaluating a pull request on the Linux (Amazon Linux, CentOS, Debian, Ubuntu), Windows, and FreeBSD environments where I run tests regularly. Will you submit the pull request?

Add Comment

mark.earl.waite@gmail.com (JIRA)

unread,

Sep 2, 2019, 6:08:02 PM9/2/19

to jenkinsc...@googlegroups.com

Mark Waite edited a comment on

JENKINS-59139

Re: don't set GIT_LFS_SKIP_SMUDGE=1 anymore while cloning

Your description seems reasonable to me. I want to see the results of evaluating a pull request on the Linux (Amazon Linux, CentOS, Debian, Ubuntu), Windows, and FreeBSD environments where I run tests regularly. Will you submit the pull request?

[PR 2511 to git-lfs|https://github.com/git-lfs/git-lfs/pull/2511] seems to indicate that specific versions of command line git and of git lfs are needed in order to gain the benefit of the change.  Can you confirm that command line git versions 2.15 and later are needed and that git lfs 2.3.0 and later are needed?

If so, then the pull request will need to check the command line git version and the git lfs version before removing the GIT_LFS_SKIP_SMUDGE environment variable.  If that conditional is not included in the change, then many operating systems that deliver older versions of command line git or older versions of git lfs will suffer a performance loss from removing that environment variable.

||Operating System||CLI git version delivered||
|Amazon Linux 2    |2.17|
|CentOS 6               |1.7.1|
|CentOS 7               |1.8|
|Debian 9                |2.11|
|Debian 10              |2.20|
|Ubuntu 16              |2.7|
|Ubuntu 18              |2.17|
|FreeBSD 12             |2.21|

Add Comment

flokli@flokli.de (JIRA)

unread,

Sep 2, 2019, 11:31:01 PM9/2/19

to jenkinsc...@googlegroups.com

Florian Klink commented on

JENKINS-59139

Re: don't set GIT_LFS_SKIP_SMUDGE=1 anymore while cloning

I did some quick digging, git >= 2.15 and git-lfs >= 2.3.0 sounds reasonable, by looking at the tags containing commits part of those PRs.

`git-client-plugin` currently lacks a way to determine the version of git-lfs, but that should not be too hard to add.

However, changing the logic might be a bit more involved if we want to do the version check as described above (and I think it makes sense to do so).

Currently, the `GitLFSPull` `GitSCMExtension` ("Git LFS pull after checkout") simply calls `CliGitAPIImpl.lfsRemote` with `repos.get(0).getName()` - and inside `CliGitAPIImpl`, all LFS-related functionality is conditional on `(lfsRemote != null)`.

If we now want to rely on the smudge filter for recent enough git(-lfs) versions, and only have the `GitLFSPull` `GitSCMExtension` to trigger `git lfs pull` for old git(-lfs) versions,`GitLFSPull` should be changed to simply flip a boolean in `CliGitAPIImpl`, and the logic in there should be made around that. Does that sound right to you?