[JIRA] (JENKINS-38699) Confine git commands to workspace directory

14 views
Skip to first unread message

io.jenkins.973288.david@distelpluis.nl (JIRA)

unread,
Oct 4, 2016, 10:42:08 AM10/4/16
to jenkinsc...@googlegroups.com
David Bouman created an issue
 
Jenkins / Improvement JENKINS-38699
Confine git commands to workspace directory
Issue Type: Improvement Improvement
Assignee: Mark Waite
Components: git-plugin
Created: 2016/Oct/04 2:41 PM
Environment: jenkins: 2.23 / Debian apt-get install
git-plugin: 3.0.0
git: 2.1.4
Labels: git
Priority: Minor Minor
Reporter: David Bouman

Please consider adding the means to confine the execution of a git command to a job's workspace directory.

What I'm looking for is the behaviour implied by the GIT_CEILING_DIRECTORIES environment variable.

https://git-scm.com/book/en/v2/Git-Internals-Environment-Variables#Repository-Locations

GIT_CEILING_DIRECTORIES controls the behavior of searching for a .git directory. If you access directories that are slow to load (such as those on a tape drive, or across a slow network connection), you may want to have Git stop trying earlier than it might otherwise, especially if Git is invoked when building your shell prompt.

In other words, a git command should never look for a `.git` directory outside of the job's workspace directory.

Why?

This would prevent any git command from clobbering any repositories further up the tree (case in point: Jenkins's $HOME) when a workspace's repo has somehow been messed up.

"Messed up" includes but is not limited to enabling the "Delete workspace before build starts" feature with some poorly understood exclude patterns that still left a complete directory structure including the .git directory in the workspace, but no files.

I've currently mitigated this problem by configuring GIT_CEILING_DIRECTORIES to ${HOME}/workspace in "Manage Jenkins / Configure System / Global properties / Environment variables", but I feel this would better be handled right out of the box.

Steps to reproduce

  1. System:
    1. cd into the Jenkins home directory (e.g., /var/lib/jenkins);
    2. git init && git add . && git commit -m `initial commit` (don't bother that lot should normally be .gitignore-ed);
  2. UI:
    1. Create a project that checks out from a git repo;
    2. For completeness sake, I've got the "Advanced clone behaviours / Shallow clone / depth 0" enabled but I do not expect that to make a difference;
    3. Trigger a build at least once to create a non-empty workspace.
    4. Enable "Build Environment / Delete workspace before build starts" and configure an "exclude" pattern to simply: .git;
    5. Trigger build again;
  3. System:
    1. Observe that your Jenkins home directory is now completely messed up; fortunately recoverably so; a git checkout master should fix it — at first glance at least; you should probably have to remove a remote and related branches too!

Fix:

(after cleaning up)

  1. UI:
    1. Go to "Manage Jenkins+ / Configure System / Global properties / Environment variables"
    2. Add GIT_CEILING_DIRECTORIES with value ${HOME}/workspace
    3. Trigger build again

The console log should now include something like this

16:29:32.561  > git rev-parse --is-inside-work-tree # timeout=10
16:29:32.573 ERROR: Workspace has a .git repository, but it appears to be corrupt.
16:29:32.575 hudson.plugins.git.GitException: Command "git rev-parse --is-inside-work-tree" returned status code 128:
16:29:32.575 stdout: 
16:29:32.575 stderr: fatal: Not a git repository (or any of the parent directories): .git
...
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
Atlassian logo

mark.earl.waite@gmail.com (JIRA)

unread,
Oct 4, 2016, 10:51:16 AM10/4/16
to jenkinsc...@googlegroups.com
Mark Waite assigned an issue to Unassigned
Change By: Mark Waite
Assignee: Mark Waite

mrichar2@paychex.com (JIRA)

unread,
Apr 1, 2020, 12:06:03 PM4/1/20
to jenkinsc...@googlegroups.com
Mark R commented on Improvement JENKINS-38699
 
Re: Confine git commands to workspace directory

Looks like a good feature. We ran into this because we have a git mechanism that backs up job config.xmls, nextBuildNumber, and a few other files to git nightly. Every now and then a pipeline jobs git gets corrupted somehow (not sure how) and it picks up the git workspace at the root jobs level. This causes it to checkout git at the root jobs level and causes problems.

The workaround of setting the GIT_CEILING_DIRECTORIES worked though we did find it is case sensitive.

This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)
Atlassian logo

io.jenkins.973288.david@distelpluis.nl (JIRA)

unread,
Apr 1, 2020, 3:43:03 PM4/1/20
to jenkinsc...@googlegroups.com

Given this issue was reported almost four years ago, I no longer understand why I suggested ${HOME}/workspace as a good value for GIT_CEILING_DIRECTORIES.

In its current version (2.229) Jenkins's home directory does not contain a workspace subdirectory. Perhaps some previous version had a different directory layout, but now it would seem saner to set GIT_CEILING_DIRECTORIES to ${HOME}/jobs.

mrichar2@paychex.com (JIRA)

unread,
Apr 1, 2020, 6:25:02 PM4/1/20
to jenkinsc...@googlegroups.com
Mark R edited a comment on Improvement JENKINS-38699
Looks like a good feature. We ran into this because we have a git mechanism that backs up job config.xmls, nextBuildNumber, and a few other files to git nightly. Every now and then a pipeline jobs git gets corrupted somehow (not sure how) and it picks up the git workspace at the root jobs level. This causes it to checkout git at the root jobs level and causes problems.

The workaround of setting the GIT_CEILING_DIRECTORIES worked though we did find it is case sensitive (we set to home/jobs) .

mrichar2@paychex.com (JIRA)

unread,
Apr 1, 2020, 6:26:03 PM4/1/20
to jenkinsc...@googlegroups.com
Mark R commented on Improvement JENKINS-38699

Yes we set it to ${HOME}/jobs. I was wondering where workspace came from.

mrichar2@paychex.com (JIRA)

unread,
Apr 1, 2020, 6:27:03 PM4/1/20
to jenkinsc...@googlegroups.com
Mark R edited a comment on Improvement JENKINS-38699
Looks like a good feature. We ran into this because we have a git mechanism that backs up job config.xmls, nextBuildNumber, and a few other files to git nightly. Every now and then a pipeline jobs git gets corrupted somehow (not sure how) and it picks up the git workspace at the root jobs level. This causes it to checkout git at the root jobs level and causes problems.

The workaround of setting the GIT_CEILING_DIRECTORIES worked though we did find it is case sensitive (we set to home ${HOME} /jobs).

io.jenkins.973288.david@distelpluis.nl (JIRA)

unread,
Apr 2, 2020, 2:23:02 AM4/2/20
to jenkinsc...@googlegroups.com
David Bouman updated an issue
 
Change By: David Bouman
Please consider adding the means to confine the execution of a git command to a job's workspace directory.

What I'm looking for is the behaviour implied by the *{{GIT_CEILING_DIRECTORIES}}* environment variable.

[https://git-scm.com/book/en/v2/Git-Internals-Environment-Variables#
Repository _repository_locations|https://git - Locations scm.com/book/en/v2/Git-Internals-Environment-Variables#_repository_locations ]
{quote}

*{{GIT_CEILING_DIRECTORIES}}* controls the behavior of searching for a {{ .git }} directory. If you access directories that are slow to load (such as those on a tape drive, or across a slow network connection), you may want to have Git stop trying earlier than it might otherwise, especially if Git is invoked when building your shell prompt.
{quote}
[https://git-scm.com/docs/git#Documentation/git.txt-codeGITCEILINGDIRECTORIEScode]
{quote}This should be a colon-separated list of absolute paths. If set, it is a list of directories that Git should not chdir up into while looking for a repository directory (useful for excluding slow-loading network directories). It will not exclude the current working directory or a {{GIT_DIR}} set on the command line or in the environment.
{quote}
In other words, a {{ git }} command should never look for a ` {{ .git ` }} directory outside of the job's workspace directory.

h3. Why?

This would prevent any
{{ git }} command from clobbering any repositories further up the tree (case in point: Jenkins's *{{$HOME}}*) when a workspace's repo has somehow been messed up.

"Messed up" includes
, but is not limited to enabling configuring the "_Delete workspace before build starts_" feature with some poorly understood exclude patterns that still left delete files but cause a complete directory structure \- including the {{.git}} directory \- to be left in the workspace , but no files .

I've currently mitigated this problem by configuring *{{GIT_CEILING_DIRECTORIES}}* to {{$\{HOME\}/
workspace jobs }} in "+Manage Jenkins+ / +Configure System+ / +Global properties+ / +Environment variables+", but I feel this would better be handled right out of the box.

h3. Steps to reproduce

# System:
## {{cd}} into the Jenkins home directory (e.g., {{/var/lib/jenkins}});
## {{git init && git add . && git commit -m `initial commit`}} (don't bother
for now that lot should normally be {{.gitignore}}-ed);
# UI:
## Create a project that checks out from a git repo;
## For completeness sake, I've got the _"+Advanced clone behaviours+ / +Shallow clone+ / +depth 0+"_ enabled but I do not expect that to make a difference;
## Trigger a build at least once to create a non-empty workspace.
## Enable _"+Build Environment+ / +Delete workspace before build starts+"_ and configure an _"+exclude+"_ pattern to simply: {{.git}};
## Trigger build again;
# System:


# # * Observe that your Jenkins home directory is now completely messed up; fortunately
#* Fortunately
recoverably so ; a : A {{git checkout master}} should fix it --- at first glance at least; you should 'd probably have to remove a remote and related branches too!

h3. Fix:

(after cleaning up)

# UI:
## Go to _"Manage Jenkins+ / +Configure System+ / +Global properties+ / +Environment variables+"_
## Add {{GIT_CEILING_DIRECTORIES}} with value {{$
\ {HOME \ }/ workspace jobs }}
## Trigger build again

The A subsequent build's console log should now include something like this

{noformat}

16:29:32.561  > git rev-parse --is-inside-work-tree # timeout=10
16:29:32.573 ERROR: Workspace has a .git repository, but it appears to be corrupt.
16:29:32.575 hudson.plugins.git.GitException: Command "git rev-parse --is-inside-work-tree" returned status code 128:
16:29:32.575 stdout:
16:29:32.575 stderr: fatal: Not a git repository (or any of the parent directories): .git
...
{noformat}

io.jenkins.973288.david@distelpluis.nl (JIRA)

unread,
Apr 2, 2020, 2:24:03 AM4/2/20
to jenkinsc...@googlegroups.com
David Bouman commented on Improvement JENKINS-38699
 
Re: Confine git commands to workspace directory

I've adjusted the issue description.

io.jenkins.973288.david@distelpluis.nl (JIRA)

unread,
Apr 2, 2020, 2:27:03 AM4/2/20
to jenkinsc...@googlegroups.com
David Bouman updated an issue
Change By: David Bouman
Please consider adding the means to confine the execution of a git command to a job's workspace directory.

What I'm looking for is the behaviour implied by the *{{GIT_CEILING_DIRECTORIES}}* environment variable.


{quote}*{{GIT_CEILING_DIRECTORIES}}* controls the behavior of searching for a {{.git}} directory. If you access directories that are slow to load (such as those on a tape drive, or across a slow network connection), you may want to have Git stop trying earlier than it might otherwise, especially if Git is invoked when building your shell prompt.
{quote}
[https://git-scm.com/docs/git#Documentation/git.txt-codeGITCEILINGDIRECTORIEScode]
{quote}This should be a colon-separated list of absolute paths. If set, it is a list of directories that Git should not chdir up into while looking for a repository directory (useful for excluding slow-loading network directories). It will not exclude the current working directory or a {{GIT_DIR}} set on the command line or in the environment.
{quote}
In other words, a {{git}} command should never look for a {{.git}} directory outside of the job's workspace directory.
h3. Why?

This would prevent any {{git}} command from clobbering any repositories further up the tree (case in point: Jenkins's *{{$HOME}}*) when a workspace's repo has somehow been messed up.

"Messed up" includes, but is not limited to configuring the "_Delete workspace before build starts_" feature with some poorly understood exclude patterns that delete files but cause a complete directory structure \- including the {{.git}} directory \- to be left in the workspace.

I've currently mitigated this problem by configuring *{{GIT_CEILING_DIRECTORIES}}* to {{$\{HOME\}/jobs}} in "+Manage Jenkins+ / +Configure System+ / +Global properties+ / +Environment variables+", but I feel this would better be handled right out of the box.


h3. Steps to reproduce

# System:
## {{cd}} into the Jenkins home directory (e.g., {{/var/lib/jenkins}});
## {{git init && git add . && git commit -m `initial commit`}} (don't bother for now that lot should normally be {{.gitignore}}-ed);
# UI:
## Create a project that checks out from a git repo;
## For completeness sake, I've got the _"+Advanced clone behaviours+ / +Shallow clone+ / +depth 0+"_ enabled but I do not expect that to make a difference;
## Trigger a build at least once to create a non-empty workspace.
## Enable _"+Build Environment+ / +Delete workspace before build starts+"_ and configure an _"+exclude+"_ pattern to simply: {{.git}};
## Trigger build again;
# System:
#* Observe that your Jenkins home directory is now completely messed up;
#* Fortunately recoverably so: A {{git checkout master}} should fix it — at first glance at least; you'd probably have to remove a remote and related branches too!


h3. Fix:

(after cleaning up)
# UI:
## Go to _" + Manage Jenkins+ / +Configure System+ / +Global properties+ / +Environment variables+"_
## Add {{GIT_CEILING_DIRECTORIES}} with value {{${HOME}/jobs}}
## Trigger build again


A subsequent build's console log should now include something like this
{noformat}
16:29:32.561  > git rev-parse --is-inside-work-tree # timeout=10
16:29:32.573 ERROR: Workspace has a .git repository, but it appears to be corrupt.
16:29:32.575 hudson.plugins.git.GitException: Command "git rev-parse --is-inside-work-tree" returned status code 128:
16:29:32.575 stdout:
16:29:32.575 stderr: fatal: Not a git repository (or any of the parent directories): .git
...
{noformat}
Reply all
Reply to author
Forward
0 new messages