[JIRA] (JENKINS-60204) git plugin 4.0.0 can not shallow clone a submodule

50 views
Skip to first unread message

mark.earl.waite@gmail.com (JIRA)

unread,
Nov 19, 2019, 2:12:02 PM11/19/19
to jenkinsc...@googlegroups.com
Mark Waite updated an issue
 
Jenkins / Bug JENKINS-60204
git plugin 4.0.0 can not shallow clone a submodule
Change By: Mark Waite
Summary: git plugin 4.0.0 can not handle submodules shallow clone a submodule
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

mark.earl.waite@gmail.com (JIRA)

unread,
Nov 19, 2019, 2:14:03 PM11/19/19
to jenkinsc...@googlegroups.com
Mark Waite commented on Bug JENKINS-60204
 
Re: git plugin 4.0.0 can not shallow clone a submodule

René Scheibe was shallow clone for submodules an area that you submitted in 4.0.0, or was that someone else?

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 2:33:01 PM11/19/19
to jenkinsc...@googlegroups.com
Benjamin Brummer edited a comment on Bug JENKINS-60204
If shallow clone for the submodule is activated this error happens on my side. It has to be a clean build.

 

Additionally: If i activate shallow clone and the build fails the submodule-folder has just a .git. If i deactivate shallow clone again it will still fail, until i wipe the workspace again.

rene.scheibe@gmail.com (JIRA)

unread,
Nov 19, 2019, 3:10:11 PM11/19/19
to jenkinsc...@googlegroups.com

Benjamin Brummer can you reproduce this error with a) only a specific private git repository or b) also with an open source one from GitHub?
Can you provide the full XML of the configured job here (maybe with redacted repository URL and credentials)?

rene.scheibe@gmail.com (JIRA)

unread,
Nov 19, 2019, 3:24:02 PM11/19/19
to jenkinsc...@googlegroups.com
René Scheibe edited a comment on Bug JENKINS-60204
[~benbrummer] can you reproduce this error with a) only a specific private git repository or b) also with an open source one from GitHub?
Can you provide the full XML of the configured job here and a stripped down Jenkinsfile (maybe with redacted repository URL and credentials)?

mark.earl.waite@gmail.com (JIRA)

unread,
Nov 19, 2019, 3:39:02 PM11/19/19
to jenkinsc...@googlegroups.com

René Scheibe I have enabled shallow clone for the parent repository and for the submodule update in my JENKINS-60204 check job. It works as expected and does not show any failure. I still am unable to duplicate the problem as reported by Benjamin Brummer.

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 3:40:02 PM11/19/19
to jenkinsc...@googlegroups.com
Benjamin Brummer updated an issue
 
Change By: Benjamin Brummer
Attachment: works.xml

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 3:41:02 PM11/19/19
to jenkinsc...@googlegroups.com
Benjamin Brummer commented on Bug JENKINS-60204
 
Re: git plugin 4.0.0 can not shallow clone a submodule

I tried to reproduce it with a public repo without success, so a) only private repo.

works.xml

fail.xml

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 3:41:03 PM11/19/19
to jenkinsc...@googlegroups.com
Benjamin Brummer updated an issue
Change By: Benjamin Brummer
Attachment: fail.xml

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 3:47:01 PM11/19/19
to jenkinsc...@googlegroups.com
Benjamin Brummer edited a comment on Bug JENKINS-60204
I tried to reproduce it with a public repo without success, so a) only private repo.

[^works.xml]

[^fail.xml]

^[^fail.log]^

rene.scheibe@gmail.com (JIRA)

unread,
Nov 19, 2019, 4:06:02 PM11/19/19
to jenkinsc...@googlegroups.com

Mark Waite yes I have been involved in the shallow submodule update - see

I also cleaned up the original parallel submodule update implementation - see

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 4:23:05 PM11/19/19
to jenkinsc...@googlegroups.com

I checked the submodule again updated it to a different commit already and now i removed it, added it again => it looks like it is working now.

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 4:25:02 PM11/19/19
to jenkinsc...@googlegroups.com
Benjamin Brummer edited a comment on Bug JENKINS-60204
I checked the submodule again updated it to a different commit already and now .

Now i removed it, added it again => it looks like it is working now.

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 4:28:04 PM11/19/19
to jenkinsc...@googlegroups.com
Benjamin Brummer edited a comment on Bug JENKINS-60204
I checked the submodule again updated it to a different commit already.


Now i removed it, added it again => it looks like it is working now.


No clue what is going on.

rene.scheibe@gmail.com (JIRA)

unread,
Nov 19, 2019, 4:49:03 PM11/19/19
to jenkinsc...@googlegroups.com

I was first with confused by the the description that shallow submodule updates are used but the logged command didn't contain --depth=1. I guess it was edited somehow.

> git submodule update --remote impl/tool_protec # timeout=10

The provided fail.log contains the proper command logged

> git submodule update --remote --reference C:\jenkins\repo\submodule.git --depth=1 tool_protec # timeout=10

The reported git issue is not an issue of the new git-plugin 4.0.0.

hudson.plugins.git.GitException: Command "git submodule update --remote --reference C:\jenkins\repo\submodule.git --depth=1 tool_protec" returned status code 1:
stdout:
stderr: Cloning into 'C:/jenkins/workspace/submoduletest/master/tool_protec'...
fatal: Needed a single revision
Unable to find current origin/development revision in submodule path 'tool_protec'

You should also be able to reproduce it on the command line without Jenkins and this plugin involved. It's due to the fact that only fetching a single commit from the submodule repository but wanting to resolve the "origin/development" revision which is I guess not the latest commit. You can find plenty of such failed updates reported when searching the Internet for "git submodule update Unable to find current revision in submodule path".

The other issue reported in the logs is I guess due to some invalid usage in the Jenkinsfile. Please double check this.

Error when executing always post condition:
org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing

Does this sound reasonable to you Benjamin Brummer?

rene.scheibe@gmail.com (JIRA)

unread,
Nov 19, 2019, 4:52:02 PM11/19/19
to jenkinsc...@googlegroups.com
René Scheibe edited a comment on Bug JENKINS-60204
I was first with confused by the the description that shallow submodule updates are used but the logged command didn't contain {{--depth=1}}. I guess it was edited somehow or some intermediate setup while trying to reproduce the relevant job settings .
{code :java }
> git submodule update --remote impl/tool_protec # timeout=10{code}
The provided fail.log contains the proper command logged
{code :java }
> git submodule update --remote --reference C:\jenkins\repo\submodule.git --depth=1 tool_protec # timeout=10{code}
The reported git issue is not an issue of the new git-plugin 4.0.0.
{code :java }

hudson.plugins.git.GitException: Command "git submodule update --remote --reference C:\jenkins\repo\submodule.git --depth=1 tool_protec" returned status code 1:
stdout:
stderr: Cloning into 'C:/jenkins/workspace/submoduletest/master/tool_protec'...
fatal: Needed a single revision
Unable to find current origin/development revision in submodule path 'tool_protec'
{code}

You should also be able to reproduce it on the command line without Jenkins and this plugin involved. It's due to the fact that only fetching a single commit from the submodule repository but wanting to resolve the "origin/development" revision which is I guess not the latest commit. You can find plenty of such failed updates reported when searching the Internet for "git submodule update Unable to find current revision in submodule path".

The other issue reported in the logs is I guess due to some invalid usage in the Jenkinsfile. Please double check this.
{code :java }

Error when executing always post condition:
org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing
{code}
Does this sound reasonable to you [~benbrummer]?

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 4:58:02 PM11/19/19
to jenkinsc...@googlegroups.com

The missing "–depth=1" in the first log is caused, by running first with shallow clone on the submodule which leaves a somehow broken submodule directory behind. The next build was than not a clean build i guess.

 

None of the above errors happen when i downgrade to git plugin 3.12.1 with the exact same job configuration.

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 4:59:02 PM11/19/19
to jenkinsc...@googlegroups.com
Benjamin Brummer edited a comment on Bug JENKINS-60204
The missing "–depth=1" in the first log is caused, by running first with shallow clone on the submodule which leaves a somehow broken submodule directory behind. The next build was than not a clean build i guess. [comment|https://issues.jenkins-ci.org/browse/JENKINS-60204?focusedCommentId=380228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-380228]

 

None of the above errors happen when i downgrade to git plugin 3.12.1 with the exact same job configuration.

rene.scheibe@gmail.com (JIRA)

unread,
Nov 19, 2019, 5:00:02 PM11/19/19
to jenkinsc...@googlegroups.com
René Scheibe edited a comment on Bug JENKINS-60204
I was first with confused by the description that shallow submodule updates are used but the logged command didn't contain {{--depth=1}}. I guess it was edited somehow or some intermediate setup while trying to reproduce the relevant job settings.

{code:java}
> git submodule update --remote impl/tool_protec # timeout=10{code}
The provided fail.log contains the proper command logged
{code:java}
> git submodule update --remote --reference C:\jenkins\repo\submodule.git --depth=1 tool_protec # timeout=10{code}
The reported git issue is not an issue of the new git-plugin 4.0.0.
{code:java}
hudson.plugins.git.GitException: Command "git submodule update --remote --reference C:\jenkins\repo\submodule.git --depth=1 tool_protec" returned status code 1:
stdout:
stderr: Cloning into 'C:/jenkins/workspace/submoduletest/master/tool_protec'...
fatal: Needed a single revision
Unable to find current origin/development revision in submodule path 'tool_protec'
{code}
You should also be able to reproduce it on the command line without Jenkins and this plugin involved. It's due to the fact that only fetching a single commit from the submodule repository but wanting to resolve the "origin/development" revision which is I guess not the latest commit. You can find plenty of such failed updates reported when searching the Internet for "git submodule update Unable to find current revision in submodule path".

The other issue reported in the logs is I guess due to some invalid usage in the Jenkinsfile.
Please double check this.

 {code:java}
Error when executing always post condition:
org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing
{code}
From the fail.log I can only see that you are using a declarative pipeline with scripted block and parallel inside. I guess you might have missed to allocate a node in there as suggested.

Please double check this.

 

Does this sound reasonable to you [~benbrummer]?

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 5:07:03 PM11/19/19
to jenkinsc...@googlegroups.com
Benjamin Brummer edited a comment on Bug JENKINS-60204
The missing "–depth=1" in the first log is caused, by running first with shallow clone on the submodule which leaves a somehow broken submodule directory behind. The next build was than not a clean build i guess. [ comment |https://issues.jenkins-ci.org/browse/JENKINS-60204?focusedCommentId=380228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-380228]

 

 None of the above errors happen when i downgrade to git plugin 3.12.1 with the exact same job configuration.


When i switch between the original submodule and the readded submodule commit i can reproducible make the build fail/succeed.

I will test on commandline and report back. But i assume somehow the submodules where added in a strange way. Although i can not tell what is wrong and why it worked until this update.

rene.scheibe@gmail.com (JIRA)

unread,
Nov 19, 2019, 5:09:01 PM11/19/19
to jenkinsc...@googlegroups.com

git-plugin 3.12.1 didn't yet provide the shallow submodule update feature. After a downgrade of the plugin from 4.0.0 to 3.12.1 you should therefore see an entry in the "old data" section in Jenkins. Jenkins is clever enough to just ignore unknown settings. Therefore after the downgrade a git submodule update without the -- depth parameter is performed, always fetching all revisions of a submodule repository. Therefore "origin/development" can also be resolved.

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 5:21:02 PM11/19/19
to jenkinsc...@googlegroups.com

So it looks like a layer 8 problem on my side combined with inconsistent submodule handling in our company.

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 5:25:02 PM11/19/19
to jenkinsc...@googlegroups.com
Benjamin Brummer edited a comment on Bug JENKINS-60204
So it looks like a layer 8 problem on my side combined with inconsistent submodule handling in our company.


 

We always get this error when something is wrong with git.

{code:java}
Error when executing always post condition:
org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing
.....
Suppressed: org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing
Perhaps you forgot to surround the code with a step that provides this, such as: node
{code}

rene.scheibe@gmail.com (JIRA)

unread,
Nov 19, 2019, 5:39:02 PM11/19/19
to jenkinsc...@googlegroups.com
René Scheibe edited a comment on Bug JENKINS-60204
git-plugin 3.12.1 didn't yet provide the shallow submodule update feature. After a downgrade of the plugin from 4.0.0 to 3.12.1 you should therefore see an entry in the "old data" section in Jenkins. Jenkins is clever enough to just ignore unknown settings.

Therefore after the downgrade a {{git submodule update}} without the {{-- depth}} parameter is performed, always fetching all revisions of a submodule repository. Therefore "origin/development" can also be resolved.


Instead of downgrading the plugin you can just disable the new "Shallow clone" option in the [Advanced sub-modules behaviours|https://github.com/jenkinsci/git-plugin/blob/git-4.0.0/README.adoc#advanced-sub-modules-behaviours] in case you are using submodules in a way where shallow updating may not be feasible with a depth of 1. (You are using "Update tracking submodules to tip of branch" which uses the {{--remote}} parameter.)

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 5:43:05 PM11/19/19
to jenkinsc...@googlegroups.com
Benjamin Brummer edited a comment on Bug JENKINS-60204
So it looks like a layer 8 problem on my side combined with inconsistent submodule handling in our company.

 

We always get this error when something is wrong with git (e . g. Agent is running out of disk space and therefore git fails)
{code:java}
Error when executing always post condition:
org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing
.....
Suppressed: org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing
Perhaps you forgot to surround the code with a step that provides this, such as: node
{code}

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 5:45:02 PM11/19/19
to jenkinsc...@googlegroups.com
Benjamin Brummer edited a comment on Bug JENKINS-60204
So it looks like a layer 8 problem on my side combined with inconsistent submodule handling in our company.

 

We always get this error when something is wrong with git (e.g. Agent is running out of disk space and therefore git fails)

{code:java}
Error when executing always post condition:
org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing
.....
Suppressed: org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing
Perhaps you forgot to surround the code with a step that provides this, such as: node
{code}


jjjj

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 5:48:02 PM11/19/19
to jenkinsc...@googlegroups.com
Benjamin Brummer updated an issue
Change By: Benjamin Brummer
Attachment: git_status_submodule.png

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 5:50:02 PM11/19/19
to jenkinsc...@googlegroups.com
Benjamin Brummer edited a comment on Bug JENKINS-60204
So it looks like a layer 8 problem on my side combined with inconsistent submodule handling in our company.

 

We always get this error when something is wrong with git (e.g. Agent is running out of disk space and therefore git fails)
{code:java}
Error when executing always post condition:
org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing
.....
Suppressed: org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing
Perhaps you forgot to surround the code with a step that provides this, such as: node
{code}
jjjj So therefore the build failed cause a detached head status in the submodule can not work with shallow. When i readded the submodule it was tracking the master
!git_status_submodule.png|thumbnail!

The shallow clone was a missconfiguration on my side, which worked often but failed on branches with detached head submodules.

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 5:56:02 PM11/19/19
to jenkinsc...@googlegroups.com
Benjamin Brummer edited a comment on Bug JENKINS-60204
So it looks like a layer 8 problem on my side combined with inconsistent submodule handling in our company.

 

We always get this error when something is wrong with git (e.g. Agent is running out of disk space and therefore git fails)
{code:java}
Error when executing always post condition:
org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing
.....
Suppressed: org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing
Perhaps you forgot to surround the code with a step that provides this, such as: node
{code}

So therefore the build failed cause a detached head status in the submodule can not work with shallow. When i readded the submodule it was tracking the master
!git_status_submodule.png|thumbnail!

The shallow clone was a missconfiguration miss-configuration on my side, which worked often but failed on branches with detached head submodules. As it is hard to make sure that submodules are tracking a branch, which they per default will not do it is safer to not use this feature on our jenkins. This is handled inconsistently on git client side, e.g GitKraken tracks the branch by default for a submodule, therefore the build startet to work after i re-added the submodule.

benjamin.brummer@pinteam.eu (JIRA)

unread,
Nov 19, 2019, 5:59:04 PM11/19/19
to jenkinsc...@googlegroups.com
Benjamin Brummer edited a comment on Bug JENKINS-60204
So it looks like a layer 8 problem on my side combined with inconsistent submodule handling in our company.

 

We always get this error when something is wrong with git (e.g. Agent is running out of disk space and therefore git fails)
{code:java}
Error when executing always post condition:
org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing
.....
Suppressed: org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing
Perhaps you forgot to surround the code with a step that provides this, such as: node
{code}

So therefore the build failed cause because of a detached head status in the submodule , what can not work with shallow clone . When i readded the submodule it was tracking the master
!git_status_submodule.png|thumbnail!

The shallow clone was a miss-configuration on my side, which worked often but failed on branches with a detached head submodules submodule . As it is hard to make sure that submodules are tracking a branch, which they per default will not do it is safer to not use this feature on our jenkins. This is handled inconsistently on git client side, too , e.g GitKraken tracks the branch by default for a submodule, therefore the build startet to work after i re-added the submodule.
Reply all
Reply to author
Forward
0 new messages