[JIRA] (JENKINS-61187) Option to create ref-spec to target branch when cloning

2 views
Skip to first unread message

stephane@odul.com (JIRA)

unread,
Feb 21, 2020, 6:50:03 PM2/21/20
to jenkinsc...@googlegroups.com
Stephane Odul updated an issue
 
Jenkins / Improvement JENKINS-61187
Option to create ref-spec to target branch when cloning
Change By: Stephane Odul
Summary: Option to create ref-spec to target branch when cloning
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

mark.earl.waite@gmail.com (JIRA)

unread,
Feb 21, 2020, 7:38:02 PM2/21/20
to jenkinsc...@googlegroups.com
Mark Waite commented on Improvement JENKINS-61187
 
Re: Option to create ref-spec to target branch when cloning

Multibranch pipelines for most repository providers (GitHub, Bitbucket, GitLab, and Gitea) should already automatically narrow the refspec for the specific branch in the checkout. The branch source provider in the git plugin does not perform that narrowing, but users can perform the narrowing themselves in the checkout command if needed.

mark.earl.waite@gmail.com (JIRA)

unread,
Feb 21, 2020, 7:38:03 PM2/21/20
to jenkinsc...@googlegroups.com
Mark Waite assigned an issue to Unassigned
 
Change By: Mark Waite
Assignee: Mark Waite

stephane@odul.com (JIRA)

unread,
Feb 22, 2020, 3:21:03 PM2/22/20
to jenkinsc...@googlegroups.com
Stephane Odul commented on Improvement JENKINS-61187
 
Re: Option to create ref-spec to target branch when cloning

Mark Waite

Unfortunately the multi branch pipelines are not a good substitute for the use case we need since they pretty much create a separate 'job' per branch, which in turn is causing scaling issues on the master with git repos that have hundreds of branches, even if we filter the branches. They are definitely not ideal for manually triggered jobs with input parameters from users.

We do use them for our CI and Pull Requests workload, but not for the other types of jobs we use for various other tasks.

Can you clarify how users can narrow the selection during the checkout command? We tried to set the ref-spec to +refs/heads/$BRANCH:refs/remotes/origin/$BRANCH but variables do not get expanded (there is a separate Jira for this) and that variable usually starts with origin/ which would make crafting the refspec a problem anyways.

 

mark.earl.waite@gmail.com (JIRA)

unread,
Feb 22, 2020, 4:16:02 PM2/22/20
to jenkinsc...@googlegroups.com

I've created a sample job which illustrates your case. It defines a parameterized Pipeline job that is not a multibranch job, places the job definition inside Jenkins instead of in the SCM, and uses the user selected branch name parameter to narrow the refspec to exactly that branch. It also uses shallow clone of depth 1 and a reference repository to further reduce the time and bandwidth required for git fetch. Clones of individual branches of my 45 MB repository consistently complete in less than 5 seconds with that configuration.

See the parameterized Pipeline job in the lts-with-plugins branch of my docker-lfs repository.

mark.earl.waite@gmail.com (JIRA)

unread,
Feb 25, 2020, 8:32:02 AM2/25/20
to jenkinsc...@googlegroups.com
Mark Waite edited a comment on Improvement JENKINS-61187
I've created a sample job which illustrates your case.  It defines a parameterized Pipeline job that is not a multibranch job, places the job definition inside Jenkins instead of in the SCM, and uses the user selected branch name parameter to narrow the refspec to exactly that branch.  It also uses shallow clone of depth 1 and a reference repository to further reduce the time and bandwidth required for {{git fetch}}.  Clones of individual branches of my 45 MB repository consistently complete in less than 5 seconds with that configuration.

See the [parameterized Pipeline job|https://github.com/MarkEWaite/docker-lfs/blob/lts-with-plugins/ref/jobs/Bugs-Individual/jobs/JENKINS-61187-parameterized-pipeline-job-with-branch-specific-refspec/config.xml] in the [lts-with-plugins branch|https://github.com/MarkEWaite/docker-lfs/tree/lts-with-plugins] of my [docker-lfs repository|https://github.com/MarkEWaite/docker-lfs].


[~sodul] can you confirm that the examples resolve the issue you were seeing?

mark.earl.waite@gmail.com (JIRA)

unread,
Feb 25, 2020, 8:32:03 AM2/25/20
to jenkinsc...@googlegroups.com
Mark Waite edited a comment on Improvement JENKINS-61187
I've created a sample job which illustrates your case.  It defines a parameterized Pipeline job that is not a multibranch job, places the job definition inside Jenkins instead of in the SCM, and uses the user selected branch name parameter to narrow the refspec to exactly that branch.  It also uses shallow clone of depth 1 and a reference repository to further reduce the time and bandwidth required for {{git fetch}}.  Clones of individual branches of my 45 MB repository consistently complete in less than 5 seconds with that configuration.

See the [parameterized Pipeline job|https://github.com/MarkEWaite/docker-lfs/blob/lts-with-plugins/ref/jobs/Bugs-Individual/jobs/JENKINS-61187-parameterized-pipeline-job-with-branch-specific-refspec/config.xml] in the [lts-with-plugins branch|https://github.com/MarkEWaite/docker-lfs/tree/lts-with-plugins] of my [docker-lfs repository|https://github.com/MarkEWaite/docker-lfs].

[~sodul] can you confirm that the examples resolve example resolves the issue you were seeing?

mark.earl.waite@gmail.com (JIRA)

unread,
Mar 6, 2020, 11:11:04 AM3/6/20
to jenkinsc...@googlegroups.com
Mark Waite closed an issue as Won't Do
 

After two weeks without a response, I'm closing this on the assumption that the example I provided is a working example for the described issue.

Change By: Mark Waite
Status: Open Closed
Resolution: Won't Do
This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)
Atlassian logo

stephane@odul.com (JIRA)

unread,
Mar 6, 2020, 11:35:04 AM3/6/20
to jenkinsc...@googlegroups.com
Stephane Odul commented on Improvement JENKINS-61187
 
Re: Option to create ref-spec to target branch when cloning

Sorry for not replying sooner, unfortunately were not able to try the given example yet due to more pressing issues on our side.

We actually extended our Git wrapper to massage more of the git command, add retry logic and handle adding a depth when performing pull request jobs.

We run into a bug where the value of the git command ends up being an empty string in one of several stages. This is random and rare and only when we have the custom installer configured. Once we have more time to investigate  we will file additional reports for these bugs.

 

Thank you for your help Mark Waite

Reply all
Reply to author
Forward
0 new messages