[JIRA] (JENKINS-56404) Impossible exclude redundant/double fetches

[kanstantsin.sha@gmail.com (JIRA)]

Kanstantsin Shautsou created an issue

Jenkins / JENKINS-56404 Impossible exclude redundant/double fetches Issue Type: Bug Assignee: Mark Waite Components: git-client-plugin, git-plugin Created: 2019-03-05 01:41 Priority: Minor Reporter: Kanstantsin Shautsou When hasGitRepo is false retrieveChanges tries firstly to Clone repository and then fetch according to available list of repositories. https://github.com/jenkinsci/git-plugin/blob/e9d15cd3a390f1d1849b9d6cbee707bc85213ab0/src/main/java/hudson/plugins/git/GitSCM.java#L1104-L1110 But when it clones it picks first repository name and injects ALL branches fetch https://github.com/jenkinsci/git-client-plugin/blob/2b217bd90a93462112176febbb5f30e05141c674/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java#L704-L712 When refspecs is empty - it fetch all branches (git default), when it null it fetches all branches. The workaround is to set some refspec from branch i.e. https://github.com/jenkinsci/git-plugin/blob/a7ad6eb38a00397c4a656b53d99bfbad8ea3b2cc/src/main/java/jenkins/plugins/git/GitSCMSourceDefaults.java#L108-L115 But, later it will be fetched anyway. Solution: Don't fetch anything during clone operation (useful part is clean, fetch etc). OR pass repositories to CloneComand and remove separate fetch iteration . Add Comment

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

[kanstantsin.sha@gmail.com (JIRA)]

Kanstantsin Shautsou updated an issue

Jenkins / JENKINS-56404 Impossible exclude redundant/double fetches

Change By: Kanstantsin Shautsou When hasGitRepo is false , retrieveChanges () tries firstly to Clone repository and then fetch according to available list of repositories.

https://github.com/jenkinsci/git-plugin/blob/e9d15cd3a390f1d1849b9d6cbee707bc85213ab0/src/main/java/hudson/plugins/git/GitSCM.java#L1104-L1110 But when it clones it picks first repository name and injects  ALL branches fetch https://github.com/jenkinsci/git-client-plugin/blob/2b217bd90a93462112176febbb5f30e05141c674/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java#L704-L712

When So, when refspecs is empty - it fetch all branches (git default), when it null it fetches all branches.

The workaround is to set some refspec from branch i.e. https://github.com/jenkinsci/git-plugin/blob/a7ad6eb38a00397c4a656b53d99bfbad8ea3b2cc/src/main/java/jenkins/plugins/git/GitSCMSourceDefaults.java#L108-L115 But, later it will be fetched anyway. Solution: Don't fetch anything during clone operation (useful part is clean, fetch etc). OR pass repositories to CloneComand and remove separate fetch iteration .

Add Comment

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite assigned an issue to Unassigned

Jenkins / JENKINS-56404 Impossible exclude redundant/double fetches

Change By: Mark Waite Assignee: Mark Waite

Add Comment

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite commented on JENKINS-56404

Re: Impossible exclude redundant/double fetches As far as I can tell from testing, when a refspec is defined and the clone option "honor refspec on inital clone" is enabled, the refspec is honored in both fetches that occur. Do you have a case where the clone option "honor refspec on initial clone" is not having the desired result? It is an intentional choice to retain compatibility with the existing (flawed) behavior of using a default refspec which fetches all branches. Too many users depend on fetching all branches without realizing they have that dependency. If the default behavior were changed, they would need to adjust the definitions of those jobs. Multibranch pipeline intentionally changed its behavior relatively early in development. Originally it used the default refspec for each job defined by the multibranch pipeline. A breaking change was accepted that uses a narrow refspec for exactly the branch dedicated to the specific job. Even that early stage breaking change caused some challenges for user, but the challenge was considered worth it due to the savings provided to multibranch pipeline jobs by the narrow refspec.

Add Comment

[kanstantsin.sha@gmail.com (JIRA)]

Kanstantsin Shautsou commented on JENKINS-56404

Re: Impossible exclude redundant/double fetches

"It is an intentional choice to retain compatibility with the existing (flawed) behavior of using a default refspec which fetches all branches. Too many users depend on fetching all branches without realizing they have that dependency. If the default behavior were changed, they would need to adjust the definitions of those jobs." That absolutely unrelated, refspec must tbe added only when repositories are not defined, but they are DEFINED and ignored, code is taking ONLY origin name from the first repository (refspec).

", the refspec is honored in both fetches that occur. "

It shouldn't do two fetches, on huge instances it's only adding unneeded load on git server - this is a BUG. When refspecs (repositories) are clearly specified it's fetching ANYWAY all branches, because it takes from the list of repositories. In current situatiton it's IMPOSSIBLE to avoid double fetches: impossible to inherite and reimplement retrieve method, impossible to exclude fetch during clone or exclude all fetches that happen later. This is critical for performance and huge instances.

Add Comment

[kanstantsin.sha@gmail.com (JIRA)]

Kanstantsin Shautsou updated an issue

Jenkins / JENKINS-56404 Impossible exclude redundant/double fetches

Change By: Kanstantsin Shautsou Priority: Minor Critical

Add Comment

[rishabhbudhouliya+jenkins@gmail.com (JIRA)]

Rishabh Budhouliya commented on JENKINS-56404

Re: Impossible exclude redundant/double fetches

I have attempted to fix this issue:  https://github.com/jenkinsci/git-plugin/pull/845

Add Comment

This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)