[JIRA] (JENKINS-47978) Lightweight checkout not working for branches that contain forward slash '/'

[bjorn.rohlen@gmail.com (JIRA)]

bjorn rohlen commented on JENKINS-47978

Re: Lightweight checkout not working for branches that contain forward slash '/' This fix seriously broke something. We've been at it for weeks – it still will not find Jenkinsfile in subdirectories.  Started by user P*** S**** (p*.**) [Thu Jan 25 10:00:01 CET 2018] Starting branch indexing... Connecting to http://***.***.**.se using allowed.user/******Repository type: Git Looking up asd/asd_stuff for branches Checking branch feature/asd-things from asd/asd_stuff ‘javaproject/asd-stuff-autotest/Jenkinsfile’ not found Skipped Checking branch feature/ASD-123-test-doit from asd/asd_stuff ‘javaproject/asd-stuff-autotest/Jenkinsfile’ not found Skipped ... ... although the file is clearly present and worked before this patch. Add Comment

This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)

[bjorn.rohlen@gmail.com (JIRA)]

bjorn rohlen edited a comment on JENKINS-47978

Re: Lightweight checkout not working for branches that contain forward slash '/'

This fix seriously broke something. We've been at it for weeks a week – it still will not find Jenkinsfile in subdirectories.  Started by user P*** S**** (p**.***)

[Thu Jan 25 10:00:01 CET 2018] Starting branch indexing...

Connecting to [ http://***.***.**.se |http://%2A%2A%2A.%2A%2A%2A.%2A%2A.se/] using allowed.user/******Repository type: Git

Looking up asd/asd_stuff for branches Checking branch feature/asd-things from asd/asd_stuff       ‘javaproject/asd-stuff-autotest/Jenkinsfile’ not found     Skipped Checking branch feature/ASD-123-test-doit from asd/asd_stuff       ‘javaproject/asd-stuff-autotest/Jenkinsfile’ not found     Skipped ... ... although the file is clearly present and worked before this patch.

Add Comment

[bjorn.rohlen@gmail.com (JIRA)]

bjorn rohlen edited a comment on JENKINS-47978

Re: Lightweight checkout not working for branches that contain forward slash '/'

This fix seriously broke something. We've been at it for a week – it still will not find Jenkinsfile in subdirectories.

Started by user P*** S**** (p**.***) [Thu Jan 25 10:00:01 CET 2018] Starting branch indexing... Connecting to [http://***.***.**.se|http://%2A%2A%2A.%2A%2A%2A.%2A%2A.se/] using allowed.user/******Repository type: Git Looking up asd/asd_stuff for branches Checking branch feature/asd-things from asd/asd_stuff ‘javaproject/asd-stuff-autotest/Jenkinsfile’ not found Skipped Checking branch feature/ASD-123-test-doit from asd/asd_stuff ‘javaproject/asd-stuff-autotest/Jenkinsfile’ not found Skipped ...

... although the file is clearly present and worked before this patch. Jenkinsfile in root path of a repo will work just fine, in a subdirectory, not so fine.

Add Comment

[bjorn.rohlen@gmail.com (JIRA)]

bjorn rohlen reopened an issue

.. broke other things containing slash '/', see comment. Jenkins / JENKINS-47978

Lightweight checkout not working for branches that contain forward slash '/'

Change By: bjorn rohlen Resolution: Fixed Status: Resolved Reopened

Add Comment

[bjorn.rohlen@gmail.com (JIRA)]

bjorn rohlen edited a comment on JENKINS-47978

Re: Lightweight checkout not working for branches that contain forward slash '/'

This fix seriously broke something. We've been at it for a week – it still will not find Jenkinsfile in subdirectories.  Started by user P*** S**** (p**.***) [Thu Jan 25 10:00:01 CET 2018] Starting branch indexing... Connecting to [http://***.***.**.se|http://%2A%2A%2A.%2A%2A%2A.%2A%2A.se/] using allowed.user/******Repository type: Git Looking up asd/asd_stuff for branches Checking branch feature/asd-things from asd/asd_stuff ‘javaproject/asd-stuff-autotest/Jenkinsfile’ not found Skipped Checking branch feature/ASD-123-test-doit from asd/asd_stuff ‘javaproject/asd-stuff-autotest/Jenkinsfile’ not found Skipped ... ... although the file is clearly present and worked before this patch. Jenkinsfile in root path of a repo will work just fine, in a subdirectory, not so fine.

To clarify: If you specify Jenkinsfile in a subdirectory, that will no longer work. I've reproduced this numerous times before re-opening this issue on my own repos.

Add Comment

[bjorn.rohlen@gmail.com (JIRA)]

bjorn rohlen edited a comment on JENKINS-47978

Re: Lightweight checkout not working for branches that contain forward slash '/'

REPOENING ISSUE .. broke other things containing slash '/', see previous comment.

Add Comment

[bjorn.rohlen@gmail.com (JIRA)]

bjorn rohlen edited a comment on JENKINS-47978

Re: Lightweight checkout not working for branches that contain forward slash '/'

REPOENING REOPENING ISSUE

.. broke other things containing slash '/', see previous comment.

Add Comment

[vivek.pandey@gmail.com (JIRA)]

Vivek Pandey commented on JENKINS-47978

Re: Lightweight checkout not working for branches that contain forward slash '/'

bjorn rohlen Ok, so to confirm Jenkinsfile is getting picked correctly when at its located at repository/branch root. If placed inside a subdirectory its not detected. I checked with github branch source as well, Jenkinsfile discovery doesn't get picked from subdirectory. Looks like this is consistent compared to github branch source. Stephen Connolly can you shade some light as whats expected behavior here?

Add Comment

[stephen.alan.connolly@gmail.com (JIRA)]

Stephen Connolly commented on JENKINS-47978

Re: Lightweight checkout not working for branches that contain forward slash '/'

If the SCMSourceCriteria is looking for a file at a non-root path, that is supposed to work. If it is not working for GitHub then that is a bug in GitHub, as is not working for Bitbucket here. https://github.com/jenkinsci/workflow-multibranch-plugin/pull/59 was the PR that enabled the use case in multibranch pipelines and is aligned with the architectural designs of SCM API.

Add Comment

[stephen.alan.connolly@gmail.com (JIRA)]

Stephen Connolly commented on JENKINS-47978

Re: Lightweight checkout not working for branches that contain forward slash '/'

it is possible that GitHub Branch Source has a regression, but https://github.com/jenkinsci/github-branch-source-plugin/blob/master/src/test/java/org/jenkinsci/plugins/github_branch_source/GitHubSCMFileSystemTest.java#L193 is passing (yes based on an evil replayed capture of real github), so unless https://github.com/jenkinsci/github-branch-source-plugin/blob/master/src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubSCMProbe.java#L135-L151 has a bug, we should be detecting the Jenkinsfile at the path and creating the branch, and then loading the file from the lightweight checkout (which the test "claims" is working)   So it looks like GitHub is working with child paths... of course I don't trust the wiremock tests and would love to rip them out... but GitHub has that rate limit that makes testing live a nightmare!

Add Comment

[bjorn.rohlen@gmail.com (JIRA)]

bjorn rohlen commented on JENKINS-47978

Re: Lightweight checkout not working for branches that contain forward slash '/'

Well, unfortunately it does not, as confirmed by Vivek and myself. This is still an issue and breaking projects.

Add Comment

[atikhonova@parallels.com (JIRA)]

Anna Tikhonova commented on JENKINS-47978

Re: Lightweight checkout not working for branches that contain forward slash '/'

I can confirm that the issue is not in Bitbucket branch source plugin. I have tested 2.2.10 (which is the latest at the time of writing) with Bitbucket Server and Jenkinsfile placed in build/test.jenkinsfile in my repository. Flawless.

Add Comment

[atikhonova@parallels.com (JIRA)]

Anna Tikhonova edited a comment on JENKINS-47978

Re: Lightweight checkout not working for branches that contain forward slash '/'

I can confirm that the issue there is not an issue in Bitbucket branch source plugin. I have tested 2.2.10 (which is the latest at the time of writing) with Bitbucket Server and Jenkinsfile placed in build/test.jenkinsfile in my repository. Flawless With 'master' branch it is flawless . However, when I put it to branch 'feature/test', in the console output I have: {code:java} Lightweight checkout support not available, falling back to full checkout. Checking out git ssh://localhost:7999/bitbucket/~atikhonova/test.git into /var/lib/jenkins/workspace/test@script to read build/test.jenkinsfile {code}

Add Comment

[atikhonova@parallels.com (JIRA)]

Anna Tikhonova edited a comment on JENKINS-47978

Re: Lightweight checkout not working for branches that contain forward slash '/'

I can confirm that the original issue is still there . I don't know if it is an issue in Bitbucket branch source plugin or elsewhere . I have tested 2.2.10 (which is the latest at the time of writing) with Bitbucket Server and Jenkinsfile placed in build/test.jenkinsfile in my repository. With 'master' branch it is flawless. However, when I put it to branch 'feature/test', in the console output I have:

{code:java} Lightweight checkout support not available, falling back to full checkout. Checking out git ssh://localhost:7999/bitbucket/~atikhonova/test.git into /var/lib/jenkins/workspace/test@script to read build/test.jenkinsfile {code}

Add Comment

[atikhonova@parallels.com (JIRA)]

Anna Tikhonova commented on JENKINS-47978

Re: Lightweight checkout not working for branches that contain forward slash '/'

The workaround is to put 'refs/heads/feature/test' in Branch Specifier field of Git SCM. Not 'feature/test', not '*/feature/test'. You have to specify your git ref at full as this is the only supported case. See the breaking code in git-plugin (src/main/java//jenkins/plugins/git/GitSCMFileSystem.java): @Extension(ordinal = Short.MIN_VALUE) public static class BuilderImpl extends SCMFileSystem.Builder { @Override public boolean supports(SCM source) { return source instanceof GitSCM && ((GitSCM) source).getUserRemoteConfigs().size() == 1   && ((GitSCM) source).getBranches().size() == 1   && ((GitSCM) source).getBranches().get(0).getName().matches( "^((\\Q" + Constants.R_HEADS + "\\E.*)|([^/]+)|(\\*/[^/*]+(/[^/*]+)*))$" // HERE !! ); // we only support where the branch spec is obvious

Add Comment

[atikhonova@parallels.com (JIRA)]

Anna Tikhonova edited a comment on JENKINS-47978

Re: Lightweight checkout not working for branches that contain forward slash '/'

The workaround is to put 'refs/heads/feature/test' in Branch Specifier field of Git SCM. Not 'feature/test', not '*/feature/test'. You have to specify your git ref at full as this is the only supported case. See the breaking code in git-plugin (src/main/java//jenkins/plugins/git/GitSCMFileSystem.java):

{code:java}

@Extension(ordinal = Short.MIN_VALUE) public static class BuilderImpl extends SCMFileSystem.Builder {   @Override   public boolean supports(SCM source) {     return source instanceof GitSCM       && ((GitSCM) source).getUserRemoteConfigs().size() == 1       && ((GitSCM) source).getBranches().size() == 1       && ((GitSCM) source).getBranches().get(0).getName().matches(       "^((\\Q" + Constants.R_HEADS + "\\E.*)|([^/]+)|(\\*/[^/*]+(/[^/*]+)*))$"        // HERE !!     );

// we only support where the branch spec is obvious{code} I think this simple implementation is good and the only issue here is the match failing silently. Probably, this method should put a warning to the Console why lightweight checkout failed.

Add Comment

[atikhonova@parallels.com (JIRA)]

Anna Tikhonova edited a comment on JENKINS-47978

Re: Lightweight checkout not working for branches that contain forward slash '/'

The workaround is to put 'refs/heads/feature/test' in Branch Specifier field of Git SCM. Not 'feature/test', not '*/feature/test'. You have to specify your git ref at full as this is the only supported case. See the breaking code in git-plugin (src/main/java//jenkins/plugins/git/GitSCMFileSystem.java): {code:java} @Extension(ordinal = Short.MIN_VALUE) public static class BuilderImpl extends SCMFileSystem.Builder {   @Override   public boolean supports(SCM source) {     return source instanceof GitSCM       && ((GitSCM) source).getUserRemoteConfigs().size() == 1       && ((GitSCM) source).getBranches().size() == 1       && ((GitSCM) source).getBranches().get(0).getName().matches(       "^((\\Q" + Constants.R_HEADS + "\\E.*)|([^/]+)|(\\*/[^/*]+(/[^/*]+)*))$"        // HERE !!     );   // we only support where the branch spec is obvious{code}

I think this simple implementation is good and the only issue here is the match failing silently. Probably, this method should put a warning to the Console explaining why lightweight checkout failed.

Add Comment

[josephp90@gmail.com (JIRA)]

Joseph Petersen resolved as Fixed

Recently tested it with 2.2.13 at least it seems fixed.

Jenkins / JENKINS-47978

Lightweight checkout not working for branches that contain forward slash '/'

Change By: Joseph Petersen Status: Reopened Resolved Assignee: Vivek Pandey Joseph Petersen Resolution: Fixed

Add Comment

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

[josephp90@gmail.com (JIRA)]

Joseph Petersen assigned an issue to Joseph Petersen

Jenkins / JENKINS-47978 Lightweight checkout not working for branches that contain forward slash '/'

Change By: Joseph Petersen Assignee: Joseph Petersen (old)

Add Comment

This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)