[JIRA] (JENKINS-48420) Provide lightweight checkout capability for Subversion

10 views
Skip to first unread message

jmalopoy@opentext.com (JIRA)

unread,
Mar 16, 2018, 4:12:02 AM3/16/18
to jenkinsc...@googlegroups.com
Jesús Malo Poyatos commented on Improvement JENKINS-48420
 
Re: Provide lightweight checkout capability for Subversion

Andreas Magnusson: Thanks for the link to the GitHub project. Did you send the PR?
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
Atlassian logo

tflanigan@meditech.com (JIRA)

unread,
Jun 7, 2018, 2:53:01 PM6/7/18
to jenkinsc...@googlegroups.com

I am currently trying to migrate some of my organization's existing jobs to use Jenkinsfiles instead. We'd like to store the Jenkinsfiles in some of our SVN paths since new branches are regularly created which need jobs created automatically, so managing jobs with a multibranch pipeline job scanning for Jenkinsfiles is a great solution for us, but I am currently running into this SVN limitation.

Although we allow plenty of space on our agents for checkouts, our repos are large enough that we risk running out of space on our master Jenkins machine if a fully recursive checkout is done there simply to read the Jenkinsfile. Will this feature be available relatively soon?

michiel.hendriks@mp-objects.com (JIRA)

unread,
Jul 20, 2018, 5:09:04 AM7/20/18
to jenkinsc...@googlegroups.com

This commit contains the fixes for lightweight checkouts, right? https://github.com/Zune01/subversion-plugin/commit/100d8fbefb3d68733b26439daf94cded6cd9fbb8

(We really need this fix as we are going to run out of inodes)
This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396)

michiel.hendriks@mp-objects.com (JIRA)

unread,
Jul 20, 2018, 6:33:02 AM7/20/18
to jenkinsc...@googlegroups.com

I've tested the changes against the current plugin versions and it looks like everything works correctly. I've issued a pull request for these changes: https://github.com/jenkinsci/subversion-plugin/pull/221

michiel.hendriks@mp-objects.com (JIRA)

unread,
Jul 20, 2018, 8:48:04 AM7/20/18
to jenkinsc...@googlegroups.com

Looks like one of the added test cases fails on the buildserver (work for me locally): SubversionSCMFileSystemTest#lastModified_Smokes

Not sure what the added value of this test is as it mostly relies on svnkit's "lastModified" response. But there is clearly a timing issue going on. i.e. buildserver not being fast enough for this test to fall within the given time window.

 

michiel.hendriks@mp-objects.com (JIRA)

unread,
Sep 12, 2018, 6:09:02 AM9/12/18
to jenkinsc...@googlegroups.com

Kevin Cathcart created a new pull request with fixed unit test: https://github.com/jenkinsci/subversion-plugin/pull/223

It passes all tests. (So I've closed the #221 PR in favor of the #223)
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

kuisathaverat@gmail.com (JIRA)

unread,
Sep 17, 2018, 10:34:07 AM9/17/18
to jenkinsc...@googlegroups.com
Change By: Ivan Fernandez Calvo
Status: Fixed but Unreleased Resolved

kuisathaverat@gmail.com (JIRA)

unread,
Sep 17, 2018, 10:34:07 AM9/17/18
to jenkinsc...@googlegroups.com
Status: Open Fixed but Unreleased
Assignee: Ivan Fernandez Calvo
Resolution: Fixed
Released As: 2.12.0

ziggystar@gmx.de (JIRA)

unread,
Sep 19, 2018, 6:24:01 AM9/19/18
to jenkinsc...@googlegroups.com
Thomasero Vulturera commented on Improvement JENKINS-48420
 
Re: Provide lightweight checkout capability for Subversion

I just did an update to the new version. It breaks my jobs in a Pipeline-Multibranch job because the sparce checkout apparently cannot follow svn:externals. There is also no switch to disable sparse checkout, so I had to downgrade the plugin.

I have 50+ permanent branches, and I want all of them to use the same Jenkinsfile. So I put the script into a separate folder and import that folder into my branch folder and have a symlink into the external:

shared/
  Jenkinsfile
<branch>/ (with shared/ as svn:external under shared/)
  Jenkinsfile -> shared/Jenkinsfile

I could get rid of this setup with file externals (don't remember why this didn't work). But still, the Subversion plugin in sparse mode needs to follow externals for my usecase to work.

 

ziggystar@gmx.de (JIRA)

unread,
Sep 19, 2018, 6:25:03 AM9/19/18
to jenkinsc...@googlegroups.com
Thomasero Vulturera edited a comment on Improvement JENKINS-48420
I just did an update to the new version. It breaks my jobs in a Pipeline-Multibranch job because the sparce sparse checkout apparently cannot follow svn:externals. There is also no switch option within the Pipeline-Multibranch plugin to disable sparse checkout, so I had to downgrade the plugin.


I have 50+ permanent branches, and I want all of them to use the same Jenkinsfile. So I put the script into a separate folder and import that folder into my branch folder and have a symlink into the external:

{{shared/}}
{{  Jenkinsfile}}
{{<branch>/ (with shared/ as svn:external under shared/)}}
{{  Jenkinsfile -> shared/Jenkinsfile}}

I could get rid of this setup with file externals (don't remember why this didn't work). But still, the Subversion plugin in sparse mode needs to follow externals for my usecase to work.

 

kuisathaverat@gmail.com (JIRA)

unread,
Sep 19, 2018, 12:51:03 PM9/19/18
to jenkinsc...@googlegroups.com

I think it could have easy workaround with a property here https://github.com/KevinCathcart/subversion-plugin/blob/d70e95d6cfe2ea660449417fa54abdeec51b203e/src/main/java/jenkins/scm/impl/subversion/SubversionSCMFileSystem.java#L74-L82 I will make the PR and release a minor version with the workaround

kuisathaverat@gmail.com (JIRA)

unread,
Sep 20, 2018, 6:43:03 AM9/20/18
to jenkinsc...@googlegroups.com

I just releases 2.12.1, it contains a new property to disable lightweight checkout -Djenkins.scm.impl.subversion.SubversionSCMFileSystem.disable=<true/false> for SCM API 2.0 so it would use the default implementation.

kuisathaverat@gmail.com (JIRA)

unread,
Sep 20, 2018, 6:44:03 AM9/20/18
to jenkinsc...@googlegroups.com
I just releases 2.12.1, it contains a new property to disable lightweight checkout -Djenkins.scm.impl.subversion.SubversionSCMFileSystem.disable=<true/false> for SCM API 2.0 so it would use the default implementation.


By default the SCM API 2.0  is used (the new implementation here), set the property means to use the implementation on <2.12.0 versions

philip.steiner@mdacorporation.com (JIRA)

unread,
Sep 20, 2018, 6:40:03 PM9/20/18
to jenkinsc...@googlegroups.com

Jobs on my Jenkins v2.107.3 master are configured with "Lightweight Checkout" selected (default when SVN is added to job). After upgrading to 2.12.1, the jobs started throwing the following SVN exception. If I deselect "Lightweight Checkout" in the job configuration, then the SVN checkout step succeeds.

org.tmatesoft.svn.core.SVNException: svn: E160013: '/svn/gpd/!svn/bc/59394/CI/dev/trunk/CI/GPD60/AE/jenkinsfile' path not found: 404 Not Found (https://svnyvr1.mda.ca)
{{ at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)}}
{{ at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)}}
{{ at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)}}
{{ at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)}}
{{ at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:920)}}
{{ at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doGet(DAVConnection.java:335)}}
{{ at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getFile(DAVRepository.java:289)}}
{{ at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getFile(DAVRepository.java:251)}}
{{ at jenkins.scm.impl.subversion.SubversionSCMFile.content(SubversionSCMFile.java:82)}}
Caused: java.io.IOException: failed to fetch file: CI/GPD60/AE/jenkinsfile
{{ at jenkins.scm.impl.subversion.SubversionSCMFile.content(SubversionSCMFile.java:85)}}
{{ at jenkins.scm.api.SCMFile.contentAsString(SCMFile.java:338)}}
{{ at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:110)}}
{{ at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:67)}}
{{ at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:303)}}
{{ at hudson.model.ResourceController.execute(ResourceController.java:97)}}
{{ at hudson.model.Executor.run(Executor.java:429)}}
Finished: FAILURE

 

philip.steiner@mdacorporation.com (JIRA)

unread,
Sep 20, 2018, 6:41:03 PM9/20/18
to jenkinsc...@googlegroups.com
Philip Steiner edited a comment on Improvement JENKINS-48420
Jobs on my Jenkins v2.107.3 master are configured with "Lightweight Checkout" selected (default when SVN is added to job). After upgrading SVNPlugin from v2.11.1 to 2 v2 .12.1, the jobs started throwing the following SVN exception. If I deselect "Lightweight Checkout" in the job configuration, then the SVN checkout step succeeds.

{{org.tmatesoft.svn.core.SVNException: svn: E160013: '/svn/gpd/!svn/bc/59394/CI/dev/trunk/CI/GPD60/AE/jenkinsfile' path not found: 404 Not Found ( [ https://svnyvr1.mda.ca |https://svnyvr1.mda.ca/] )}}
 \ {{ at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)}}
\ {{ at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)}}
 \ {{ at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)}}
 \ {{ at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)}}
 \ {{ at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:920)}}
 \ {{ at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doGet(DAVConnection.java:335)}}
 \ {{ at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getFile(DAVRepository.java:289)}}
 \ {{ at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getFile(DAVRepository.java:251)}}
 \ {{ at jenkins.scm.impl.subversion.SubversionSCMFile.content(SubversionSCMFile.java:82)}}
{{Caused: java.io.IOException: failed to fetch file: CI/GPD60/AE/jenkinsfile}}
 \ {{ at jenkins.scm.impl.subversion.SubversionSCMFile.content(SubversionSCMFile.java:85)}}
 \ {{ at jenkins.scm.api.SCMFile.contentAsString(SCMFile.java:338)}}
 \ {{ at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:110)}}
 \ {{ at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:67)}}
 \ {{ at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:303)}}
 \ {{ at hudson.model.ResourceController.execute(ResourceController.java:97)}}
 \ {{ at hudson.model.Executor.run(Executor.java:429)}}
{{Finished: FAILURE}}

 

philip.steiner@mdacorporation.com (JIRA)

unread,
Sep 20, 2018, 6:57:04 PM9/20/18
to jenkinsc...@googlegroups.com
Philip Steiner edited a comment on Improvement JENKINS-48420
Jobs After upgrading SVNPlugin from v2.11.1 to v2.12.1, pipeline jobs on my Jenkins v2.107.3 master started throwing an SVN exception (see example below).

The pipeline jobs are configured defined as "Pipeline script from SCM" with "Lightweight Checkout" selected (default when SVN is added to job). After upgrading SVNPlugin from v2

Check-out Strategy is "Always check out a fresh copy" . 11 Quiet checkout is selected . 1 to v2.12.1, the jobs started throwing the following SVN exception.

If I deselect "Lightweight Checkout" in the job configuration, then the SVN pipeline checkout step succeeds.


{{org.tmatesoft.svn.core.SVNException: svn: E160013: '/svn/gpd/!svn/bc/59394/CI/dev/trunk/CI/GPD60/AE/jenkinsfile' path not found: 404 Not Found ([https://svnyvr1.mda.ca|https://svnyvr1.mda.ca/])}}
\{{ at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)}}
\{{ at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)}}
\{{ at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)}}
\{{ at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)}}
\{{ at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:920)}}
\{{ at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doGet(DAVConnection.java:335)}}
\{{ at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getFile(DAVRepository.java:289)}}
\{{ at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getFile(DAVRepository.java:251)}}
\{{ at jenkins.scm.impl.subversion.SubversionSCMFile.content(SubversionSCMFile.java:82)}}
{{Caused: java.io.IOException: failed to fetch file: CI/GPD60/AE/jenkinsfile}}
\{{ at jenkins.scm.impl.subversion.SubversionSCMFile.content(SubversionSCMFile.java:85)}}
\{{ at jenkins.scm.api.SCMFile.contentAsString(SCMFile.java:338)}}
\{{ at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:110)}}
\{{ at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:67)}}
\{{ at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:303)}}
\{{ at hudson.model.ResourceController.execute(ResourceController.java:97)}}
\{{ at hudson.model.Executor.run(Executor.java:429)}}
{{Finished: FAILURE}}

 

jimmy.praet@telenet.be (JIRA)

unread,
Sep 21, 2018, 5:08:02 AM9/21/18
to jenkinsc...@googlegroups.com

Since upgrading to v2.12.x, our jenkins pipeline builds always showed "No changes".

I configured -Djenkins.scm.impl.subversion.SubversionSCMFileSystem.disable=true as a workaround.

kuisathaverat@gmail.com (JIRA)

unread,
Sep 21, 2018, 9:25:04 AM9/21/18
to jenkinsc...@googlegroups.com

Philip Steiner Is the jenkinsfile in an external? if so you will need to set -Djenkins.scm.impl.subversion.SubversionSCMFileSystem.disable=true property.

philip.steiner@mdacorporation.com (JIRA)

unread,
Sep 21, 2018, 3:28:07 PM9/21/18
to jenkinsc...@googlegroups.com

Ivan Fernandez Calvo - I don't think there are any externals. The files all reside in the same subdirectory of the repository, and as far as I know, there are no external references (I am not an SVN expert).

I have worked around the error for now by deselecting "Lightweight checkout" on all jobs that use these pipeline files.

Forgive this question, I am just an end-user on this Jenkins master, with no admin access - where would I ask the sysadmin to set the property you suggested? Is that in a configuration file for the Jenkins master?

kuisathaverat@gmail.com (JIRA)

unread,
Sep 24, 2018, 3:58:03 AM9/24/18
to jenkinsc...@googlegroups.com

Philip Steiner you have detailed steps to make it on this KB

philip.steiner@mdacorporation.com (JIRA)

unread,
Oct 9, 2018, 1:06:02 PM10/9/18
to jenkinsc...@googlegroups.com

Thanks Ivan I will forward the KB to my sys admin

philip.steiner@mdacorporation.com (JIRA)

unread,
Oct 9, 2018, 1:06:04 PM10/9/18
to jenkinsc...@googlegroups.com
Philip Steiner edited a comment on Improvement JENKINS-48420
Thanks Ivan [~ifernandezcalvo] I will forward the KB to my sys admin

robert.orrock@gameloft.com (JIRA)

unread,
Jun 18, 2019, 9:17:03 PM6/18/19
to jenkinsc...@googlegroups.com

Philip Steiner did the java argument fix your issue?

philip.steiner@mdacorporation.com (JIRA)

unread,
Jun 19, 2019, 12:12:03 PM6/19/19
to jenkinsc...@googlegroups.com

Robert Orrock Yes it did! 

arnaud.richard@st.com (JIRA)

unread,
Jul 26, 2019, 4:00:09 AM7/26/19
to jenkinsc...@googlegroups.com

I'm having the same issue than Philip Steiner. I'm sure I don't have externals. What is the fix ?
Reply all
Reply to author
Forward
0 new messages