[JIRA] Created: (JENKINS-10222) Jenkins cleaning out workspace unnecessarily when polling SCM (SVN)

[giles@fantasticthinking.com (JIRA)]

Jenkins cleaning out workspace unnecessarily when polling SCM (SVN)
-------------------------------------------------------------------

Key: JENKINS-10222
URL: https://issues.jenkins-ci.org/browse/JENKINS-10222
Project: Jenkins
Issue Type: Bug
Components: scm-sync-configuration
Environment: Windows
Reporter: Giles Dermody
Assignee: fcamblor

I am using a SVN repository which is being checked out at the beginning of every build in a Jenkins job. It is set to "poll SCM" for updates. However, regardless of whether there were changes or not it is cleaning out the workspace and downloading the whole repository in it's entirety every time which is unnecessary.

Builds are taking much longer than they should as the SVN poll should only be checking out newly-updated files. Sample log:

Checking out a fresh workspace because the workspace is not file://PICKLE/Repositories/project
Cleaning workspace.....

--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[globule71]

Je suis absent du bureau du 30 Juin au 15 Juillet,
 
En cas d'urgence merci de contacter directement le support : [hidden email]

Ou une des personnes suivantes :
[hidden email]
[hidden email]

---

View this message in context: Absence du bureau : [JIRA] Created: (JENKINS-10222) Jenkins cleaning out workspace unnecessarily when polling SCM (SVN)
Sent from the Jenkins issues mailing list archive at Nabble.com.

[fcamblor@java.net (JIRA)]

[ https://issues.jenkins-ci.org/browse/JENKINS-10222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

fcamblor updated JENKINS-10222:
-------------------------------

Component/s: subversion
(was: scm-sync-configuration)

Switched component to subversion plugin as it is not related to scm-sync-configuration plugin

> Jenkins cleaning out workspace unnecessarily when polling SCM (SVN)
> -------------------------------------------------------------------
>
> Key: JENKINS-10222
> URL: https://issues.jenkins-ci.org/browse/JENKINS-10222
> Project: Jenkins
> Issue Type: Bug

> Components: subversion

[globule71]

Je suis absent du bureau du 30 Juin au 15 Juillet,
 
En cas d'urgence merci de contacter directement le support : [hidden email]

Ou une des personnes suivantes :
[hidden email]
[hidden email]

---

View this message in context: Absence du bureau : [JIRA] Updated: (JENKINS-10222) Jenkins cleaning out workspace unnecessarily when polling SCM (SVN)

[fcamblor@java.net (JIRA)]

[ https://issues.jenkins-ci.org/browse/JENKINS-10222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

fcamblor reassigned JENKINS-10222:
----------------------------------

Assignee: (was: fcamblor)

Changed assignee since not related to scm-sync-config plugin

> Jenkins cleaning out workspace unnecessarily when polling SCM (SVN)
> -------------------------------------------------------------------
>
> Key: JENKINS-10222
> URL: https://issues.jenkins-ci.org/browse/JENKINS-10222
> Project: Jenkins
> Issue Type: Bug

> Components: subversion

> Environment: Windows
> Reporter: Giles Dermody
>

[jeff@paynesplace.com (JIRA)]

[ https://issues.jenkins-ci.org/browse/JENKINS-10222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=156712#comment-156712 ]

Jeff Payne commented on JENKINS-10222:
--------------------------------------

I found one way to reproduce this: if I check out more than one path and I don't specify a target directory, it appears that the subsequent check-outs corrupt the files in the .svn subdirectory. Specifying a different target for each make it work correctly.

I also googled and found an example where the user used different capitalization for the host name than was stored in the .svn, and had the same effect.

Hope this helps!

> Jenkins cleaning out workspace unnecessarily when polling SCM (SVN)
> -------------------------------------------------------------------
>
> Key: JENKINS-10222
> URL: https://issues.jenkins-ci.org/browse/JENKINS-10222
> Project: Jenkins
> Issue Type: Bug

> Components: subversion

> Environment: Windows
> Reporter: Giles Dermody
>

> I am using a SVN repository which is being checked out at the beginning of every build in a Jenkins job. It is set to "poll SCM" for updates. However, regardless of whether there were changes or not it is cleaning out the workspace and downloading the whole repository in it's entirety every time which is unnecessary.
> Builds are taking much longer than they should as the SVN poll should only be checking out newly-updated files. Sample log:
> Checking out a fresh workspace because the workspace is not file://PICKLE/Repositories/project
> Cleaning workspace.....

--
This message is automatically generated by JIRA.

If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa

[damien.finck67@online.fr (JIRA)]

[ https://issues.jenkins-ci.org/browse/JENKINS-10222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=163655#comment-163655 ]

Damien Finck commented on JENKINS-10222:
----------------------------------------

Hello,

I have exactly the same problem.

In Jenkins, I have about 30 projects.
For two of my projects, I have this problem for some time.

The first one has this configuration:
- An SVN repository on "https://<company>.fr:433/svn/<project>" in the folder "trunk" of my workspace
- "Use 'svn update' as much as possible"
And at each build, Jenkins said "Checking out a fresh workspace Because The workspace is not "https://<company>.fr:443/svn/<project>" "

For the second project that is the problem, it is configured this way:
- 3 SVN repositories
* https://<company>.fr/svn/<project1> in the folder "project1"
* https://<company>.fr:433/svn/<project2> in the "Project2"
* https://<company>.fr:433/svn/<project3> in the "project3"
- "Use 'svn update' as much as possible"
And at each build, Jenkins said "Checking out a fresh workspace Because The workspace is not "https://<company>.fr:443/svn/<project3>" "
I have a problem only on the third repositorie.

I tried to remove the first project and recreate it: Same problem.

I read in a forum user who said: "I had a similar problem .... but It Was Because The name of the SVN server in my path Was in CAPS. I changed it to lower case and it worked. ... "
So, I changed my path from https://<company>.fr:433/svn/<folder> to https://<company>.fr/svn/<folder> and it works in my two projects .

It like if repositorie has once time a bug, Jenkins do each other time a "svn checkout" on this repositorie instead of an "svn update".

Sincerely,
Damien

> Jenkins cleaning out workspace unnecessarily when polling SCM (SVN)
> -------------------------------------------------------------------
>
> Key: JENKINS-10222
> URL: https://issues.jenkins-ci.org/browse/JENKINS-10222
> Project: Jenkins
> Issue Type: Bug

> Components: subversion

> Environment: Windows
> Reporter: Giles Dermody
>

> I am using a SVN repository which is being checked out at the beginning of every build in a Jenkins job. It is set to "poll SCM" for updates. However, regardless of whether there were changes or not it is cleaning out the workspace and downloading the whole repository in it's entirety every time which is unnecessary.
> Builds are taking much longer than they should as the SVN poll should only be checking out newly-updated files. Sample log:
> Checking out a fresh workspace because the workspace is not file://PICKLE/Repositories/project
> Cleaning workspace.....

--
This message is automatically generated by JIRA.

If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa

[damien.finck67@online.fr (JIRA)]

Damien Finck commented on JENKINS-10222

Jenkins cleaning out workspace unnecessarily when polling SCM (SVN)

Hello, I think I found why Jenkins did a svn checkout: This is when a build is canceled while Jenkins did an svn update. Sincerely, Damien

This message is automatically generated by JIRA.

If you think it was sent incorrectly, please contact your JIRA administrators.

[damien.finck67@online.fr (JIRA)]

Damien Finck edited a comment on JENKINS-10222

Jenkins cleaning out workspace unnecessarily when polling SCM (SVN)

Hello,

I have exactly the same problem. In Jenkins, I have about 30 projects. For two of my projects, I have this problem for some time. The first one has this configuration:

An SVN repository on "https://<company>.fr:433/svn/<project>" in the folder "trunk" of my workspace

"Use 'svn update' as much as possible" And at each build, Jenkins said "Checking out a fresh workspace Because The workspace is not "https://<company>.fr:443/svn/<project>" "

For the second project that is the problem, it is configured this way:

3 SVN repositories

https://<company>.fr/svn/<project1> in the folder "project1"

https://<company>.fr:433/svn/<project2> in the "Project2"

https://<company>.fr:433/svn/<project3> in the "project3"

"Use 'svn update' as much as possible"

And at each build, Jenkins said "Checking out a fresh workspace Because The workspace is not "https://<company>.fr:443/svn/<project3>" " I have a problem only on the third repositorie.

I tried to remove the first project and recreate it: Same problem. I read in a forum user who said: "I had a similar problem .... but It Was Because The name of the SVN server in my path Was in CAPS. I changed it to lower case and it worked. ... " So, I changed my path from https://<company>.fr:433/svn/<folder> to https://<company>.fr/svn/<folder> and it works in my two projects . It like if repositorie has once time a bug, Jenkins do each other time a "svn checkout" on this repositorie instead of an "svn update".

Version : Jenkins : 1.474 Jenkins Subversion Plug-in : 1.42 Subversion version (in Jenkins configuration) : 1.7 Sincerely, Damien

This message is automatically generated by JIRA.

If you think it was sent incorrectly, please contact your JIRA administrators.

[andreas@semanticedge.de (JIRA)]

Andreas Schoen commented on JENKINS-10222

Jenkins cleaning out workspace unnecessarily when polling SCM (SVN)

I had the same problem with a project with the Repository URL of the following pattern: svn://<servername>/applications/xxx//trunk/yyy The polling message was: Workspace doesn't contain svn://<servername>/applications/xxx//trunk/yyy. Need a new build. However, the polling did work successfully (no changes, at revision...) when I set the Repository URL to svn://<servername>/applications/xxx/trunk/yyy (without the second slash after xxx). It seems that the repository url in the working copy is normalized (e.g. removing double slashes, port number) by the svn client, but the check if the working copy matches the project does not perform this kind of normalization. Jenkins ver. 1.471 Subversion Plugin version 1.4

This message is automatically generated by JIRA.

If you think it was sent incorrectly, please contact your JIRA administrators.

[damien.finck67@online.fr (JIRA)]

Damien Finck commented on JENKINS-10222

Jenkins cleaning out workspace unnecessarily when polling SCM (SVN)

@Andreas : I agree with you. I try to add double '/' in URL and this problem come back. A good idea is to add a message when a URL is not normalized. Jenkins v1.491 Subversion Plugin v1.43

This message is automatically generated by JIRA.

If you think it was sent incorrectly, please contact your JIRA administrators.

[matthew.b.layman@lmco.com (JIRA)]

Matt Layman commented on JENKINS-10222

Jenkins cleaning out workspace unnecessarily when polling SCM (SVN)

This error message comes from the hudson.scm.subversion.UpdateUpdater.java at line 96. The reason for the failure is because svnInfo.url != url. So I built a test version of the plugin and logged some extra information so I could find out what 'svnInfo.url' was and what 'url' was. My job configuration uses the 'svn update' option and my SVN URL looks something like this (I've had to modify the URL because the path is on an intranet where info is proprietary): https://<some IP address>/svn/REPO/directory/branches/some_branch/@HEAD In this scenario, I got the following: svnInfo.url = https://<some IP address>/svn/REPO/directory/branches/some_branch url = https://<some IP address>/svn/REPO/directory/branches/some_branch/ The observable difference is that svnInfo.url has no trailing slash and url has a trailing slash. Therefore, I presume that the fix is to modify one of those two variables in some appropriate manner so that they agree with each other. If I have time to investigate the appropriate place to add (or remove) the slash, I'll report back.

This message is automatically generated by JIRA.

If you think it was sent incorrectly, please contact your JIRA administrators.

[matthew.b.layman@lmco.com (JIRA)]

Matt Layman commented on JENKINS-10222

Jenkins cleaning out workspace unnecessarily when polling SCM (SVN)

I did some more research and have concluded that changing svnInfo.url to introduce a slash when present would be very difficult (even though it would be more correct since including a trailing slash in a URL is perfectly valid syntax). The problem is that the SVNInfo object is constructed with a File object. I ran a test and confirmed that none of the string getters from java.io.File will return a path with a trailing slash even if the trailing slash is provided in the File constructor. Thus, the SVNInfo object has no way of knowing if a user provided the trailing slash in a path or not. This means that the only feasible way to fix this is either to: Change ModuleLocation.getURL() so that it won't return a trailing slash (which could have unforseen impacts to other code which might depend on the trailing slash). OR... Change UpdateUpdater.isUpdatable so that when getURL is called, a trailing slash is removed if it exists, in order to normalize url and svnInfo.url. By the way, for those looking for a workaround, remove the trailing slash from the URL in your job. This has the effect of removing it from url so that it will match svnInfo.url. My earlier example would look like: https://<some IP address>/svn/REPO/directory/branches/some_branch@HEAD Of course, all of this may not be the only reason that url and svnInfo.url wouldn't match, but it is definitely one reason.

This message is automatically generated by JIRA.

If you think it was sent incorrectly, please contact your JIRA administrators.

[alexander@roehnsch.de (JIRA)]

Pjotr Vladrisk commented on JENKINS-10222

Jenkins cleaning out workspace unnecessarily when polling SCM (SVN)

From what I see, the comments so far describe two problems. multiple checkouts onto the workspace collide svn URLs do not match internally when ending on a slash Personally, I had problem No. 1. Jeff Payne's solution worked for that. Also, that problem yielded the very same error message as given in the description section above, while problem No. 2 seems to give a different error message under circumstances.

This message is automatically generated by JIRA.

If you think it was sent incorrectly, please contact your JIRA administrators.

[alan.birtles@eu.sony.com (JIRA)]

Alan Birtles commented on JENKINS-10222

Jenkins cleaning out workspace unnecessarily when polling SCM (SVN)

just had a similar issue, in our case we had the svn url defined as http://server:80/svn/repo, I think the svn checkout strips out the port number then the polling thinks that the workspace doesn't contain the same working copy.

This message is automatically generated by JIRA.

If you think it was sent incorrectly, please contact your JIRA administrators.

[alghe.global@gmail.com (JIRA)]

Alexandru Gheorghe commented on JENKINS-10222

Jenkins cleaning out workspace unnecessarily when polling SCM (SVN)

Same issue as Damien Finck described we're encoutering with latest Jenkins version to this date: v1.602: subversion plugin: 2.5 multiple repositories: some fetched to different folders the rest fetched to one folder each time there's a trigger (timed at 15 minutes) even if there are no changes polling takes place the checkout will delete files and add ones on the first poll of the first repository, then the second does the same; what is interesting is that in the end it will balance out and it'll work but each time the build is triggered it will fetch the files in the workspace at each 15 minutes a new build takes 11MB so that in a couple of days we got over 1GB of disk space occupied; not something wanted

This message is automatically generated by JIRA.

If you think it was sent incorrectly, please contact your JIRA administrators.

[alghe.global@gmail.com (JIRA)]

Alexandru Gheorghe edited a comment on JENKINS-10222

Jenkins cleaning out workspace unnecessarily when polling SCM (SVN)

Same issue as Damien Finck described we're encoutering with latest Jenkins version to this date: v1.602:

subversion plugin: 2.5 multiple repositories: some fetched to different folders the rest fetched to one folder each time there's a trigger (timed at 15 minutes) even if there are no changes polling takes place the checkout will delete files and add ones on the first poll of the first repository, then the second does the same; what is interesting is that in the end it will balance out and it'll work but each time the build is triggered it will fetch the files in the workspace at each 15 minutes a new build takes 11MB so that in a couple of days we got over 1GB of disk space occupied; not something wanted

Tried to append @HEAD at the URL but no luck: https://url/svn/branch@HEAD , it will indeed go to HEAD and poll there (unlike before where the revision was the date at which it was started) but still with "No changes" in the subversion polling log it still gets the files and checks out as if it goes back 1 revision and sees new changes.

This message is automatically generated by JIRA.

If you think it was sent incorrectly, please contact your JIRA administrators.