[JIRA] Commented: (JENKINS-8059) Subversion Plugin does break native svn command line authentication, credentials missing after rewriting auth cache file
abayer@java.net (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148490#comment-148490 ]
abayer commented on JENKINS-8059:
---------------------------------
Not sure why this is assigned to me, honestly - I'm reassigning to kohsuke and will ping him to take a look today.
> Subversion Plugin does break native svn command line authentication, credentials missing after rewriting auth cache file
> ------------------------------------------------------------------------------------------------------------------------
>
> Key: JENKINS-8059
> URL: https://issues.jenkins-ci.org/browse/JENKINS-8059
> Project: Jenkins
> Issue Type: Bug
> Components: subversion
> Environment: Linux, Ubuntu Hardy and Jaunty, Solaris
> Reporter: tkrah
> Assignee: abayer
> Priority: Blocker
>
> Hudson does rewrite the authentication file under ${user.home}/.subversion/auth/svn.simple/$file and does not insert credentials stuff.
> I am using a custom build which does use the command line client (in addition to the normal svn usage of the hudson project) - so the credentials are important to be in there. My custom project is broken every time hudson does rewrite those authentication cache file from subversion.
> Is it possible to configure hudson not to do this rewrite or to insert those credentials when the rewrite does happen?
> The only workaround found is to set the immutable bit (removing write privileges is not enough) as root user to the file in question which hudson is not able to workaround (which is expected here and good).
> Project does build but the log grows with these exception trace:
> Nov 10, 2010 10:54:47 AM hudson.scm.SubversionSCM$CheckOutTask invoke
> INFO: Failed to estimate the remote time stamp
> org.tmatesoft.svn.core.SVNException: svn: Cannot rename file '/home/hudson/.subversion/auth/svn.simple/auth.d17e3535-2c01-0010-81b2-1f57b38e3f1e.tmp' to '/home/hudson/.subversion/auth/svn.simple/1755861b3f63d264955a25532195c4f8'
> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:64)
> at org.tmatesoft.svn.core.internal.wc.SVNFileUtil.rename(SVNFileUtil.java:552)
> at org.tmatesoft.svn.core.internal.wc.SVNWCProperties.setProperties(SVNWCProperties.java:352)
> at org.tmatesoft.svn.core.internal.wc.DefaultSVNAuthenticationManager$PersistentAuthenticationProvider.saveAuthentication(DefaultSVNAuthenticationManager.java:810)
> at org.tmatesoft.svn.core.internal.wc.DefaultSVNAuthenticationManager.acknowledgeAuthentication(DefaultSVNAuthenticationManager.java:276)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:606)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:275)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:263)
> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:516)
> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:98)
> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1001)
> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.info(DAVRepository.java:724)
> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:698)
> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:596)
> at hudson.FilePath.act(FilePath.java:753)
> at hudson.FilePath.act(FilePath.java:735)
> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:589)
> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:537)
> at hudson.model.AbstractProject.checkout(AbstractProject.java:1119)
> at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:479)
> at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:411)
> at hudson.model.Run.run(Run.java:1324)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
> at hudson.model.ResourceController.execute(ResourceController.java:88)
> at hudson.model.Executor.run(Executor.java:139)
> Of cause it can not rename the file, the immutable bit is set and only the root user is able to change this or any process which got the CAP_LINUX_IMMUTABLE capability bit set - of cause my hudson process does not get this privilege.
> So anything i can do to get rid of those subversion problem without this "workaround"?
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
kk@kohsuke.org (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148491#comment-148491 ]
Kohsuke Kawaguchi commented on JENKINS-8059:
--------------------------------------------
So just to make sure, files in svn.simple are written but without a password in it?
> Subversion Plugin does break native svn command line authentication, credentials missing after rewriting auth cache file
> ------------------------------------------------------------------------------------------------------------------------
>
> Key: JENKINS-8059
> URL: https://issues.jenkins-ci.org/browse/JENKINS-8059
> Project: Jenkins
> Issue Type: Bug
> Components: subversion
> Environment: Linux, Ubuntu Hardy and Jaunty, Solaris
> Reporter: tkrah
> Assignee: Kohsuke Kawaguchi
scm_issue_link@java.net (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148498#comment-148498 ]
scm_issue_link commented on JENKINS-8059:
-----------------------------------------
Code changed in jenkins
User: kohsuke
Path:
src/main/java/hudson/scm/SVNAuthStoreHandlerImpl.java
src/main/java/hudson/scm/SubversionSCM.java
src/main/java/hudson/scm/UserProvidedCredential.java
http://jenkins-ci.org/commit/subversion-plugin/3131a2d662790fbe2418f4b4c8148b94c4333358
Log:
[FIXED JENKINS-8059] store the password in the file.
> Subversion Plugin does break native svn command line authentication, credentials missing after rewriting auth cache file
> ------------------------------------------------------------------------------------------------------------------------
>
> Key: JENKINS-8059
> URL: https://issues.jenkins-ci.org/browse/JENKINS-8059
> Project: Jenkins
> Issue Type: Bug
> Components: subversion
> Environment: Linux, Ubuntu Hardy and Jaunty, Solaris
> Reporter: tkrah
> Assignee: Kohsuke Kawaguchi
> Priority: Blocker
> Attachments: subversion.hpi
cbos.ec@gmail.com (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148507#comment-148507 ]
Cees Bos commented on JENKINS-8059:
-----------------------------------
Before the issue popped up, the credentials were not stored at all on the filesystem. (Plugin version 1.17)
The credentials were only used internal in the SVNKit classes.
Preferable it is not to stored at all and just keep them in memory.
I think that is just a setting on
Credentials used from Jenkins might be different from the credentials used by the user from filesystem.
Once the credentials are in the file, SVN does not ask for credentials anymore, but uses the stored credentials.
Then the user reuses the credentials from Jenkins.
costincaraivan@gmail.com (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148508#comment-148508 ]
Costin Caraivan commented on JENKINS-8059:
------------------------------------------
What would be the cost, in terms of development and testing time if this would be left as a global Jenkins option?
In my opinion there are 2 user scenarios for this:
1. Store the credentials in the filesystem - this way, if you somehow need to also use the command line SVN client (for example for more complex release processes), it is already configured from Jenkins, because Jenkins has to do a checkout and already needs the credentials.
2. Do not store the credentials in the filesystem, if the Jenkins machine is also a development machine, where you don't want to commit with Jenkins' presumably separate SVN account.
Maybe something under Subversion -> Advanced? With the default of not storing the credentials (since that was the previous behavior).
bmjsmith@gmail.com (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148512#comment-148512 ]
Brian Smith commented on JENKINS-8059:
--------------------------------------
Sent this to the mailing list, adding here for completeness.
??
Hi folks
I'm trying to test the plugin now. I need to know what the circumstances are under which jenkins will write the credentials.
So far I've:
- Cleared the existing credentials at ~/.subversion/auth/svn.simple
- Entered new credentials using the URL <context_root>/scm/subversionSCM/enterCredential
- Run jobs with SCM configured to point at the same repository
The jobs complete successfully but no credentials are written out to the filesystem.
We had found previously that we couldn't identify what caused the write to happen, it might have been timed (often 12:01) or it might have been caused by one of our timed jobs.
Anyone have some insight?
Thanks
Brian??
nate@mediatemple.net (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148530#comment-148530 ]
Nate Jones commented on JENKINS-8059:
-------------------------------------
IIRC, a couple of the times that the auth file was rewritten on our installation the svn update that jenkins did included a new directory (with new files).
I don't remember if this was common to every single occurrence, but I do remember noticing it.
mwebber@java.net (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148533#comment-148533 ]
mwebber commented on JENKINS-8059:
----------------------------------
Brian Smith wondered why Jenkins did not write the credentials to ~/.subversion/auth/svn.simple.
The answer is that Jenkins does not store the credentials there. If you want to save credentials so Jenkins jobs can issue the "svn" command from a script, then you need to log on as the Jenkins user, and issue any svn command (I use svn log at our site) that requires the credentials to be entered. Enter your credentials and say "yes" to save them.
mwebber@java.net (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148534#comment-148534 ]
mwebber commented on JENKINS-8059:
----------------------------------
I've tested the plugin for a day, and for me it fixes the problem, and has not caused any adverse side-effects.
I think the change can be released. Note that Jenkins bundles the subversion plugin, so the next version of Jenkins needs to updated to include the new version.
Thanks
nate@mediatemple.net (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148535#comment-148535 ]
Nate Jones commented on JENKINS-8059:
-------------------------------------
To clarify my use case:
We need to run svn commands during the job, so we pre-cache the svn credentials in ~/.subversion/auth/svn.simple, which worked well until Jenkins overwrote them, seemingly at random. When this happened, the password was missing, and so our jobs would fail. Downgrading to the 1.17 version of the subversion plugin solved the issue for us.
We're planning on trying the new version soon. I'll report back if it works.
endolf@java.net (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148536#comment-148536 ]
endolf commented on JENKINS-8059:
---------------------------------
Jenkins does write over the files in ~/.subversion/auth/svn.simple, that is why this whole problem started. The question was when it does this.
jose.i.arevalo@jpmorgan.com (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148537#comment-148537 ]
Jose Arevalo commented on JENKINS-8059:
---------------------------------------
Please make sure that the fix supports multiple SVN URL in the same Jenkins instance.
mwebber@java.net (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148539#comment-148539 ]
mwebber commented on JENKINS-8059:
----------------------------------
bq. Jenkins does write over the files in ~/.subversion/auth/svn.simple, that is why this whole problem started. The question was when it does this.
It happens during the svn checkout at the start of the job. This is actually mentioned in a previous comment, although I can understand why you might not have spotted it.
bmjsmith@gmail.com (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148541#comment-148541 ]
Brian Smith commented on JENKINS-8059:
--------------------------------------
{quote}It happens during the svn checkout at the start of the job. This is actually mentioned in a previous comment, although I can understand why you might not have spotted it.{quote}
This is what I thought too - however having done the test above I did not get any credentials written out. I will test further and see if this is something that differs between master/slave or by O/S. I was just hoping someone knew definitively what would provoke them being written.
mwebber@java.net (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148542#comment-148542 ]
mwebber commented on JENKINS-8059:
----------------------------------
Brian Smith said:
bq. This is what I thought too - however having done the test above I did not get any credentials written out.
Are you testing with the fix or without it? If you are testing with the fix, then you won't get the credentials in ~/.subversion/auth/svn.simple overwritten, that is the whole point of the fix.
If you are testing without the fix, then the credentials are overwritten sometimes, but not always. We would sometimes go a day, then the problem suddenly appeared.
In your test, it may be that if here aren't any credentials cached, then nothing is overwritten.
bmjsmith@gmail.com (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148544#comment-148544 ]
Brian Smith commented on JENKINS-8059:
--------------------------------------
I am testing with the fixed subversion.hpi attached.
I thought the change KK made was that the credentials would now be written out with the password? See https://github.com/jenkinsci/subversion-plugin/commit/3131a2d662790fbe2418f4b4c8148b94c4333358
$user.home and $hudson.home are the same location on our system.
Note that since installing the new subversion hpi I've restarted all the slaves as well as the master, and reset the svn credentials on all boxes to be correct such that the user running jenkins can do command line operations using them without having to enter anything.
Running jobs from jenkins with SCM pointed at the correct subversion repo, I can't get credential overwrites to happen on the master. I can sometimes (seemingly at random) on the slaves and the files get written back incorrectly - they look like this:
{code:title=/app/hudson/.subversion/auth/svn.simple/b2cdb1d858f85ffe65eac66c9b1073d8}K 15
svn:realmstring
V 59
<http://subversion.example.co.uk:80> Subversion Repository
K 8
username
V 10
hudson_svn
END
{code}
Am I barking up the wrong tree somehow?
bmjsmith@gmail.com (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148613#comment-148613 ]
Brian Smith commented on JENKINS-8059:
--------------------------------------
An update also sent to the mailing list:
{quote}
I spent time today working on this and can confirm that on it's own the fix did not change anything for us. SVN credentials were still being written out without the password every time they were written. The conditions under which writes happened were still somewhat sporadic but seemed to be common when a job ran on a node for the first time having a non existent workspace.
I enabled a lot of logging on svnkit to no real avail other than being able to determine when credentials were presented, there was nothing recorded around when credentials were stored in the filesystem cache, and there was little correlation between the two.
However I did get this to work by following KK's hints and configuring the svn clients on each machine this way::
~/.subversion/servers
[global]
store-passwords = yes
store-plaintext-passwords = yes
Note that these parameters were not set at all thus defaulting to yes and no respectively.
After this was done, every time (so far!) jenkins/svnkit wrote credentials they were written with the passwords present in full as expected. We have around a hundred jobs long running acceptance test and release jobs on a nightly cycle which do a variety of jenkins level svn operations and o/s level svn operations - that should give this a good test and I'll let you know if anything goes wrong.
I'm unsure if this works without the change in the latest subversion.hpi. If it's important I may be able to downgrade and check.
I am not certain why the changes made in jenkins take no effect, I would have expected that since these parameters were previously unspecified the ones supplied by jenkins to svnkit ought to take precedence but I think that is a question for svnkit. From a brief look I think it's likely a bug somewhere under DefaultSVNAuthenticationManager. I think the change KK mentioned to allow control over where jenkins stores credentials is probably still useful, particularly if it controls where the overall config directory for svn lives, as this will enable people to use different svn client configurations entirely for the same o/s user for jenkins and command line.
{quote}
bmjsmith@gmail.com (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148653#comment-148653 ]
Brian Smith commented on JENKINS-8059:
--------------------------------------
Having been through this again it seems I was running the old plugin because I hadn't pinned the new one and it was being overwritten. Apologies. After re-testing this is working for us too.
mwebber@java.net (JIRA)
[ https://issues.jenkins-ci.org/browse/JENKINS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=148654#comment-148654 ]
mwebber commented on JENKINS-8059:
----------------------------------
Hopefully KK will update the bundled plugin in the next release of Jenkins.