[JIRA] Created: (JENKINS-9589) SSH sessions kept open by Subersion checkout

72 views
Skip to first unread message

sjm217-jenkins@srcf.ucam.org (JIRA)

unread,
May 4, 2011, 10:05:18 AM5/4/11
to jenkinsc...@googlegroups.com
SSH sessions kept open by Subersion checkout
--------------------------------------------

Key: JENKINS-9589
URL: https://issues.jenkins-ci.org/browse/JENKINS-9589
Project: Jenkins
Issue Type: Bug
Components: subversion
Environment: Jenkins ver. 1.410
Jenkins subversion plugin 1.25
Subversion server 1.1
Reporter: Steven Murdoch


After running Jenkins for a few days there are a large number of SSH processes running on the server:
root 19732 0.0 0.6 11632 3092 ? Ss May02 0:00 \_ sshd: secsvn [priv]
secsvn 19736 0.0 0.6 11632 3332 ? S May02 0:00 | \_ sshd: secsvn@notty
secsvn 19737 0.0 0.3 6152 2052 ? Ss May02 0:00 | \_ svnserve -t --tunnel-user=sjm217 -r ...
root 19756 0.0 0.6 11632 3092 ? Ss May02 0:00 \_ sshd: secsvn [priv]
secsvn 19760 0.0 0.6 11632 3332 ? S May02 0:00 | \_ sshd: secsvn@notty
secsvn 19761 0.0 0.3 6148 1944 ? Ss May02 0:00 | \_ svnserve -t --tunnel-user=sjm217 -r ...
root 19780 0.0 0.5 11632 2676 ? Ss May02 0:00 \_ sshd: secsvn [priv]
secsvn 19784 0.0 0.5 11632 2944 ? S May02 0:00 | \_ sshd: secsvn@notty
secsvn 19785 0.0 0.3 6152 1928 ? Ss May02 0:00 | \_ svnserve -t --tunnel-user=sjm217 -r ...
...

Perhaps related is the following error in the log
04-May-2011 15:03:08 hudson.model.Run run
INFO: cheri #10 main build action completed: SUCCESS

04-May-2011 15:03:08 hudson.scm.SubversionSCM$CheckOutTask checkClockOutOfSync
INFO: Failed to estimate the remote time stamp
org.tmatesoft.svn.core.SVNException: svn: 'stat' not implemented
svn: Unknown command 'stat'
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:126)
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:110)
at org.tmatesoft.svn.core.internal.io.svn.SVNRepositoryImpl.handleUnsupportedCommand(SVNRepositoryImpl.java:1368)
at org.tmatesoft.svn.core.internal.io.svn.SVNRepositoryImpl.info(SVNRepositoryImpl.java:1212)
at hudson.scm.SubversionSCM$CheckOutTask.checkClockOutOfSync(SubversionSCM.java:732)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:709)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:691)
at hudson.FilePath.act(FilePath.java:756)
at hudson.FilePath.act(FilePath.java:738)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:684)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:633)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1181)
at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:536)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:424)
at hudson.model.Run.run(Run.java:1374)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:145)
Caused by: org.tmatesoft.svn.core.SVNException: svn: Unknown command 'stat'
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:64)
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:51)
at org.tmatesoft.svn.core.internal.io.svn.SVNReader.handleFailureStatus(SVNReader.java:269)
at org.tmatesoft.svn.core.internal.io.svn.SVNReader.parse(SVNReader.java:248)
at org.tmatesoft.svn.core.internal.io.svn.SVNConnection.read(SVNConnection.java:260)
at org.tmatesoft.svn.core.internal.io.svn.SVNConnection.authenticate(SVNConnection.java:160)
at org.tmatesoft.svn.core.internal.io.svn.SVNRepositoryImpl.authenticate(SVNRepositoryImpl.java:1265)
at org.tmatesoft.svn.core.internal.io.svn.SVNRepositoryImpl.info(SVNRepositoryImpl.java:1191)
... 14 more
Caused by: org.tmatesoft.svn.core.SVNErrorMessage: svn: Unknown command 'stat'
at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:200)
at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:146)
at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:89)
at org.tmatesoft.svn.core.internal.io.svn.SVNReader.getErrorMessage(SVNReader.java:283)
at org.tmatesoft.svn.core.internal.io.svn.SVNReader.handleFailureStatus(SVNReader.java:261)
... 19 more


--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira


sjm217-jenkins@srcf.ucam.org (JIRA)

unread,
May 4, 2011, 10:29:17 AM5/4/11
to jenkinsc...@googlegroups.com

[ https://issues.jenkins-ci.org/browse/JENKINS-9589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Steven Murdoch updated JENKINS-9589:
------------------------------------

Summary: SSH sessions kept open by Subversion checkout (was: SSH sessions kept open by Subersion checkout)

> SSH sessions kept open by Subversion checkout
> ---------------------------------------------

sjm217-jenkins@srcf.ucam.org (JIRA)

unread,
Jun 1, 2011, 5:51:43 PM6/1/11
to jenkinsc...@googlegroups.com

[ https://issues.jenkins-ci.org/browse/JENKINS-9589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=149101#comment-149101 ]

Steven Murdoch commented on JENKINS-9589:
-----------------------------------------

Here's a thread dump of what I think are the threads which are keeping the session open:

Thread-21

"Thread-21" Id=79 Group=main RUNNABLE (in native)
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:146)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108)
at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232)
at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:672)
at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:470)
at java.lang.Thread.run(Thread.java:636)


Thread-22

"Thread-22" Id=80 Group=main WAITING on com.trilead.ssh2.channel.Channel@3a552b48
at java.lang.Object.wait(Native Method)
- waiting on com.trilead.ssh2.channel.Channel@3a552b48
at java.lang.Object.wait(Object.java:502)
at com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:985)
at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58)
at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:70)
at com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:48)


Thread-30

"Thread-30" Id=100 Group=main RUNNABLE (in native)
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:146)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108)
at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232)
at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:672)
at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:470)
at java.lang.Thread.run(Thread.java:636)


Thread-31

"Thread-31" Id=101 Group=main WAITING on com.trilead.ssh2.channel.Channel@677556a0
at java.lang.Object.wait(Native Method)
- waiting on com.trilead.ssh2.channel.Channel@677556a0
at java.lang.Object.wait(Object.java:502)
at com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:985)
at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58)
at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:70)
at com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:48)


Thread-39

"Thread-39" Id=111 Group=main RUNNABLE (in native)
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:146)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108)
at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232)
at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:672)
at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:470)
at java.lang.Thread.run(Thread.java:636)


Thread-40

"Thread-40" Id=112 Group=main WAITING on com.trilead.ssh2.channel.Channel@27c4c54d
at java.lang.Object.wait(Native Method)
- waiting on com.trilead.ssh2.channel.Channel@27c4c54d
at java.lang.Object.wait(Object.java:502)
at com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:985)
at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58)
at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:70)
at com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:48)


Thread-48

"Thread-48" Id=128 Group=main RUNNABLE (in native)
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:146)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108)
at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232)
at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:672)
at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:470)
at java.lang.Thread.run(Thread.java:636)


Thread-49

"Thread-49" Id=129 Group=main WAITING on com.trilead.ssh2.channel.Channel@68c04835
at java.lang.Object.wait(Native Method)
- waiting on com.trilead.ssh2.channel.Channel@68c04835
at java.lang.Object.wait(Object.java:502)
at com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:985)
at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58)
at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:70)
at com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:48)


Thread-57

"Thread-57" Id=138 Group=main RUNNABLE (in native)
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:146)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79)
at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108)
at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232)
at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:672)
at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:470)
at java.lang.Thread.run(Thread.java:636)


Thread-58

"Thread-58" Id=139 Group=main WAITING on com.trilead.ssh2.channel.Channel@31b90043
at java.lang.Object.wait(Native Method)
- waiting on com.trilead.ssh2.channel.Channel@31b90043
at java.lang.Object.wait(Object.java:502)
at com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:985)
at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58)
at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:70)
at com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:48)

> SSH sessions kept open by Subversion checkout
> ---------------------------------------------

Stefan.Moj@gmail.com (JIRA)

unread,
Mar 1, 2012, 4:40:25 AM3/1/12
to jenkinsc...@googlegroups.com

[ https://issues.jenkins-ci.org/browse/JENKINS-9589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=159698#comment-159698 ]

Stefan Moj commented on JENKINS-9589:
-------------------------------------

Did you resolve the issue or find a workaround? Seems like i ran into the same issue. We are using SVN version 1.6.12. It looks like that problem occurs since i set up post-commit hook as described [here. | https://wiki.jenkins-ci.org/display/JENKINS/Subversion+Plugin]

> SSH sessions kept open by Subversion checkout
> ---------------------------------------------

If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa

sjm217-jenkins@srcf.ucam.org (JIRA)

unread,
Mar 12, 2012, 2:40:24 PM3/12/12
to jenkinsc...@googlegroups.com

[ https://issues.jenkins-ci.org/browse/JENKINS-9589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=160167#comment-160167 ]

Steven Murdoch commented on JENKINS-9589:
-----------------------------------------

I haven't found a workaround for this, and it is still causing a problem. What I ended up doing was having a cron job restart the Jenkins build master every night. It's far from ideal but at least keeps the problem under control.

> SSH sessions kept open by Subversion checkout
> ---------------------------------------------

If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa

evernat@free.fr (JIRA)

unread,
Mar 13, 2013, 6:06:53 AM3/13/13
to jenkinsc...@googlegroups.com
evernat commented on Bug JENKINS-9589

Is it still a problem with recent versions of Jenkins and of the subversion plugin?
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.

hudson.8467364@mstier.de (JIRA)

unread,
Mar 20, 2013, 5:45:53 AM3/20/13
to jenkinsc...@googlegroups.com
Mark S. commented on Bug JENKINS-9589

I have this problem now, too. I'm using svn+ssh:// with private key. The dangling connections/dangling svnserve processes are seemingly produced by concurrent svn accesses, ie. a revision check by jenkins blocked by a manual svn commit. I'm also using the svn post-commit hook to trigger project rebuilds, so the likelyhood of jenkins getting into conflict with commits is rather high. When I do NOT do anything as a regular user with the repository, the problem seems to not exist...
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.

hudson.8467364@mstier.de (JIRA)

unread,
Mar 20, 2013, 5:47:53 AM3/20/13
to jenkinsc...@googlegroups.com
Change By: Mark S. (20/Mar/13 9:47 AM)
Environment: Jenkins ver. 1. 410 506
Jenkins subversion plugin 1. 25 45
Subversion server 1. 1 6.17
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.

jesse.johnson@silabs.com (JIRA)

unread,
Jul 2, 2013, 10:42:58 AM7/2/13
to jenkinsc...@googlegroups.com

I have this exact same problem. SVNKIT is supposed to time out open connections after 10 minutes by default but does not appear to be doing so, OR the subversion plugin is setting the connection close to something much higher.

I get an average of 30 open connections from svn commit trigger back to jenkins that just seem to hang around.

SVN polling seems to login and out properly.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.

esfd286@gmail.com (JIRA)

unread,
Jun 13, 2014, 9:58:03 AM6/13/14
to jenkinsc...@googlegroups.com
 
Todd Rhoades edited a comment on Bug JENKINS-9589

Possibly related? http://issues.tmatesoft.com/issue/SVNKIT-467

Also, I haven't fully confirmed this, but it appears to happen when the post commit hook triggers and "No jobs found matching the modified files."

Running
Jenkins 1.532.2
subversion plugin 1.45
svnserve 1.6.15
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.

esfd286@gmail.com (JIRA)

unread,
Jun 13, 2014, 9:54:04 AM6/13/14
to jenkinsc...@googlegroups.com
Todd Rhoades commented on Bug JENKINS-9589

Possibly related? http://issues.tmatesoft.com/issue/SVNKIT-467

Also, I haven't fully confirmed this, but it appears to happen when the post commit hook triggers and "No jobs found matching the modified files."
This message is automatically generated by JIRA.

If you think it was sent incorrectly, please contact your JIRA administrators.

daniel@beckweb.net (JIRA)

unread,
Jun 13, 2014, 12:05:04 PM6/13/14
to jenkinsc...@googlegroups.com
Daniel Beck commented on Bug JENKINS-9589

Needs to be reproduced in Subversion Plugin 2.x. 1.45 was released 17 months ago.
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.

esfd286@gmail.com (JIRA)

unread,
Jun 13, 2014, 1:49:03 PM6/13/14
to jenkinsc...@googlegroups.com
 
Todd Rhoades edited a comment on Bug JENKINS-9589

Possibly related? http://issues.tmatesoft.com/issue/SVNKIT-467

Also, I haven't fully confirmed this, but it appears to happen when the post commit hook triggers and "No jobs found matching the modified files." Confirmed this was not the case. It happened on a check-in of a single file that was found in an existing job.

Running
Jenkins 1.532.2
subversion plugin 1.45
svnserve 1.6.15
This message is automatically generated by JIRA.

If you think it was sent incorrectly, please contact your JIRA administrators.

daniel@beckweb.net (JIRA)

unread,
Jun 28, 2014, 8:28:04 AM6/28/14
to jenkinsc...@googlegroups.com
Daniel Beck commented on Bug JENKINS-9589

Please add a comment if this issue still occurs on more recent versions of Subversion plugin.
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.

daniel@beckweb.net (JIRA)

unread,
Aug 9, 2014, 4:29:52 PM8/9/14
to jenkinsc...@googlegroups.com
Daniel Beck resolved Bug JENKINS-9589 as Cannot Reproduce

No response to comments asking for updated information in two months, so resolving as Cannot Reproduce.
Change By: Daniel Beck (09/Aug/14 8:29 PM)
Status: Open Resolved
Resolution: Cannot Reproduce
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
Reply all
Reply to author
Forward
0 new messages