Some users are always occupying ssh channels

Yingchun Li

unread,

Mar 26, 2025, 3:05:53 AMMar 26

to Repo and Gerrit Discussion

Hi,

Recently, I encountered a strange issue where some users consistently occupy SSH channels without releasing them (for at least 3 days).

These connections can be observed using the command "gerrit show-connections",

and the issue is also visible in the JavaMelody threads section, where certain users appear to be waiting for specific conditions. A thread dump generated using jstack shows:

"SSH git-receive-pack /mcu-sw (user1)" #207 [3055257] prio=1 os_prio=0 cpu=3793846.27ms elapsed=2912013.20s tid=0x00007f325c090ba0 nid=3055257 waiting on condition [0x00007f324f2f1000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java...@21.0.1/Native Method)
- parking to wait for <0x00007f36f3055f10> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(java...@21.0.1/LockSupport.java:371)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java...@21.0.1/AbstractQueuedSynchronizer.java:519)
at java.util.concurrent.ForkJoinPool.unmanagedBlock(java...@21.0.1/ForkJoinPool.java:3780)
at java.util.concurrent.ForkJoinPool.managedBlock(java...@21.0.1/ForkJoinPool.java:3725)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java...@21.0.1/AbstractQueuedSynchronizer.java:1707)
at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:144)
at java.io.InputStream.readNBytes(java...@21.0.1/InputStream.java:509)
at org.eclipse.jgit.util.IO.readFully(IO.java:142)
at org.eclipse.jgit.transport.PacketLineIn.readLength(PacketLineIn.java:316)
at org.eclipse.jgit.transport.PacketLineIn.readString(PacketLineIn.java:180)
at org.eclipse.jgit.transport.ReceivePack.recvCommands(ReceivePack.java:1375)
at org.eclipse.jgit.transport.ReceivePack.service(ReceivePack.java:2264)
at org.eclipse.jgit.transport.ReceivePack.receive(ReceivePack.java:2200)
at com.google.gerrit.sshd.commands.Receive.runImpl(Receive.java:98)
at com.google.gerrit.sshd.AbstractGitCommand.service(AbstractGitCommand.java:109)
at com.google.gerrit.sshd.AbstractGitCommand$1.run(AbstractGitCommand.java:74)
at com.google.gerrit.sshd.BaseCommand$TaskThunk.run(BaseCommand.java:492)
- locked <0x00007f36f3056720> (a com.google.gerrit.sshd.BaseCommand$TaskThunk)
at com.google.gerrit.server.logging.LoggingContextAwareRunnable.run(LoggingContextAwareRunnable.java:113)
at java.util.concurrent.Executors$RunnableAdapter.call(java...@21.0.1/Executors.java:572)
at java.util.concurrent.FutureTask.run(java...@21.0.1/FutureTask.java:317)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(java...@21.0.1/ScheduledThreadPoolExecutor.java:304)
at com.google.gerrit.server.git.WorkQueue$Task.run(WorkQueue.java:703)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java...@21.0.1/ThreadPoolExecutor.java:1144)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java...@21.0.1/ThreadPoolExecutor.java:642)
at java.lang.Thread.runWith(java...@21.0.1/Thread.java:1596)
at java.lang.Thread.run(java...@21.0.1/Thread.java:1583)

I am certain that these users are not actively pushing code at the moment. Although I can terminate the connections using the gerrit close-connection command, I would like to understand the root cause of this issue, as it tends to reoccur occasionally.

Could someone provide guidance on how to debug and resolve this problem? thanks a lot.

My Gerrit version is 3.10.4

Br,

Yingchun

Matthias Sohn

unread,

Mar 26, 2025, 8:31:21 AMMar 26

to Yingchun Li, Repo and Gerrit Discussion

Did you configure

Br,
Yingchun

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/repo-discuss/0e20e304-f605-416b-94ae-ca7b8ce84ef1n%40googlegroups.com.

Yingchun Li

unread,

Mar 26, 2025, 11:13:50 PMMar 26

to Repo and Gerrit Discussion

No, I didn't config these two options, they should be default.

Matthias Sohn

unread,

Mar 29, 2025, 3:58:22 PMMar 29

to Yingchun Li, Repo and Gerrit Discussion

Try to set sshd.idleTimeout which is disabled by default. We set it to 10min.

Br,
Yingchun

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/repo-discuss/0e20e304-f605-416b-94ae-ca7b8ce84ef1n%40googlegroups.com.

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/repo-discuss/0e9078fe-e88b-4e76-9918-7c66c408c9aan%40googlegroups.com.

Yingchun Li

unread,

Mar 31, 2025, 7:40:38 AMMar 31

to Repo and Gerrit Discussion

Thanks, I will try.

Yingchun Li

unread,

Apr 12, 2025, 5:31:16 AMApr 12

to Repo and Gerrit Discussion

after setting the timeout about one weeks , there are no extra users always occupying the ssh channels, Thank you Matthias.

Reply all

Reply to author

Forward