SshChannelNotFoundException: Received SSH_MSG_CHANNEL_WINDOW_ADJUST on unassigned channel 0 (last assigned=null)

Yingchun Li

unread,

Apr 8, 2021, 2:29:01 AM4/8/21

to Repo and Gerrit Discussion

Hi,

When clone or fetch from the gerrit server, sometimes it will fails like:

jenkins@robot-pc:~/workspace/wifi_app-trigger$ git fetch origin

remote: Counting objects: 82020, done

remote: Finding sources: 100% (19107/19107)

Connection to 192.168.1.11 closed by remote host. 294.00 KiB/s

fatal: The remote end hung up unexpectedly

fatal: early EOF

fatal: index-pack failed

And check the server side error log, it shows:

[2021-04-08T14:07:12.426+08:00] [Index-Batch-3[java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@733fe72b[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@3f375ef1[Wrapped task = com.google.gerrit.server.logging.LoggingContextAwareRunnable@136fd25d]]]] WARN com.google.gerrit.server.logging.LoggingContextAwareRunnable : Logging context is not empty: LoggingContext{tags=null, forceLogging=null, performanceLogging=null, performanceLogRecords=MutablePerformanceLogRecords{performanceLogRecords=[]}}

[2021-04-08T14:08:11.187+08:00] [sshd-SshDaemon[21b75073](port=22)-nio2-thread-2] WARN org.apache.sshd.server.session.ServerSessionImpl : exceptionCaught(ServerSessionImpl[jenkins@/192.168.10.99:48596])[state=Opened] SshChannelNotFoundException: Received SSH_MSG_CHANNEL_WINDOW_ADJUST on unassigned channel 0 (last assigned=null)

The issue will happen frequently if the net speed is not high(should be about 200KB, as displayed by git message)

My gerrit server version is 3.3.0, and the git version in client side is 2.17.1.

By the way, I found a similar post in:

https://phabricator.wikimedia.org/T263293

But seem there is no result still.

Br,

Yingchun

Matthias Sohn

unread,

Apr 8, 2021, 3:17:45 AM4/8/21

to Yingchun Li, Repo and Gerrit Discussion

this looks similar to https://issues.apache.org/jira/browse/SSHD-942

-Matthias

David Ostrovsky

unread,

Apr 8, 2021, 3:17:55 AM4/8/21

to Repo and Gerrit Discussion

On Thursday, April 8, 2021 at 8:29:01 AM UTC+2 sword.l...@gmail.com wrote:

Hi,
When clone or fetch from the gerrit server, sometimes it will fails like:

jenkins@robot-pc:~/workspace/wifi_app-trigger$ git fetch origin
remote: Counting objects: 82020, done
remote: Finding sources: 100% (19107/19107)
Connection to 192.168.1.11 closed by remote host. 294.00 KiB/s
fatal: The remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed

And check the server side error log, it shows:

[2021-04-08T14:07:12.426+08:00] [Index-Batch-3[java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@733fe72b[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@3f375ef1[Wrapped task = com.google.gerrit.server.logging.LoggingContextAwareRunnable@136fd25d]]]] WARN com.google.gerrit.server.logging.LoggingContextAwareRunnable : Logging context is not empty: LoggingContext{tags=null, forceLogging=null, performanceLogging=null, performanceLogRecords=MutablePerformanceLogRecords{performanceLogRecords=[]}}
[2021-04-08T14:08:11.187+08:00] [sshd-SshDaemon[21b75073](port=22)-nio2-thread-2] WARN org.apache.sshd.server.session.ServerSessionImpl : exceptionCaught(ServerSessionImpl[jenkins@/192.168.10.99:48596])[state=Opened] SshChannelNotFoundException: Received SSH_MSG_CHANNEL_WINDOW_ADJUST on unassigned channel 0 (last assigned=null)

The issue will happen frequently if the net speed is not high(should be about 200KB, as displayed by git message)

It's this issue: [1], that was fixed in: [2]. The fix is included since gerrit 3.1.0,

so that I actually surprised you still see it on gerrit 3.3.0.

[1] https://issues.apache.org/jira/browse/SSHD-942

[2] https://gerrit-review.googlesource.com/c/gerrit/+/238384

Matthias Sohn

unread,

Apr 8, 2021, 3:23:16 AM4/8/21

to David Ostrovsky, Repo and Gerrit Discussion

would it make sense to try disabling this experimental feature by setting sshd.enableChannelIdTracking = false ?

Yingchun Li

unread,

Apr 8, 2021, 9:34:26 AM4/8/21

to Repo and Gerrit Discussion

Should the option be set to false?

[sshd]
enableChannelIdTracking = false

Seems the option is true default.

Anyway, I set it to false and test if okay.

Antoine Musso

unread,

Apr 8, 2021, 3:55:25 PM4/8/21

to Repo and Gerrit Discussion

Le 08/04/2021 à 09:17, David Ostrovsky a écrit :
> It's this issue: [1], that was fixed in: [2]. The fix is included
> since gerrit 3.1.0,
> so that I actually surprised you still see it on gerrit 3.3.0.
>
> [1] https://issues.apache.org/jira/browse/SSHD-942
> [2] https://gerrit-review.googlesource.com/c/gerrit/+/238384

Hello,

I can confirm it happens on Gerrit 3.2.7 (Wikimedia install), though it
is not wide spread and only affects a limited subset of users. Either a
ssh client incompatibility or a routinely slow network (but I could not
reproduce).

There is some bug report from 2009 (Gerrit 2.0.15) with someone
mentioning it is due to SSH ControlMaster being set to auto/on
(https://github.com/CandyShop/gerrit/issues/234
<https://github.com/CandyShop/gerrit/issues/234>), then I could not get
any of our users to check whether they used that feature or not. So it
is not that helpful.

cheers,

--
Antoine "hashar" Musso
Release Engineering

Yingchun Li

unread,

Apr 8, 2021, 11:10:48 PM4/8/21

to Repo and Gerrit Discussion

On Friday, April 9, 2021 at 3:55:25 AM UTC+8 Antoine Musso wrote:

Le 08/04/2021 à 09:17, David Ostrovsky a écrit :
> It's this issue: [1], that was fixed in: [2]. The fix is included
> since gerrit 3.1.0,
> so that I actually surprised you still see it on gerrit 3.3.0.
>
> [1] https://issues.apache.org/jira/browse/SSHD-942
> [2] https://gerrit-review.googlesource.com/c/gerrit/+/238384

Hello,

I can confirm it happens on Gerrit 3.2.7 (Wikimedia install), though it
is not wide spread and only affects a limited subset of users. Either a
ssh client incompatibility or a routinely slow network (but I could not
reproduce).

Some of my users said it fail with high possibility when the network was about 200KB.

There is some bug report from 2009 (Gerrit 2.0.15) with someone
mentioning it is due to SSH ControlMaster being set to auto/on
(https://github.com/CandyShop/gerrit/issues/234
<https://github.com/CandyShop/gerrit/issues/234>), then I could not get
any of our users to check whether they used that feature or not. So it
is not that helpful.

According to this thread

https://bugs.chromium.org/p/gerrit/issues/detail?id=11491&

enableChannelIdTracking = false

will help, I am testing with this setting, you can try.

Reply all

Reply to author

Forward