Fwd: sshd "getpeername failed: Transport endpoint is not connected" error

[Andrey Klimentev]

Hello, everybody.

I've recently encountered a problem with OpenSSH server. Could you
help me to troubleshoot it?

I've configured 2 IP interfaces[1]: one with a public IP adress and
one with a private address. When I connect[2] through the public
interface (ens34), SSH works fine, but when I connect[3] through the
private interface (ens32), I receive a rather cryptic message on my
client - "ssh_exchange_identification: read: Connection reset by
peer". And this on my server - getpeername failed: Transport endpoint
is not connected".

I am using OpenSSH_7.1p1, OpenSSL 1.0.2e 3 Dec 2015. It is the latest
version available in Arch Linux repo.

--
[1] http://pastebin.com/8BF0N84t - interface conf and routing table
[2] http://pastebin.com/QptpeF7f - remote log and strace
[3] http://pastebin.com/niFJac46 - local log and strace
_______________________________________________
openssh-unix-dev mailing list
openssh-...@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev

[Darren Tucker]

On Wed, Dec 9, 2015 at 7:43 PM, Andrey Klimentev <andrei...@gmail.com> wrote:
> Hello, everybody.
>
> I've recently encountered a problem with OpenSSH server. Could you
> help me to troubleshoot it?
>
> I've configured 2 IP interfaces[1]: one with a public IP adress and
> one with a private address. When I connect[2] through the public
> interface (ens34), SSH works fine, but when I connect[3] through the
> private interface (ens32), I receive a rather cryptic message on my
> client - "ssh_exchange_identification: read: Connection reset by
> peer".

"reset by peer" usually means the TCP connection has been torn down
via a TCP RST packet. This can be the other end crashing (although
from your logs that doesn't look to be the case here) or something
like a firewall sending a RST.

Do you have any firewall or packet filter in that network path?
iptables anti-spoofing rules?

> And this on my server - getpeername failed: Transport endpoint
> is not connected".

From the failed traces:

debug1: rexec start in 4 out 4 newsock 4 pipe -1 sock 7
debug1: inetd sockets after dupping: 3, 3
debug1: getpeername failed: Transport endpoint is not connected
debug1: get_remote_port failed

Since this happens immediately after the server reexecs itself,
another possiblity is that somehow the descriptors are being marked
close-on-exec. You can test this theory by adding "-r" to sshd's
command line to disable reexec.

--
Darren Tucker (dtucker at zip.com.au)
GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4 37C9 C982 80C7 8FF4 FA69
Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.

[Andrey Klimentev]

Thanks for suggestion. That was, indeed, a problem on our network.
Sorry for bothering you.

BTW, "-r" option is not included in a man page, it doesn't seem to
have any effect as well.

--
С уважением, Андрей Климентьев.

[Darren Tucker]

On Fri, Dec 11, 2015 at 6:23 PM, Andrey Klimentev
<andrei...@gmail.com> wrote:
> Thanks for suggestion. That was, indeed, a problem on our network.
> Sorry for bothering you.
>
> BTW, "-r" option is not included in a man page,

We consider -r (and -R which is also used by the reexec code) to be
implementation details that end users should not rely on.

> it doesn't seem to have any effect as well.

It does, but the effect is not user visible. Look for
"send_rexec_state: done" in the debug output with and without -r.

--
Darren Tucker (dtucker at zip.com.au)
GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4 37C9 C982 80C7 8FF4 FA69
Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.

[Nico Kadel-Garcia]

On Mon, Dec 14, 2015 at 6:02 PM, Darren Tucker <dtu...@zip.com.au> wrote:
> On Fri, Dec 11, 2015 at 6:23 PM, Andrey Klimentev
> <andrei...@gmail.com> wrote:
>> Thanks for suggestion. That was, indeed, a problem on our network.
>> Sorry for bothering you.
>>
>> BTW, "-r" option is not included in a man page,
>
> We consider -r (and -R which is also used by the reexec code) to be
> implementation details that end users should not rely on.

So you refuse to document it at all, and insist that admins must read
the source code? I've actually seen people do this, but it seems
unfriendly.