recv() for buffered data after peer disconnected

[termin...@gmail.com]

Hello

I have a question about how calls to send(), recv(), shutdown() and close() are meant to work.

At some point in my application my client needs to abort. This is much like exiting the process, but actually it is a plugin so the not really exit, but still clean things up.

In my clean-up procedure I would still like to send the server some messages with simple clean-up and stop commands, before I really close the socket.

So my question is can server read old data from a socket if the peer has already disconnected, but server did not disconnect and still has buffered data to read ?

And if so, is there any way for server to detect that it is reading old buffered data, and peer has already disconnected ?

Or, suppose server is reading old buffered data and now wants to send a response to my clean-up command. Will server immediately discover the disconnect upon the attempt to send() ? Can server still continue to read buffered data after send() returns 'connection reset by peer' ?

Is there some place where I can find such information ?

My client is a Vim plugin written in python that communicates with DBGp protocol to a debug server like the Xdebug php module for debugging.

Thank you,
Timothy Madden

[Rainer Weikusat]

termin...@gmail.com writes:
> I have a question about how calls to send(), recv(), shutdown() and
> close() are meant to work.
>
> At some point in my application my client needs to abort. This is
> much like exiting the process, but actually it is a plugin so the
> not really exit, but still clean things up.
>
> In my clean-up procedure I would still like to send the server some
> messages with simple clean-up and stop commands, before I really
> close the socket.
>
> So my question is can server read old data from a socket if the peer
> has already disconnected, but server did not disconnect and still
> has buffered data to read ?

Generally, yes (assuming this refers to 'TCP or something similar'):
The server will receive all data which was sent in the order it was
sent, followed by 'an EOF indication' (read returning 0).

> And if so, is there any way for server to detect that it is reading
> old buffered data, and peer has already disconnected ?

You can envision each byte in a TCP stream as an event which happens
at a 'time X' (X being the distance between this byte and the first
byte which was sent) relative to the 'time frame' of this TCP
stream. This implies that, by the time the server reads the last
message, 'the client disconnected' is still 'a future event which
hasn't happened yet' and 'the server' doesn't have any information
regarding "what will happen in future", it can only observer what's
happening now. The fact that there's a 'reference time frame'
independent of the TCP stream and that 'the client disconnected' is a
past event at the time the server reads the last message relative to
this other reference time frame is of no concern here because 'the
server' cannot learn 'faster' about past events relative to the other
reference time frame than this information can be communicated to it:
The 'information propagation speed' of the TCP stream is equivalent to
'the speed of light' in the physical universe here. Maybe the star
exploded already. But until the light generated by the explosion
reached the observer which is going to happen after all light
generated before the explosion has reached it, the observer can't
tell.

NB: This is simplified because it ignores the pseudo 'out of band'
communication facility TCP also provides.

> Or, suppose server is reading old buffered data and now wants to
> send a response to my clean-up command. Will server immediately
> discover the disconnect upon the attempt to send() ?

If the connection was shut down in this direction, a send attempt will
result in an error. That's a different bytestream.

> Can server still continue to read buffered data after send() returns
> 'connection reset by peer' ?

Send should return EPIPE in this case except if the connection was
actually reset. If it was reset, probably not but I haven't tested
this.

[termin...@gmail.com]

On Wednesday, February 13, 2013 2:20:16 PM UTC+2, Rainer Weikusat wrote:
> termin...@gmail.com writes:
>
> > I have a question about how calls to send(), recv(), shutdown() and
>
> > close() are meant to work.

[...]

> > Or, suppose server is reading old buffered data and now wants to
>
> > send a response to my clean-up command. Will server immediately
>
> > discover the disconnect upon the attempt to send() ?
>
>
>
> If the connection was shut down in this direction, a send attempt will
>
> result in an error. That's a different bytestream.
>
>
>
> > Can server still continue to read buffered data after send() returns
>
> > 'connection reset by peer' ?
>
>
>
> Send should return EPIPE in this case except if the connection was
>
> actually reset. If it was reset, probably not but I haven't tested
>
> this.

The TCP connection was closed normally by the peer (not sure if a normal close() is different than 'connection reset').

Server reads first message from the buffer, and tries to send() back a response. Server gets an error here, since the OS already knows socket has been closed.

Now can server still read the next message from the buffer with recv() ?

Thank you,
Timothy Madden

[Rainer Weikusat]

termin...@gmail.com writes:
> On Wednesday, February 13, 2013 2:20:16 PM UTC+2, Rainer Weikusat wrote:
>> termin...@gmail.com writes:

[...]

>>> Can server still continue to read buffered data after send() returns
>>> 'connection reset by peer' ?
>>
>> Send should return EPIPE in this case except if the connection was
>> actually reset. If it was reset, probably not but I haven't tested
>> this.
>
> The TCP connection was closed normally by the peer (not sure if a
> normal close() is different than 'connection reset').

Well, I am 'sure' that this is something different (a FIN for an
orderly close and a RST for a connection reset) and since you were
asking the question, have considered to believe in the answers you
get?

> Server reads first message from the buffer, and tries to send() back
> a response. Server gets an error here, since the OS already knows
> socket has been closed.
>
> Now can server still read the next message from the buffer with
> recv()?

If it wasn't killed by a SIGPIPE, for an orderly shutdown, probably yes
(didn't test it). For a connection reset, this depends on whether the
reset is an 'out of band' event. This seems not entirely improbable,
however, I have neither checked nor tested that.

[Rainer Weikusat]

Rainer Weikusat <rwei...@mssgmbh.com> writes:

[...]

> If it wasn't killed by a SIGPIPE, for an orderly shutdown, probably yes
> (didn't test it). For a connection reset, this depends on whether the
> reset is an 'out of band' event. This seems not entirely improbable,
> however, I have neither checked nor tested that.

Meanwhile, it occurred to me that this is total nonsense:
TCP-connections are logically composed of two independent bytestream
and the process on one endpoint can't 'close' the incoming
bytestream. It can 'close' the outgoing one by doing something which
causes a FIN to be sent. When the process on the other reaches the FIN
in its incoming bytestream, it will see 'an EOF indicator' (read
returning zero) but it can theoretically continue to send data for as
long as it sees fit. Once it also 'closes' its outgoing bytestream, a
FIN will be sent in the other direction. Once the other end receives
this FIN in its incoming bytestream, it also sees an EOF and the
connection is 'fully closed' (simplification). The TCP specification
(RFC793) actually requires the process which sent the first FIN to
keep reading data until the other FIN has been received.

For UNIX(*), this is what shutdown(fd, SHUT_WR) provides. Doing a
close or shutdown(fd, SHUT_RDWR) instead means the process which
issued to call can't receive any more data on this socket but TCP has
no way to communicate this to the other endpoint. The RFC1122 text on
this is

A host MAY implement a "half-duplex" TCP close sequence, so
hat an application that has called CLOSE cannot continue to
read data from the connection. If such a host issues a CLOSE
call while received data is still pending in TCP, or if new
data is received after CLOSE is called, its TCP SHOULD send a
RST to show that data was lost.

Reception of a RST is supposed to 'abort' the connection, leading to
an ECONNRESET error. But the RST needs a round trip to
arrive. Because of this, a send initiated by one end will
usually succeed not matter how the other 'closed the
connection' and the process on the 'not yet closed' side should be
able to receive 'buffered data' at least until the RST arrived.

[Barry Margolin]

In article <87y5es8...@sapphire.mobileactivedefense.com>,

This is why the default behavior is to use a signal, SIGPIPE, to
indicate that the RST was received -- it happens asynchronously, when
the RST arrives later. If you disable, ignore, or block the signal, you
don't find out about the reset until you perform an I/O operation on the
socket after the RST is received; the next operation (or an operation
that's blocked on the socket when the RST is received) will report
ECONNRESET.

--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[termin...@gmail.com]

On Wednesday, February 13, 2013 10:51:01 PM UTC+2, Barry Margolin wrote:
> In article <87y5es8...@sapphire.mobileactivedefense.com>,
>
> Rainer Weikusat <rwei...@mssgmbh.com> wrote:
>
>
>
> > Rainer Weikusat <rwei...@mssgmbh.com> writes:
>
> >
>
> > [...]
>
> >
>
> > > If it wasn't killed by a SIGPIPE, for an orderly shutdown, probably yes
>
> > > (didn't test it). For a connection reset, this depends on whether the
>
> > > reset is an 'out of band' event. This seems not entirely improbable,
>
> > > however, I have neither checked nor tested that.
>
> >
>

[...]

>
> This is why the default behavior is to use a signal, SIGPIPE, to
>
> indicate that the RST was received -- it happens asynchronously, when
>
> the RST arrives later. If you disable, ignore, or block the signal, you
>
> don't find out about the reset until you perform an I/O operation on the
>
> socket after the RST is received; the next operation (or an operation
>
> that's blocked on the socket when the RST is received) will report
>
> ECONNRESET.

I am still confused about it, but you are saying that:

An attempt by the server to send() after the client has already
shutdown(SHUT_RDWR) normally results in a RST on the server
side and would raise SIGPIPE.

By default SIGPIPE kills the process, and frankly from my experience a connection reset does not kill my process (either client or server).

This raises one more question: if server has 100 connected sockets and SIGPIPE is raised, how does server pick the socket with the RST ? How would server know it was a socket in the first place, and not other file descriptor ?

And if SIGPIPE will not kill server, I still have the question: can the rest of the input data in the socket buffer be read after the RST and the SIGPIPE, if server calls recv() later on its socket ?


Thank you,
Timothy Madden

[Rainer Weikusat]

termin...@gmail.com writes:
> On Wednesday, February 13, 2013 10:51:01 PM UTC+2, Barry Margolin wrote:

[...]

>> This is why the default behavior is to use a signal, SIGPIPE, to
>> indicate that the RST was received -- it happens asynchronously, when
>> the RST arrives later. If you disable, ignore, or block the signal, you
>> don't find out about the reset until you perform an I/O operation on the
>> socket after the RST is received; the next operation (or an operation
>> that's blocked on the socket when the RST is received) will report
>> ECONNRESET.
>
> I am still confused about it, but you are saying that:
>
> An attempt by the server to send() after the client has already
> shutdown(SHUT_RDWR) normally results in a RST on the server
> side and would raise SIGPIPE.
>
> By default SIGPIPE kills the process, and frankly from my experience
> a connection reset does not kill my process (either client or
> server).

I don't think the description of SIGPIPE in the text was
correct. SIGPIPE is usually raised when an attempt to send data
accross a 'pipe-like' communication channel was made but the recipient
has gone away. For TCP, this should happen on a write attempt after
the socket was fully closed but not the file descriptor, ie, because a
RST was received (I tested this with AF_UNIX because I don't have the
time to write a more elaborate test program now). ECONNRESET is an
error reported by a read attempt on a connection-oriented socket whose
connection was reset. And this is still something different than
'closing the connection'.

> This raises one more question: if server has 100 connected sockets
> and SIGPIPE is raised, how does server pick the socket with the RST
> ? How would server know it was a socket in the first place, and not
> other file descriptor ?

Not at all, but that's a different question.

> And if SIGPIPE will not kill server, I still have the question: can
> the rest of the input data in the socket buffer be read after the
> RST and the SIGPIPE, if server calls recv() later on its socket ?

For AF_UNIX sockets, yes, at least on Linux[*]. But similar to the
"what if the client disconnected already" proposition, the 'server'
has no way of knowing if the connection was reset until the reset
'occurs' within the TCP stream. This going to happen some time after
the final write of the server because this data needs to travel to the
remote endpoint and the RST must come back. Since either data or RST
could be lost in transmission, this could possibly be a *long* time
after this final write, ie, your question doesn't really make sense.

[*]
---------------------
#include <sys/socket.h>
#include <stdio.h>
#include <stdlib.h>
#include <signal.h>

int main(void)
{
int fds[2], rc, d;

rc = socketpair(AF_UNIX, SOCK_STREAM, 0, fds);
if (rc == -1) {
perror("socketpair");
exit(1);
}

switch (fork()) {
case -1:
perror("fork");
exit(1);

case 0:
close(*fds);

write(fds[1], "ha!", 3);

close(fds[1]);
pause();
}

close(fds[1]);
sleep(1);

signal(SIGPIPE, SIG_IGN);

rc = write(*fds, &d, sizeof(d));
if (rc == -1) perror("write");

rc = read(*fds, &d, sizeof(d));
if (rc == -1) {
perror("read");
exit(1);
}

fprintf(stderr, "read returned %d\n", rc);
return 0;
}

[Jorgen Grahn]

On Thu, 2013-02-14, termin...@gmail.com wrote:
...

> I am still confused about it, but you are saying that:
>
> An attempt by the server to send() after the client has already
> shutdown(SHUT_RDWR) normally results in a RST on the server
> side and would raise SIGPIPE.
>
> By default SIGPIPE kills the process, and frankly from my experience
> a connection reset does not kill my process (either client or server).

It's a bit confusing to speak about "client" and "server" here; as far
as TCP is concerned they are peers, once the "client" has connected ok
to the "server".

(There are of course traffic patterns /on top of/ TCP, e.g. a HTTP
server is really a server.)

/Jorgen

--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .