Sockets: TIME_WAIT/FIN_WAIT hanging around
Nick Bishop
clients connect, there is an exchange, then the server closes the
connection.
The "problem" is when I use netstat, I get TIME_WAIT entries.
My sequence is basically
m_listener = new TcpListener(10020);
m_listener.Start();
mySocket = m_listener.AcceptSocket();
mySocket.Receive and Send
mySocket.Shutdown(SocketShutdown.Both);
// The TIME_WAIT appears here
mySocket.Close();
m_listener.Stop();
// Program termination here.
Another problem I have is getting FIN_WAIT (or similar) but I haven't
been able to reliably reproduce this. However, I'd like to know what
the usual causes are, because I'm evolving the program.
To diagnose the TIME_WAIT problem, I reduced my program to the minimum
required to demonstrate the problem (complete program below) and
reproduced the problem with telnet as the client.
There are no problems if the client terminates the connection (^] then
quit in telnet). If I use Task Manager to brutally kill the telnet
process, I get
10054 An existing connection was forcibly closed by the remote host
and there are no problems.
Using telnet from a different machine makes no difference to the
behaviour (except that netstat only gives one ESTABLISHED entry).
System Information
==================
Visual Studio .NET installed on Windows 2000
Microsoft Visual C# .NET 55603-651-0000007-18445
Item Value
OS Name Microsoft Windows 2000 Professional
Version 5.0.2195 Service Pack 2 Build 2195
OS Manufacturer Microsoft Corporation
System Name MELWKS183
System Manufacturer Dell Computer Corporation
System Model Dimension 2350
System Type X86-based PC
Processor x86 Family 15 Model 1 Stepping 3 GenuineIntel ~1700 Mhz
BIOS Version Phoenix - AwardBIOS v6.00PG
Windows Directory C:\WINNT
Locale United States
Time Zone AUS Eastern Standard Time
Total Physical Memory 260,592 KB
Available Physical Memory 62,792 KB
Total Virtual Memory 893,036 KB
Available Virtual Memory 521,540 KB
Page File Space 632,444 KB
Problem reproduction
====================
Compile the program. Locate the exe and double-click. This brings up
a console (referred to as 1).
Get another two consoles (2 & 3).
In (2) I type & get Y:\>netstat -a -n | find "10020"
TCP 0.0.0.0:10020 0.0.0.0:0 LISTENING
In (3) I type: telnet localhost 10020
In (2) I type & get Y:\>netstat -a -n | find "10020"
TCP 0.0.0.0:10020 0.0.0.0:0 LISTENING
TCP 127.0.0.1:1680 127.0.0.1:10020 ESTABLISHED
TCP 127.0.0.1:10020 127.0.0.1:1680 ESTABLISHED
In (3) I type Hello and press ENTER
I get the echo of Hello, followed by Canned Response
In (2) I type & get Y:\>netstat -a -n | find "10020"
TCP 0.0.0.0:10020 0.0.0.0:0 LISTENING
TCP 127.0.0.1:1680 127.0.0.1:10020 ESTABLISHED
TCP 127.0.0.1:10020 127.0.0.1:1680 ESTABLISHED
In (1) I see Smash the enter key for Socket.Shutdown
so I do so (gently)
In (3) I get Connection to host lost
In (2) I type & get Y:\>netstat -a -n | find "10020"
TCP 0.0.0.0:10020 0.0.0.0:0 LISTENING
* TCP 127.0.0.1:10020 127.0.0.1:1680 TIME_WAIT ***
In (1) I see Smash the enter key for Socket.Close
so I do so (gently)
In (2) I type & get Y:\>netstat -a -n | find "10020"
TCP 0.0.0.0:10020 0.0.0.0:0 LISTENING
* TCP 127.0.0.1:10020 127.0.0.1:1680 TIME_WAIT ***
In (1) I see Smash the enter key for TcpListener.Stop
so I do so (gently)
In (2) I type & get Y:\>netstat -a -n | find "10020"
* TCP 127.0.0.1:10020 127.0.0.1:1680 TIME_WAIT ***
In (1) I see Smash the enter key for program termination
so I do so (gently)
In (2) I type & get Y:\>netstat -a -n | find "10020"
* TCP 127.0.0.1:10020 127.0.0.1:1680 TIME_WAIT ***
If I wait a while, the TIME_WAIT disappears.
Complete program
================
// You may like to cut and paste this into notepad if it doesn't
// look pretty here.
using System;
using System.Net.Sockets;
using System.Threading;
namespace NickBishop
{
/// <summary>
/// Summary description for Class1.
/// </summary>
class Class1
{
static TcpListener m_listener;
static Socket mySocket;
const int BUFFER_SIZE = 128;
static int bytesRecdToDate; // Cumulative
static int bytesRecdNow; // Increment
static bool isLineRecd;
static byte[] theData;
static void sendAndReceiveData()
{
try
{
bytesRecdNow = mySocket.Receive(theData, bytesRecdToDate,
BUFFER_SIZE - bytesRecdToDate, SocketFlags.None);
while (!isLineRecd && bytesRecdNow > 0 && BUFFER_SIZE -
bytesRecdToDate > 2)
{
// Search for a linefeed character
int idx;
for (idx = 0; idx < bytesRecdNow; ++idx)
{
if (theData[idx + bytesRecdToDate] == '\n')
{
// Truncate received string so linefeed is last char
bytesRecdNow = idx +1;
isLineRecd = true;
break; // for
}
}
// Echo character(s) back
mySocket.Send(theData, bytesRecdToDate, bytesRecdNow,
SocketFlags.None);
bytesRecdToDate += bytesRecdNow;
bytesRecdNow = 0;
if (!isLineRecd)
bytesRecdNow = mySocket.Receive(theData, bytesRecdToDate,
BUFFER_SIZE - bytesRecdToDate, SocketFlags.None);
}
// Canned response
byte[] buf = System.Text.Encoding.ASCII.GetBytes("\r\nCanned
response\r\n");
mySocket.Send(buf);
}
catch (SocketException e)
{
System.Console.Write(e.ErrorCode.ToString());
System.Console.WriteLine(" " + e.Message);
}
}
/// <summary>
/// The main entry point for the application.
/// </summary>
[STAThread]
static void Main(string[] args)
{
//
// TODO: Add code to start application here
//
m_listener = new TcpListener(10020);
m_listener.Start();
mySocket = m_listener.AcceptSocket();
bytesRecdToDate = 0; // Cumulative
// bytesRecdNow; // Increment
isLineRecd = false;
theData = new byte[BUFFER_SIZE];
sendAndReceiveData();
System.Console.Write("Smash the enter key for Socket.Shutdown ");
System.Console.ReadLine();
mySocket.Shutdown(SocketShutdown.Both);
System.Console.Write("Smash the enter key for Socket.Close ");
System.Console.ReadLine();
mySocket.Close();
System.Console.Write("Smash the enter key for TcpListener.Stop ");
System.Console.ReadLine();
m_listener.Stop();
System.Console.Write("Smash the enter key for program termination
");
System.Console.ReadLine();
}
}
}
// End of program
Nick Bishop
Email replies ignored.
Rich Blum
> I wrote a server program which accepts multiple connections. The
> clients connect, there is an exchange, then the server closes the
> connection.
>
> The "problem" is when I use netstat, I get TIME_WAIT entries.
>
> been able to reliably reproduce this. However, I'd like to know what
> the usual causes are, because I'm evolving the program.
>
TIME_WAIT and FIN_WAIT entries are part of the TCP states as
described in RFC 793. From the RFC, the definition of TIME-WAIT is:
"TIME-WAIT - represents waiting for enough time to pass to be sure
the remote TCP received the acknowledgment of its connection
termination request."
The TIME-WAIT state is normal, and is used to ensure there are no
"left over" packets floating around the network before the connection
is closed. If you follow the state diagram shown in the RFC, you will
notice that the device that initiates closing the TCP connection
(sends the first FIN) enters the TIME-WAIT state, while the device
that receives the first FIN does not. Depending on whether your server
or the client initiates closing the session will determine which one
enters the TIME_WAIT state. No other socket can bind to that address
until the TIME-WAIT state is over (usually 60 seconds).
If you need to reuse a socket that is in TIME_WAIT state, you can
use the ReuseAddress socket level option on your socket before the
Bind():
Socket sock = new Socket(....);
sock.SetSocketOption(SocketOptionLevel.Socket,
SocketOptionName.ReuseAddress,
1);
On the other hand, the FIN_WAIT state represents a more difficult
problem. According to the RFC there are two FIN_WAIT states:
" FIN-WAIT-1 - represents waiting for a connection termination
request
from the remote TCP, or an acknowledgment of the connection
termination request previously sent.
FIN-WAIT-2 - represents waiting for a connection termination
request
from the remote TCP."
It is normal for a connection to enter the FIN_WAIT state,
however, if you have a connection that stays in the FIN_WAIT state,
this can mean that the client did not properly terminate the TCP
session.
Hope this info helps you out some.
Rich Blum
Author of "C# Network Programming" (Sybex)
http://www.sybex.com/sybexbooks.nsf/Booklist/4176
Nick Bishop
> Nick -
>
> TIME_WAIT and FIN_WAIT entries are part of the TCP states as
> described in RFC 793. From the RFC, the definition of TIME-WAIT is:
>
> "TIME-WAIT - represents waiting for enough time to pass to be sure
> the remote TCP received the acknowledgment of its connection
> termination request."
>
> The TIME-WAIT state is normal, and is used to ensure there are no
> "left over" packets floating around the network before the connection
> is closed. If you follow the state diagram shown in the RFC, you will
> notice that the device that initiates closing the TCP connection
> (sends the first FIN) enters the TIME-WAIT state, while the device
> that receives the first FIN does not. Depending on whether your server
> or the client initiates closing the session will determine which one
> enters the TIME_WAIT state. No other socket can bind to that address
> until the TIME-WAIT state is over (usually 60 seconds).
>
> If you need to reuse a socket that is in TIME_WAIT state, you can
> use the ReuseAddress socket level option on your socket before the
> Bind():
I have some additional questions, but first an observation ...
Thanks for the detailed explanation. I wanted to eliminate bad
programming or a missed step as a cause. While you use class Socket
directly, my program uses class TcpListener, which I suspect sets the
ReuseAddress option under the covers, because I'm able to restart the
program even if there is a TIME_WAIT hanging around.
I've searched other newsgroups for explanations of FIN_WAIT_x
conditions (mostly in unix groups) and it seems that the same
ReuseAddress option will work around that problem too (they also
explain root causes).
*****
Question:
I see the TIME_WAIT entries when my C-sharp program (see original
post) closes the connection, but I don't see TIME_WAIT entries when
telnet closes the connection first. I achieve this in one of two ways
1) In telnet, press CTRL-^, then at the telnet prompt, type QUIT
2) Use task manager to kill the telnet process brutally
Why the discrepancy?
2nd question:
Your explanation assumes one of the parties sends its FIN first. What
happens if both parties send their FIN (and the two FINS cross in the
post, so to speak, quite possible over a slow network)?
Nick Bishop
email replies ignored
Rich Blum
> Question:
> I see the TIME_WAIT entries when my C-sharp program (see original
> post) closes the connection, but I don't see TIME_WAIT entries when
> telnet closes the connection first. I achieve this in one of two ways
> 1) In telnet, press CTRL-^, then at the telnet prompt, type QUIT
> 2) Use task manager to kill the telnet process brutally
> Why the discrepancy?
>
session. In a normal Telnet session, the client usually types
something to exit the host session (such as typing 'exit' on a Unix
host), then the server initiates closing the TCP session by sending
the FIN packet, and goes into TIME-WAIT state. When you type 'QUIT' on
the Telnet session, you are forcing the client to terminate the TCP
session, and send the first FIN packet. Thus the server will go into
the CLOSE-WAIT state and not use TIME-WAIT.
> 2nd question:
> Your explanation assumes one of the parties sends its FIN first. What
> happens if both parties send their FIN (and the two FINS cross in the
> post, so to speak, quite possible over a slow network)?
>
The TCP state diagram in RFC 793 accounts for this. When a device
sends a FIN packet it enters FINWAIT-1 state. If it receives a FIN
packet without an ACK of its FIN (if the other end tries to close the
session as well) it enters the CLOSING state, waiting for the ACK from
its FIN packet. This is true on both ends of the connection. Both ends
should then send an ACK packet to properly close the session (in this
scenario both ends should enter the TIME-WAIT state before closing).
Hope this helps.
Nick Bishop
> It depends on what sends the first FIN packet to close the
> session. In a normal Telnet session, the client usually types
> something to exit the host session (such as typing 'exit' on a Unix
> host), then the server initiates closing the TCP session by sending
> the FIN packet, and goes into TIME-WAIT state. When you type 'QUIT' on
> the Telnet session, you are forcing the client to terminate the TCP
> session, and send the first FIN packet. Thus the server will go into
> the CLOSE-WAIT state and not use TIME-WAIT.
Presumably we should see TIME-WAIT at the client end, WHICH I DON'T,
but I'm not going to chase this down anymore. I'll put it down to a
Microsoftism.
> > (question)
> The TCP state diagram in RFC 793 accounts for this. When a device
> sends a FIN packet it enters FINWAIT-1 state. If it receives a FIN
> packet without an ACK of its FIN (if the other end tries to close the
> session as well) it enters the CLOSING state, waiting for the ACK from
> its FIN packet. This is true on both ends of the connection. Both ends
> should then send an ACK packet to properly close the session (in this
> scenario both ends should enter the TIME-WAIT state before closing).
> Hope this helps.
Thanks for your answers, Rich. This makes me a bit wiser (and less
paranoid that I've muffed something up).
Nick Bishop
(the same Nick Bishop as on http://www.theage.com.au -> Business ->
OpenTel news stories for 19 July 2003)
Email replies ignored.