TcpAckFrequency test results
Mike Moore
here is what the VSAM Redirector Loader statistics showed.
TcpAckFrequency = default:
4,771,938 records
13,361 seconds for Overall Duration
7,446 seconds for Transfer Duration
TcpAckFrequency = 1:
4,781,173 records
11,487 seconds for Overall Duration
2,652 seconds for Transfer Duration
Looks like the TcpAckFrequency parameter has a huge affect on transfer
times.
Mike Moore
IT Manager
Alabama Judicial Datacenter
300 Dexter Ave
Montgomery, AL 36104
334-954-5025
Frank Swarbrick
I have a Windows XP Pro SP2 workstation, and did the following test:
- Connected from my workstation to "TCP/IP for VSE Internal FTPDAEMN 01.05 F
20080714 05.43".
- Set Wireshark Network Protocol Analyzer to listen for traffic between my
workstation and the VSE FTP server host.
- Did a GET of a large file in ASCII mode.
My observation is that Windows sends an ACK for every two TCP packets
received. This is confirmed by the following statement in Microsoft
Knowledge Base issue 328890:
"Typically, an acknowledgment is sent for every other TCP segment that is
received on a connection unless the delayed ACK timer (200 milliseconds)
expires. You can adjust the delayed ACK timer by editing the following
registry entry."
Further, MS says that with this fix:
"If you set the value to 1, every packet is acknowledged immediately because
there is only one outstanding TCP ACK as a segment is just received. The
value of 0 (zero) is not valid and is treated as the default, 2. The only
time the ACK number is 0 is when a segment is not received and the host is
not going to acknowledge the data."
My confusion is that Jeff Barnard states "My Linux systems do not have this
problem and have higher transfer rates too."
My understanding is that delayed ACKs are common on most (all?) TCP/IP
platforms. So I am puzzled as to why this Windows option is necessary. And
in fact, I don't understand why it is causing the issue it appears to be
causing.
First, here is my understanding of how a delayed ACK works.
Delayed ACK: This says that a receiver should delay sending an ack for a
short period of time in anticipation of the receiver's application
generating a response. If the receiver's application generates a response
quickly enough then the ACK can be piggybacked on the response. In other
words, instead of the receiver doing ACK followed almost immediately by ACK
with data, it will just send ACK with data.
In a 'sink stream' (I just made up that name) protocol such as FTP where
there is only the TCP level ack, with no application level ACK, I don't see
why a delayed ACK would cause problems (as longer as the sender TCP is not
waiting for the ACK!). It sends the ack for both packets as soon as it
receives a second packet, even if the receiving application has no data to
send.
It is my understanding the the BSI stack does not have this wait, other than
perhaps some implementation of the Nagle Algorithm (see below). So why is
this delayed ACK causing a problem? The sender should just keep sending,
not caring about acks. It should only stop sending of the receiver has
closed down the receive window.
Nagle says: If new data is to be sent and there is still unacknowledged data
on this connection, hold off sending the data until either 1) there is now
enough data to fill an entire segment (MSS), or 2) we receive the ACK of the
previously sent data.
Here's an interesting page that talks about an issue with Delayed ACKs and
Nagle:
http://www.stuartcheshire.org/papers/NagleDelayedAck/
This example is for a "request/response" protocol, however, in that the
sender waits for the receiving application to send an application level
response and not just the TCP level ack. I don't believe(!) that FTP should
suffer from this.
So what am I missing? Why does this fix to Windows "solve" this issue, and
why is it apparently not an issue at all on Linux?
Frank
Jeffrey Barnard
> So what am I missing? Why does this fix to Windows "solve" this issue, and
> why is it apparently not an issue at all on Linux?
Your email and the threads of the last few days talk about 2 things ... and
one does not have much (if anything) to do with the other.
1) Having an application issue a send() for data and having the send() not
complete until the data has been ACKed by the remote host TCP/IP stack.
BSI's stack does not do this. When a send() is done the data is queued into
the stack's transmit buffer and the send completes.
Based on the content of this thread CSI's does wait but has options to
disable this feature (at least part of the time).
I can understand why CSI did this but it can be a performance issue.
However, the issue would be generally mild and related to latency for the
most part.
2) Transfer of data to a Windows PC and not getting the throughput desired.
This is a Windows TCP/IP issue and a solution is available from Microsoft.
Adding a registry entry for TcpAckFrequency = 1 does make a nice
improvement in data transfer throughput when transferring data to a Windows
PC. Note that this is a fairly new TCP/IP configuration option so unless
the Windows box is pretty current it may not support this option (hotfixes,
etc may be required).
Both of these items can have an impact on performance. Will they both
effect you? It depends ... ;)
So, back to the original question, Why does TcpAckFrequency solve the
problem and why is this not a issue on Linux boxes?
First, the TCP/IP stacks on Windows and Linux are quite different. The
design of the TCP/IP stack on Windows has this problem and the design of
the TCP/IP stack on Linux does not. They each implement delayed ACK quite
differently.
When Windows receives a packet it will wait for 200ms for another packet to
arrive before sending an ACK. Why? Windows hopes that it can send a single
ACK for both packets. Why does this slow down a transfer? 200ms is a very
long time at network speeds. Why would this come into play at all for an
FTP? If a host is sending data and has sent one packet but there is not
enough space available in the TCP Window to send the next packet, the
sender will wait for an ACK of the data already sent. At the same time,
Windows is waiting for 200ms for another packet to come in. It does not
have to happen very often for these 200ms delays to add up.
If I understand the process on Linux (and I have not done a lot of research
on this) the delay is adaptive and much shorter. The Linux TCP/IP stack
waits to send a delayed ACK based on how often packets are arriving. This
results in a variable delay based on how fast packet are arriving. On high
speed networks this delay may be only 1ms (or even less).
FWIW, The BSI stack implements delayed ACKs in a completely different way
that works well based on the design of our stack.
Does this help?
Regards,
Jeff
--
Jeffrey C Barnard
Barnard Software, Inc. http://www.bsiopti.com
BSI 407-323-4773 Support 407-688-7123
Frank Swarbrick
Thanks for the detailed information.
I am having a heck of a time finding information about how Linux handles the
delayed ACK feature. This article is somewhat interesting:
http://itknowledgeexchange.techtarget.com/itanswers/does-the-delayed-ack-fea
ture-make-useless-the-tcp-window-size/
It even mentions the Windows knowledge base reference.
It still doesn't really explain what Linux is doing, though.
Ah well. I may post a query to comp.protocols.tcp-ip and see if anyone can
explain it.
Thanks!
Frank
Jeffrey Barnard
This page ...
http://myhsc.pbwiki.com/Inside%20Linux%20TCP::Delayed%20Ack
does a pretty good job describing Linux TCP ACK processing.
Basically works as I stated. Linux TCP has a quick ACK mode and a variable
delayed ACK process. It also knows that under some conditions (windows size
changes) ACKs should not be delayed. Overall, a very nice method.
Frank Swarbrick
It does look like Linux is much smarter about when to delay the ACK and when
not to. Good for them! :-)
n 8/20/2008 at 4:50 AM, in message <48ABF6F2...@bsiopti.com>, Jeffrey