Jumbo Frames and TCP Segmentation Offload

[Miguel Sanders]

Hi folks

I was wondering if there is an additional performance gain in enabled
TCP segmentation offload in a jumbo frame network. What's your opinion
on that?
Also, what are the caveats of TCP segmentation offload?

Thanks

Miguel

[Rick Jones]

Miguel Sanders <miguel...@telenet.be> wrote:
> I was wondering if there is an additional performance gain in
> enabled TCP segmentation offload in a jumbo frame network. What's
> your opinion on that?

I used to call TSO "Poor Man's Jumbo Frames" since it offered
send-side benefits similar to that of Jumbo Frames but not receive
side.

TSO is sending side only. It does not reduce the burden on the
reciever. If the receiver does not support LRO (Large Receive
Offload) then enabling JumboFrames will help the receiver
considerably during bulk transfers.

Otherwise, adding JumboFrames to TSO (or TSO to JumboFrames) does have
a benefit, but it is rapidly approaching diminishing returns. One
benefit to JumboFrames (not sure if it also happens with LRO) is a
reduction in the number of ACK segments per KB transferred. In the
presence of CKO (a pre-req for TSO) and TSO and perhaps copy
avoidance, ACK processing starts to dominate in bulk transfer. The
TSO sender can send up to 64K at a time down the stack, so has one
"send" as far as CPU util is concerned, but will receive upwards of 22
ACKs from the reciever with the typical 1500 byte MTU and
corresponding MSS.

If one happens to be sending large UDP datagrams, TSO does nothing for
that, but JumboFrames can reduce the IP fragmentation and so help.
With UDP though there is no MSS exchange so one must be certain that
both ends can do JumboFrames. With TCP's MSS exchange, if you enable
JumboFrames at one end, but not the other, you get your backside
covered and traffic will still flow, based on the smaller MTU.

> Also, what are the caveats of TCP segmentation offload?

It adds a little bit of overhead to the sending path to decide when to
do a large send and by how much. It may also make traffic a bit more
bursty.

It does virtually nothing for small sends, particularly small
request/response exchanges.

rick jones
--
A: Because it fouls the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?