When are payload lengths greater than the negotiated MSS?
Ashwin Rao
The following traces are of a connection between a client and server
running on the same machine. The connection is over a local loop and
the MTU is limited to 1500 bytes for the local loop device.
During connection establishment the MSS exchanged is 1460 as shown
in the three way handshake sniffed using tcpdump.
17:22:25.111903 IP (tos 0x8, ttl 64, id 4549, offset 0, flags [DF],
proto TCP (6), length 60) 127.0.0.1.50930 > 127.0.0.1.15001: S, cksum
0xdc50 (correct), 339494931:339494931(0) win 5840 <mss
1460,sackOK,timestamp 22998 0,nop,wscale 7>
17:22:25.312009 IP (tos 0x8, ttl 64, id 0, offset 0, flags [DF], proto
TCP (6), length 60) 127.0.0.1.15001 > 127.0.0.1.50930: S, cksum 0xb2a4
(correct), 336116516:336116516(0) ack 339494932 win 5792 <mss
1460,sackOK,timestamp 23198 22998,nop,wscale 7>
17:22:25.512392 IP (tos 0x8, ttl 64, id 4550, offset 0, flags [DF],
proto TCP (6), length 52) 127.0.0.1.50930 > 127.0.0.1.15001: ., cksum
0xf652 (correct), ack 1 win 46 <nop,nop,timestamp 23398 23198>
After some time the packets with length (2948) greater than 1460 bytes
(even after accounting for the TCP header sans options) are being
exchanged over the same socket. Following are three packets from the
same connection
17:22:34.846900 IP (tos 0x8, ttl 64, id 59157, offset 0, flags [DF],
proto TCP (6), length 2948) 127.0.0.1.15001 > 127.0.0.1.50930: .
5828:8724(2896) ack 113 win 46 <nop,nop,timestamp 32733 32533>
17:22:34.868313 IP (tos 0x8, ttl 64, id 59159, offset 0, flags [DF],
proto TCP (6), length 2948) 127.0.0.1.15001 > 127.0.0.1.50930: .
8724:11620(2896) ack 113 win 46 <nop,nop,timestamp 32754 32554>
17:22:34.952985 IP (tos 0x8, ttl 64, id 59161, offset 0, flags [DF],
proto TCP (6), length 2948) 127.0.0.1.15001 > 127.0.0.1.50930: .
11620:14516(2896) ack 113 win 46 <nop,nop,timestamp 32839 32608>
Data payloads with length greater than the MSS negotiated are being
exchanged. I would like to know under which conditions are payloads
greater than the MSS exchanged? Further, is there a way to limit the
payloads to the specified MSS value?
Regarding the TCP implementation, the kernel used is the linux 2.6.27
kernel.
Regards,
Ashwin
Martijn Lievaart
> Data payloads with length greater than the MSS negotiated are being
> exchanged. I would like to know under which conditions are payloads
> greater than the MSS exchanged? Further, is there a way to limit the
> payloads to the specified MSS value?
I've seen this happening with NICs that do tcp offloading. In that case,
the frame you see in tcpdump is NOT what goes over the wire, but what the
OS send to the card. FWIW, this was on AIX.
HTH,
M4
Dick Wesseling
The original post states that this occurred on the loopback device:
David Schwartz
> The following traces are of a connection between a client and server
> running on the same machine. The connection is over a local loop and
> the MTU is limited to 1500 bytes for the local loop device.
The MTU setting doesn't have the same effect on devices that don't
have media as it does on devices that do. The loopback device actually
directly connects the two endpoints of a stream connection together,
and at that point, they no longer really use the loopback device so
its MTU no longer affects them.
DS
Barry Margolin
<99716537-b4f3-4bcd...@j9g2000prh.googlegroups.com>,
David Schwartz <dav...@webmaster.com> wrote:
But MSS should still apply.
--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE don't copy me on replies, I'll read them in the group ***
David Schwartz
Pascal Hambourg
Dick Wesseling a ï¿œcrit :
> Martijn Lievaart <m...@rtij.nl.invlalid> writes:
>> On Thu, 26 Nov 2009 09:10:35 -0800, Ashwin Rao wrote:
>>
>>> Data payloads with length greater than the MSS negotiated are being
>>> exchanged. I would like to know under which conditions are payloads
>>> greater than the MSS exchanged? Further, is there a way to limit the
>>> payloads to the specified MSS value?
>>
>> I've seen this happening with NICs that do tcp offloading. In that case,
>> the frame you see in tcpdump is NOT what goes over the wire, but what the
>> OS send to the card. FWIW, this was on AIX.
>
> The original post states that this occurred on the loopback device
Linux has TSO enabled on the loopback device since version 2.6.27.
# ethtool -k lo
[...]
tcp segmentation offload: on
glen herrmannsfeldt
(snip)
> The MTU setting doesn't have the same effect on devices that don't
> have media as it does on devices that do. The loopback device actually
> directly connects the two endpoints of a stream connection together,
> and at that point, they no longer really use the loopback device so
> its MTU no longer affects them.
Maybe not, but my FreeBSD system has MTU of 16384 on lo0, and
my linux system has 16436 on lo. If I remember back to SunOS 3.5
days, I don't remember the large MTU on lo0.
-- glen
Pascal Hambourg
>
> The MTU setting doesn't have the same effect on devices that don't
> have media as it does on devices that do. The loopback device actually
> directly connects the two endpoints of a stream connection together,
> and at that point, they no longer really use the loopback device so
> its MTU no longer affects them.
What do you mean ? Here on Linux the loopback device looks very much
like any other network device, it sends and receives packets even for a
stream connection like TCP.
Steinar Haug
> >>> exchanged. I would like to know under which conditions are payloads
> >>> greater than the MSS exchanged? Further, is there a way to limit the
> >>> payloads to the specified MSS value?
> >>
> >> I've seen this happening with NICs that do tcp offloading. In that case,
> >> the frame you see in tcpdump is NOT what goes over the wire, but what the
> >> OS send to the card. FWIW, this was on AIX.
> >
> > The original post states that this occurred on the loopback device
>
> Linux has TSO enabled on the loopback device since version 2.6.27.
>
> # ethtool -k lo
> [...]
> tcp segmentation offload: on
Um, what exactly do you offload this processing to? Since the loopback
interface is internal on the machine (internal to the operating system
if you wish), I don't really see the point of TSO ...
Steinar Haug, Nethelp consulting, sth...@nethelp.no
David Schwartz
wrote:
> What do you mean ? Here on Linux the loopback device looks very much
> like any other network device, it sends and receives packets even for a
> stream connection like TCP.
It does not send and receive packets even for a stream connection like
TCP. It cuts itself out of the loop.
DS
Arlet
It would prevent TCP segmentation by just copying the application data
from sender to receiver.
Barry Margolin
<90812451-96fa-49b2...@z10g2000prh.googlegroups.com>,
David Schwartz <dav...@webmaster.com> wrote:
Why not? Where does RFC 793 say that a sender is allowed to ignore the
MSS option depending on the link layer? Although MSS is normally used
to keep segments under the link layer MTU, it could have other uses.
Albert Manfredi
> In article
> <90812451-96fa-49b2-bdcd-698229d2d...@z10g2000prh.googlegroups.com>,
> David Schwartz <dav...@webmaster.com> wrote:
>
> > On Nov 28, 12:49 am, Barry Margolin <bar...@alum.mit.edu> wrote:
>
> > > But MSS should still apply.
>
> > Why?
>
> > DS
>
> Why not? Where does RFC 793 say that a sender is allowed to ignore the
> MSS option depending on the link layer? Although MSS is normally used
> to keep segments under the link layer MTU, it could have other uses.
I agree. I can see why the Ethernet MTU would not apply, in loopback,
but the TCP MSS is higher (in the protocol stack) than all of that,
and as far as I can tell, is still in effect. Did the MSS become
renegotiated during the session?
Bert
David Schwartz
> In article
> Why not? Where does RFC 793 say that a sender is allowed to ignore the
> MSS option depending on the link layer? Although MSS is normally used
> to keep segments under the link layer MTU, it could have other uses.
It's not TCP. It replicates TCP's interface to application level, but
that's it.
DS
Barry Margolin
<0f604894-c38b-40d7...@j24g2000yqa.googlegroups.com>,
David Schwartz <dav...@webmaster.com> wrote:
Then how did his tcpdump grab the raw packets and show the TCP header
fields?
Barry Margolin
<9521372e-20ca-4959...@j11g2000vbi.googlegroups.com>,
Albert Manfredi <bert...@hotmail.com> wrote:
I don't think MSS can be renegotiated, it's only supposed to be sent
with the SYN packets. Someone correct me if I'm wrong.
David Schwartz
> > It's not TCP. It replicates TCP's interface to application level, but
> > that's it.
> Then how did his tcpdump grab the raw packets and show the TCP header
> fields?
You know, that's a very good question.
DS
Pascal Hambourg
Tcpdump and iptables see IP packets over the loopback interface. Why
should I trust you more than them ?
Pascal Hambourg
>
> Um, what exactly do you offload this processing to? Since the loopback
> interface is internal on the machine (internal to the operating system
> if you wish), I don't really see the point of TSO ...
Don't ask me, I just reported what I read in Linux 2.6.27 changelog :
loopback: Enable TSO
This patch enables TSO since the loopback device is naturally
capable of handling packets of any size. This also means that
we won't enable GSO on lo which is good until GSO is fixed to
preserve netfilter state as netfilter treats loopback packets
in a special way.
Jorgen Grahn
> In article
> <9521372e-20ca-4959...@j11g2000vbi.googlegroups.com>,
> Albert Manfredi <bert...@hotmail.com> wrote:
>> I agree. I can see why the Ethernet MTU would not apply, in loopback,
>> but the TCP MSS is higher (in the protocol stack) than all of that,
>> and as far as I can tell, is still in effect. Did the MSS become
>> renegotiated during the session?
>
> I don't think MSS can be renegotiated, it's only supposed to be sent
> with the SYN packets. Someone correct me if I'm wrong.
That someone would have to correct Stevens too:
Some texts refer to [MSS] as a 'negotiated' option. It is not
negotiated in any way. When a connection is established, each end
has the option of announcing the MSS it expects to receive. (A MSS
option can only appear in a SYN segment.)
-- Stevens, TCP/IP Illustrated
/Jorgen
--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .
David Schwartz
wrote:
> Tcpdump and iptables see IP packets over the loopback interface. Why
> should I trust you more than them ?
I'm older than them, and better looking.
DS
Rick Jones
> On Thu, 26 Nov 2009 09:10:35 -0800, Ashwin Rao wrote:
> > Data payloads with length greater than the MSS negotiated are being
> > exchanged. I would like to know under which conditions are payloads
> > greater than the MSS exchanged? Further, is there a way to limit the
> > payloads to the specified MSS value?
> I've seen this happening with NICs that do tcp offloading.
Given that stateless offloads (CKO, TSO, LRO, etc (or "Little TOE"))
are more common than stateful (TOE or "Big TOE"), chances are it is a
"large send" taking advantage of TSO (Tcp Segmentation Offload) in the
NIC.
And when CKO (ChecKsum Offload) is being used you may also see
"invalid checksum" messages on the sending side out of tcpdump since
the checksum has been deferred to the NIC.
rick jones
--
The computing industry isn't as much a game of "Follow The Leader" as
it is one of "Ring Around the Rosy" or perhaps "Duck Duck Goose."
- Rick Jones
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...
Rick Jones
not short-circuit, it is indeed possible to point tcpdump at a
loopback device and see loopback "traffic."
rick jones
--
Process shall set you free from the need for rational thought.