MTU size for a 3g mobile broadband dongle
Mark Hobley
an Edimax 3g-6200n router, which connects to the internet via a 3g mobile
broadband stick.
Can a 3g mobile broadband stick route the 1500 byte packets arriving from
the LAN without fragmentation, or should I reduce the size of the MTU for
outbound traffic? If so, what should I reduce it to? (My internet service
provider is Three Mobile, if that matters.)
My next question is ...
Will Linux allow me to configure two MTU sizes on a single network interface,
depending on whether or not the destination address is local or remote?
If I need to reduce the MTU for outbound traffic, can I keep using a 1500
byte MTU for LAN traffic?
(The computers have only one ethernet connection onto a single subnet. There
is a single ethernet based router (Netgear) before the Edimax unit, which
forwards internet bound traffic via the Edimax unit).
Mark.
--
Mark Hobley
Linux User: #370818 http://markhobley.yi.org/
Rick Jones
> Will Linux allow me to configure two MTU sizes on a single network
> interface, depending on whether or not the destination address is
> local or remote?
No, but you can manually introduce PMTU routes up in your routing
table to accomplish that task.
rick jones
--
denial, anger, bargaining, depression, acceptance, rebirth...
where do you want to be today?
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...
Jorgen Grahn
On Mon, 2010-09-27, Morten Reistad wrote:
> In article <i7qage$ej$1...@news.eternal-september.org>,
> Mark Hobley <markh...@yahoo.donottypethisbit.co> wrote:
>>My LAN is ethernet based. However, for connectivity to the internet, I have
>>an Edimax 3g-6200n router, which connects to the internet via a 3g mobile
>>broadband stick.
>>
>>Can a 3g mobile broadband stick route the 1500 byte packets arriving from
>>the LAN without fragmentation, or should I reduce the size of the MTU for
>>outbound traffic? If so, what should I reduce it to? (My internet service
>>provider is Three Mobile, if that matters.)
>
> You expect us to answer this? You are sitting on the network. Fire
> up wireshark, or even tcpdump, and send som max-sized ping packets
> to somewhere, and then you can tell us.
Yeah.
I can add that there's most likely at least one bottleneck: at one
point your packets go over GTP over UDP over IP, on Ethernet.
*Someone* will get hurt if you use 1500-octet packets -- possibly the
operator.
>>My next question is ...
>>
>>Will Linux allow me to configure two MTU sizes on a single network interface,
>>depending on whether or not the destination address is local or remote?
>>
>>If I need to reduce the MTU for outbound traffic, can I keep using a 1500
>>byte MTU for LAN traffic?
>
> No. You can use a setsocopt to set tcp segment sizes on sockets. I would
> like to see this as a sysctl variable. One mtu for all. The linux box
> will ACCEPT bigger muts though.
You've mentioned this several times recently. What about PMTU
discovery -- doesn't that work for you? It seems to work well enough
(or never kick in) for me.
/Jorgen
--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .
Andy Furniss
> You expect us to answer this? You are sitting on the network. Fire
> up wireshark, or even tcpdump, and send som max-sized ping packets
> to somewhere, and then you can tell us.
Telling us what the somewhere he pinged has set their mtu to may be a
bit misleading as to the nature of his ISP settings, though.
Andy Furniss
> My LAN is ethernet based. However, for connectivity to the internet, I have
> an Edimax 3g-6200n router, which connects to the internet via a 3g mobile
> broadband stick.
>
> Can a 3g mobile broadband stick route the 1500 byte packets arriving from
> the LAN without fragmentation, or should I reduce the size of the MTU for
> outbound traffic? If so, what should I reduce it to? (My internet service
> provider is Three Mobile, if that matters.)
Don't know, but if it works OK for all sites then I guess so.
That's a bit oversimplistic because I have no idea about 3G devices, eg.
they may be doing some mss clamping that reduces the size of packets in
tcp connections.
Tcpdump/wireshark etc will show this - but don't just rely on looking at
one connection as you may be just seeing how they have set things up
rather than learning about your ISP.
I have seen threads where people claim better results by using smaller
MTU due to smaller packets having more chance of getting through rf
noise type loss, so experimenting if you have problems could improve
things - if you don't have problems I wouldn't bother.
>
> My next question is ...
>
> Will Linux allow me to configure two MTU sizes on a single network interface,
> depending on whether or not the destination address is local or remote?
>
> If I need to reduce the MTU for outbound traffic, can I keep using a 1500
> byte MTU for LAN traffic?
Like Rick has said you can do things with ip route. Another way would be
to use iptables to mss clamp non local tcp connections.
>
> (The computers have only one ethernet connection onto a single subnet. There
> is a single ethernet based router (Netgear) before the Edimax unit, which
> forwards internet bound traffic via the Edimax unit).
A quick test I just did using ip route shows it is possible to just
modify the default route, which I assume is all you would need.
Normally if you set mtu on eth then the tcp advertised mss will also
adjust so that incoming tcp packets are also limited in size. Using ip
route this doesn't happen, but you can use an extra commant to specify.
Testing on this PC which already has a default route setup to
192.168.0.1, the command -
ip route change dev eth0 default via 192.168.0.1 mtu 1000 advmss 960
seems to do the trick and leaves traffic to local net unchanged.
Andy Furniss
Jorgen Grahn wrote:
> I can add that there's most likely at least one bottleneck: at one
> point your packets go over GTP over UDP over IP, on Ethernet.
> *Someone* will get hurt if you use 1500-octet packets -- possibly the
> operator.
They shouldn't get hurt if they know what they are doing - most recent
ISP type kit should be able to run MTU > 1500 to handle this.
In the UK alot of ISPs use the dominant teleco for transport and they
use L2TP - and explicitly state in the specs that the ISP must run a
higher MTU to allow for this.
> You've mentioned this several times recently. What about PMTU
> discovery -- doesn't that work for you? It seems to work well enough
> (or never kick in) for me.
Does your ISP restrict your inbound to < 1500, though. If it doesn't you
are not likely to have problems.
In the case of pppoe type ISPs the 1492 they (may) run is overcome by
the fact that consumer dsl modem/routers mss clamp TCP avoids PMTUD
black hole faliures.
Rick Jones
True, the reply coming-back fragmented means either the remote, or the
ISP(s) in the middle had a smaller-than-echo-size MTU. However, if
the reply comes-back unfragmented we know that both the remote and
ISP(s) in the middle had an MTU of at least that size.
rick jones
--
firebug n, the idiot who tosses a lit cigarette out his car window
Mark Hobley
> You expect us to answer this? You are sitting on the network. Fire up
> wireshark, or even tcpdump, and send som max-sized ping packets to
> somewhere, and then you can tell us.
What is the simplest way to send a max-sized ping packet? Is there
something I can type at the command line to do this?
jack
> On Mon, 27 Sep 2010 21:29:04 +0200, Morten Reistad wrote:
>
>> You expect us to answer this? You are sitting on the network. Fire up
>> wireshark, or even tcpdump, and send som max-sized ping packets to
>> somewhere, and then you can tell us.
>
> What is the simplest way to send a max-sized ping packet? Is there
> something I can type at the command line to do this?
>
man ping
....
-s packetsize
Specifies the number of data bytes to be sent. The default is 56,
which translates into 64 ICMP data bytes when combined with the 8 bytes
of ICMP header data.
.....
HTH
Andy Furniss
> On Mon, 27 Sep 2010 21:29:04 +0200, Morten Reistad wrote:
>
>> You expect us to answer this? You are sitting on the network. Fire up
>> wireshark, or even tcpdump, and send som max-sized ping packets to
>> somewhere, and then you can tell us.
>
> What is the simplest way to send a max-sized ping packet? Is there
> something I can type at the command line to do this?
>
> Mark.
>
ping -s 1472 will send a 1500 packet (1472 + 8 bytes ICMP header + 20 IP
header)
Mark Hobley
> ....
> -s packetsize
> Specifies the number of data bytes to be sent. The default is 56,
> which translates into 64 ICMP data bytes when combined with the 8 bytes
> of ICMP header data.
Right. I didn't realize that ping could carry a payload. Cheers, my results
are as follows:
On a system with PMTUD enabled, the maximum value for the -s switch is 1384:
ping -s 1384 www.yahoo.com
PING any-fp.wa1.b.yahoo.com (67.195.160.76) 1384(1412) bytes of data.
1392 bytes from ir1.fp.vip.ac4.yahoo.com (67.195.160.76): icmp_seq=2 ttl=45 time=305 ms
If I disable PMTUD, I can use a larger maximum value as follows:
echo 1 > /proc/sys/net/ipv4/ip_no_pmtu_disc
ping www.yahoo.com -s 1472
PING any-fp.wa1.b.yahoo.com (67.195.160.76) 1472(1500) bytes of data.
1480 bytes from ir1.fp.vip.ac4.yahoo.com (67.195.160.76): icmp_seq=1 ttl=45 time=1455 ms
So, am I right in thinking that datagrams bigger than 1412 cannot be carried
without fragmentation, and that I would be better to reduce my MTU to a value of
1412 for externally bound traffic?
Mark Hobley
I have some more weirdness now. If I ping from one host to another, connected
via ethernet, I would expect a maximum datagram size of 1500. However, I
actually get a result of 65535, as follows:
# PMTUD is disabled on both hosts:
echo 0 > /proc/sys/net/ipv4/ip_no_pmtu_disc
ping -s 65507 miranda
PING miranda.markhobley.yi.org (10.0.0.30) 65507(65535) bytes of data.
65515 bytes from miranda.markhobley.yi.org (10.0.0.30): icmp_seq=1 ttl=64
time=13.8 ms
Does this mean that jumbo frames have become enabled on my network?
Jorgen Grahn
On Wed, 2010-09-29, Mark Hobley wrote:
> On Wed, 29 Sep 2010 17:16:12 +0000, Mark Hobley wrote:
>
> I have some more weirdness now. If I ping from one host to another, connected
> via ethernet, I would expect a maximum datagram size of 1500. However, I
> actually get a result of 65535, as follows:
>
> # PMTUD is disabled on both hosts:
> echo 0 > /proc/sys/net/ipv4/ip_no_pmtu_disc
>
> ping -s 65507 miranda
> PING miranda.markhobley.yi.org (10.0.0.30) 65507(65535) bytes of data.
> 65515 bytes from miranda.markhobley.yi.org (10.0.0.30): icmp_seq=1 ttl=64
> time=13.8 ms
>
> Does this mean that jumbo frames have become enabled on my network?
No, it means you should install and use[1] tcpdump to see what this
actually translates to on the Ethernet level. Hint: IP fragmentation.
/Jorgen
[1] If you have root access on either machine, that is.
Mark Hobley
> ["Followup-To:" header set to comp.protocols.tcp-ip.]
>
> On Wed, 2010-09-29, Mark Hobley wrote:
>> On Wed, 29 Sep 2010 17:16:12 +0000, Mark Hobley wrote:
>>
>> I have some more weirdness now. If I ping from one host to another,
>> connected via ethernet, I would expect a maximum datagram size of 1500.
>> However, I actually get a result of 65535, as follows:
>>
>> # PMTUD is disabled on both hosts:
>> echo 0 > /proc/sys/net/ipv4/ip_no_pmtu_disc
>>
>> ping -s 65507 miranda
>> PING miranda.markhobley.yi.org (10.0.0.30) 65507(65535) bytes of data.
>> 65515 bytes from miranda.markhobley.yi.org (10.0.0.30): icmp_seq=1
>> ttl=64 time=13.8 ms
>>
>> Does this mean that jumbo frames have become enabled on my network?
>
> No, it means you should install and use[1] tcpdump to see what this
> actually translates to on the Ethernet level. Hint: IP fragmentation.
>
Right. I have done a tcpdump, at first examining the internet bound stuff:
With PMTUD enabled:
21:13:47.557102 IP venus.markhobley.yi.org > ir1.fp.vip.ac4.yahoo.com: ICMP echo request, id 52485, seq 4, length 1392
21:13:47.811766 IP ir1.fp.vip.ac4.yahoo.com > venus.markhobley.yi.org: ICMP echo reply, id 52485, seq 4, length 1392
With PMTUD disabled:
21:19:24.453588 IP venus.markhobley.yi.org > ir1.fp.vip.re1.yahoo.com: ICMP echo request, id 58885, seq 6, length 1480
21:19:24.976825 IP ir1.fp.vip.re1.yahoo.com > venus.markhobley.yi.org: ICMP echo reply, id 58885, seq 6, length 1480
So, with PMTUD disabled, I can send and receive packets of 1480 to and from yahoo,
presumably fragmentation and reassembly is taking place en-route and I cannot see this.
With PMTUD enabled, my maximum packet size if 1392.
Now, if PMTUD is disabled, why do I now have a limit of 1480? I would have expected packets
larger than this to become fragmented and transmitted successfully.
This actually does happen on the LAN. Here are those 65535 long being fragmented at
1480 for transportation over ethernet:
21:34:00.635725 IP venus.markhobley.yi.org > miranda.markhobley.yi.org: ICMP echo request, id 64005, seq 2, length 1480
21:34:00.635760 IP venus.markhobley.yi.org > miranda.markhobley.yi.org: icmp
blah blah blah ...
21:34:00.640468 IP venus.markhobley.yi.org > miranda.markhobley.yi.org: icmp
21:34:00.643685 IP miranda.markhobley.yi.org > venus.markhobley.yi.org: ICMP echo reply, id 64005, seq 2, length 1480
21:34:00.643804 IP miranda.markhobley.yi.org > venus.markhobley.yi.org: icmp
blah blah blah ...
21:34:00.648989 IP miranda.markhobley.yi.org > venus.markhobley.yi.org: icmp
21:34:04.638350 ARP, Request who-has venus.markhobley.yi.org tell miranda.markhobley.yi.org, length 46
21:34:04.638376 ARP, Reply venus.markhobley.yi.org is-at 00:11:2f:bc:df:19 (oui Unknown), length 28
I guess upstream have set a limit of 1480, even though they have to fragment to carry this.
There is another strange thing I noticed. Why does miranda make an arp request for venus at
the end of the exchange? I would have thought that miranda already knows the mac address
of venus, because she has just been communicating with it.
Rick Jones
> ....
> -s packetsize
> Specifies the number of data bytes to be sent. The default is 56,
> which translates into 64 ICMP data bytes when combined with the 8 bytes
> of ICMP header data.
> .....
I guess there is an opportunity to fix the "linux" ping manpage then -
that would be 64 IP data bytes when the 56 bytes of ICMP data are
added to the 8 bytes of ICMP header...
Since comp.protocols.tcp-ip is included, here is how for the ping
which ships with HP-UX :)
SYNOPSIS
ping [-oprv] [-f address-family] [-i address] [-I interval] [-t ttl]
host [-n count [-m timeout]]
ping [-oprv] [-f address-family] [-i address] [-I interval] [-t ttl]
host packet-size [ [-n] count [-m timeout]]
rick jones
--
portable adj, code that compiles under more than one compiler
Andy Furniss
> Right. I have done a tcpdump, at first examining the internet bound stuff:
Remember - don't just test one address, try the f.root-servers.net that
Morten suggested.
> So, with PMTUD disabled, I can send and receive packets of 1480 to and from yahoo,
> presumably fragmentation and reassembly is taking place en-route and I cannot see this.
If you use the -v switch for tcpdump you will see whether the replies
have DF set or not.
>
> With PMTUD enabled, my maximum packet size if 1392.
>
> Now, if PMTUD is disabled, why do I now have a limit of 1480?
That 1480 is IP data so add 20 = 1500 it's a bit hit or miss whether
you'll get a server on the internet to answer to a > 1500.
ICMPs with DF set aren't the same as other packets with DF set because
AIUI ICMP packets aren't allowed to generate further ICMPs so there will
be no PMTUD to help them get through.
I would have expected packets
> larger than this to become fragmented and transmitted successfully.
> 1500 works for me to f.root-servers.net but not yahoo, even if the
target doesn't answer tcpdump should show you outgoing fragged echo request.
Rick Jones
> arp request for venus at the end of the exchange? I would have
> thought that miranda already knows the mac address of venus, because
> she has just been communicating with it.
If it happens all the time at the end of the exchange, that would be
odd. If you just see it once and a while, that stems from ARP
maintaining a cache, and periodically refreshing the entries. ARP
does not see or know that traffic is being exchanged successfully,
thus the periodic refreshes.
rick jones
--
A: Because it fouls the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
Rick Jones
> ICMPs with DF set aren't the same as other packets with DF set because
> AIUI ICMP packets aren't allowed to generate further ICMPs so there will
> be no PMTUD to help them get through.
I think there must be an exception in there somewhere - until it was
found to be a vector for an amplification attack, years ago HP-UX did
a "hybrid" PathMTU discovery by default. When it first started
talking to a remote IP, it would send IP datagrams with DF cleared -
however, at the same time, it would send an ICMP Echo Request to the
remote with the DF bit set. It would lather, rinse and repeat until
it could get the Echo reply without generating any datagram too big
messages. I believe that was to "deal" with black holes where the
ICMP Datagram Too Big messages were being filtered.
rick jones
--
denial, anger, bargaining, depression, acceptance, rebirth...
where do you want to be today?
Andy Furniss
> In comp.protocols.tcp-ip Andy Furniss<sp...@andyfurniss.entadsl.com> wrote:
>
>> ICMPs with DF set aren't the same as other packets with DF set because
>> AIUI ICMP packets aren't allowed to generate further ICMPs so there will
>> be no PMTUD to help them get through.
>
> I think there must be an exception in there somewhere
Ugh, yes, I really should have tested that rather than relying on
something I once read.
On my LAN with vanilla Linux kernel settings I can indeed get a frag
needed in response to an echo request.