Sending UDP payload of size more than 1472 bytes

[Ash]

I have been testing some basic network functions in Android using
native code and I found that if my UDP client sends a payload of size
more than 1472 bytes, it does not reach to the destination (or
reassembled on IP layer at the destination). In my socket program the
sendto API still returns success (i.e. no of bytes sent).

I tried tracing with Wireshark but I could not even find a single IP
trace.

Is this a bug? a feature/ or configuration/ MTU specific?

Regards,
Ash

[David Given]

Ash wrote:
> I have been testing some basic network functions in Android using
> native code and I found that if my UDP client sends a payload of size
> more than 1472 bytes, it does not reach to the destination (or
> reassembled on IP layer at the destination). In my socket program the
> sendto API still returns success (i.e. no of bytes sent).

There's a Linux kernel option that enables UDP MTU discovery. If turned
on, then the kernel will try to discover the minimum MTU to each target
IP address and return an error code if you try to send a packet that's
too big. (If turned off it'll fragment UDP packets instead.) You might
be running into this, although it's suspicious that you're not seeing an
error code --- it's supposed to return an EMSGSIZE error in this situation.

(1472 bytes is the amount of UDP payload you get in a 1500 byte ethernet
frame.)

See the udp and ip man pages for details.

--
┌─── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ───── http://www.cowlark.com ─────
│
│ "They laughed at Newton. They laughed at Einstein. Of course, they
│ also laughed at Bozo the Clown." --- Carl Sagan

[Roman ( T-Mobile USA)]

Use ifconfig to find out about the MTU size of your interface.

--
Roman Baumgaertner
Sr. SW Engineer-OSDC
·T· · ·Mobile· stick together
The views, opinions and statements in this email are those of the
author solely in their individual capacity, and do not necessarily
represent those of T-Mobile USA, Inc.

[Jason Proctor]

UDP is datagram service. IP itself won't do fragmentation and
reassembly of UDP packets, that's what TCP is there for. each
individual datagram has to be self-contained, as any packet might be
lost in transport and the receiver has to be able to determine state
from anywhere in the "stream".

1500 bytes is the length of an ethernet frame, but it's become
something of a standard. hence it's no surprise that you're not
seeing any joy sending UDP packets over that length.

why are you using UDP? :-)

(usually it's used for situations where the loss of a packet here and
there is not that important, like streaming A/V content etc.)

>Use ifconfig to find out about the MTU size of your interface.

--
jason.vp.engineering.particle

[Pawel Veselov]

Hi,

I thought he said "basic" network functions (Ash?), like send/sendto...
EMSGSIZE is only supposed to be returned when the packet transmission
is required to be atomic, and this requirement is imposed by the
socket. UDP is as well fragmentable as TCP, IMHO you need to care for
fragmentation only when you dealing with raw IP sockets (can't really
do that on Android, being non-root and all).

Btw, AFAIK, packets, while fragmented, are IP packets, as
fragmentation information is a part of IP header. Kernel is supposed
to read all fragmented packets, in any order, including the master one
that has UDP header, and assemble them, or throw them away, if all
fragments aren't received within a timeout.

Being a stickler about these things, I've ran this
(http://manticore.2y.net/temp/udp-send.c) on a Linux desktop, with 10k
byte buffer, and enjoyed tcpdump displaying nicely fragmented packets
on both sending and receiving sides (and dissappointed in netcat,
since for some reason it only reads first 1000bytes). Interfaces on
both systems had 1,500 MTU.

So, the questions to Ash are: what syscalls are you using to send
data, what network are they coming over, and where are you running
tcpdump?

I wouldn't be surprised if the carrier network, or default device
configuration blackholes fragmented UDP packets, for example, but I
see no reason why wouldn't this work over WiFi, or USB link.

P.S.
Yes, using UDP is ineffective due to packet loss, especially
ineffective if you fragment them, but that's not really the point, it
should still work reasonably reliable.

Thanks,
Pawel.

On Thu, Sep 10, 2009 at 10:04 AM, Roman ( T-Mobile USA)
<roman.bau...@t-mobile.com> wrote:
>
> Use ifconfig to find out about the MTU size of your interface.
>

[skipped]

[Ash]

I am testing with Android emulator (E) running on Ubuntu host (S1). My
UDP client is running on Android emulator (E) and it is trying to send
UDP packets to another system (S2). I am running wireshark on the host
system (S1).

S1 and S2 are on LAN. S1 to S2 ping time is less than 0.5ms. And if
the UDP payload size is less than 1472 it reaches to the S2 100 out of
100 times. The MTU size on S1 and S2 is 1500.

For convenience, I am attaching my test program.

http://fykfpa.bay.livefilestore.com/y1pbt54e7DumqTdNn7qsdyyyejv_kqjxeV0LtaUJTgzeaj_6DWWALvP5aCSNKZdpK80I2MPxLOsNnejKCLHuoBBuMsISEMbQQ0t/testUdp.c?download

Thanks
Ash

> <roman.baumgaert...@t-mobile.com> wrote:
>
> > Use ifconfig to find out about the MTU size of your interface.
>
> [skipped]
>
>
>
>
>
> > On Sep 10, 3:43 am, David Given <d...@cowlark.com> wrote:
> > > Ash wrote:
> > > > I have been testing some basic network functions in Android using
> > > > native code and I found that if my UDP client sends a payload of size

> > > > more than1472bytes, it does not reach to the destination (or

> > > > reassembled on IP layer at the destination). In my socket program the
> > > > sendto API still returns success (i.e. no of bytes sent).
>
> > > There's a Linux kernel option that enables UDP MTU discovery. If turned
> > > on, then the kernel will try to discover the minimum MTU to each target
> > > IP address and return an error code if you try to send a packet that's
> > > too big. (If turned off it'll fragment UDP packets instead.) You might
> > > be running into this, although it's suspicious that you're not seeing an
> > > error code --- it's supposed to return an EMSGSIZE error in this situation.
>

> > > (1472bytes is the amount of UDP payload you get in a 1500 byte ethernet
> > > frame.)
>
> > > See the udp and ip man pages for details.- Hide quoted text -
>
> - Show quoted text -

[Pawel Veselov]

Ash,

I'll see what I can figure out.
In the mean time, could you run tcpdump on the emulator, and see what
it shows up?
Also, make sure that you don't filter 'udp' protocol, best is to see
all the traffic, since UDP frags may not be recognized as UDP packets.

tcpdump for android can be found here:
http://junxian-huang.blogspot.com/2009/03/finally-tcpdump-on-gphone-g1-android.html

Thanks,
Pawel.

[David Turner]

Or just use the -tcpdump <file> option when starting the emulator :-)

[Pawel Veselov]

Mmm, unless there is a problem getting data from inside the emulated
kernel to the emulated interface :)

--
With best of best regards
Pawel S. Veselov

[Pawel]

This seems to be working fine on the device, I can push through 5k UDP
packets with no problem.
So, I would recommend you trying the device, may be there is some code
in the emulator routing that breaks UDP fragmentation.

Thanks,
Pawel.

On Sep 11, 8:57 am, Ash <ashutoshkagra...@gmail.com> wrote:
> I am testing with Android emulator (E) running on Ubuntu host (S1). MyUDPclient is running on Android emulator (E) and it is trying to sendUDPpackets to another system (S2). I am running wireshark on the host

> system (S1).
>
> S1 and S2 are on LAN.  S1 to S2 ping time is less than 0.5ms. And if

> theUDPpayload size is less than 1472 it reaches to the S2 100 out of

> 100 times. The MTU size on S1 and S2 is 1500.
>
> For convenience, I am attaching my test program.
>

> http://fykfpa.bay.livefilestore.com/y1pbt54e7DumqTdNn7qsdyyyejv_kqjxe...

>
> Thanks
> Ash
>
> On Sep 11, 5:49 am, Pawel Veselov <pawel.vese...@gmail.com> wrote:
>
> > Hi,
>
> > I thought he said "basic" network functions (Ash?), like send/sendto...
> > EMSGSIZE is only supposed to be returned when the packet transmission
> > is required to be atomic, and this requirement is imposed by the

> > socket.UDPis as well fragmentable as TCP, IMHO you need to care for

> > fragmentation only when you dealing with raw IP sockets (can't really
> > do that on Android, being non-root and all).
>
> > Btw, AFAIK, packets, while fragmented, are IP packets, as
> > fragmentation information is a part of IP header. Kernel is supposed
> > to read all fragmented packets, in any order, including the master one

> > that hasUDPheader, and assemble them, or throw them away, if all

> > fragments aren't received within a timeout.
>
> > Being a stickler about these things, I've ran this
> > (http://manticore.2y.net/temp/udp-send.c) on a Linux desktop, with 10k
> > byte buffer, and enjoyed tcpdump displaying nicely fragmented packets
> > on both sending and receiving sides (and dissappointed in netcat,
> > since for some reason it only reads first 1000bytes). Interfaces on
> > both systems had 1,500 MTU.
>
> > So, the questions to Ash are: what syscalls are you using to send
> > data, what network are they coming over, and where are you running
> > tcpdump?
>
> > I wouldn't be surprised if the carrier network, or default device

> > configuration blackholes fragmentedUDPpackets, for example, but I

> > see no reason why wouldn't this work over WiFi, or USB link.
>
> > P.S.

> >   Yes, usingUDPis ineffective due to packet loss, especially

> > ineffective if you fragment them, but that's not really the point, it
> > should still work reasonably reliable.
>
> > Thanks,
> >   Pawel.
>
> > On Thu, Sep 10, 2009 at 10:04 AM, Roman ( T-Mobile USA)
>
> > <roman.baumgaert...@t-mobile.com> wrote:
>
> > > Use ifconfig to find out about the MTU size of your interface.
>
> > [skipped]
>
> > > On Sep 10, 3:43 am, David Given <d...@cowlark.com> wrote:
> > > > Ash wrote:
> > > > > I have been testing some basic network functions in Android using

> > > > > native code and I found that if myUDPclient sends a payload of size

> > > > > more than1472bytes, it does not reach to the destination (or
> > > > > reassembled on IP layer at the destination). In my socket program the
> > > > > sendto API still returns success (i.e. no of bytes sent).
>

> > > > There's a Linux kernel option that enablesUDPMTU discovery. If turned

> > > > on, then the kernel will try to discover the minimum MTU to each target
> > > > IP address and return an error code if you try to send a packet that's

> > > > too big. (If turned off it'll fragmentUDPpackets instead.) You might

> > > > be running into this, although it's suspicious that you're not seeing an
> > > > error code --- it's supposed to return an EMSGSIZE error in this situation.
>

> > > > (1472bytes is the amount ofUDPpayload you get in a 1500 byte ethernet
> > > > frame.)
>
> > > > See theudpand ip man pages for details.- Hide quoted text -

[Michael Richardson]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jason Proctor wrote:
> UDP is datagram service. IP itself won't do fragmentation and
> reassembly of UDP packets, that's what TCP is there for. each
> individual datagram has to be self-contained, as any packet might be
> lost in transport and the receiver has to be able to determine state
> from anywhere in the "stream".

what?

It's true that IPv6 won't do fragmentation, but IPv4 certainly does.
It's done it for UDP packets since the dawn of time.
I think you should read RFC791 and RFC792.

- --
] He who is tired of Weird Al is tired of life! | firewalls [
] Michael Richardson, Sandelman Software Works, Ottawa, ON |net architect[
] m...@sandelman.ottawa.on.ca http://www.sandelman.ottawa.on.ca/ |device driver[
Kyoto Plus: watch the video <http://www.youtube.com/watch?v=kzx1ycLXQSE>
then sign the petition.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Finger me for keys

iQEVAwUBSszV84CLcPvd0N1lAQJm6gf8DOKUACACsM9DnzuGbdbjorXlWMmhTnch
OGv5rxgDcrF6fWc7hrXXTXA0zjYfaWrl4pm3AvbSWyOXFI5ACSY/ZNSIY6NW68qX
OWJys7ts31sPPKank6yDgTOSIPRMVvfSTpYU8j+TJaGx6ypQ2uB/JmKMjuPtE4xx
5nsHH2vbR5CkFyDWnRjTFk7scRFzdtG48QYSIRnFwqLXEot6MwcVfSXX/YgmGxPF
v9pNDPuuA5Vtmm7XP3VAQst/tjAj/EwvDt3gEhRDQyegYqqMOFd5FN0p/eaA7rcX
F/C2ncArI/OwKZ8aICEubcbREMcpt4EmDQYauQoKjF0jYVU5Pt4Ppw==
=9cl1
-----END PGP SIGNATURE-----