Max size of unreliable DataChannel packages (MTU)?

[Jesús Leganés Combarro]

On reliable DataChannels there's no a limit on the size of the data that you send (seems there's a limit of 2GB but due to browser limitations, not directly implied by DataChannels themselves) since internally the data is splitted in chunks of 64KB, but when using UDP-like, pure unreliable DataChannel connections (maxRetries=0 and maxTimeout=0), what's the max size of a single message packet (MTU) that we can be sure will get to its destination in just one fragment?

I'm currently taking in account the standard Ethernet MTU (1500 bytes aprox.) as the maximum size of the packets I'm transmiting in my application, but I'm not sure if this still applies to WebRTC DataChannels, and if so, if they can be bigger or (as I suspect) if they are smaller... Of courses I can be able to use reliable connections, but since I'm sending packets on the order of this size (1-1.5KB) and I'm already doing congestion control at the application layer, I would like to send small UDP-like messages (ideally in just one ethernet datagram) to increase performance of my application.

[Shachar]

Hi Jesus, as you know we've been dealing with the same issue in sharefest.me for a while now.

Our experiments, show that the maximum size of a single MTU is about 1300bytes.

[pir...@gmail.com]

Hi Shachar, I know you had it some time ago with sharefest, and I'm having it now with WebP2P.io :-) Sorry for the delay, I'm really busy with work and classes and development is going slow.

Does the size of 1300 bytes come from some specification, or did you found it by try-and-error? I remember Michelle Bu found this way the size of 1073 byted on RTP DataChannes. Can I be able to use securely the size of 1300 as a constant in my code, or could I have problems and should use a smaller size?

Send from my Samsung Galaxy Note II

--
 
---
You received this message because you are subscribed to a topic in the Google Groups "discuss-webrtc" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/discuss-webrtc/LZsm-jbP0zA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to discuss-webrt...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Shachar]

It's specified in the spec, but I also talked with ff/chrome guys and vreified with trial & error. (It's not exactly 1300, I don't remember the exact number, because I had to substract the overhead we as an application add to the raw data chunk size. Anywhere between 1200 to 1270 you should be fine.

[pir...@gmail.com]

It's specified in the spec, but I also talked with ff/chrome guys and vreified with trial & error. (It's not exactly 1300, I don't remember the exact number, because I had to substract the overhead we as an application add to the raw data chunk size. Anywhere between 1200 to 1270 you should be fine.

Great then :-) I was asking for the full size of the MTU since I'm going to use ArrayBuffers and a fixed-size header, so I will be able to get a reliable max raw data chunk size to transmit and would be able to take it in account. Thanks for the info! :-)

--
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un monton de sitios diferentes, simplemente escribe un sistema operativo Unix."
– Linus Tordvals, creador del sistema operativo Linux

[Jesús Leganés Combarro]

Looking for the specs where the actual size is defined I've find this links:

https://groups.google.com/forum/#!topic/discuss-webrtc/EW9a--eRLR0

1) Shouldn't the MTU be set even lower to account for the fact that we're tunneling SCTP over DTLS and UDP (~30~50 bytes and 8 bytes respectively). Round it up, and we're left with 1200 bytes?
2) Assuming we have 1200 bytes to work with, there is also the 28 byte SCTP header (with single data chunk), which means.. ~1172 bytes. Round it down, let's say 1150 bytes for application data?

https://code.google.com/p/chromium/codesearch#chromium/src/third_party/libjingle/source/talk/media/sctp/sctpdataengine.cc&l=115

static const size_t kSctpMtu = 1280;

http://tools.ietf.org/html/draft-ietf-rtcweb-data-channel-06#page-8

The initial Path MTU at the IP layer MUST NOT exceed 1200 bytes for IPv4 and 1280 for IPv6. Taking an overhead of 20 bytes for IPv4, 40 bytes for IPv6, 8 bytes for UDP, 13 + X for DTLS and 28 bytes for SCTP into account, this results in an SCTP payload of 1131 - X when IPv4 is used and 1192 - X bytes when IPv6 is used.

So here we are with three different, related size numbers: 1280 bytes on Chrome source code (256*5, that's in the order of 1270 that you told before), 1200 bytes for IPv4 and 1280 for IPv6 on the spec, and some other numbers down up to 1131. Definitely, MTU seems to be something under 1300 bytes (1280?), but what one should be considered as max allowed application data on that MTU-constrained packet? Something as low as 1131-X (that means exactly a loss of 25% on packet headers from 1500 bytes of the Ethernet MTU... and also is not a constant :-/ ), or that number is too conservative and in fact you can send some more bytes?

[Justin Uberti]

Chrome assumes wire MTU of 1280. http://tools.ietf.org/html/draft-ietf-rtcweb-data-channel-06#page-8 talks about the max SCTP payload size; as indicated in that doc, there's a bunch of headers that must be added to the SCTP payload size, the total of which must be less than the wire MTU.

If you absolutely must avoid any fragmentation, you need to use something like 1192 bytes (I forget the details of the 'X' calculation) as your max SCTP payload. If you can tolerate having messages split into two packets in < 1% of cases, you could go up to ~1320. We did some tests and MTU < 1400, while possible, is very uncommon.

Note that the actual throughput boost by going from 1192 to 1320 is pretty nominal, < 1%. Typically it's not worth worrying about.

[pir...@gmail.com]

I don't need to "absolutely" avoid any fragmentation, it's just I
consider it would improve performance if I can be able to send all the
data I need in just one packet, that's the reason why I asked for the
limits. It's lower that I expected, but after reviewing the design of
my messages and shrinking a little bit superfluous data I think I have
been able to fit all of them :-)

Thanks for the info Justin! :-D

2014-02-04 Justin Uberti <jub...@google.com>:

> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "discuss-webrtc" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/discuss-webrtc/LZsm-jbP0zA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> discuss-webrt...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

--
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."

- Linus Tordvals, creador del sistema operativo Linux

[Oleg Moskalenko]

You actually do want to "absolutely avoid any fragmentation" if you do want a decent media quality. When you have fragmentation in the media stream, the quality suffers big time. Sending packets which are just under MTU is the way to achieve the best performance.

Oleg

[Raju]

Chrome assumes wire MTU of 1280. http://tools.ietf.org/html/draft-ietf-rtcweb-data-channel-06#page-8 talks about the max SCTP payload size; as indicated in that doc, 

Does Chrome always use MTU of 1280? or is it just a starting MTU and it will be adjusted per PathMTU discovery mechanism suggested in Req. 9 of http://tools.ietf.org/html/draft-ietf-rtcweb-data-channel-06#page-8 (which basically comes from SCTP PathMTU of SCTP RFC http://tools.ietf.org/html/rfc4960#section-7.3)

In mobile environments like 4G LTE, the immediate-link MTU and PathMTU size is lower to accommodate tunneling of IP pkts in core network. In such environments, using PathMTU avoids IP fragmentation altogether.
 

-Raju

[Justin Uberti]

Chrome always uses a MTU of 1280 right now. I didn't quite follow your comment about path MTU - are you concerned that 1280 is too high or too low?

[Oleg Moskalenko]

My understanding is that in some environments the link MTU and the path MTU differ and Path MTU may be lower. Ideally, the Chrome should have to perform some kind of path MTU discovery...

Oleg

[Justin Uberti]

yes, path MTU may be different than the interface MTU, but it should never be lower than 1280. Path MTU discovery would allow the MTU used by the SCTP stack to be slightly larger, but the performance benefit from doing this is negligible (<1% bandwidth savings).

--

 
---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.

[Raju]

To avoid the terminology confusion, I want to make it clear that Chrome’s 1280 is user pkt size, which is calculated based on 1500 MTU (of Ethernet) size.

I think the 1280 calculation is lower for some access networks like 4G LTE (EPC) and scenarios like IPSec use (which is very common for telecommuting use cases). Overall, I think it is not optimal as it is not considering the various tunnelings and overheads that may be added by access networks (4G LTE GPRS tunneling) and/or applications (e.g. IPSec).

Determining packet size based on Ethernet MTU only takes care of IP fragmentation if the underlying network is Ethernet without any tunneling (or other reasons for reduced MTU size) all the way to the destination. As we know, please note that the purpose here is to avoid IP fragmentation, not just the next hop, but all the hops covering until the destination. PathMTU (RFC 4821), as the name indicates,  calculates the actual MTU size of the entire path from source ip to destination ip, independent of access networks and other tunneling that may be used. The mentioned PathMTU RFC outlines procedures to be used.

As an additional resource for 4G LTE reference, I want point to 3GPP TS 23.060 page 341 “Figure C.1: Overhead for MTU calculation” .

Other access technologies like Cable (http://www.cablelabs.com/specs) may use similar tunneling mechanism there by “reducing” the MTU size available to the applications.

The above discussion was started for data channel transport, however I also want to point out that the video RTP size should also consider this PathMTU size to avoid IP fragmentation as they traverse the entire path to the destination.If such PathMTU is not considered then in some cases (like IPSec over LTE) every video RTP pkt might be subject to IP fragmentation. Using Path MTU RFC 4821 fixes all these by calculating the true end-to-end MTU size which avoids IP fragmentation altogether.

My 2 cents!

 -Raju

[Oleg Moskalenko]

1280 is the minimum MTU in IPv6 networks. In IPv4, it can go much lower, the min practical figure is 576 bytes. Of course usually it is much larger.

Oleg

[pir...@gmail.com]

> Determining packet size based on Ethernet MTU only takes care of IP fragmentation if the underlying network is Ethernet without any tunneling (or other reasons for reduced MTU size) all the way to the destination. As we know, please note that the purpose here is to avoid IP fragmentation, not just the next hop, but all the hops covering until the destination. PathMTU (RFC 4821), as the name indicates,  calculates the actual MTU size of the entire path from source ip to destination ip, independent of access networks and other tunneling that may be used. The mentioned PathMTU RFC outlines procedures to be used.
>

Would it be possible/makes sense to be able to tell to Javascript what's the current pathMTU? Maybe with the Stats API?

[Raju]

Would it be possible/makes sense to be able to tell to Javascript what's the current pathMTU? Maybe with the Stats API?

IMHO, application/javascript should be agnostic of the underlying PathMTU size. 

In theory, application gives it's entire data to the browser in one single API call. Browser then hands it over to SCTP layer, which then have to consider PathMTU while sending DATA chunks. Other end reassembles the DATA chunks and gives one single msg to application.

This approach works well if there are no issues of one single data channel (one SCTP stream) sending single large data monopolizing the SCTP link while other data channels starving.

SCTP NDATA (http://tools.ietf.org/html/draft-stewart-tsvwg-sctp-ndata-01) solves this issue. With NDATA, I believe browser still hands the application data as is to the SCTP layer which in turn multiplexes data from other data channels so that no one starves.

But in practice until NDATA is finalized, it would be better if the application knows this limitation and not send large data or deal with the issue of other, if any, data channels starving.

Another approach to solve NOT having NDATA support is, browsers use PPID (payload protocol identifier) of SCTP data chunk hdr and fragment the user data, send PPID  as "partial", "last" (similar to ip fragmentation, except here reassembly does not require any seq # as SCTP does ordered delivery) and on the other end deliver entire data to the application when "last" PPID is received. http://tools.ietf.org/html/draft-ietf-rtcweb-data-channel-06#page-3 does define some of these PPIDs which browsers can make use of. I just noticed that this mechanism is what exactly documented in http://tools.ietf.org/html/draft-ietf-rtcweb-data-channel-06#section-6.6 

Please note that this draft refers "browser" as "application" to cover other users (like native clients) of webrtc.

So, browsers may most likely use this later approach (either based on size of the data and/or # of data channels created; for one data channel this approach is not needed unless user adds another data channel immediately after sending large data on 1st data channel) and all of this is hidden under the data channel send API.

Either way, javascript do not have to know the PathMTU size.

-Raju

[pir...@gmail.com]

>> Would it be possible/makes sense to be able to tell to Javascript what's the current pathMTU? Maybe with the Stats API?
>
> IMHO, application/javascript should be agnostic of the underlying PathMTU size. 

Yeah, I know and think this abstraction it's a great idea, I just was asking for it for an optimization corner case.

P.D.: great explanation, very detailed :-)

[Oleg Moskalenko]

On Wednesday, February 5, 2014 11:57:29 AM UTC-8, Raju wrote:

Would it be possible/makes sense to be able to tell to Javascript what's the current pathMTU? Maybe with the Stats API?

IMHO, application/javascript should be agnostic of the underlying PathMTU size. 

I

I am not sure that this is realistic in all cases. Sometimes the payload is application-specific and "abstract" chunking is impossible. For example, encoded RTP/RTCP traffic is difficult to cut into pieces. The RTP traffic must be encoded for a specific payload size if you want any decent performance.

[Justin Uberti]

576 is the minimum theoretical MTU in IPv4 networks. I have never seen it actually go below 1280 (and anything below 1400 is extremely rare).

[Justin Uberti]

Chrome is not basing its MTU off of 1500 Ethernet MTU. It uses a much lower MTU that provides sufficient headroom in all real-world cases, including 4G/IPSec.

The 1280 constant mentioned above is a sanity check to make sure we never emit a packet that would be fragmented.

--
 
---

You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.

[Oleg Moskalenko]

That is probably the best compromise approach.

[Raju]

Chrome is not basing its MTU off of 1500 Ethernet MTU. It uses a much lower MTU that provides sufficient headroom in all real-world cases, including 4G/IPSec.

The 1280 constant mentioned above is a sanity check to make sure we never emit a packet that would be fragmented.

Agreed that it's safe approach. But, IMHO that is probably not an optimal approach as in some cases you could actually be getting higher MTU size. To get optimal values and maximum throughput Req9 of http://tools.ietf.org/html/draft-ietf-rtcweb-data-channel-06#section-4 says PathMTU support is a MUST:

   Req. 9:   The data channel transport protocol SHOULD avoid IP
      fragmentation.  It MUST support PMTU (Path MTU) discovery and MUST
      NOT rely on ICMP or ICMPv6 being generated or being passed back,
      especially for PMTU discovery.

-Raju

[Raju]

IMHO, application/javascript should be agnostic of the underlying PathMTU size. 

I am not sure that this is realistic in all cases. Sometimes the payload is application-specific and "abstract" chunking is impossible. For example, encoded RTP/RTCP traffic is difficult to cut into pieces. The RTP traffic must be encoded for a specific payload size if you want any decent performance. 

If javascript application is taking care of RTP/RTCP (in some way as today browsers take care of audio/video RTP/RTCP) then application sends the RTP/RTCP (or some othe type of data) just like any other data. Browser then applies similar MTU specific logic to avoid IP fragmentation.

BTW, both VP8 (http://tools.ietf.org/html/draft-ietf-payload-vp8-10#section-4.6.4) and H.264 (http://tools.ietf.org/html/rfc3984#section-5.7) fragment video pkts across RTP pkts to avoid IP fragmentation. 

-Raju

[Justin Uberti]

Agree. But the throughput gains from doing so are minimal, < 1%, so it's not the top priority.

[Raju]

Agree. But the throughput gains from doing so are minimal, < 1%, so it's not the top priority.

Ok, understood. Thanks.