Bad router QoS settings can disrupt QUIC

[Nelson Minar]

I just fixed a problem where Google Chrome wasn't working well with Google sites and wanted to share my experience in case it helps anyone.

The problem was that Google sites were loading very slowly in Chrome using QUIC. HTTP requests everywhere were working fine, but Google search and Google Groups using QUIC were super-slow. I think the problem was my router's QoS settings. The default QoS config for this firmware has a rule which is "unidentified UDP protocols are labeled Crawl and limited to 5% of bandwidth". QUIC was unidentified. Removing that rule fixed the problem.

The QoS rule in the router is definitely stupid but it may not be entirely uncommon. I had the rule as a default from Tomato v1.32 (Toastman). I think some of the newer Tomato/Shibby builds have a rule like this as well. QoS is off by default in Tomato, but if you turn it on you get this Crawl classification. And while Tomato is hackerware and people who use it should know better, I suspect many of its QoS ideas have made their way into commercial products. Newer ASUS routers, for instance, have a Tomato-derived firmware installed.

Every time I see a new UDP protocol on the Internet I worry what all may break. I love UDP but a whole lot of Internet infrastructure these days is optimized only for TCP (or worse, HTTP). I don't think QUIC is a bad idea at all, I'm just curious what UDP misconfigurations it will uncover.

[Ian Swett]

Thanks for that discovery.  As you said, the internet is very much optimized for TCP, so we knew we'd hit some cases like these, but they are very difficult to reliably detect client side, though we are starting to look into it.

If others have evidence of this in other router firmwares, it would be great to know.

--
You received this message because you are subscribed to the Google Groups "QUIC Prototype Protocol Discussion group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to proto-quic+...@chromium.org.
To post to this group, send email to proto...@chromium.org.
For more options, visit https://groups.google.com/a/chromium.org/d/optout.

[Nelson Minar]

Resurrecting an ancient topic, I reported on a problem with an old version of the Tomato / Shibby router firmware, QoS, and QUIC connections getting heavily throttled.

I recently installed a new version of Tomato / Shibby (v130) and can confirm it now has a special QoS protocol rule for QUIC. It's classifying UDP traffic on port 443 as "WWW", which by default is allowed up to 70% of bandwidth. That's consistent with other web traffic. (There's also a second rule that UDP 443 connections with more than 512kbytes get assigned a lower tier of service, although that's still up to 70% of bandwidth.) The firmware still has a catch-all rule that unidentified UDP traffic gets throttled to 5%. And I didn't see a rule for UDP port 80, so QUIC on port 80 might still be heavily throttled.

I should re-iterate Tomato / Shibby is hackerware not used by many people. And QoS is turned off by default. But if someone does install it and enable QoS they will get these traffic shaping rules for QUIC. I think the current state is reasonable enough, certainly better than what I originally found.

[Ian Swett]

Thanks for the update, that sounds like major progress.  

As Ryan emailed a yesterday, insecure QUIC is no longer supported, we now only support HTTPS over port 443, so the new QoS rules should be perfect.  I realized that email didn't mention 80 and 443 explicitly, so I just sent a note about that.

--

[Dave Taht]

What are the underlying qdiscs in their qos system?

Can you paste some

tc -s qdisc show the_inbound_device_name_whatever_it_is

statistics?

On Thu, Jul 30, 2015 at 2:15 AM, Nelson Minar <nel...@monkey.org> wrote:

> --
> You received this message because you are subscribed to the Google Groups
> "QUIC Prototype Protocol Discussion group" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to proto-quic+...@chromium.org.
> To post to this group, send email to proto...@chromium.org.
> For more options, visit https://groups.google.com/a/chromium.org/d/optout.

--
Dave Täht
worldwide bufferbloat report:
http://www.dslreports.com/speedtest/results/bufferbloat
And:
What will it take to vastly improve wifi for everyone?
https://plus.google.com/u/0/explore/makewififast

[Nelson Minar]

On Thu, Jul 30, 2015 at 3:25 AM, Dave Taht <dave...@gmail.com> wrote:

What are the underlying qdiscs in their qos system? Can you paste some tc -s qdisc show the_inbound_device_name_whatever_it_is statistics?

Oops, sorry, I already uninstalled the firmware. The stock firmware has an elaborate set of some 50 classification rules and 8 or more classes that you get if you enable QoS. Only reason I looked at it closely as I did is those rules used to break QUIC, although as I said they've now fixed it. If you want to follow up:

Source code for firmware:

  http://tomato.groov.pl/?page_id=166

Commit that added the QUIC rule (28 May 2015)

  http://repo.or.cz/w/tomato.git/commit/6638a4dc715d1d5af4006eb1907894619c8543f8

Discussion of the firmware's QOS rules and QUIC

  http://www.linksysinfo.org/index.php?threads/qos-development-thread.31886/#post-255209

[Dave Taht]

On Thu, Jul 30, 2015 at 4:09 PM, Nelson Minar <nel...@monkey.org> wrote:
> On Thu, Jul 30, 2015 at 3:25 AM, Dave Taht <dave...@gmail.com> wrote:
>>
>> What are the underlying qdiscs in their qos system? Can you paste some tc
>> -s qdisc show the_inbound_device_name_whatever_it_is statistics?
>>
>
> Oops, sorry, I already uninstalled the firmware. The stock firmware has an
> elaborate set of some 50 classification rules and 8 or more classes that you
> get if you enable QoS. Only reason I looked at it closely as I did is those
> rules used to break QUIC, although as I said they've now fixed it. If you
> want to follow up:

In my world, fq_codel and the successor, cake, needs no specialized rules
except for bittorrent, as near as I can tell.

> Source code for firmware:
> http://tomato.groov.pl/?page_id=166
>
> Commit that added the QUIC rule (28 May 2015)
>
> http://repo.or.cz/w/tomato.git/commit/6638a4dc715d1d5af4006eb1907894619c8543f8
>
> Discussion of the firmware's QOS rules and QUIC
>
> http://www.linksysinfo.org/index.php?threads/qos-development-thread.31886/#post-255209
>