Slow Chrome file download over HTTP3 (QUIC)

[Harry Zhang]

Hi Team,

I tested file download speed over HTTP2 and HTTP3 using Chrome (v99.0.4844.82) / Chromium (v102.0.4963.0). It looks like HTTP3 is significantly slower. For example, it takes 10s vs. 25s to download a 1GB binary file over HTTP2 vs HTTP3. The throughput is stable (880Mbps vs. 335Mbps).

The network is Ethernet. The client machine (Ubuntu 18.04, 4.15.0-166-generic) and the server machine (Ubuntu 18.04, 4.18.0-041800-generic) are two hops away.

Any ideas? Could the problems come from the QUIC protocol, Chrome/Chromium, or machine configurations?

Thanks,

Harry

[K. Moon]

I don't know what the current state of HTTP3 performance is, but there's really not enough information here to pinpoint the problem at all. For example, if there's a problem with the server's implementation of HTTP3, you would need to test with a different HTTP3 client to isolate that.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/041e6c30-0c0c-42f9-8d8d-eab246f6526dn%40chromium.org.

[K. Moon]

Thanks, that data is helpful. Given that curl's performance is fine, I agree it's most likely a Chromium-specific issue in this case, then.

On Thu, Apr 7, 2022, 5:15 PM Harry <harryx...@gmail.com> wrote:

Sorry I forgot to mention the HTTP3-capable server. It is OpenLiteSpeed.

I actually tested it with about HTTP3 client, curl complied with HTTP3 support (ngtcp2). For curl - OpenLiteSpeed, the results of HTTP/2 and HTTP/3 file download are quite similar: 9.7s vs 10.5s (throughput 889Mbps vs. 846Mbps on average).

So I personally was guessing there could be some issues with Chromium's implementation of QUIC (or some specific parameter configurations) or Chromium as a multi-functional "browser" is more complicated and different from curl which is more like a simple file downloader.

[Harry Zhang]

Any hints on what part (of Chromium source) I should look at to pinpoint the issue? I am thinking to tune some QUIC/UDP-related parameters.

Thanks!

[Harry]

Sorry I forgot to mention the HTTP3-capable server. It is OpenLiteSpeed.

I actually tested it with about HTTP3 client, curl complied with HTTP3 support (ngtcp2). For curl - OpenLiteSpeed, the results of HTTP/2 and HTTP/3 file download are quite similar: 9.7s vs 10.5s (throughput 889Mbps vs. 846Mbps on average).

So I personally was guessing there could be some issues with Chromium's implementation of QUIC (or some specific parameter configurations) or Chromium as a multi-functional "browser" is more complicated and different from curl which is more like a simple file downloader.

On Thu, Apr 7, 2022 at 8:08 PM K. Moon <km...@chromium.org> wrote:

[Yutaka Hirano]

+net-dev 

How are CPU usage of Chromium (especially, "Utility: Network Service" process) in both cases?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/28f2ecf6-1d58-4171-b591-4b58ca1975f3n%40chromium.org.

[Nick Harper]

One thing you can do experiment with Chromium's QUIC stack is to build and use its standalone QUIC client. https://www.chromium.org/quic/playing-with-quic/ has some instructions on that. If you're making code changes in a Chromium checkout, most of the QUIC code is in https://source.chromium.org/chromium/chromium/src/+/main:net/third_party/quiche/src/quic/?q=f:quiche&ss=chromium (some of it's in //net/quic, but that's mostly code to interface between Google's quiche library and Chromium).

On Thu, Apr 7, 2022 at 9:06 PM Harry Zhang <harryx...@gmail.com> wrote:

Thanks for the pointer. I just checked the CPU usage for Utility: Network Service. Upon visually check, during HTTP/2 download, it is around 80% while during HTTP/3 download, it is always around 100%. Any takeaway here? Maybe the high overhead of QUIC does not allow Chromium to receive at high speed?

By the way, does Chromium provide any command/scripts to log down such stats?

You received this message because you are subscribed to the Google Groups "net-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to net-dev+u...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/net-dev/de26d40c-b196-425a-a14f-6b5a6927f7f2n%40chromium.org.

[Harry Zhang]

Thanks for the pointer. I just checked the CPU usage for Utility: Network Service. Upon visually check, during HTTP/2 download, it is around 80% while during HTTP/3 download, it is always around 100%. Any takeaway here? Maybe the high overhead of QUIC does not allow Chromium to receive at high speed?

By the way, does Chromium provide any command/scripts to log down such stats?

On Thursday, April 7, 2022 at 11:23:51 PM UTC-4 Yutaka Hirano wrote:

[Harry Zhang]

I tried the quic_client provided in the Chromium source. With the same setup (two hops to my HTTP server, 1G link, etc. as mentioned before), I was able to reach over 820Mbps download throughput, which is similar to the performance of Chrome during HTTP/2 downloads (~880Mbps). Then it looks like the problems come from Chromium itself instead of the quiche's QUIC stack implementation?

Do you know I can save the SSL session keys when using quic_client? For Chrome/cURL, I can save it by setting an env variable before running them ("export SSLKEYLOGFILE=/path/to/sslkey.log").

Also, is there any way to run quic_client over HTTP/2? For example, pass some specific parameters to quic_client or run a separate binary (something like http2_client)?

[Harry Zhang]

Regarding the QUIC code, it seems that the part you mentioned (https://source.chromium.org/chromium/chromium/src/+/main:net/third_party/quiche/src/quic/?q=f:quiche&ss=chromium) is not publicly available at this point but I do find another folder relevant (https://source.chromium.org/chromium/chromium/src/+/main:net/third_party/quiche/src/quiche/quic/). Maybe the directory structure was recently modified?

On Friday, April 8, 2022 at 1:12:58 PM UTC-4 nha...@chromium.org wrote:

[Sapphire Yu]

Hi Harry,

I'm running into the same issue here. Did you find out how to save SSL key so that I can decrypt the packets?

Thanks,