Are QUIC client and server provided in Chromium performant?

[Arash Molavi]

The Playing with QUIC page reads:

"note: both the client and server are meant mainly for integration testing: neither is performant at scale!"

What if I want to do some simple benchmarking, e.g. serve an object using the server, fetch it using the client and record the time and compare it against HTTP (basically something like what Connectify folks did here: Taking quick for a test drive).

Are QUIC client and server provided in Chromium performant for such tests? Because if they're not, any such performance tests will be unfair to QUIC!

[Ian Swett]

It has improved over time, but it's certainly not as fast as a kernel TCP implementation at the moment. 

Any bandwidth under 100Mbps should be fine based on benchmarks we've done.  The most recent benchmarks indicated the client was slower than the server.

--
You received this message because you are subscribed to the Google Groups "QUIC Prototype Protocol Discussion group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to proto-quic+...@chromium.org.
To post to this group, send email to proto...@chromium.org.
For more options, visit https://groups.google.com/a/chromium.org/d/optout.

[Jana Iyengar]

Just clarifying that the limits you hit are CPU limits. As Ian points out, you are most likely fine under 100Mbps, but are likely to hit CPU limits beyond that point.

[Romain Jacotin]

Can i conclude of this discussion that Google is using a kernel implementation of QUIC on server side, or at least an intelligent coupling between kernel and server code (user space code can for example delegate the pacing task to the kernel and just ask it to adjust the pacing rate) ?

Cheers,

Romain

[Ian Swett]

Our implementation is largely the same as the Chromium on the server side, with some OS specific optimizations(which may be released at some point), but

1) We're being conservative(I expect you could easily see 200Mbits/core from the current client-server on relatively new desktop CPUs)

2) The QuicServer appears to be slower per-core than our server.

3) The QuicClient is slow enough that it's typically the limiting factor.  One obvious optimization is that it currently doesn't use recvmmsg, it uses recvmsg.

What we're hoping to avoid is a characterization of QUIC based on a 1Gbps egress test, which isn't an interesting test for most of our users anyway.

--

[Romain Jacotin]

Thank you for your answer Ian (was also thinking of sendmmsg to reduce syscalls, but that's far from a pure Go implementation)

[Arash Molavi]

Just wanted to add that the QuicClient which is part of Chromium is definitely the bottleneck. I did some simple head-to-head comparison of QUIC and HTTP/2 by downloading files of different sizes from Google Drive. When I use Google Chrome as my client, QUIC's and HTTP/2's download times (extracted from devtools) are close enough. However, if I use QuicClient provided in Chromium and get (or curl for none-QUIC downloads), quic download times can be an order of magnitude worse!

Which makes me think at this point, the only fair setup for head-to-head comparisons is to use Google Chrome for client side and Google Servers for server side. Any thoughts on this?

[Gaetano Carlucci]

Hi,

 

in the case of the results presented in this paper http://dl.acm.org/citation.cfm?id=2695664.2695706&coll=DL&dl=ACM&CFID=533410066&CFTOKEN=94965030

the QUIC server we used was the one extracted from Chromium whereas the QUIC client was Chromium itself.

This benchmark worked well for our purpose and at the bitrate considered (less than 10Mbps). 

2015-08-04 17:23 GMT+02:00 Arash Molavi <arash....@gmail.com>:

Just wanted to add that the QuicClient which is part of Chromium is definitely the bottleneck. I did some simple head-to-head comparison of QUIC and HTTP/2 by downloading files of different sizes from Google Drive. When I use Google Chrome as my client, QUIC's and HTTP/2's download times (extracted from devtools) are close enough. However, if I use QuicClient provided in Chromium and get (or curl for none-QUIC downloads), quic download times can be an order of magnitude worse!

Which makes me think at this point, the only fair setup for head-to-head comparisons is to use Google Chrome for client side and Google Servers for server side. Any thoughts on this?

--