Hi all,
I have been working on a research project where we have been testing the file transfer performance of a few protocols including QUIC over long distance high bandwidth networks. I have some preliminary results from some tests that I did that compare QUIC with TCP, as well as some questions if someone is willing to answer them.
I started with the quic-server and quic-client, but found HTTP to be unnecessary for what was exclusively file transfer. I made a modified version of the quic-server and quic-client that is just a clone of the existing SPDY streams with the HTTP components removed, and can send arbitrarily large files. I had hoped that this would improve throughput slightly, but it seems to be the same as the original quic-server and quic-client.
All my transfers were sent from the University of Saskatchewan in Saskatoon, Canada, and I tested different receiving ends in Waterloo Ontario, Sherbrooke Quebec, Auckland New Zealand, and another machine at the University of Saskatchewan (Saskatoon, Canada). The maximum bandwidth of every connection is 1Gbit/sec.
Round trip times:
U of S: 1ms
Waterloo: 35ms
Sherbrooke: 40ms
Auckland: 180ms
I tested throughput of single streams as well as multiple competing streams.
Units are all in Megabits per second.
1 QUIC stream
UofS: 41.7233
Waterloo: 41.446
Sherbrooke: 40.4536
Auckland: 38.9536
2 competing QUIC streams
UofS: 40.3249
Waterloo: 45.0905
Sherbrooke: 42.7394
Auckland: 38.4414
4 competing QUIC streams
UofS: 21.3517
Waterloo: 35.4937
Sherbrooke: 24.7738
Auckland: 31.1733
1 TCP stream
UofS: 236.565
Waterloo: 300.809
Sherbrooke: 268.247
Auckland: 80.3132
2 competing TCP streams
UofS: 186.822
Waterloo: 201.329
Sherbrooke: 189.109
Auckland: 80.3061
4 competing TCP streams
UofS: 93.1884
Waterloo: 96.5582
Sherbrooke: 102.322
Auckland: 62.6127
1 QUIC stream, 1 TCP stream
UofS: tcp: 193.821500, quic: 30.571950
Waterloo: tcp: 212.380750, quic: 33.172700
Sherbrooke: tcp: 121.629550, quic: 29.944550
Auckland: tcp: 60.834225, quic: 28.134875
2 QUIC streams, 2 TCP streams
UofS: tcp: 89.198750, quic: 27.389675
Waterloo: tcp: 55.140150, quic: 34.871375
Sherbrooke: tcp: 64.906875, quic: 29.896775
Auckland: tcp: 52.522150, quic: 28.242350
4 QUIC streams, 4 TCP streams
UofS: tcp: 25.190400, quic: 12.482325
Waterloo: tcp: 14.821700, quic: 16.252675
Sherbrooke: tcp: 17.351000, quic: 14.555875
Auckland: tcp: 13.298825, quic: 15.776350
In most cases TCP had higher throughput, but in situations of high congestion and larger round trip times (four QUIC streams, four TCP streams to Auckland), QUIC starts to perform better.
I used a 500MB tar file for all the tests, but I also tried a 1.2GB file which gave similar results.
I used dumpcap to capture outgoing packets on the machine doing the sending, and wireshark/tshark to find the average bandwidth. I did multiple transfers throughout the day and took the average of them, but there are still some wierd values (transfers to Waterloo sometimes perform better than transfers to the local U of S network).
I also tried changing some flags and constants in quic_flags_list.h and quic_connection.cc. In quic_connection.cc, I changed kDefaultRetransmittablePacketsBeforeAck from 2 to 20, and saw a noticable increase in speed:
1 QUIC stream
UofS: 68.3129
Waterloo: 66.5002
Sherbrooke: 62.5588
Auckland: 61.6268
2 QUIC streams
UofS: 61.772
Waterloo: 64.5282
Sherbrooke: 59.3394
Auckland: 59.5028
4 QUIC streams
UofS: 35.7902
Waterloo: 53.0485
Sherbrooke: 35.5793
Auckland: 50.0515
1 QUIC stream, 1 TCP stream
UofS: tcp: 190.115250, udp: 50.875500
Waterloo: tcp: 221.220250, udp: 49.034500
Sherbrooke: tcp: 218.880500, udp: 43.976825
Auckland: tcp: 61.341775, udp: 47.311275
2 QUIC streams, 2 TCP streams
UofS: tcp: 105.374000, udp: 40.930250
Waterloo: tcp: 72.324425, udp: 50.957400
Sherbrooke: tcp: 97.856250, udp: 38.281200
Auckland: tcp: 54.357875, udp: 48.186700
4 QUIC streams, 4 TCP streams
UofS: tcp: 37.996725, udp: 21.136800
Waterloo: tcp: 25.026875, udp: 28.635550
Sherbrooke: tcp: 29.878325, udp: 20.490775
Auckland: tcp: 24.937275, udp: 27.463325
I also tried using BBR for congestion control. The performance seemed to be slightly better Cubic with the kDefaultRetransmittablePacketsBeforeAck set to 2:
1 QUIC stream
UofS: 49.0245
Waterloo: 48.9832
Sherbrooke: 42.8096
Auckland: 23.9718
2 competing QUIC streams
UofS: 39.498
Waterloo: 44.0193
Sherbrooke: 37.9645
Auckland: 23.675
4 competing QUIC stream
UofS: 20.9513
Waterloo: 36.2447
Sherbrooke: 20.6761
Auckland: 21.98
1 QUIC streams, 1 TCP streams
UofS: tcp: 292.518000, udp: 47.321600
Waterloo: tcp: 229.384000, udp: 42.630550
Sherbrooke: tcp: 267.684000, udp: 40.822400
Auckland: tcp: 79.150450, udp: 24.795650
2 QUIC streams, 2 TCP streams
UofS: tcp: 136.782000, udp: 34.992700
Waterloo: tcp: 93.637250, udp: 45.999450
Sherbrooke: tcp: 109.088150, udp: 35.688450
Auckland: tcp: 75.388550, udp: 23.777550
4 QUIC streams, 4 TCP streams
UofS: tcp: 43.921250, udp: 17.456000
Waterloo: tcp: 25.748300, udp: 27.228700
Sherbrooke: tcp: 26.704600, udp: 16.596000
Auckland: tcp: 27.802950, udp: 19.001650
I have a few questions:
- Do these numbers seem reasonable?
- Is it possible to get better performance out of BBR in QUIC?
I've only enabled BBR in quic_flags_list.h, but I noticed quite a few other flags relating to BBR in the same file and am not sure which ones are worth investigating.
- Are there other parameters and constants worth looking into that might lead to better performance on long distance high bandwidth connections?
Thanks,
- Carl Hofmeister