BBRv3 experiments

[Mihai Mazilu]

Hello all,

I am an undergraduate student at University of Sussex in the UK and my dissertation was about the performance of BBR within an LEO network context through both simulation (ns3) and emulation (mininet). The start of this project involved implementing BBRv3 in ns3 so that it can be used in Hypatia (a LEO mobility model which uses ns3 for the network simulation). The implementation, while not complete due to the lack of ECN and other features not possible to replicate in ns3, was still successful, resembling its behavior to that of the real deal.

The reason for my writing is that the lasts section of my dissertation consisted of a more systematic analysis of BBRv3 and v1 compared to cubic, which produced some interesting plots that I thought were worth sharing here. These experiments are ran using mininet on a dumbbell topology under various network conditions.  Each datapoint shown in the following plots consisted of 5 experiments averaged across a wide parameter space, across 2850 experiments taking over 200 hours to run, all using the bbrv3 kernel.

Here are the experiments and the plots:

**** bbr is BBRv3 and bbr1 is BBRv1 as they are named in the output of "sysctl net.ipv4.tcp_available_congestion_control"

Experiment 1

This experiments surprisingly shows some performance degradation for BBRv3 in the higher BDP and RTT scenario

Experiment 2

This experiment showcases that the change of the "BBR_BW_PROBE_DOWN" pacing rate increase from 0.75 to 0.91 does improve fairness in higher BDP queue scenarios  as outlined here.

Experiment 3

This experiment shows that the loss response mechanism is successful. It also shows some performance degradation for BBRv3 in the higher BDP scenario.

Experiment 4

This experiment shows the loss management mechanism working well, however when non congestive loss is present, it sees heavy performance degradation.

Experiment 5

This experiment shows he improvement in the loss-based coexistence targeted at the lower RTT parameter space. I have also ran additional experiments with a version of BBRv3 with a 1 second and 7 second bbr_bw_probe_base_us (+ bbr_bw_probe_rand_us left unchanged at 1 second) probe interval. The seven second performs best across the parameter space, however it does come at a cost (shown in the next experiment).

Here is the goodput evolution of the 20ms data-point, averaged over 5 runs

Experiment 6

Here we can see that the convergence performance of 7 second BBRv3 suffers. BBRv3 1 second performs better but still suffers from the inopportune join time especially in the high BDP scenarios.

***** Because of the probe_bw timer resetting when doing a RTT probe, it would lead to BBR not probing bandwidth every 7 seconds, as the time to probe bandwidth MAY exceed the time to probe RTT, as such this reset was removed for all of the BBR versions tested to make it more fair.

Any feedback is appreciated. In a perfect world I would have liked to also test BBRv2 to see the generational improvement better, but due to time constraints it was not possible.

Thank you so much for reading this far, this protocol has been interesting to use and try to understand as it is cleverly designed, and it got me a good grade on my dissertation which i am happy about.  I am by no means an expert but I hope my input is somewhat informative. Keep up the good work on the protocol, I can't wait for it to be in the mainline kernel.

Best,

Mihai

[Mihai Mazilu]

Here are the figures in PDF format.