[Brian Tierney]

Hi BBR team and all:

We at ESnet have set up a 40G test host directly connected to our backbone network to explore bbr2 vs cubic for large science data transfers, which typically use parallel streams.

Some early results are available in this presentation:
https://docs.google.com/presentation/d/1ePT33Vs0SHI5nITRX29DVY3XdKeV6rH3XGllqY8ZenQ/edit?usp=sharing

In general I think these are all expected results, but we were surprised to see just how much bbr2 impacts cubic on high RRT paths when there is contention between the two. (see slide 8)

Do these results all make sense? Is this expected behavior?

Raw data used to generate these plots (as well as tcpdumps) are available at:
https://downloads.es.net/public/bbr2/

--

Brian Tierney, http://www.es.net/tierney
http://fasterdata.es.net

[Brian Tierney]

Sorry, with a subject heading this time. Doh.

[Tergel Munkhbat]

Dear Brian,

Thanks for sharing your data.

This is Tergel Munkhbat from Korea. A similar test has been also conducted over KREONET - national research and education network. The test consisted of around 30 PerfSonar nodes which are located over the KREONET domestic as well as international REN. Basically, most of our test results are very similar to your results.  However, in some high packet-loss and high RTT paths, we were surprised that BBR was approximately 60 times higher performance than HTCP or Cubic.

Sincerely,

Tergel

--
You received this message because you are subscribed to the Google Groups "BBR Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bbr-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bbr-dev/CADbZS-DmmUO16cRF2xWXDVj9QhM8k%2Btv69q%2BxyqpvPr-chOXcw%40mail.gmail.com.

[Neal Cardwell]

Hi Brian,

Many thanks for all these interesting and useful results!

I agree that in general these are all expected results. Regarding the impact of bbr2 on CUBIC on high-RTT paths when there is contention between the two (see slide 8): the behavior in your tests is expected given the current bbr2 design and design parameters.

The basic design approach that bbr2 takes with respect to sharing with loss-based algorithms like CUBIC and Reno is described briefly in our IETF 102 talk [slides] [video].

One big challenge in coexisting with CUBIC/Reno is that these algorithms (especially Reno but even CUBIC) need very long periods without packet loss in order to reach high speeds with long RTTs. By my quick back-of-the-envelope calculations, CUBIC would need regular runs of 21 seconds without packet loss to maintain a 1.6Gbit/sec fair share with a 100ms RTT (similar to your WAN tests in slide 8). While this is achievable in controlled testing on networks that are lightly-utilized or have low degrees of multiplexing, our experience is that on networks with a dynamic mix of short and long flows this very low level of packet loss is rarely achieved in practice, and trying to maintain it is overly constraining to congestion control design.

That said, we are always open to feedback and discussions on important issues like this. So thank you for the data and analysis and discussion!

best,

neal

--
You received this message because you are subscribed to the Google Groups "BBR Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bbr-dev+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bbr-dev/CADbZS-BYQh93PC4inCHWcZ3HATvRqz%3DuKG-xt50i6qYGsn9-4Q%40mail.gmail.com.