BBRv3 and long-term bandwidth estimates (policer detection)

[Erfan Sharafzadeh]

Hi all,

We have been noticing that BBRv1 offers pretty high single-flow throughput with a policer regardless of the policer bucket size (as long as it is not too small), but that's not the case for BBRv3.

I know that there is no explicit policer detection in BBRv3, there is lot of other stuff going on in there like trying to maintain <2% loss in general, so my questions are:

1. What prevents BBRv3 to implement a similar long-term BW estimate?

2. Are there any side-effects to BBRv1's policer detection? Are there any scenarios in which the long-term estimate won't work properly?

Thanks in advance!

Erfan

[Neal Cardwell]

On Mon, Feb 17, 2025 at 3:48 PM Erfan Sharafzadeh <e.shar...@gmail.com> wrote:

Hi all,

We have been noticing that BBRv1 offers pretty high single-flow throughput with a policer regardless of the policer bucket size (as long as it is not too small), but that's not the case for BBRv3.

Interesting. We have not encountered that. Do you have performance summaries or packet traces you can share to characterize what you are seeing with BBRv3 and policers?

 

I know that there is no explicit policer detection in BBRv3, there is lot of other stuff going on in there like trying to maintain <2% loss in general, so my questions are:

1. What prevents BBRv3 to implement a similar long-term BW estimate?

In theory, nothing prevents BBRv3 from having a long-term BW estimate (aka policer detection) that is similar to that mechanism in BBRv1. The motivations for not including a long-term BW estimate in BBRv3 are mainly: simplicity, maintainability, and generality. Notably, since BBR v2 and v3 already have a general mechanism to adapt the cwnd and pacing rate based on loss signals, it seemed best to use that general mechanism, rather than adding complexity and state to incorporate the more specific long-term BW estimate that is tuned for detecting a single connection going through a fixed-rate policer.

 

2. Are there any side-effects to BBRv1's policer detection? Are there any scenarios in which the long-term estimate won't work properly?

One side-effect is that if the policer detection triggers spuriously, then the flow could be "locked in" to an unnecessarily low throughput for bbr_lt_bw_max_rtts = 48 round trips.

One scenario in which the long-term estimate won't work properly is if multiple connections are sharing a policed bottleneck. In that case, typically the per-connection bandwidth would not be uniform enough to trigger the use of the long-term estimate.

best,

neal

[Erfan Sharafzadeh]

Thanks for all the insights!

Do you have performance summaries or packet traces you can share to characterize what you are seeing with BBRv3 and policers?

We only did a bunch of experiments with a single flow through a single policer. We are planning to go back and make sure everything is done correctly, and I'll update you!

 

In theory, nothing prevents BBRv3 from having a long-term BW estimate (aka policer detection) that is similar to that mechanism in BBRv1. The motivations for not including a long-term BW estimate in BBRv3 are mainly: simplicity, maintainability, and generality. Notably, since BBR v2 and v3 already have a general mechanism to adapt the cwnd and pacing rate based on loss signals, it seemed best to use that general mechanism, rather than adding complexity and state to incorporate the more specific long-term BW estimate that is tuned for detecting a single connection going through a fixed-rate policer.

Thanks a lot, that's insightful information about design choice! I agree that simplicity is good design. 

 

One scenario in which the long-term estimate won't work properly is if multiple connections are sharing a policed bottleneck. In that case, typically the per-connection bandwidth would not be uniform enough to trigger the use of the long-term estimate.

That makes sense! I'm gonna try this and see what happens. 

Cheers

Erfan