Implementing BBR in fast recovery

[Beyers Cronje]

I'm busy implementing BBR in our experimental protocol, but hope someone can clarify something for me. Our current implementation is modeled on RFC 6675 (SACK based loss recovery) to detect and process fast recovery. The BBR draft does not specify *when* to enter fast recovery and *what* to send during fast recovery, it only specifies how to modulate cwnd during fast recovery and after existing fast recovery. So would it be reasonable to say that I should use the methods as specified in RFC 6675 to detect duplicate ACKs and when to enter fast recovery as well as which packets to send during fast recovery, but rely on BBR instead of RFC 6675 to modulate cwnd? How does the Linux implementation handle this?

[hiren panchasara]

Someone with more clue can chime-in. Based on my limited understanding:

On 09/07/17 at 06:09P, Beyers Cronje wrote:
> I'm busy implementing BBR in our experimental protocol, but hope someone
> can clarify something for me. Our current implementation is modeled on RFC 6675
> (SACK based loss recovery) to detect and process fast recovery. The BBR
> draft does not specify *when* to enter fast recovery and *what* to send
> during fast recovery, it only specifies how to modulate cwnd during fast
> recovery and after existing fast recovery.

That's how BBR is supposed to behave.
https://tools.ietf.org/html/draft-cardwell-iccrg-bbr-congestion-control-00
provides what BBR is supposed to do.

Detect loss: Traditional (N dupacks, fack, etc) OR RACK
Modulate cwnd and a lot of other magic: BBR
What to send during recovery: rfc6675 OR much simpler version based on RACK.

> So would it be reasonable to say
> that I should use the methods as specified in RFC 6675 to detect duplicate
> ACKs and when to enter fast recovery as well as which packets to send
> during fast recovery, but rely on BBR instead of RFC 6675 to modulate cwnd?

I believe this is correct if you are following non-RACK methods to
detect loss.

> How does the Linux implementation handle this?
>

Not totally sure how Linux does it but I believe *what to send* part
becomes much simpler (at least compared to 6675's way of doing that)
when you use RACK to detect loss. Because big part of rfc6675's approach
is the guess work of determining whether a packet is lost or not which
is much simpler with RACK.

One question I have is whether PRR (Proportional rate reduction) is
still needed with BBR or not. I'd think not but would like to confirm.

Cheers,
Hiren

[Neal Cardwell]

On Thu, Sep 7, 2017 at 8:08 PM, hiren panchasara <hi...@strugglingcoder.info> wrote:

Someone with more clue can chime-in. Based on my limited understanding:

On 09/07/17 at 06:09P, Beyers Cronje wrote:
> I'm busy implementing BBR in our experimental protocol, but hope someone
> can clarify something for me. Our current implementation is modeled on RFC 6675
> (SACK based loss recovery) to detect and process fast recovery. The BBR
> draft does not specify *when* to enter fast recovery and *what* to send
> during fast recovery, it only specifies how to modulate cwnd during fast
> recovery and after existing fast recovery.

That's how BBR is supposed to behave.
https://tools.ietf.org/html/draft-cardwell-iccrg-bbr-congestion-control-00
provides what BBR is supposed to do.

Detect loss: Traditional (N dupacks, fack, etc) OR RACK
Modulate cwnd and a lot of other magic: BBR
What to send during recovery: rfc6675 OR much simpler version based on RACK.

Yes, that's the basic idea. The BBR algorithm considers loss detection (what seems lost and thus what needs to be sent or resent) to be separate from congestion control (how fast to send it) and packet scheduling (exactly when to send it). Figure 1 in this netdev 1.2 paper that Yuchung and I wrote lays out how the BBR team thinks about decomposing these problems:

  https://netdevconf.org/1.2/papers/bbr-netdev-1.2.new.new.pdf

> So would it be reasonable to say
> that I should use the methods as specified in RFC 6675 to detect duplicate
> ACKs and when to enter fast recovery as well as which packets to send
> during fast recovery, but rely on BBR instead of RFC 6675 to modulate cwnd?

I believe this is correct if you are following non-RACK methods to
detect loss.

Yes. The loss recovery module (e.g. RACK) decides when to enter fast recovery, and which packets are probably lost, and which to send during fast recovery. The congestion control module (e.g. BBR) decides how fast to pace, what cwnd to use, and how big a quantum of bytes to pass down through the network stack when transmitting (whether in loss recovery or not).

For the loss recovery part, our team would really recommend that implementors use RACK:

   https://tools.ietf.org/html/draft-ietf-tcpm-rack 

> How does the Linux implementation handle this?
>

Not totally sure how Linux does it but I believe *what to send* part
becomes much simpler (at least compared to 6675's way of doing that)
when you use RACK to detect loss. Because big part of rfc6675's approach
is the guess work of determining whether a packet is lost or not which
is much simpler with RACK.

Currently Linux TCP uses a combination of RFC6675 and RACK, but our team's plan is to move more in the direction of RACK (e.g. RACK, possibly augmented by allowing N dupacks to trigger recovery, but with RFC6675 code removed).

 

One question I have is whether PRR (Proportional rate reduction) is
still needed with BBR or not. I'd think not but would like to confirm.

PRR, which is part of congestion control, is not needed with BBR congestion control. (Though BBR currently uses some ideas similar to PRR.)

neal

 

[Beyers Cronje]

Hi Neal and Hiren,

Thank you for the informative responses, it is much clearer now. Also thank you for pointing me to RACK, wasn't aware of it and does indeed look much simpler to implement than RFC 6675. I'm busy decoupling our loss detection and congestion control, so this should set us up for easier integration with BBR.

Much appreciated.

Beyers