I have questions about the IETF 117.

[ssw]

Hello everyone,

First, I really appreciate you taking the time to read this question. My question is about bug fix #1, early stopping of probing in IETF 117 slides(link here).

Q_1) What does "early stop" mean? Does it refer to stopping before reaching fairness, stopping before the true 25% queuing, or something else?

Q_2) I wonder how inflight_hi is related to this issue. I agree that inflight_hi could limit the growth of bandwidth, but I don’t understand how inflight_hi causes the stopping of probing. Could you explain it?

Q_3) This question is not related to the questions above. In the early version of BBR v1, I observed that when other flows disappear, probing could persist for more than one RTT. I wonder if that situation was intended. If it is, the exit condition (inflight > bbr_inflight * 1.25) is more appropriate than (inflight ≥ bbr_inflight * 1.25) in the code of BBR v1?

Thank you.

Best regards,
Seoungwon

[Bless, Roland (TM)]

Hi,

On 29.07.24 at 14:10 ssw wrote:
> First, I really appreciate you taking the time to read this question. My
> question is about bug fix #1, early stopping of probing in IETF 117

> slides(link here <https://datatracker.ietf.org/meeting/117/materials/
> slides-117-ccwg-bbrv3-algorithm-bug-fixes-and-public-internet-
> deployment-00#page=4>).
>
> Q_1) _What does "early stop" mean?_ Does it refer to stopping before

> reaching fairness, stopping before the true 25% queuing, or something else?

It probably means that the flow should have increased its share some
more, so probing (ProbeBW:Up) is exited "too early" and yes this is
related to fairness. Especially, there was a latecomer disadvantage
for flows that start later, see below.

> Q_2) _I wonder how inflight_hi is related to this issue_. I agree that

> inflight_hi could limit the growth of bandwidth, but I don’t understand
> how inflight_hi causes the stopping of probing. Could you explain it?

Inflight_hi is usually set due to packet loss in the startup phase,
but rarely set to a different value afterwards.
Slide 25 from the presentation @IETF114 explains this:
inflight_target is inflight_hi + inflight_probe and
inflight_probe grows exponentially per round.
However, due to termination of ProbeBW:UP on
inflight > 1.25*estimated_bdp
a newly started flow cannot increase its inflight data
significantly enough to gain a larger share, because
it hits exactly this condition quite often.
Note that estimated_bdp corresponds to the flows
current estimated bw_share * current estimated RTT_min.
if you have bw shares a and b for flows a and b, then
their shares are a/(a+b) and b/(a+b). Now if flow a
probes first, its share would be 1.25a/(1.25a+b) in the best case
and if flow b probes afterwards, it starts from b/(1.25a+b)
and will end up at b/(a+b). So the situation does not change.

> Q_3) This question is not related to the questions above. In the early
> version of BBR v1, I observed that when other flows disappear, probing
> could persist for more than one RTT. I wonder if that situation was
> intended. If it is, the exit condition (inflight > bbr_inflight * 1.25)
> is more appropriate than (inflight ≥ bbr_inflight * 1.25) in the code of
> BBR v1?

Yes, see also slide 25: inflight_probe grows exponentially per round,
so theoretically multiple rounds are required to let the flow.

Regards,
Roland

[Neal Cardwell]

To expand a little on Roland's nice answers:

On Mon, Jul 29, 2024 at 8:10 AM ssw <ssw3...@gmail.com> wrote:

Hello everyone,

First, I really appreciate you taking the time to read this question. My question is about bug fix #1, early stopping of probing in IETF 117 slides(link here).

Q_1) What does "early stop" mean? Does it refer to stopping before reaching fairness, stopping before the true 25% queuing, or something else?

This refers to "stopping before reaching fairness".

 

Q_2) I wonder how inflight_hi is related to this issue. I agree that inflight_hi could limit the growth of bandwidth, but I don’t understand how inflight_hi causes the stopping of probing. Could you explain it?

In BBR, bandwidth probing is limited to sending at a pacing rate that is a multiple of the current estimated bandwidth. So if the estimated bandwidth gets "stuck", the bandwidth probing gets "stuck." With the old BBRv2-era logic, inflight_hi limited cwnd, and cwnd limited the measured bandwidth, and that limited the estimated bandwidth, and that limited the sending rate when probing for bandwidth. Thus the estimated bandwidth could get "stuck" at a rate that was below the fair share.

Q_3) This question is not related to the questions above. In the early version of BBR v1, I observed that when other flows disappear, probing could persist for more than one RTT. I wonder if that situation was intended. If it is, the exit condition (inflight > bbr_inflight * 1.25) is more appropriate than (inflight ≥ bbr_inflight * 1.25) in the code of BBR v1?

I would not necessarily say that it's an "intended" behavior in BBRv1. However, it is usually a reasonable thing to happen. Your suggested change in exit condition is not unreasonable, but I don't think it would be a reliable way to achieve that persistent probing behavior. That's because bandwidth (delivery rate) samples lag one round trip behind the in-flight data levels that cause them. So at the time that the flow noticed the (inflight > bbr_inflight * 1.25) condition, typically the bandwidth (delivery rate) samples would not have risen, and so typically the flow would not have entered the positive feedback loop that can cause multiple consecutive rounds of bandwidth probing in BBRv1.

A flow has to allow at least two full rounds of bandwidth probing (pacing at k*estimate_bw, where k > 1) to reliably get the larger bandwidth (delivery rate) samples that allow the flow to enter the positive feedback loop. So that's why the fix for BBRv3 is structured to allow multiple rounds of bandwidth probing.

 

Best regards,

neal

Thank you.

Best regards,
Seoungwon

--
You received this message because you are subscribed to the Google Groups "BBR Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bbr-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bbr-dev/1ae89371-afac-4360-be38-7d6dc4627ae4n%40googlegroups.com.

[ssw]

Roland and Neal, THANKS a lot!!

2024년 7월 30일 화요일 오전 12시 51분 30초 UTC+9에 Neal Cardwell님이 작성: