"Application-limited" and flow control

[matt.a...@gmail.com]

This has surely been discussed before and I'm sorry for rehashing, but I can't find any mention in the ID or in this group's archive.

BBR's definition of an application-limited episode requires the send buffer to have less than an MSS of data, which means situations where the PEER app is limiting the throughput (via flow control) are excluded. But intuitively it seems to me that "peer-app-limited" situations are the same as "local-app-limited" with respect to underestimation of available capacity. So why are they excluded?

[Neal Cardwell]

Hi Matt,

Great question! :-)

I agree it would be intuitive for "peer-app-limited" situations to be considered the same as "local-app-limited". In fact, when we were developing BBR we initially did that: we treated "peer-app-limited" situations as "application-limited". However, this did not work well in practice. What we found was that a significant proportion of TCP receivers on the public Internet spent nearly the entire connection lifetime in a "peer-app-limited" (aka receive-window-limited) state, probably because of low static receive buffer limits or disabled/non-existent receive buffer autotuning. This caused several problems for these flows:

(a) The flows never exited Startup, because there were never enough non-application-limited bandwidth samples to have confidence that the connection had discovered the available bandwidth. And as a consequence, the flow continued to pace with a high pacing_gain and keep a large cwnd with a BDP or more of data in the bottleneck queue, perpetuating the condition of being receive-window-limited rather than cwnd-limited.

(b) The flows never reduced their estimated available bandwidth, again because there were never non-application-limited bandwidth samples.

So we decided to have BBR treat cases where the delivery rate was constrained by "peer-app-limited" conditions the same as BBR treats cases where the delivery rate is constrained by in-network bottlenecks; basically, to treat receiver bottlenecks the same as network bottlenecks. This seems to have a nice conceptual symmetry and so far has seemed to work well enough in practice.

In the long run I think it would be good to get rid of this tricky "application-limited" tracking, but that will require a different framework. Something for future work. :-)

thanks,

neal

On Tue, May 31, 2022 at 3:18 PM matt.a...@gmail.com <matt.a...@gmail.com> wrote:

This has surely been discussed before and I'm sorry for rehashing, but I can't find any mention in the ID or in this group's archive.

BBR's definition of an application-limited episode requires the send buffer to have less than an MSS of data, which means situations where the PEER app is limiting the throughput (via flow control) are excluded. But intuitively it seems to me that "peer-app-limited" situations are the same as "local-app-limited" with respect to underestimation of available capacity. So why are they excluded?

--
You received this message because you are subscribed to the Google Groups "BBR Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bbr-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bbr-dev/94343f46-4e86-4f26-b840-d8dff60669a3n%40googlegroups.com.

[matt.a...@gmail.com]

Thanks, makes sense. But is this justification specific to BBR? The Rate Estimation draft is presented as a generic algorithm.

[Neal Cardwell]

Hi Matt,

I agree there are alternate approaches that could make sense, depending on the use case. 

To try to discuss this kind of consideration, I have proposed to add the following subsection to the "Tracking application-limited phases" section of the delivery rate Internet Draft:

Considerations Related to Receiver Flow Control Limits 

In some cases receiver flow control limits (such as the TCP [RFC793] advertised receive window, RCV.WND) are the factor limiting the delivery rate. This algorithm treats cases where the delivery rate was constrained by such conditions the same as it treats cases where the delivery rate is constrained by in-network bottlenecks. That is, it treats receiver bottlenecks the same as network bottlenecks. This has a conceptual symmetry and has worked well in practice for congestion control and telemetry purposes.

Please let us know what you think of this, or feel free to suggest alternate wording or approaches.

Thanks!
neal

To view this discussion on the web visit https://groups.google.com/d/msgid/bbr-dev/f3f732ca-dd95-49fe-a658-e03881062b2cn%40googlegroups.com.

[matt.a...@gmail.com]

I think that's reasonable.