help testing bbrv1 between two aws ubuntu 20 servers

[marcello gambetti]

good afternoon,
i have read and tried between two aws istances (europe and california) this ubuntu bbr settings, using iperf, but the measurement with iperf is still the same....does anyone worked with bbr before can help me?

(https://www.cyberciti.biz/cloud-computing/increase-your-linux-server-internet-speed-with-tcp-bbr-congestion-control/)

thank you

[Neal Cardwell]

Hi,

Thanks for the report! 

One possibility is that CUBIC is probably already able to fully utilize the path you are testing, probably because that path has a relatively low loss rate. In such cases, BBR will not be able to achieve higher throughput than CUBIC, because CUBIC is already fully utilizing the path.

Another possibility is that your traffic is constrained by the send buffer or receive window, in which case the throughput will be limited to roughly:

  throughput <= min(receive_window, send_buffer) / RTT

Can you please share more information:

+ What is the throughput you are achieving with CUBIC? BBR?

+ What is the round-trip time between the sender and receiver machine (for example, as reported by ping)?

+ What are the send and receive memory settings on your machine? Can you share the output of:

    sysctl net.ipv4.tcp_rmem  net.ipv4.tcp_wmem

+ To get a sense of the loss rate, you could use nstat to print SNMP counters, with something like:

   nstat > /dev/null  # reset counters

   command_to_run_iperf_test # run your test

   nstat # print counter value changes since the start of the test

thanks,

neal

--
You received this message because you are subscribed to the Google Groups "BBR Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bbr-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bbr-dev/8a7345e4-fc8d-44e2-81f5-df5b52d54c67n%40googlegroups.com.

[marcello gambetti]

good afternoon,

 done some more tests, and with the same settings, i mean same servers (srv-A, with reno, and srv-B, with bbr), same os (ubuntu 20 latest update), 10 ms lat, 2,5% packet loss, 100mbps (done inside a lab not real internet, using apfsense vm as "wan lantecy and pkt loss").

now if i use wget to download say 2GB iso from both servers, the bbr enable server reach almost 90mbps download bandwidth, very impressive, the ohter server with reno-cubic stops at 30mbps....BUT if i use iperf, the numebers are almost equal, 30mbps for BOTH the servers....am i missing somenthing about iperf vs wget dowload ?

thank you

[Neal Cardwell]

You may want to double-check that the iperf transfer is happening in the direction you expect. Those symptoms would be expected if you changed the congestion control algorithm on the machine that ended up being the receiver rather than the sender. You can run "ss -tinm" on both sides during the transfer to see which side is actually sending bulk data and what congestion control algorithm it is using.

neal

To view this discussion on the web visit https://groups.google.com/d/msgid/bbr-dev/ac26c01d-c7c2-4afc-b12a-dcd1f288c99dn%40googlegroups.com.

[Bob McMahon]

Can you provide the iperf output including the -e option? If you can sync the clocks, try --trip-times too.  

Also, there are lots of knobs to play with. Man page is here. Latest source here.

Bob

To view this discussion on the web visit https://groups.google.com/d/msgid/bbr-dev/ac26c01d-c7c2-4afc-b12a-dcd1f288c99dn%40googlegroups.com.

This electronic communication and the information and any files transmitted with it, or attached to it, are confidential and are intended solely for the use of the individual or entity to whom it is addressed and may contain information that is confidential, legally privileged, protected by privacy laws, or otherwise restricted from disclosure to anyone else. If you are not the intended recipient or the person responsible for delivering the e-mail to the intended recipient, you are hereby notified that any use, copying, distributing, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you received this e-mail in error, please return the e-mail to the sender, delete it from your computer, and destroy any printed copy of it.