Q on wscale

[Ravi Kerur]

Hello,

I am seeing tests fail with following errors

FAIL [/home/rkerur/packetdrill-unmod/packetdrill/gtests/net/tcp/notsent_lowat/notsent-lowat-default.pkt (ipv6)]
stdout:
stderr:
notsent-lowat-default.pkt:15: error handling packet: bad value outbound TCP option 3
script packet:  0.000078 S. 0:0(0) ack 1 <mss 1460,nop,nop,sackOK,nop,wscale 8>
actual packet:  0.000075 S. 0:0(0) ack 1 win 65535 <mss 1460,nop,nop,sackOK,nop,wscale 14>

As I understand wscale is derived from tcp_rmem, rmem_max. No matter what values I change tcp_rmem and rmem_max, tests always fail with above errors.

Intially after system boot values were

sudo sysctl -a | grep rmem
net.core.rmem_default = 212992
net.core.rmem_max = 536870912
net.ipv4.tcp_rmem = 4096 540000 15728640
net.ipv4.udp_rmem_min = 4096

Later I changed values to

sudo sysctl -a | grep rmem
net.core.rmem_default = 212992
net.core.rmem_max = 536870912
net.ipv4.tcp_rmem = 4096 262144 4194304
net.ipv4.udp_rmem_min = 4096

Both cases tests failed with above errors.

How do I change the expected wscale value during tests?

Thanks.

[Neal Cardwell]

Hi,

THanks for the report.

As you noted, the wscale option is a function of tcp_rmem. 

Sounds like you are talking about the test: 

  gtests/net/tcp/notsent_lowat/notsent-lowat-default.pkt 

at:

  https://github.com/google/packetdrill/blob/master/gtests/net/tcp/notsent_lowat/notsent-lowat-default.pkt

Please note that most of the tests we release follow the convention of initializing the machine state by running the bash script at  ../../common/defaults.sh (or whatever relative path is needed to reach that script):

  https://github.com/google/packetdrill/blob/master/gtests/net/common/defaults.sh

That script does its own tuning of tcp_rmem, which is likely overriding any manual tuning you are doing.

You might try tweaking the defaults.sh script.

If you still have problems after that, can you please share the exact version of the kernel you are testing?

thanks,

neal

--
You received this message because you are subscribed to the Google Groups "packetdrill" group.
To unsubscribe from this group and stop receiving emails from it, send an email to packetdrill...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packetdrill/3183e774-cff9-485e-9019-689120b4821bn%40googlegroups.com.

[Ravi Kerur]

Thanks for your help. 

Kernel version

Linux rkerur-ztest 6.1.29 #25 SMP Tue Aug  1 22:42:27 PDT 2023 x86_64 x86_64 x86_64 GNU/Linux

I have been making changes to default.sh in my tests as you have suggested and it doesn't make any difference. Irrespective of the values, tests always fails with the same error message *bad value outbound TCP option 3*

git diff common/defaults.sh
diff --git a/gtests/net/common/defaults.sh b/gtests/net/common/defaults.sh
index 8e49555..974ab74 100755
--- a/gtests/net/common/defaults.sh
+++ b/gtests/net/common/defaults.sh
@@ -5,8 +5,10 @@
 ip tcp_metrics flush all > /dev/null 2>&1
 
 # TCP min, default, and max receive and send buffer sizes.
-sysctl -q net.ipv4.tcp_rmem="4096 540000 $((15*1024*1024))"
-sysctl -q net.ipv4.tcp_wmem="4096 $((256*1024)) 4194304"
+sysctl -q net.ipv4.tcp_rmem="65536 1048576 536870912"
+sysctl -q net.ipv4.tcp_wmem="65536 1048576 26214400"
 
 # TCP timestamps.
 sysctl -q net.ipv4.tcp_timestamps=1

[David Morley]

Hi,

Just following up, not sure if you have done this already, but can you also try adding wmem_max and rmem_max to common/defaults.sh before those modified lines? I've ran into a similar issue in the past where that was due to tcp_wmem and tcp_rmem truncating when tcp_wmem_max and tcp_rmem_max were too small (not sure if this is the issue you're having here, but thought I'd chime in just in case). It'd also be great to validate after running a packetdrill test that you are seeing the values for those sysctls you expect. 

Best,

David

To view this discussion on the web visit https://groups.google.com/d/msgid/packetdrill/458ae371-fb61-4a74-8a4e-50069098426en%40googlegroups.com.

[Neal Cardwell]

Along the lines David mentions, I think the issue is that in the Linux kernel net.core.rmem_max was recently raised to a much larger value (the net.core.rmem_max = 536870912 value that you mention) than the tests expect, which causes tcp_select_initial_window() to pick the maximum wscale value of 14 rather than the previous value of 8.

I was able to reproduce your problem on a 6.4-ish kernel and get the tests to pass by running the following before the run_all.py invocation:

  sysctl -q net.core.rmem_max="$((8 * 1024 * 1024))"

That allowed all the tcp/blocking/ tests to pass when running:

    ./packetdrill/run_all.py -S -v -L -l tcp/blocking/

=>

# ./packetdrill/run_all.py -S  -L -l tcp/blocking/
Ran   12 tests:   12 passing,    0 failing,    0 timed out (11.22 sec): tcp/blocking/

We will have to work on some changes to the test scripts to update them to pass with the recent kernel changes.

thanks,

neal

To view this discussion on the web visit https://groups.google.com/d/msgid/packetdrill/CAFXTrzf2UMZyBTKs53aXk8aFgPkT9HfL2maSurGXJa3-d%3D-PDw%40mail.gmail.com.

[Ravi Kerur]

Thanks David and Neal.

I overlooked net.core.rmem_max, instead only made changes to tcp_rmem and trying to get  wscale to work. After making changes to net.core.rmem_max I am able to get the tests pass.

Thanks,

Ravi