Requesting help for running the v2alpha kernel on a vmware workstation VM

[Anup Agarwal]

Hi, 

Summary of issue:

I am trying to run BBRv2 using the custom kernel provided at https://github.com/google/bbr/blob/v2alpha. I am using a VM on vmware workstation running Ubuntu 22.04. The VM boots fine with its default kernel, i.e,  5.19.0-46-generic. When I try to boot with the v2alpha kernel, I get the error: 

"Gave up waiting for root filesystem device" (see screenshots attached).

I tried building the kernel through multiple methods and configs described below.

Any help in resolving this would be appreciated.

Steps:

1. I first tried the script https://github.com/google/bbr/blob/v2alpha/gce-install.sh directly. I only used the script to do the compilation, i.e., I commented out the steps post tar creation. I ran the modified script on the VM itself.
   1. Based on my reading of the gce-install script, I manually copied the boot/*GCE and lib/modules/*GCE files to /boot/ and /lib/modules/ respectively
   2. I ran sudo mkinitramfs -k -o initrd.img-5.13.12+v2alpha+a23c4bb59e0c+GCE 5.13.12+v2alpha+a23c4bb59e0c+GCE
   3. I did sudo update-grub and then rebooted.
   4. I get the error shown in attached screenshot: GCE-config.JPG
   5. Note, I updated /etc/grub/default to set timeout as 5 seconds (without that it won't show me option to select the newly built kernel).
2. I came across https://groups.google.com/g/bbr-dev/c/5BkIRx9k4Ns/m/Sg98jVh1AAAJ and https://groups.google.com/g/bbr-dev/c/DUoz7nGvzz0/m/njL0w_8NAwAJ, and realized that the GCE config may not work for the VM. I had two options, follow https://wiki.gentoo.org/wiki/VMware to update config or copy the working kernel config. I copied the working kernel config (I can try following the other approach, but I don't know why that would work if the copying did not work).
   1. On a fresh clone, I did cp /boot/config-$(uname -r) .config
   2. make olddefconfig
   3. Updated CONFIG_TCP_CONG_{CUBIC, DCTCP, BBR, BBR2} as y.
   4. make $MAKE_OPTS bindeb-pkg. 
      1. I set the MAKE_OPTS similar to the gce_install.sh script.
      2. In the MAKE_OPTS, I had to remove the -GCE suffix as the make process gave error that character G is not allowed).
      3. I had to run following to get around signature/cert errors
         1. scripts/config --disable SYSTEM_TRUSTED_KEYS scripts/config --disable SYSTEM_REVOCATION_KEYS
            
      4. I faced pahole not found, so I installed dwarves (https://stackoverflow.com/questions/61657707/btf-tmp-vmlinux-btf-pahole-pahole-is-not-available/61758176#61758176)
      5. I also faced an issue saying Unable to resolve "bbr_min_tso_segs" in the BTF generation (this may actually be a linking issue with the v2alpha repo). I got around this by setting  CONFIG_DEBUG_INFO_BTF=n  (https://stackoverflow.com/a/71977338/5039326)
   5. This produced .deb files in ../
   6. I did sudo apt install ./*.deb and rebooted.
   7. I selected the new kernel and got error message in screenshot: Upstream-config.JPG

Please let me know if I should provide any other information.

Any help would be appreciated.

Thanks in advance,

Anup

 

[Neal Cardwell]

Hi Anup,

Thanks for the post!

What happens if you try something like the following after the "sudo apt install ./*.deb"?

cd /boot

for v in $(ls vmlinuz-* | sed s/vmlinuz-//g); do
   sudo mkinitramfs -k -o initrd.img-${v} ${v}
done
sudo update-grub
sudo reboot

Does that help?

cheers,

neal

--
You received this message because you are subscribed to the Google Groups "BBR Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bbr-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bbr-dev/6d340c4f-ba20-4a28-a637-762475f1130an%40googlegroups.com.

[Anup Agarwal]

Hi Neal,

Thanks for the prompt response.

I tried running the commands you suggested, I get the same error on boot. I got some warnings when I ran the mkinitramfs command (attached in txt file).

I am considering trying the following things:

1. I am not super sure if external kernel modules get packaged into one of the .deb files (though I selected the CC modules to be built inside the kernel, i.e, =y instead of =m). I will try using the gce_install.sh steps for compiling with the working 5.19 kernel config (instead of the make bindeb-pkg).
   
2. When I power off after failed boot, I get a "install VMware tools notification". Maybe there are some params I need to set in the config for vmware tools that is not covered in the 5.19 config. I can look into this deeper. Typically vmware tools are installed after OS install on a fresh VM.
3. I mount a host directory to the VM, I can try removing this as this feature uses vmware tools modules.

I will update here if any of these steps help. Please let me know if I should try something else.

Thanks,

Anup

[Anup Agarwal]

Follwing the gce_install.sh steps for compilation with the v5.19 kernel default ubuntu config (+ enabling BBR2 DCTCP etc. params in the config) worked.

I am able to boot in into the custom kernel and bbr2 shows up in sysctl net.ipv4.tcp_available_congestion_control.

 

I don't know why the bindeb-pkg compilation flow did not work.

Anyway, thanks for all the help!

- Anup