Strange bridge behaviour

[Rudolph Bott]

Hey List,

we have recently built a six-node-cluster (Ganeti 3 with KVM/DRBD on Debian Bullseye) which hosts around 80-90 instances per node. We did run into some multicast issues (which do not exist on clusters of the same configuration but with ~30-40 instances per node).

Sometimes guests would see all multicast traffic flooded, sometimes they would see only relevant multicast traffic (e.g. due to the mcast groups they joined) but common to all would be intermittent drops of multicast traffic.

Network-wise we are using vlan aware bridges (as in the other clusters) and the network equipment/configuration is also similar to other environments we operate (the switches do not do any multicast/igmp snooping and simply flood multicast traffic). We do not have high levels of multicast traffic, mainly some corosync/keycloak stuff.

The workaround/fix was to disable multicast snooping on the bridges and have them flood the traffic to all instances. This works, but is/was not necessary on other clusters:

echo 0 > /sys/devices/virtual/net/$BRIDGE/bridge/multicast_snooping

Hence our question:

Do you have any experience using linux bridges with "many" ports? I've read that there is a hardcoded limit of 1024 ports on a bridge, but we are not anywhere near that. We do not see any other traffic drops/issues, just multicast-related.

Cheers,

Rudi

--

 Rudolph Bott - bo...@sipgate.de

 Telefon: +49 (0)211-63 55 55-55

 Telefax: +49 (0)211-63 55 55-22

 sipgate GmbH - Gladbacher Str. 74 - 40219 Düsseldorf

 HRB Düsseldorf 39841 - Geschäftsführer: Thilo Salmon, Tim Mois

 Steuernummer: 106/5724/7147, Umsatzsteuer-ID: DE219349391

 www.sipgate.de - www.sipgate.co.uk