VMA UDP Flow Steering with lots of Multicast Groups

[Andrew Jameson]

Hi VMA experts,

I have a dual threaded application using VMA to receive UDP packets from multicast groups at a reasonably high data rate (27 Gb/s) with a ConnectX3 on a 40 Gb port.

When the incoming data are split into something like 16,32 or 64 multicast groups, the application experience good performance, dropping very few packets.

When the same incoming data are partitioned into a larger number of multicast groups (256 or 512) the performance drops substantially and the same application drops about 10% of the incoming UDP packets. 

Non VMA applications that are receiving the subscribing to same data stream on similar Mellanox NICs also experience issues with the larger number of MC groups. However, for these non-VMA applications, when changing the log_num_mgm_entry_size to -7 in the mlx4_core module, the issues appear to go away, and good capture performance is experienced.

If I try to apply this setting to my environment/application I receive a VMA error message at startup:

VMA WARNING: * VMA will not operate properly while flow steering option is disabled!                       *

VMA WARNING: * Please restart your VMA applications after running the following:                           *

VMA WARNING: * WARNING: the following steps will restart your network interface!                           *

VMA WARNING: * 1. "echo options mlx4_core log_num_mgm_entry_size=-1 > /etc/modprobe.d/mlx4_core.conf"    *

VMA WARNING: * 2. "/etc/init.d/openibd restart"                                                          *

VMA WARNING: * Read more about the Flow Steering support in the VMA's User Manual                          *

From what I have been able to read the "-7" argument means: "configure optimised steering mode to improve performance with the following limitation: VLAN filtering is not supported with this mode. This is the recommended mode in case VLAN filter is not needed."

So this still sounds like a device managed steering mode that I would like to try and use with my VMA application. Is there any way to enable this feature?

Or perhaps there may be other ways to resolve this issue using VMA environment variables?

Any advice greatly appreciated!

Andrew