DPDK not loading with ConnectX-6

Lior Asher

unread,

Apr 14, 2021, 7:26:04 AM4/14/21

to TRex Traffic Generator

Hi Hanoch,

I have two ConnectX-6 adapters with latest driver 5.3-0.0.0.1 installed (with fw from bundle 20.30.1004)

We run on Ubuntu 16.04

We are running TRex server (latest v2.89) with:

sudo ./t-rex-64 --stl -i --software --mbuf-factor 0.2 --iom 1

Although TRex successfully starts, running ifconfig we see MLNX ethernet interfaces still appear in ifconfig output. We expect is to disappear from ifconfig output.

Running dpdk_nic_bind status shows:

sudo ./dpdk_nic_bind.py -s

Network devices using DPDK-compatible driver

============================================

0000:04:00.0 'MT28908 Family [ConnectX-6]' drv=mlx5_core unused=

0000:04:00.1 'MT28908 Family [ConnectX-6]' drv=mlx5_core unused=

0000:09:00.0 'MT28908 Family [ConnectX-6]' drv=mlx5_core unused=

0000:09:00.1 'MT28908 Family [ConnectX-6]' drv=mlx5_core unused=

Network devices using kernel driver

===================================

0000:04:00.0 'MT28908 Family [ConnectX-6]' if=enp4s0f0 drv=mlx5_core unused=

0000:04:00.1 'MT28908 Family [ConnectX-6]' if=enp4s0f1 drv=mlx5_core unused=

0000:05:00.0 'I211 Gigabit Network Connection' if=enp5s0 drv=igb unused= *Active*

0000:09:00.0 'MT28908 Family [ConnectX-6]' if=enp9s0f0 drv=mlx5_core unused=

0000:09:00.1 'MT28908 Family [ConnectX-6]' if=enp9s0f1 drv=mlx5_core unused=

Other network devices

=====================

<none>

How can we make sure DPDK to take over the interfaces?

If I leave TRex up and running (untouched), I see notice a few unexpected packets exiting the NIC ports from time to time. I believe this is probably done by the OS as the interfaces are still discoverable to the OS.

How can we resolve this?

Thanks,

Lior.

Lior Asher

unread,

Apr 14, 2021, 7:35:40 AM4/14/21

to TRex Traffic Generator

Typo in driver version. We have installed: 5.3-1.0.0.1

ב-יום רביעי, 14 באפריל 2021 בשעה 14:26:04 UTC+3, Lior Asher כתב/ה:

hanoh haim

unread,

Apr 18, 2021, 2:29:10 AM4/18/21

to Lior Asher, TRex Traffic Generator

Hi Lior,

Mellanox driver works in a different way than the other DPDK driver, try to read about that.

It is a hybrid mode that only the software queue is there. You should see the kernel driver in case of mlx5.

This is the reason for the dependency hell that requires a specific kernel driver and specific ipv library (OFED versio