TRex Server dpdk port binding issue with vmxnet3_v4_rss_configure(): Set RSS fields (v4) failed: 1

[balaje...@gmail.com]

Hi,
I'm using TRex v2.77 and trying to generate UPD traffic by starting the TRex server with "sudo ./t-rex-64 -i". But I end up with DPDK messages:

sudo ./t-rex-64 -i
Starting Scapy server..... Scapy server is started
Trying to bind to vfio-pci ...
Trying to compile and bind to igb_uio ...
ERROR: We don't have precompiled igb_uio.ko module for your kernel version
Will try compiling automatically...
Success.

/usr/bin/python3 dpdk_nic_bind.py --bind=igb_uio 0000:13:00.0 0000:14:00.0
The ports are bound/configured.
Starting TRex v2.77 please wait ...
set driver name net_vmxnet3
driver capability : TSO
set dpdk queues mode to ONE_QUE
Number of ports found: 2
zmq publisher at: tcp://*:4500
vmxnet3_v4_rss_configure(): Set RSS fields (v4) failed: 1
vmxnet3_dev_start(): Failed to configure v4 RSS
vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
EAL: Error - exiting with code: 1
Cause: rte_eth_dev_start: err=-22, port=0
Killing Scapy server... Scapy server is killed

I hope that the interfaces are correctly configured.

Network devices using DPDK-compatible driver
============================================
0000:13:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused=vmxnet3,vfio-pci,uio_pci_generic
0000:14:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused=vmxnet3,vfio-pci,uio_pci_generic

Network devices using kernel driver
===================================
0000:03:00.0 'VMXNET3 Ethernet Controller' if=ens160 drv=vmxnet3 unused=igb_uio,vfio-pci,uio_pci_generic *Active*
0000:0b:00.0 'VMXNET3 Ethernet Controller' if=ens192 drv=vmxnet3 unused=igb_uio,vfio-pci,uio_pci_generic
0000:0d:00.0 'VMXNET3 Ethernet Controller' if=ens194 drv=vmxnet3 unused=igb_uio,vfio-pci,uio_pci_generic

Other network devices
=====================
<none>

Config:

- port_limit : 2
version : 2
interfaces: ['13:00.0', '14:00.0']
port_info:
- ip: 10.10.11.3
default_gw: 10.10.11.4
- ip: 10.10.11.4
default_gw: 10.10.11.3

TRex VM is located on the VMware cluster. Has anyone faced the same issue?
Thanks for any hint

-Jegan

[hanoh haim]

Hi 

V2.80 has a newer DPDK driver version. Worth checking it out

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/9cf906fd-a7c3-4564-be34-007cf9a29cf0%40googlegroups.com.

--

Hanoh

Sent from my iPhone

[balaje...@gmail.com]

Hi Hanoch,

Thanks for your reply.

I'm still getting the same error with v2.80.

jen@prod-trex:/opt/trex/v2.80$ sudo ./t-rex-64 -i

Starting Scapy server..... Scapy server is started

Trying to bind to igb_uio ...

/usr/bin/python3 dpdk_nic_bind.py --bind=igb_uio 0000:13:00.0 0000:14:00.0
The ports are bound/configured.

Starting TRex v2.80 please wait ...

set driver name net_vmxnet3
driver capability : TSO
set dpdk queues mode to ONE_QUE
Number of ports found: 2
zmq publisher at: tcp://*:4500
vmxnet3_v4_rss_configure(): Set RSS fields (v4) failed: 1
vmxnet3_dev_start(): Failed to configure v4 RSS
vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
EAL: Error - exiting with code: 1
Cause: rte_eth_dev_start: err=-22, port=0
Killing Scapy server... Scapy server is killed

I have two VMWare Host machines. One machine located locally and another Machine located in a remote location. In the local machine, I tested successfully with v2.77. But on the other machine, it is not working. Both machines are using same nic cards.

Thanks
Jegan

[hanoh haim]

looking into the code, it seems you have a version 4.0 of the VXNET and it suppose to support RSS but return error in this case.

if (VMXNET3_VERSION_GE_4(hw)) {

/* Check for additional RSS */

ret = vmxnet3_v4_rss_configure(dev);

if (ret != VMXNET3_SUCCESS) {

PMD_INIT_LOG(ERR, "Failed to configure v4 RSS");

return ret;

}

}

I would just patch and remove this code .

Thanks

Hanoh

[balaje...@gmail.com]

Hi Hanoch,

Thanks for your reply.

Where can I find the file to comment on the code block?

Regards,
Jegan

[besi7...@gmail.com]

Hi,

The code is located in the following code file:

trex-core/src/dpdk/drivers/net/vmxnet3/vmxnet3_ethdev.c
Thanks, 

Bes

[balaje...@gmail.com]

Hi Hanoch,

Both VMs are configured with the same vmxnet3 driver version (1.4.14.0-k-NAPI):

# Working environment (sanprod-trex)

root@sanprod-trex:~# ethtool -i ens160
driver: vmxnet3
version: 1.4.14.0-k-NAPI
firmware-version:
expansion-rom-version:
bus-info: 0000:03:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no

# Non-Working environment (prod-trex)

root@prod-trex:~# ethtool -i ens160
driver: vmxnet3
version: 1.4.14.0-k-NAPI
firmware-version:
expansion-rom-version:
bus-info: 0000:03:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no

Why I'm facing the issue only in the prod-trex environment?

Best Regards,
Jegan

[balaje...@gmail.com]

Hi Bes,

Thanks for your reply,

I'm using Ubuntu VM and my TRex files are located in "/opt/trex/v2.80$". I couldn't find the "trex-core/src/dpdk/drivers/net/vmxnet3/vmxnet3_ethdev.c" inside the trex folder.
I downloaded the TRex by using the command: "sudo wget --no-check-certificate https://trex-tgn.cisco.com/trex/release/v2.80.tar.gz"

Is there any other library that I need to use here?

Best Regards,
Jegan

[besi7...@gmail.com]

Look here:

https://github.com/cisco-system-traffic-generator/trex-core/blob/master/src/dpdk/drivers/net/vmxnet3/vmxnet3_ethdev.c

Thanks,

Bes

[balaje...@gmail.com]

Hi Bes,

The file is not locally available with the TRex. How can I use this file with the downloaded TRex file?

Best Regards,
Jegan

[hanoh haim]

You should compile the code. See instruction in the wiki 

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/ab0bc0e7-819b-498c-852f-08b6979d83dc%40googlegroups.com.

[Mike Dehaan]

I was getting this same error. Removing that section of code and rebuilding fixed it. 

[hanoh haim]

Hi Mike, did you remove this?

[Mike Dehaan]

Hanoch,

Correct, that's the only thing I removed. 

I was also getting the repeated lines of "vmxnet3_dev_start(): Device activation: UNSUCCESSFUL" when trying to launch. I saw that there were two If statements that threw that error. (Listed below). I did not remove those, but I didn't get the "Device activation: UNSUCCESSFUL" error anymore either. 

if (ret != 0) {

        PMD_INIT_LOG(ERR, "Device activation: UNSUCCESSFUL");

        return -EINVAL;

    }

*/Line 763 - 785

if (ret != VMXNET3_SUCCESS) {

        PMD_INIT_LOG(ERR, "Device queue init: UNSUCCESSFUL");

        return ret;

    }