DPDK vdev boning

Haklat Haklat

unread,

Mar 30, 2021, 7:44:22 AM3/30/21

to TRex Traffic Generator

Hi,

has anyone experience from trying DPDK vdev bonding ports with TREX? Does e.g. Intel i40e DPDK bonding drivers come with TREX distribution, or would it be something that need to be compiled separately or with some TREX compilation switch somehow?

BR//Håkan

hanoh haim

unread,

Mar 30, 2021, 9:10:34 AM3/30/21

to Haklat Haklat, TRex Traffic Generator

You will need to add the driver to the makefile

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/7d45b7ab-58af-4e30-b208-db26ef531a5en%40googlegroups.com.

--

Hanoh

Sent from my iPhone

Haklat Haklat

unread,

Apr 1, 2021, 3:58:33 AM4/1/21

to TRex Traffic Generator

Hi,

think I managed to build the drivers (adding lines to ws_main.py). But got, below when trying to launch trex server. Do I need to register the drivers to TREX somehow?:

[root@trex-i40e-bonded-trexexperiment-v1 scripts]# ./t-rex-64 -i

Starting Scapy server..... Scapy server is started

The ports are bound/configured.

Starting TRex v2.88 please wait ...

Error: driver net_bonding is not supported. Please consult the documentation for a list of supported drivers

Killing Scapy server... Scapy server is killed

Had this in my trex_cnf.yaml:

interfaces : ['--vdev=net_bonding0,mode=2,slave=0000:5e:02.1,slave=0000:5e:0a.5,xmit_policy=l34,mac=00:50:56:1f:01:73', 'dummy']

Bonding drivers:

[root@trex-i40e-bonded-trexexperiment-v1 lib64]# cd /scratch/trex-core/linux_dpdk/build_dpdk/src/dpdk/drivers/net/bonding

[root@trex-i40e-bonded-trexexperiment-v1 bonding]# ls -al

total 7700

drwxr-xr-x 2 root root 4096 Mar 31 15:49 .

drwxr-xr-x 21 root root 261 Mar 31 15:36 ..

-rw-r--r-- 1 root root 691192 Mar 31 15:44 rte_eth_bond_8023ad.c.14.o

-rw-r--r-- 1 root root 903064 Mar 31 15:48 rte_eth_bond_8023ad.c.19.o

-rw-r--r-- 1 root root 440952 Mar 31 15:36 rte_eth_bond_8023ad.c.4.o

-rw-r--r-- 1 root root 909224 Mar 31 15:41 rte_eth_bond_8023ad.c.9.o

-rw-r--r-- 1 root root 101248 Mar 31 15:45 rte_eth_bond_alb.c.14.o

-rw-r--r-- 1 root root 109536 Mar 31 15:49 rte_eth_bond_alb.c.19.o

-rw-r--r-- 1 root root 101608 Mar 31 15:36 rte_eth_bond_alb.c.4.o

-rw-r--r-- 1 root root 109824 Mar 31 15:40 rte_eth_bond_alb.c.9.o

-rw-r--r-- 1 root root 131200 Mar 31 15:45 rte_eth_bond_api.c.14.o

-rw-r--r-- 1 root root 225048 Mar 31 15:48 rte_eth_bond_api.c.19.o

-rw-r--r-- 1 root root 131560 Mar 31 15:36 rte_eth_bond_api.c.4.o

-rw-r--r-- 1 root root 225432 Mar 31 15:40 rte_eth_bond_api.c.9.o

-rw-r--r-- 1 root root 92376 Mar 31 15:45 rte_eth_bond_args.c.14.o

-rw-r--r-- 1 root root 102944 Mar 31 15:48 rte_eth_bond_args.c.19.o

-rw-r--r-- 1 root root 92736 Mar 31 15:36 rte_eth_bond_args.c.4.o

-rw-r--r-- 1 root root 103288 Mar 31 15:40 rte_eth_bond_args.c.9.o

-rw-r--r-- 1 root root 192664 Mar 31 15:45 rte_eth_bond_flow.c.14.o

-rw-r--r-- 1 root root 121040 Mar 31 15:48 rte_eth_bond_flow.c.19.o

-rw-r--r-- 1 root root 125288 Mar 31 15:37 rte_eth_bond_flow.c.4.o

-rw-r--r-- 1 root root 121768 Mar 31 15:40 rte_eth_bond_flow.c.9.o

-rw-r--r-- 1 root root 530336 Mar 31 15:45 rte_eth_bond_pmd.c.14.o

-rw-r--r-- 1 root root 934304 Mar 31 15:48 rte_eth_bond_pmd.c.19.o

-rw-r--r-- 1 root root 402624 Mar 31 15:37 rte_eth_bond_pmd.c.4.o

-rw-r--r-- 1 root root 933224 Mar 31 15:41 rte_eth_bond_pmd.c.9.o

BR//Håkan

hanoh haim

unread,

Apr 1, 2021, 4:07:11 AM4/1/21

to Haklat Haklat, TRex Traffic Generator

Yes, you need to do this. Try to find similar commits that add a driver to dpdk

To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/3baae69c-2398-4b18-bbd6-9343c01349d3n%40googlegroups.com.

Haklat Haklat

unread,

Apr 1, 2021, 1:53:30 PM4/1/21

to TRex Traffic Generator

Hi,

got a bit further, but failed with:

assert: ../../src/dpdk_trex_port_attr.cpp:173 virtual void ::update_device_info() Assertion 'ret' failed.

[root@trex-i40e-bonded-trexexperiment-v1 scripts]# ./t-rex-64 -i -v 7

Starting Scapy server..... Scapy server is started

The ports are bound/configured.

Starting TRex v2.88 please wait ...

Using configuration file /etc/trex_cfg.yaml

port limit : not configured

port_bandwidth_gb : 10

port_speed : 0

port_mtu : 0

if_mask : None

prefix : trex1

is low-end : 0

stack type :

limit_memory : 2048

thread_per_dual_if : 1

if : --vdev=net_bonding0,mode=2,slave=0000:5e:02.4,slave=0000:5e:0a.5,xmit_policy=l34,mac=00:50:56:1f:01:73, dummy,

enable_zmq_pub : 1

zmq_pub_port : 4500

m_zmq_rpc_port : 4501

m_rx_desc : 4096

m_tx_desc : 4096

src : 00:00:00:00:00:00

dest : 00:00:00:00:00:00

memory per 2x10G ports

MBUF_64 : 16380

MBUF_128 : 8190

MBUF_256 : 8190

MBUF_512 : 8190

MBUF_1024 : 8190

MBUF_2048 : 4095

MBUF_4096 : 128

MBUF_9K : 512

TRAFFIC_MBUF_64 : 65520

TRAFFIC_MBUF_128 : 32760

TRAFFIC_MBUF_256 : 8190

TRAFFIC_MBUF_512 : 8190

TRAFFIC_MBUF_1024 : 8190

TRAFFIC_MBUF_2048 : 32760

TRAFFIC_MBUF_4096 : 128

TRAFFIC_MBUF_9K : 512

MBUF_DP_FLOWS : 524288

MBUF_GLOBAL_FLOWS : 5120

master thread : 20

rx thread : 60

dual_if : 0

socket : 0

[ 14 16 18 54 56 58 ]

CTimerWheelYamlInfo does not exist

flags : 18010f00

write_file : 0

verbose : 7

realtime : 1

flip : 0

cores : 1

single core : 0

flow-flip : 0

no clean close : 0

zmq_publish : 1

vlan mode : 1

client_cfg : 0

mbuf_cache_disable : 0

cfg file :

mac file :

out file :

client cfg file :

duration : 0

factor : 1

mbuf_factor : 1

latency : 0 pkt/sec

zmq_port : 4500

telnet_port : 4501

expected_ports : 2

tw_bucket_usec : 20.000000 usec

tw_buckets : 1024 usec

tw_levels : 3 usec

port : 0 dst:00:00:00:00:00:00 src:00:00:00:00:00:00 vlan:840

port : 1 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 2 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 3 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 4 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 5 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 6 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 7 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 8 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 9 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 10 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 11 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 12 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 13 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 14 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 15 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 16 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 17 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 18 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 19 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 20 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 21 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 22 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 23 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 24 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 25 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 26 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 27 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 28 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 29 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 30 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

port : 31 dst:00:00:00:01:00:00 src:00:00:00:00:00:00 vlan:0

Total Memory :

MBUF_64 : 81900

MBUF_128 : 40950

MBUF_256 : 16380

MBUF_512 : 16380

MBUF_1024 : 16380

MBUF_2048 : 36855

MBUF_4096 : 1024

MBUF_DP_FLOWS : 524288

MBUF_GLOBAL_FLOWS : 5120

get_each_core_dp_flows : 524288

Total memory : 248.40 Mbytes

core_list : 20,60,14

sockets : 0

active sockets : 1

ports_sockets : 1

0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

phy | virt

14 1

DPDK args

xx -l 20,60,14 -n 4 --log-level 8 --main-lcore 20 --vdev=net_bonding0,mode=2,slave=0000:5e:02.4,slave=0000:5e:0a.5,xmit_policy=l34,mac=00:50:56:1f:01:73 --no-pci --no-huge -m 2048 --legacy-mem --file-prefix trex1

EAL: Detected 80 lcore(s)

EAL: Detected 2 NUMA nodes

EAL: Detected static linkage of DPDK

EAL: Multi-process socket /var/run/dpdk/trex1/mp_socket

EAL: Selected IOVA mode 'VA'

EAL: Probing VFIO support...

EAL: VFIO support initialized

EAL: No legacy callbacks, legacy socket not created

set driver name net_bonding

driver capability :

set dpdk queues mode to ONE_QUE

DPDK devices 1 : 1

-----

0 : vdev net_bonding0

-----

Number of ports found: 2 (dummy among them: 1)

if_index : 0

driver name : net_bonding

min_rx_bufsize : 0

max_rx_pktlen : 16128

max_rx_queues : 1024

max_tx_queues : 1024

max_mac_addrs : 16

rx_offload_capa : 0x0

tx_offload_capa : 0x0

rss reta_size : 0

flow_type_rss : 0x2003ffffc

tx_desc_max : 65535

tx_desc_min : 0

rx_desc_max : 65535

rx_desc_min : 0

zmq publisher at: tcp://*:4500

rx_data_q_num : 1

rx_drop_q_num : 0

rx_dp_q_num : 0

rx_que_total : 1

--

rx_desc_num_data_q : 4096

rx_desc_num_drop_q : 4096

rx_desc_num_dp_q : 4096

total_desc : 4096

--

tx_desc_num : 4096

assert: ../../src/dpdk_trex_port_attr.cpp:173 virtual void ::update_device_info() Assertion 'ret' failed.

*** traceback follows ***

1 0x576546 __assert_fail + 694

2 0x4f5324 ./_t-rex-64() [0x4f5324]

3 0x5c2d26 DpdkTRexPortAttr::DpdkTRexPortAttr(unsigned char, unsigned char, bool, bool, bool, bool, bool) + 566

4 0x60b7fa CTRexExtendedDriverVirtBase::create_port_attr(unsigned char, unsigned char) + 74

5 0x4ec5c4 CPhyEthIF::Create(unsigned char, unsigned char) + 78

6 0x4ec707 CGlobalTRex::device_start() + 219

7 0x4f4328 CGlobalTRex::Create() + 384

8 0x4f46ea main_test(int, char**) + 758

9 0x7fee8d1a7555 __libc_start_main + 245

10 0x50fab1 ./_t-rex-64() [0x50fab1]

*** addr2line information follows ***

trex_watchdog.cpp:152 (discriminator 1)

ostream:530

dpdk_trex_port_attr.cpp:172

trex_driver_virtual.cpp:34

main_dpdk.cpp:5708

main_dpdk.cpp:3518

main_dpdk.cpp:3663

main_dpdk.cpp:6528

??:0

??:?

./t-rex-64: line 106: 556 Aborted (core dumped) ./_$(basename $0) $INPUT_ARGS $EXTRA_INPUT_ARGS