Trex STL - RX rate is very low than expected, and rx_cpu_util is high

[Akshay Dorwat]

Hi, 

I am using Trex STL to run 10mpps UDP traffic (4million flows and 4 streams), but I observe only about 4.5mpps RX rate at the receive port and we see ierror counters going up and rx cpu utilization is very high. 

Based on multiple threads i came across:

1. I disabled the service mode on the port. This improved performance by 0.5 mpps.

2. Added rx_thread_id in the trex config yaml. Using core same as port numa node. Which resulted in increase of rx rate by another 1 mpps.

But we are still below 10mpps packet receive rate. Appreciate if someone can help the missing element. 

Thanks,

Akshay

[Akshay Dorwat]

Please find below trex config and global statistics:

cat /etc/trex_cfg_stl.yaml 

### Config file generated by dpdk_setup_ports.py 

### - version: 2 

 interfaces: ['d8:00.0', 'dummy', 'dummy', 'd8:00.1'] 

 zmq_pub_port: 4501 

 zmq_rpc_port: 4500 

 port_info: 

   - dest_mac: 00:ae:10:02:00:01 

     src_mac: 00:ae:cd:00:00:01 

   - dest_mac: 00:00:00:00:00:00 

     src_mac: 00:00:00:00:00:00 

  - dest_mac: 00:00:00:00:00:00 

    src_mac: 00:00:00:00:00:00 

  - dest_mac: 00:ae:cd:00:00:01 

   src_mac: 00:ae:10:02:00:01 

 platform: 

   master_thread_id: 0 

  latency_thread_id: 1 

  rx_thread_id: 18 

 dual_if: 

    - socket: 1 

         threads: [19,20,21,22,23,24,25,26,27,28,29,30] 

   - socket: 1 

        threads: [2,3,4,5,6,7,8,9,10,11,12,13] 

Global Statistitcs connection : 

localhost, Port 4500 

total_tx_L2 : 9.61 Gb/sec ▼▼▼ 

version : STL @ v2.64

total_tx_L1 : 10.24 Gb/sec ▼▼▼ 

cpu_util. : 7.84% @ 16 cores (8 per dual port) 

total_rx : 11.16 Gb/sec 

rx_cpu_util. : 87.14% / 4.58 Mpkt/sec 

total_pps : 3.93 Mpkt/sec ▼▼▼ 

async_util. : 0.08% / 1.52 KB/sec 

drop_rate : 0 b/sec total_cps. : 0 cps/sec 

queue_full : 0 pkts

[Besart Dollma]

Hi,

A lot of things can be done to improve the performance.

1. Prefer 64 byte packets.
2. In case you are using VM try and cache.

3. In case you have latency packets, please remove them, this might be the case of the high rx_cpu_util.

Let us know if any of these help.

Thanks,