Stream statistic do not support high load traffic

Elisa Piccardo

unread,

Apr 11, 2022, 4:22:39 AM4/11/22

to TRex Traffic Generator

Hello,

I'm sending IPv6 traffic on 100 Gps port 0 (sending about 20Gbps) in two streams with different DSCP and flow id.

All the traffic is received in port 1 when looking at port statistics.

I'm using 8 cores and the CPU_util is around 8% .

However when I look at stream statistic (the purpose is to make QoS test), not all traffic is received in streams, I suppose because the rx_cpu_util is above 99%.

Is it possible to increase the rx_cpu power, maybe assigning more cores in rx?

Below my config file and the traffic profile.

Thanks a lot

Elisa Piccardo

Port statistics:

stream statistics:

### Config file generated by dpdk_setup_ports.py ###

- version: 2
interfaces: ['41:00.0', '41:00.1']
c : 8
stack: linux_based
port_info:
- dest_mac: 1c:34:da:ef:8f:00 # MAC OF LOOPBACK TO IT'S DUAL INTERFACE
src_mac: 0c:42:a1:73:8c:1e
- dest_mac: 00:e0:4b:6b:cf:5e # MAC OF LOOPBACK TO IT'S DUAL INTERFACE
src_mac: 0c:42:a1:73:8c:1f

platform:
master_thread_id: 0
latency_thread_id: 13
rx_thread_id: 14
dual_if:
- socket: 0
threads: [1,2,3,4,5,6,7,8,9,10,11,12,48,49,50,51,52,53,54,55,56,57,58,59]

- socket: 1
threads: [24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47]

*************************************************************************************
class STLS1(object):
"""
Create flow stat stream of UDP packet.
Can specify using tunables following params:
Packet length (fsize)
Packet group id (pg_id)
Packet type(pkt_type)
Number of streams (num_streams)
"""
def __init__ (self):
self.fsize = 1500
self.pg_id = 0
self.pkt_type = "ether"
self.num_streams = 2
self.dscp = 0

def _create_stream (self):
size = self.fsize - 4; # HW will add 4 bytes ethernet CRC

pkt_types1 = {"ether" :
Ether() / IPv6(src = "fc02::1:2", dst = "fc02::2:2", tc=3) / UDP(dport = 12, sport = 1025),
"vlan":
Ether() / Dot1Q(vlan=11) / IPv6(src = "fc02::1:2", dst = "fc02::2:2", tc=3) / UDP(dport = 12, sport = 1025)
, "qinq":
Ether(type=0x88A8) / Dot1Q(vlan=19) / Dot1Q(vlan=11) / IPv6(src = "fc02::1:2", dst = "fc02::2:2", tc=3) / UDP(dport = 12, sport = 1025)
}

pkt_types2 = {"ether" :
Ether() / IPv6(src = "fc02::1:2", dst = "fc02::2:2", tc=5) / UDP(dport = 12, sport = 1025),
"vlan":
Ether() / Dot1Q(vlan=11) / IPv6(src = "fc02::1:2", dst = "fc02::2:2", tc=5) / UDP(dport = 12, sport = 1025)
, "qinq":
Ether(type=0x88A8) / Dot1Q(vlan=19) / Dot1Q(vlan=11) / IPv6(src = "fc02::1:2", dst = "fc02::2:2", tc=5) / UDP(dport = 12, sport = 1025)
}

if self.pkt_type not in pkt_types1.keys():
print ("Wrong pkt_type1 given. Allowed values are " + format(pkt_types1.keys()))
return []

if self.pkt_type not in pkt_types2.keys():
print ("Wrong pkt_type2 given. Allowed values are " + format(pkt_types2.keys()))
return []

pad = max(0, size - len(self.pkt_type)) * 'x'
pkt1 = STLPktBuilder(pkt = pkt_types1[self.pkt_type]/pad)

#pad = max(0, size - len(self.pkt_type)) * 'x' #attualemte non serve, se si vuole fare pacchetti di lunghezza diversi togliere il commento
pkt2 = STLPktBuilder(pkt = pkt_types2[self.pkt_type]/pad)

streams = []
for pg_id_add in range(0, self.num_streams):
if pg_id_add == 0:
pkt = pkt1
else:
pkt = pkt2
streams.append(STLStream(packet = pkt, mode = STLTXCont(pps=10000000), flow_stats = STLFlowStats(pg_id = self.pg_id + pg_id_add)))

return streams

def get_streams (self, tunables, **kwargs):
parser = argparse.ArgumentParser(description='Argparser for {}'.format(os.path.basename(__file__)),
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument('--fsize',
type = int,
default = 64,
help="define the packet's length in the stream.")
parser.add_argument('--pg_id',
type = int,
default = 1,
help="define the packet group id (pg_id).")
parser.add_argument('--pkt_type',
type = str,
default = "ether",
choices={'ether', 'vlan', 'qinq'},
help="define the packet type.")
parser.add_argument('--num_streams',
type = int,
default = 2,
help="define the number of streams.")
args = parser.parse_args(tunables)

self.fsize = args.fsize
self.pg_id = args.pg_id
self.pkt_type = args.pkt_type
self.num_streams = args.num_streams
return self._create_stream()

# dynamic load - used for trex console or simulator
def register():
return STLS1()

Besart Dollma

unread,

Apr 12, 2022, 2:17:32 AM4/12/22

to TRex Traffic Generator

Ciao Elisa,

To answer you question, it is not possible, there is only one Rx core.

However, in your use case packets shouldn't be redirected to Rx, they can be handled in Data Planes.

So the first step should be understanding why are you packets being redirected?

Have you run TRex with --software or have you enabled service mode?

Thanks,

Elisa Piccardo

unread,

Apr 12, 2022, 3:15:42 AM4/12/22

to TRex Traffic Generator

Hi,

thanks for your answer.

I'm not using --software or service mode.

I've run trex using ./t-rex-64 -i

and ./console for the console.

I'm using tui command to see statistics.

Service mode is disabled

Anything I can do to improve?

Elisa

hanoh haim

unread,

Apr 12, 2022, 6:23:00 AM4/12/22

to Elisa Piccardo, TRex Traffic Generator

Hi Elisa,

This is expected (assuming you are using mlx5 driver), only i40e driver can do flow-stats in hardware. In default mode all the flow-stats are sent to rx core and this is the bottleneck

One workaround is to use full software mode, but this will have a negative effect on even without flow-stats as all the packets will be processed by Rx path (add --software)

Thanks

Hanoh

Elisa Piccardo

unread,

Apr 13, 2022, 3:21:34 AM4/13/22

to TRex Traffic Generator

Thanks Hanon for your answer.

I've tried with --software, but I cannot receive more than 5 Gbps (2,5 Gbps per stream) and I loose the other traffic.

Is it possible to install i40e driver or it depends by hardware?

Elisa

hanoh haim

unread,

Apr 14, 2022, 2:33:59 AM4/14/22

to Elisa Piccardo, TRex Traffic Generator

Hi,

How many cores are you using with --software, it should support much more traffic.

The i40e DPDK driver requires Intel XL710 NIC

Thanks

Hanoh

Reply all

Reply to author

Forward