I am using trex traffic generator with intel XXV710 25g NIC,
Two xxv710 NICs connected back to back directly, using trex to TX and
RX multi streams packet with different IP address (expected that different IP
address Packets can route to different queues by RSS or flow director function of NIC)
On a single port , for TX, it can reach with 25Gbps linerate (37.21Mpps),
But for RX, it only can get 14.29Gbps (27.90Mpps), but it is expected that both RX and TX should reach linerate (37.21Mpps), as with IXIA traffic
Generator can do.
After do some investigation, I found that trex can use multi TX queues to send traffic,
But it seems that trex only can use 1 RX queue to receive traffic, so the RX side rate
Was limited.
So How to configure trex to use multi RX queues of NIC port? Any help is apprecitated.
My trex configure file as below,
#cat /etc/trex_cfg.yaml
- port_limit : 2
version : 2
interfaces : ["0000:81:00.0","0000:83:00.0"]
prefix : trex1
limit_memory : 2048
c : 8
port_info :
- dest_mac : '3c:fd:fe:a7:7a:30'
src_mac : '3c:fd:fe:a7:79:d8'
- dest_mac : '3c:fd:fe:a7:79:d8'
src_mac : '3c:fd:fe:a7:7a:30'
platform :
master_thread_id : 23
latency_thread_id : 24
dual_if :
- socket : 1
threads : [25, 26, 27, 28, 29, 30, 31, 32]
Best Regards
Yulong Pei
--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/ce153f5d-cd67-4dcf-993a-eaa22cf572e5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
I also tried,
start --port 0 1 -f stl/my_pkt.py -m 100% --pin
Best Regards
Yulong Pei
From: Pei, Yulong
Sent: Wednesday, November 29, 2017 6:35 PM
To: 'hanoh haim' <hhaim...@gmail.com>
Cc: TRex Traffic Generator <trex...@googlegroups.com>
Subject: RE: [trex-tgn] How to configure trex to use multi RX queues of NIC port?
Hi ,
1. trex config file,
# cat /etc/trex_cfg.yaml
- port_limit : 2
version : 2
interfaces : ["0000:81:00.0","0000:83:00.0"]
prefix : trex1
limit_memory : 2048
c : 8
port_info :
- dest_mac : '3c:fd:fe:a7:7a:30'
src_mac : '3c:fd:fe:a7:79:d8'
- dest_mac : '3c:fd:fe:a7:79:d8'
src_mac : '3c:fd:fe:a7:7a:30'
platform :
master_thread_id : 23
latency_thread_id : 24
dual_if :
- socket : 1
threads : [25, 26, 27, 28, 29, 30, 31, 32]
2. start trex server
#cd /opt/trex-core-2.28/
#./t-rex-64 –i
3. start trex console and start to send the streams
# ./trex-console
start --port 0 1 -f stl/my_pkt.py -m 100%
#cat stl/my_pkt.py
from trex_stl_lib.api import *
class STLS1(object):
def __init__ (self):
self.fsize = 64 # the size of the packet
def create_stream (self):
# Create base packet and pad it to size
size = self.fsize - 4; # HW will add 4 bytes ethernet FCS
base_pkt = Ether()/IP(src="16.0.0.1",dst="2.1.1.1")/UDP(dport=12,sport=1025)
base_pkt1 = Ether()/IP(src="16.0.0.2",dst="2.1.1.2")/UDP(dport=18,sport=1018)
pad = max(0, size - len(base_pkt)) * 'x'
return STLProfile( [ STLStream( isg = 0.0, # start in delay in usec
packet = STLPktBuilder(pkt = base_pkt/pad),
mode = STLTXCont(),
),
STLStream( isg = 0.0,
packet = STLPktBuilder(pkt = base_pkt1/pad),
mode = STLTXCont(),
),
]).get_streams()
def get_streams (self, direction = 0, **kwargs):
# create 1 stream
return self.create_stream()
# dynamic load - used for trex console or simulator
def register():
return STLS1()
Best Regards
Yulong Pei
Hi ,
1. trex config file,
# cat /etc/trex_cfg.yaml
- port_limit : 2
version : 2
interfaces : ["0000:81:00.0","0000:83:00.0"]
prefix : trex1
limit_memory : 2048
c : 8
port_info :
- dest_mac : '3c:fd:fe:a7:7a:30'
src_mac : '3c:fd:fe:a7:79:d8'
- dest_mac : '3c:fd:fe:a7:79:d8'
src_mac : '3c:fd:fe:a7:7a:30'
platform :
master_thread_id : 23
latency_thread_id : 24
dual_if :
- socket : 1
threads : [25, 26, 27, 28, 29, 30, 31, 32]
2. start trex server
From: hanoh haim [mailto:hhaim...@gmail.com]
Hi Hanoh,
I tried that latest trex v2.32 version today, also has the issue.
What do you mean “Check that you are not in service mode”?
Is there any way to do some debugging to root cause the issue ?
Thanks & Best Regards
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/ce153f5d-cd67-4dcf-993a-eaa22cf572e5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Hanoh
Sent from my iPhone
--
Hanoh
Sent from my iPhone
--
Hanoh
Sent from my iPhone
Hi Hanoh,
Just download the code,
git clone https://github.com/cisco-system-traffic-generator/trex-core.git
cd trex-core/
cd linux_dpdk/
./b configure
./b build
cd scripts/
./t-rex-64 –i
./trex-console
start --port 0 1 -f stl/my_pkt.py -m 100% --pin
the result is same as trex v2.32 and v2.28.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/ce153f5d-cd67-4dcf-993a-eaa22cf572e5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Hanoh
Sent from my iPhone
--
Hanoh
Sent from my iPhone
--
Hanoh
Sent from my iPhone
--
Hanoh
Sent from my iPhone
--
You received this message because you are subscribed to a topic in the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/trex-tgn/NepxT6j8on4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/CA%2BYxBoLBjoDmWtuAaNV6Y2bPau9mzLd%3DTLK4B2yck41qQBOPFg%40mail.gmail.com.
Hi Hanoh,
Just download the code,
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/ce153f5d-cd67-4dcf-993a-eaa22cf572e5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Hanoh
Sent from my iPhone
--
Hanoh
Sent from my iPhone
--
Hanoh
Sent from my iPhone
--
Hanoh
Sent from my iPhone
--
You received this message because you are subscribed to a topic in the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/trex-tgn/NepxT6j8on4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to trex-tgn+unsubscribe@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/CA%2BYxBoLBjoDmWtuAaNV6Y2bPau9mzLd%3DTLK4B2yck41qQBOPFg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Hi,
Attached trex_tui_output.png
Best Regards
Yulong Pei
From: hanoh haim [mailto:hhaim...@gmail.com]
Sent: Thursday, November 30, 2017 5:09 PM
To: Pei, Yulong <yulon...@intel.com>
Cc: TRex Traffic Generator <trex...@googlegroups.com>
Subject: Re: [trex-tgn] How to configure trex to use multi RX queues of NIC port?
Can you send the output of the Console TUI in this case?
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/ce153f5d-cd67-4dcf-993a-eaa22cf572e5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Hanoh
Sent from my iPhone
--
Hanoh
Sent from my iPhone
--
Hanoh
Sent from my iPhone
--
Hanoh
Sent from my iPhone
--
You received this message because you are subscribed to a topic in the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/trex-tgn/NepxT6j8on4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/CA%2BYxBoLBjoDmWtuAaNV6Y2bPau9mzLd%3DTLK4B2yck41qQBOPFg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/ce153f5d-cd67-4dcf-993a-eaa22cf572e5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Hanoh
Sent from my iPhone
--
Hanoh
Sent from my iPhone
--
Hanoh
Sent from my iPhone
--
Hanoh
Sent from my iPhone
--
You received this message because you are subscribed to a topic in the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/trex-tgn/NepxT6j8on4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to trex-tgn+unsubscribe@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/CA%2BYxBoLBjoDmWtuAaNV6Y2bPau9mzLd%3DTLK4B2yck41qQBOPFg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--
Hanoh
Sent from my iPhone
Hi ,
My test environment is 2 XXV710 25G NIC CARDs, each card have 2 ports, I am using
1 port of first card to connect to 1 port of second card.
(for this setup, if using ixia, with 64byte packet , TX and RX both can reach 25G linerate 37.21 Mpps).
Run the command:
$start -f stl/bench.py -t size=64,vm=cached --port 0 -m 100% --force
The tui output result as the attachment trex_tui_0_1.png.
Tx 37.25 Mpps is good
Rx 27.95 Mpps, but it should 37.25Mpps.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/ce153f5d-cd67-4dcf-993a-eaa22cf572e5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Hanoh
Sent from my iPhone
--
Hanoh
Sent from my iPhone
--
Hanoh
Sent from my iPhone
--
Hanoh
Sent from my iPhone
--
You received this message because you are subscribed to a topic in the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/trex-tgn/NepxT6j8on4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/CA%2BYxBoLBjoDmWtuAaNV6Y2bPau9mzLd%3DTLK4B2yck41qQBOPFg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--
Hanoh
Sent from my iPhone
--
Hanoh
Sent from my iPhone
--
You received this message because you are subscribed to a topic in the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/trex-tgn/NepxT6j8on4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/CA%2BYxBoJj0v-5srLJKAzBTmcbFLb50VLLHyJ124tQ1tZasp-cuA%40mail.gmail.com.
# ./dpdk_setup_ports.py -t
+----+------+---------+-------------------+-----------------------------------------+---------+----------+----------+
| ID | NUMA | PCI | MAC | Name | Driver | Linux IF | Active |
+====+======+=========+===================+=========================================+=========+==========+==========+
+----+------+---------+-------------------+-----------------------------------------+---------+----------+----------+
| 6 | 1 | 81:00.0 | 3c:fd:fe:a7:79:d8 | Device 158b | igb_uio | | |
+----+------+---------+-------------------+-----------------------------------------+---------+----------+----------+
| 7 | 1 | 81:00.1 | 3c:fd:fe:a7:79:d9 | Device 158b | i40e | ens801f1 | |
+----+------+---------+-------------------+-----------------------------------------+---------+----------+----------+
| 8 | 1 | 83:00.0 | 3c:fd:fe:a7:7a:30 | Device 158b | igb_uio | | |
+----+------+---------+-------------------+-----------------------------------------+---------+----------+----------+
| 9 | 1 | 83:00.1 | 3c:fd:fe:a7:7a:31 | Device 158b | i40e | ens802f1 | |
+----+------+---------+-------------------+-----------------------------------------+---------+----------+----------+
The red two was used by trex, both at NUMA node 1.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/CA%2BYxBoJF7JGGALVvpT84N4CFKxPMK-6d%2BcMxwwm-Je64hjDCbg%40mail.gmail.com.
Hi Hanoh,
Today we tested Trex on XL710 40G NIC and XXV710 25G NIC,
With 64byte packet streams, both XL710 40G and XXV710 25G , RX pps only can reach ~27.9 Mpps.
Attached detail test steps in “Trex back to back test for XL710 and XXV710.docx”
But we also run the same test for it with DPDK and IXIA traffic generator, the expected Rx pps can be gotten.
So it seems that Trex have some kind of issue on i40e driver when do packet RX, can you help to fix it ?
$start -f stl/bench.py -t size=64,vm=cached --port 0 -m 63% --force
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/188971FCDA171749BED5DA74ABF3E6F03C0F11E8%40SHSMSX104.ccr.corp.intel.com.
For more options, visit https://groups.google.com/d/optout.
For mine, firmware-version: 6.01 0x80003221 1.1691.0
What is the output of trex-console command:
stats -x
?
Hi Yulong,
Could you check this branch? It is a quick potential fix with 3 drop queues.
I couldn’t check it because we don't have XL710 with FW=6.0.1 only 5.0.4.
--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/CAJysGCQYKV9_ZLy5koQunw5dgtL%2B7OvbOGWL%3DS_6%3DBxULr0o9w%40mail.gmail.com.
Hi Hanoh,
Thanks for your help , I have done the test it today,
#git clone -b xl710_drop_queue https://github.com/hhaim/trex-core.git
#cd linux_dpdk
#./b configure
#./b build
# cd scripts
Run the same test as previous, the result also same with previous TX 37.24 Mpps /RX 27.90 Mpps, no RX improvement.
Best Regards
Yulong Pei
From: hanoh haim [mailto:hhaim...@gmail.com]
Sent: Friday, December 1, 2017 7:40 PM
To: Yaroslav Brustinov <y.bru...@gmail.com>
Cc: Pei, Yulong <yulon...@intel.com>; Xu, Qian Q <qian...@intel.com>; TRex Traffic Generator <trex...@googlegroups.com>
Subject: Re: [trex-tgn] How to configure trex to use multi RX queues of NIC port?
Hi Yulong,
Could you check this branch? It is a quick potential fix with 3 drop queues.
I couldn’t check it because we don't have XL710 with FW=6.0.1 only 5.0.4.
on X710 there is no issue as queue is for 10gb
https://github.com/hhaim/trex-core/tree/xl710_drop_queue
thanks,
Hanoh
On Fri, Dec 1, 2017 at 1:02 PM, Yaroslav Brustinov <y.bru...@gmail.com> wrote:
What is the output of trex-console command:
stats -x
?
--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/CAJysGCQYKV9_ZLy5koQunw5dgtL%2B7OvbOGWL%3DS_6%3DBxULr0o9w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Hi,
I am using Trex in a virtual environment with SRIOV. When I try to assign 2 cores to the traffic ports, it gives an error " Error: the number of cores should be 1 when the driver support only one tx queue and one rx queue. Please use -c 1 "
I am getting line rate upto 256bytes packet size. For 64bytes I am only getting around 4Gbps.
root@trex:~/trex/v2.35# sudo ./t-rex-64 -i -c 6
Killing Scapy server... Scapy server is killed
Starting Scapy server.... Scapy server is started
The ports are bound/configured.
Starting TRex v2.35 please wait ...
set driver name net_ixgbe_vf
driver capability : TCP_UDP_OFFLOAD TSO
Number of ports found: 2
Error: the number of cores should be 1 when the driver support only one tx queue and one rx queue. Please use -c 1
root@trex:~/trex/v2.35# ./dpdk_setup_ports.py -s
Network devices using DPDK-compatible driver
============================================
0000:00:06.0 '82599 Ethernet Controller Virtual Function' drv=igb_uio unused=ixgbevf,vfio-pci,uio_pci_generic
0000:00:07.0 '82599 Ethernet Controller Virtual Function' drv=igb_uio unused=ixgbevf,vfio-pci,uio_pci_generic
Network devices using kernel driver
===================================
0000:00:03.0 'Virtio network device' if=ens3 drv=virtio-pci unused=igb_uio,vfio-pci,uio_pci_generic *Active*
Other network devices
=====================
Any thoughts?
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/CAJysGCQYKV9_ZLy5koQunw5dgtL%2B7OvbOGWL%3DS_6%3DBxULr0o9w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--
Hanoh
Sent from my iPhone
--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/188971FCDA171749BED5DA74ABF3E6F03C0F1AED%40SHSMSX104.ccr.corp.intel.com.
Sent: Friday, March 2, 2018 2:15 PM
To: Pei, Yulong <yulon...@intel.com>
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/CAJysGCQYKV9_ZLy5koQunw5dgtL%2B7OvbOGWL%3DS_6%3DBxULr0o9w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--
Hanoh
Sent from my iPhone
--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/188971FCDA171749BED5DA74ABF3E6F03C0F1AED%40SHSMSX104.ccr.corp.intel.com.
For more options, visit https://groups.google.com/d/optout.
Hi Hanoh,
Intel firmware team guys said that Trex should enable the receive q since drop q with firmware 6.01 has limitation.
Do you consider to enable receive q for XL710 and XXV710 ?