Mellanox connectx5 Performance Issues.

[george Bates]

Hi Folks,

Was hoping someone can assist me with some performance issues with running 100GbE Line Rate at benchmarked frame sizes that are displayed on the Mellanox performance results on the T-Rex website?

I have setup everything based on the instructions on the Trex Website and I am able to reach 100GbE from 1514-512Byte packet size but I can not get Line rate at anything below 512B. I see that CPU OCC reaches 100% with the smaller frame sizes.

I have 2 NUMA Nodes with a Intel(R) Xeon(R) Gold 5217 CPU @ 3.00GHz . Each Node has 8 cores and 4 Banks of 16G DDR4 Memory totaling 128G for the Server.

The config is just external loopback across NUMA Nodes at this time.

OS=Centos7.6 Bare Metal

TREX version 2.58

stateless with udp_1pkt.py set to different size frames for each  test.

running each frame size script up to 100GbE L1 Line Rate.

I also tried the stl/bench.py script and CPU OCC goes up to 100% once I attempt > 60Gbps.

Any assistance would be great.

Thanks,

trex-platform cfg:

- port_limit      : 2
  version         : 2
#List of interfaces. Change to suit your setup. Use ./dpdk_setup_ports.py -s to see available options
  interfaces    : ["37:00.0","86:00.0"]
  port_info       :  # Port IPs. Change to suit your needs. In case of loopback, you can leave as is.
          - ip         : 1.1.1.1
            default_gw : 2.2.2.2
          - ip         : 2.2.2.2
            default_gw : 1.1.1.1
  platform:
      master_thread_id: 0
      latency_thread_id: 15
      dual_if:
        - socket: 0
          threads: [1,2,3,4,5,6,7]
        - socket: 1
          threads: [8,9,10,11,12,13,14]

numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 6 7 16 17 18 19 20 21 22 23
node 0 size: 65185 MB
node 0 free: 56000 MB
node 1 cpus: 8 9 10 11 12 13 14 15 24 25 26 27 28 29 30 31
node 1 size: 65535 MB
node 1 free: 56526 MB
node distances:
node   0   1
  0:  10  21
  1:  21  10

[hanoh haim]

Hi George, 

Your setup, configuration should support full line rate @64B

However, you are doing the benchmark incorrectly - the profile you are using has one flow, one packet size. 

Try following this document and use stl/bench.py 

https://trex-tgn.cisco.com/trex/doc/images/64_util.png

and full doc is here:

https://trex-tgn.cisco.com/trex/doc/trex_stateless_bench.html

thanks

Hanoh

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/7a12665f-c896-48e3-99f5-163d91699b2a%40googlegroups.com.

--

Hanoh

Sent from my iPhone