Not able to use all available cores - try to reduce `-c value ` or 'c:' in the config file error

[shashank varshney]

Hi experts,

I have 2 socket machine with 24 physical cores (48 hyperthreaded cores) per socket.

So I have total 96 hyper-threaded cores available on the system. 

[root@arch-4 v2.97]# lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              96
On-line CPU(s) list: 0-95
Thread(s) per core:  2
Core(s) per socket:  24
Socket(s):           2
NUMA node(s):        2
Vendor ID:           GenuineIntel
BIOS Vendor ID:      Intel(R) Corporation
CPU family:          6
Model:               85
Model name:          Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz
BIOS Model name:     Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz
Stepping:            7
CPU MHz:             2384.478
CPU max MHz:         4000.0000
CPU min MHz:         1000.0000
BogoMIPS:            4800.00
Virtualization:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            1024K
L3 cache:            36608K
NUMA node0 CPU(s):   0-23,48-71
NUMA node1 CPU(s):   24-47,72-95

I am trying to use all cores and following errors are occurring whenever I go for -c parameter more than 23 cores:

DPDK args
 xx  -l  0,25,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,50,51,52,74,75,76  -n  4  --log-level  8  --main-lcore  0  -a  0000:18:00.0  -a  0000:18:00.1  -a  0000:af:00.0  -a  0000:af:00.1  --legacy-mem  --socket-mem  8192,8192
EAL: Detected 96 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: 100000 hugepages of size 2097152 reserved, but no mounted hugetlbfs found for that size
 EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL:   using IOMMU type 1 (Type 1)
EAL: Probe PCI driver: net_ice (8086:1592) device: 0000:18:00.0 (socket 0)
ice_load_pkg_type(): Active package is: 1.3.28.0, ICE OS Default Package (double VLAN mode)
EAL: Probe PCI driver: net_ice (8086:1592) device: 0000:18:00.1 (socket 0)
ice_load_pkg_type(): Active package is: 1.3.28.0, ICE OS Default Package (double VLAN mode)
EAL: Probe PCI driver: net_ice (8086:1592) device: 0000:af:00.0 (socket 1)
ice_load_pkg_type(): Active package is: 1.3.28.0, ICE OS Default Package (single VLAN mode)
EAL: Probe PCI driver: net_ice (8086:1592) device: 0000:af:00.1 (socket 1)
ice_load_pkg_type(): Active package is: 1.3.28.0, ICE OS Default Package (single VLAN mode)
EAL: No legacy callbacks, legacy socket not created
                input : [18:00.0, 18:00.1, af:00.0, af:00.1]
                 dpdk : [0000:18:00.0, 0000:18:00.1, 0000:af:00.0, 0000:af:00.1]
             pci_scan : [0000:18:00.0, 0000:18:00.1, 0000:af:00.0, 0000:af:00.1]
                  map : [ 0, 1, 2, 3]
 TRex port mapping
 -----------------
 TRex vport: 0 dpdk_rte_eth: 0
 TRex vport: 1 dpdk_rte_eth: 1
 TRex vport: 2 dpdk_rte_eth: 2
 TRex vport: 3 dpdk_rte_eth: 3
 set driver name net_ice
 driver capability  : TCP_UDP_OFFLOAD  TSO  SLRO
 set dpdk queues mode to MULTI_QUE
 DPDK devices 4 : 4
-----
 0 : vdev 0000:18:00.0
 1 : vdev 0000:18:00.1
 2 : vdev 0000:af:00.0
 3 : vdev 0000:af:00.1
-----
 Number of ports found: 4
EAL: Error - exiting with code: 1
  Cause:  Your configuration require  50 DP cores but the maximum supported is 48 - try to reduce `-c value ` or 'c:' in the config file
Killing Scapy server... Scapy server is killed

I am unable to use all cores available in the system.

Following is the trex_cfg file:

- port_limit      : 4
  version         : 2
 #List of interfaces. Change to suit your setup. Use ./dpdk_setup_ports.py -s to see available options
  interfaces    : ["18:00.0","18:00.1","af:00.0","af:00.1"]
  #interfaces    : ["af:00.0","dummy","af:00.1"]
  limit_memory    : 8192 # optional 7
  memory :
       traffic_mbuf_64 : 3400000
       #traffic_mbuf_128 : 3400000
       #traffic_mbuf_256 : 3400000
       #traffic_mbuf_2048 : 3400000
  rx_desc         : 4096 # optional 17
  tx_desc         : 4096 # optional 17
  port_info       :  # Port IPs. Change to suit your needs. In case of loopback, you can leave as is.
                 - dest_mac         : '00:00:00:00:01:01'  # Arch-4 VF NIC
                   src_mac          : 'b4:96:91:a4:03:20'  # CVL SRC NIC
                 - dest_mac         : '00:00:00:00:01:01'  # Arch-4 VF NIC
                   src_mac          : 'b4:96:91:a4:03:20'  # CVL SRC NIC
                 - dest_mac         : '00:00:00:00:01:00'  # Arch-4 VF
                   src_mac          : 'b4:96:91:a4:00:80'  # CVL SRC NIC
                 - dest_mac         : '00:00:00:00:01:00'  # Arch-4 VF
                   src_mac          : 'b4:96:91:a4:00:80'  # CVL SRC NIC
  c : 25
  platform :
        master_thread_id  : 0
        latency_thread_id : 25
        dual_if   :
                - socket   : 0
                  #threads  : [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71]
                  threads  : [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,50,51,52]
                  #threads  : [2,3,4,5,6,7,8,9,10,11]
                - socket   : 1
                  #threads  : [26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95]
                  threads  : [26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,74,75,76]
                  #threads  : [26,27,28,29,30,31,32,33,34,35]

What can be the resolution of this issue so that I can utilize all cores available in the system?

Thanks and regards

Shashank Varshney

[Mohsen Meamarian]

Hi,

We have the same problem, any solution guys?

Best,