First time run - "not enough huge-pages memory"

[avi...@gmail.com]

Hi

H/W
DL380 G9 dual CPU 2680v4 , Silicom NIC X710 SFP+ Quad, connected to CPU 0

Centos 7.3
When running the command
"./t-rex-64 -f cap2/dns.yaml -c 4 -m 1 -d 10 -l 1000"

the following output:
Trying to bind to igb_uio ...
/bin/python dpdk_nic_bind.py --bind=igb_uio 0000:05:00.0 0000:05:00.1 0000:05:00.2 0000:05:00.3
The ports are bound/configured.
Starting TRex v2.25 please wait ...
set driver name net_i40e
no client generator pool configured, using default pool
no server generator pool configured, using default pool
zmq publisher at: tcp://*:4500
Number of ports found: 4
ERROR there is not enough huge-pages memory in your system
EAL: Error - exiting with code: 1
Cause: Cannot init mbuf pool _2048-pkt-const

Configuration file:
cat /etc/trex_cfg.yaml
### Config file generated by dpdk_setup_ports.py ###

- port_limit: 4
version: 2
interfaces: ['05:00.0', '05:00.1', '05:00.2', '05:00.3']
port_info:
- ip: 1.1.1.1
default_gw: 2.2.2.2
- ip: 2.2.2.2
default_gw: 1.1.1.1

- ip: 3.3.3.3
default_gw: 4.4.4.4
- ip: 4.4.4.4
default_gw: 3.3.3.3

platform:
master_thread_id: 0
latency_thread_id: 12
dual_if:
- socket: 0
threads: [1,2,3,4,5,24,25,26,27,28,29]

- socket: 0
threads: [6,7,8,9,10,11,30,31,32,33,34]

Please help

Thanks,

[hanoh haim]

Do you have another DPDK application running on the same machine?

Hanoh

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/0ea20f19-1090-41e2-995f-b13ad08978ae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Hanoh

Sent from my iPhone

[avi...@gmail.com]

Hi.

How can i check if another application is using DPDK?

Thanks,

[hanoh haim]

Could you send the output for

1) free -h

2) same trex command with “-v 7” added

3) same with “—limit-ports 2” added 

Thanks, 

Hanoh

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/080b9aed-41c0-4504-ab2f-39086c26277d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[avi...@gmail.com]

Hi

[root@localhost v.cl2017]# free -h
total used free shared buff/cache available
Mem: 219G 2.4G 216G 12M 1.0G 216G
Swap: 31G 0B 31G
[root@localhost v.cl2017]#
[root@localhost v.cl2017]# ./t-rex-64 -f cap2/dns.yaml -v 7

The ports are bound/configured.
Starting TRex v2.25 please wait ...

Using configuration file /etc/trex_cfg.yaml
port limit : 4
port_bandwidth_gb : 10
if_mask : None
thread_per_dual_if : 1
if : 05:00.0, 05:00.1, 05:00.2, 05:00.3,
enable_zmq_pub : 1
zmq_pub_port : 4500
m_zmq_rpc_port : 4501
src : 00:00:00:00:00:00
dest : 00:00:00:00:00:00
src : 00:00:00:00:00:00
dest : 00:00:00:00:00:00
src : 00:00:00:00:00:00
dest : 00:00:00:00:00:00
src : 00:00:00:00:00:00
dest : 00:00:00:00:00:00
memory per 2x10G ports
MBUF_64 : 16380
MBUF_128 : 8190
MBUF_256 : 8190
MBUF_512 : 8190
MBUF_1024 : 8190
MBUF_2048 : 4095
MBUF_4096 : 128
MBUF_9K : 512
TRAFFIC_MBUF_64 : 65520
TRAFFIC_MBUF_128 : 32760
TRAFFIC_MBUF_256 : 8190
TRAFFIC_MBUF_512 : 8190
TRAFFIC_MBUF_1024 : 8190
TRAFFIC_MBUF_2048 : 65520
TRAFFIC_MBUF_4096 : 128
TRAFFIC_MBUF_9K : 512
MBUF_DP_FLOWS : 524288
MBUF_GLOBAL_FLOWS : 5120
master thread : 0
rx thread : 12
dual_if : 0
socket : 0
[ 1 2 3 4 5 24 25 26 27 28 29 ]
dual_if : 1
socket : 0
[ 6 7 8 9 10 11 30 31 32 33 34 ]
CTimerWheelYamlInfo does not exist
flags : 8010f00
write_file : 0
verbose : 7
realtime : 1
flip : 0
cores : 1
single core : 0
flow-flip : 0
no clean close : 0
zmq_publish : 1
vlan mode : 0
client_cfg : 0
mbuf_cache_disable : 0
cfg file : cap2/dns.yaml
mac file :
out file :
client cfg file :
duration : 3600
factor : 1
mbuf_factor : 0
latency : 0 pkt/sec
zmq_port : 4500
telnet_port : 4501
expected_ports : 4
tw_bucket_usec : 20.000000 usec
tw_buckets : 1024 usec
tw_levels : 3 usec
port : 0 dst:00:00:00:00:00:00 src:00:00:00:00:00:00
port : 1 dst:00:00:00:00:00:00 src:00:00:00:00:00:00
port : 2 dst:00:00:00:00:00:00 src:00:00:00:00:00:00
port : 3 dst:00:00:00:00:00:00 src:00:00:00:00:00:00
port : 4 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 5 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 6 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 7 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 8 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 9 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 10 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 11 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 12 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 13 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 14 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 15 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
Total Memory :
MBUF_64 : 67485
MBUF_128 : 33742
MBUF_256 : 9172
MBUF_512 : 9172
MBUF_1024 : 9172
MBUF_2048 : 66011
MBUF_4096 : 2048
MBUF_9K : 2048
MBUF_DP_FLOWS : 62914
MBUF_GLOBAL_FLOWS : 614
get_each_core_dp_flows : 31457
Total memory : 27.52 Mbytes
core_mask 1043
sockets : 0
active sockets : 1
ports_sockets : 1
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
phy | virt
1 1
6 2
DPDK args
xx -c 0x1043 -n 4 --log-level 8 --master-lcore 0 -w 0000:05:00.0 -w 0000:05:00.1 -w 0000:05:00.2 -w 0000:05:00.3
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 2 on socket 0
EAL: Detected lcore 3 as core 3 on socket 0
EAL: Detected lcore 4 as core 4 on socket 0
EAL: Detected lcore 5 as core 5 on socket 0
EAL: Detected lcore 6 as core 8 on socket 0
EAL: Detected lcore 7 as core 9 on socket 0
EAL: Detected lcore 8 as core 10 on socket 0
EAL: Detected lcore 9 as core 11 on socket 0
EAL: Detected lcore 10 as core 12 on socket 0
EAL: Detected lcore 11 as core 13 on socket 0
EAL: Detected lcore 12 as core 0 on socket 1
EAL: Detected lcore 13 as core 1 on socket 1
EAL: Detected lcore 14 as core 2 on socket 1
EAL: Detected lcore 15 as core 3 on socket 1
EAL: Detected lcore 16 as core 4 on socket 1
EAL: Detected lcore 17 as core 5 on socket 1
EAL: Detected lcore 18 as core 8 on socket 1
EAL: Detected lcore 19 as core 9 on socket 1
EAL: Detected lcore 20 as core 10 on socket 1
EAL: Detected lcore 21 as core 11 on socket 1
EAL: Detected lcore 22 as core 12 on socket 1
EAL: Detected lcore 23 as core 13 on socket 1
EAL: Detected lcore 24 as core 0 on socket 0
EAL: Detected lcore 25 as core 1 on socket 0
EAL: Detected lcore 26 as core 2 on socket 0
EAL: Detected lcore 27 as core 3 on socket 0
EAL: Detected lcore 28 as core 4 on socket 0
EAL: Detected lcore 29 as core 5 on socket 0
EAL: Detected lcore 30 as core 8 on socket 0
EAL: Detected lcore 31 as core 9 on socket 0
EAL: Detected lcore 32 as core 10 on socket 0
EAL: Detected lcore 33 as core 11 on socket 0
EAL: Detected lcore 34 as core 12 on socket 0
EAL: Detected lcore 35 as core 13 on socket 0
EAL: Detected lcore 36 as core 0 on socket 1
EAL: Detected lcore 37 as core 1 on socket 1
EAL: Detected lcore 38 as core 2 on socket 1
EAL: Detected lcore 39 as core 3 on socket 1
EAL: Detected lcore 40 as core 4 on socket 1
EAL: Detected lcore 41 as core 5 on socket 1
EAL: Detected lcore 42 as core 8 on socket 1
EAL: Detected lcore 43 as core 9 on socket 1
EAL: Detected lcore 44 as core 10 on socket 1
EAL: Detected lcore 45 as core 11 on socket 1
EAL: Detected lcore 46 as core 12 on socket 1
EAL: Detected lcore 47 as core 13 on socket 1
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 48 lcore(s)
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or directory)
EAL: VFIO modules not loaded, skipping VFIO support...
EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or directory)
EAL: Setting up physically contiguous memory...
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f8f34a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x3000000 bytes
EAL: Virtual area found at 0x7f8f31800000 (size = 0x3000000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f8f31400000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f8f31000000 (size = 0x200000)
EAL: Ask a virtual area of 0xc800000 bytes
EAL: Virtual area found at 0x7f8f24600000 (size = 0xc800000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f8f24200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f8f23e00000 (size = 0x200000)
EAL: Ask a virtual area of 0xfc00000 bytes
EAL: Virtual area found at 0x7f8f14000000 (size = 0xfc00000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f8f13c00000 (size = 0x200000)
EAL: Requesting 128 pages of size 2MB from socket 0
EAL: Requesting 128 pages of size 2MB from socket 1
EAL: TSC frequency is ~2497115 KHz
EAL: Master lcore 0 is ready (tid=36652980;cpuset=[0])
EAL: lcore 1 is ready (tid=133fe700;cpuset=[1])
EAL: lcore 6 is ready (tid=12bfd700;cpuset=[6])
EAL: lcore 12 is ready (tid=123fc700;cpuset=[12])
EAL: PCI device 0000:05:00.0 on NUMA socket 0
EAL: probe driver: 8086:1572 net_i40e
EAL: PCI memory mapped at 0x7f8f10bfc000
EAL: PCI memory mapped at 0x7f8f36668000
PMD: eth_i40e_dev_init(): FW 5.0 API 1.5 NVM 05.00.05 eetrack 80002af7
PMD: i40e_dcb_init_configure(): Failed to stop lldp
EAL: PCI device 0000:05:00.1 on NUMA socket 0
EAL: probe driver: 8086:1572 net_i40e
EAL: PCI memory mapped at 0x7f8f0b000000
EAL: PCI memory mapped at 0x7f8f36660000
PMD: eth_i40e_dev_init(): FW 5.0 API 1.5 NVM 05.00.05 eetrack 80002af7
PMD: i40e_dcb_init_configure(): Failed to stop lldp
EAL: PCI device 0000:05:00.2 on NUMA socket 0
EAL: probe driver: 8086:1572 net_i40e
EAL: PCI memory mapped at 0x7f8f0a000000
EAL: PCI memory mapped at 0x7f8f36615000
PMD: eth_i40e_dev_init(): FW 5.0 API 1.5 NVM 05.00.05 eetrack 80002af7
PMD: i40e_dcb_init_configure(): Failed to stop lldp
EAL: PCI device 0000:05:00.3 on NUMA socket 0
EAL: probe driver: 8086:1572 net_i40e
EAL: PCI memory mapped at 0x7f8f09000000
EAL: PCI memory mapped at 0x7f8f3660d000
PMD: eth_i40e_dev_init(): FW 5.0 API 1.5 NVM 05.00.05 eetrack 80002af7
PMD: i40e_dcb_init_configure(): Failed to stop lldp
set driver name net_i40e
input : [0000:05:00.0, 0000:05:00.1, 0000:05:00.2, 0000:05:00.3]
dpdk : [0000:05:00.0, 0000:05:00.1, 0000:05:00.2, 0000:05:00.3]
pci_scan : [0000:05:00.0, 0000:05:00.1, 0000:05:00.2, 0000:05:00.3]
map : [ 0, 1, 2, 3]
TRex port mapping
-----------------
TRex vport: 0 dpdk_rte_eth: 0
TRex vport: 1 dpdk_rte_eth: 1
TRex vport: 2 dpdk_rte_eth: 2
TRex vport: 3 dpdk_rte_eth: 3

no client generator pool configured, using default pool
no server generator pool configured, using default pool

flags : 8010f00
write_file : 0
verbose : 7
realtime : 1
flip : 0
cores : 1
single core : 0
flow-flip : 0
no clean close : 0
zmq_publish : 1
vlan mode : 0
client_cfg : 0
mbuf_cache_disable : 0
cfg file : cap2/dns.yaml
mac file :
out file :
client cfg file :
duration : 3600
factor : 1
mbuf_factor : 0
latency : 0 pkt/sec
zmq_port : 4500
telnet_port : 4501
expected_ports : 4
tw_bucket_usec : 20.000000 usec
tw_buckets : 1024 usec
tw_levels : 3 usec
port : 0 dst:00:00:00:00:00:00 src:00:00:00:00:00:00
port : 1 dst:00:00:00:00:00:00 src:00:00:00:00:00:00
port : 2 dst:00:00:00:00:00:00 src:00:00:00:00:00:00
port : 3 dst:00:00:00:00:00:00 src:00:00:00:00:00:00
port : 4 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 5 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 6 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 7 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 8 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 9 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 10 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 11 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 12 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 13 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 14 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
port : 15 dst:00:00:00:01:00:00 src:00:00:00:00:00:00
Total Memory :
MBUF_64 : 67485
MBUF_128 : 33742
MBUF_256 : 9172
MBUF_512 : 9172
MBUF_1024 : 9172
MBUF_2048 : 66011
MBUF_4096 : 2048
MBUF_9K : 2048
MBUF_DP_FLOWS : 62914
MBUF_GLOBAL_FLOWS : 614
get_each_core_dp_flows : 31457
Total memory : 27.52 Mbytes

zmq publisher at: tcp://*:4500
Number of ports found: 4

if_index : 0
driver name : net_i40e
min_rx_bufsize : 1024
max_rx_pktlen : 9728
max_rx_queues : 192
max_tx_queues : 192
max_mac_addrs : 64
rx_offload_capa : 0x2f
tx_offload_capa : 0x1fbf
rss reta_size : 512
flow_type_rss : 0x7ef8
port 0: FW ver 05.00.05
port 1: FW ver 05.00.05
port 2: FW ver 05.00.05
port 3: FW ver 05.00.05

ERROR there is not enough huge-pages memory in your system
EAL: Error - exiting with code: 1
Cause: Cannot init mbuf pool _2048-pkt-const

[root@localhost v.cl2017]#
[root@localhost v.cl2017]# ./t-rex-64 -f cap2/dns.yaml —limit-ports 2

The ports are bound/configured.
Starting TRex v2.25 please wait ...
set driver name net_i40e
no client generator pool configured, using default pool
no server generator pool configured, using default pool
zmq publisher at: tcp://*:4500
Number of ports found: 4
ERROR there is not enough huge-pages memory in your system
EAL: Error - exiting with code: 1
Cause: Cannot init mbuf pool _2048-pkt-const

[root@localhost v.cl2017]#

Thanks for the assistance

[hanoh haim]

For some reason you don’t have contiguous memory (2GB) 

Have a look here how to solve this using grub

http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html

Hanoh

To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/937614f2-4a8e-45da-aec8-9f5997bd7c91%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--