8 Port TREX configuration

santhosh kumar

unread,

May 23, 2016, 5:35:33 PM5/23/16

to TRex Traffic Generator

Hi,

I get the following error while running Trex with 8 ports.

root@dut1:~/trex_2.0/v2.00# ./t-rex-64 --cfg /etc/trex_8pcfg.yaml -f dpdk_cfg/l3fwd_64.yaml -c 8

Starting TRex v2.00 please wait ...

WARNING not enough mbuf memory for this configuration trying to auto update

2 : 1.333008

WARNING not enough mbuf memory for this configuration trying to auto update

3 : 1.333008

ERROR number of dual if 4 is higher than defined in configuration file 2

ERROR, the number of threads per dual ports should be the same for all dual ports

# cat /etc/trex_8pcfg.yaml

- version : 2

port_limit : 8

interfaces : ["04:00.0","04:00.1","06:00.0","06:00.1","07:00.0","07:00.1","07:00.2","07:00.3"]

c : 15

# for system of 1Gb/sec NIC or VM enable this

port_bandwidth_gb : 10 # port bandwidth 10Gb/sec , for VM put here 1 for XL710 put 40

platform :

master_thread_id : 16

latency_thread_id : 17

dual_if :

- socket : 0

threads : [0,1,2,3,4,5,6,7]

- socket : 1

threads : [8,9,10,11,12,13,14,15]

port_info : # set eh mac addr

- dest_mac : [0x68,0x05,0xCA,0x19,0xEF,0xD2]

src_mac : [0x02,0x00,0x00,0x00,0x00,0x00] # port 0

- dest_mac : [0x68,0x05,0xCA,0x19,0xEF,0xD3]

src_mac : [0x02,0x00,0x00,0x00,0x00,0x01] # port 1

- dest_mac : [0x68,0x05,0xCA,0x19,0xEF,0xD0]

src_mac : [0x02,0x00,0x00,0x00,0x00,0x02] # port 2

- dest_mac : [0x68,0x05,0xCA,0x19,0xEF,0xD1]

src_mac : [0x02,0x00,0x00,0x00,0x00,0x03] # port 3

- dest_mac : [0x3C,0xFD,0xFE,0x9C,0xE6,0xD8]

src_mac : [0x02,0x00,0x00,0x00,0x00,0x04] # port 4

- dest_mac : [0x3C,0xFD,0xFE,0x9C,0xE6,0xD9]

src_mac : [0x02,0x00,0x00,0x00,0x00,0x05] # port 5

- dest_mac : [0x3C,0xFD,0xFE,0x9C,0xE6,0xDA]

src_mac : [0x02,0x00,0x00,0x00,0x00,0x06] # port 6

- dest_mac : [0x3C,0xFD,0xFE,0x9C,0xE6,0xDB]

src_mac : [0x02,0x00,0x00,0x00,0x00,0x07] # port 7

Thanks

santhosh

hanoh haim

unread,

May 24, 2016, 12:50:07 AM5/24/16

to santhosh kumar, TRex Traffic Generator

Hi,

1. Better to set master at core zero

2. Try to reduce the -c to 4 let me know if it helps

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/a10927bf-233d-49d0-a3f3-abc9b8d16a22%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Hanoh

Sent from my iPhone

hanoh haim

unread,

May 24, 2016, 1:20:59 AM5/24/16

to santhosh kumar, TRex Traffic Generator

Hi,

The platform section is wrong.

1. If you have 16 cores for DP you can have c=4 at max (number of dual ports are 4 multiply by 4)

2. In platform section you should have for each dual port a section >> 4 sections. you have only two section.

3. If you remove this section it will work with default value that might not be optimized for NUMA

We have similar setup we will send our config file.

Thanks,

Hanoh

Yaroslav Brustinov

unread,

May 24, 2016, 9:35:47 AM5/24/16

to hanoh haim, santhosh kumar, TRex Traffic Generator

Hi,

Our NIC 1 is on first NUMA and NIC 2 is on second NUMA.

lscpu output:

...

NUMA node0 CPU(s): 0-7,16-23

NUMA node1 CPU(s): 8-15,24-31

/etc/trex_cfg.yaml:

- port_limit : 8

version : 2

interfaces : ["06:00.0", "06:00.1", "06:00.2", "06:00.3", "84:00.0", "84:00.1", "84:00.2", "84:00.3"] # list of the interfaces to bind run ./dpdk_nic_bind.py --status

c : 3

port_info : # set eh mac addr

- dest_mac : [0x00, 0xe0, 0xed, 0x5d, 0x84, 0x65]

src_mac : [0x00, 0xe0, 0xed, 0x5d, 0x84, 0x64]

- dest_mac : [0x00, 0xe0, 0xed, 0x5d, 0x84, 0x64]

src_mac : [0x00, 0xe0, 0xed, 0x5d, 0x84, 0x65]

- dest_mac : [0x00, 0xe0, 0xed, 0x5d, 0x84, 0x67]

src_mac : [0x00, 0xe0, 0xed, 0x5d, 0x84, 0x66]

- dest_mac : [0x00, 0xe0, 0xed, 0x5d, 0x84, 0x66]

src_mac : [0x00, 0xe0, 0xed, 0x5d, 0x84, 0x67]

- dest_mac : [0x00, 0xe0, 0xed, 0x5d, 0x84, 0x69]

src_mac : [0x00, 0xe0, 0xed, 0x5d, 0x84, 0x68]

- dest_mac : [0x00, 0xe0, 0xed, 0x5d, 0x84, 0x68]

src_mac : [0x00, 0xe0, 0xed, 0x5d, 0x84, 0x69]

- dest_mac : [0x00, 0xe0, 0xed, 0x5d, 0x84, 0x6b]

src_mac : [0x00, 0xe0, 0xed, 0x5d, 0x84, 0x6a]

- dest_mac : [0x00, 0xe0, 0xed, 0x5d, 0x84, 0x6a]

src_mac : [0x00, 0xe0, 0xed, 0x5d, 0x84, 0x6b]

platform :

master_thread_id : 0

latency_thread_id : 8

dual_if :

- socket : 0

threads : [1, 2, 3]

- socket : 0

threads : [4, 5, 6]

- socket : 1

threads : [9, 10, 11]

- socket : 1

threads : [12, 13, 14]

To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/CA%2BYxBoJPom15RhiAt2CBOgQMADp8ZYrtbKh%3D_YtTGoLcSWqdgg%40mail.gmail.com.

santhosh kumar

unread,

May 24, 2016, 6:34:43 PM5/24/16

to TRex Traffic Generator, santh...@gmail.com

Thanks Hanoh,

After removing the platform section I don't see any memory error.

However I see a new error, our NIC1 is X710 and NIC2 is X520 quad port cards and both are connected to NUMA node0.

Starting TRex v2.00 please wait ...

no client generator pool configured, using default pool

no server generator pool configured, using default pool

zmq publisher at: tcp://*:4500

Number of ports found: 8

ERROR all device should have the same type rte_i40e_pmd != rte_ixgbe_pmd

Thanks

santhosh

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/a10927bf-233d-49d0-a3f3-abc9b8d16a22%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Hanoh
Sent from my iPhone

santhosh kumar

unread,

May 24, 2016, 6:38:16 PM5/24/16

to TRex Traffic Generator, hhaim...@gmail.com, santh...@gmail.com

Thanks for the configuration.

both our NIC s are from the same NUMA.

NUMA node0 CPU(s): 0-27

- version : 2

port_limit : 8

interfaces : ["04:00.0","04:00.1","06:00.0","06:00.1","07:00.0","07:00.1","07:00.2","07:00.3"]

c : 4

# for system of 1Gb/sec NIC or VM enable this

port_bandwidth_gb : 10 # port bandwidth 10Gb/sec , for VM put here 1 for XL710 put 40

platform :

master_thread_id : 0

latency_thread_id : 0

dual_if :

- socket : 0

threads : [1,2,3,4]

- socket : 0

threads : [5,6,7,8]

- socket : 0

threads : [9,10,11,12]

- socket : 0

threads : [13,14,15,16]

I tried to change the platform cfg as above and facing a diff issue now.

./t-rex-64 --cfg /etc/trex_8pcfg.yaml -f dpdk_cfg/l3fwd_64.yaml -m 2 -c 4 -d 3

Starting TRex v2.00 please wait ...

no client generator pool configured, using default pool

no server generator pool configured, using default pool

zmq publisher at: tcp://*:4500

Number of ports found: 8

ERROR all device should have the same type rte_i40e_pmd != rte_ixgbe_pmd

NIC1 is fortville an NIC2 is Niantic here.

Thanks

santhosh

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/a10927bf-233d-49d0-a3f3-abc9b8d16a22%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Hanoh
Sent from my iPhone

santhosh kumar

unread,

May 27, 2016, 7:10:45 PM5/27/16

to TRex Traffic Generator, santh...@gmail.com

Hi All,

We are able to send traffic on 8 ports however the global stats are shown for 4 ports alone. I want to know if this can be increased to 8 ports ?

-Per port stats table

per port - limited to 4

ports | 0 | 1 | 2 | 3

-----------------------------------------------------------------------------------------

opackets | 337497731 | 337497773 | 337500452 | 337500483

obytes | 510971482978 | 510971548076 | 510975602568 | 510975649502

ipackets | 0 | 0 | 0 | 0

ibytes | 510958183998 | 510958305118 | 510965180192 | 510965302826

ierrors | 0 | 0 | 0 | 0

oerrors | 0 | 0 | 0 | 0

Tx Bw | 9.84 Gbps | 9.84 Gbps | 9.84 Gbps | 9.84 Gbps

Stopping traffic on port(s) [0, 1, 2, 3, 4, 5, 6, 7]: [SUCCESS]

12.34 [ms]

trex>start -f dpdk_cfg/profile/1518B.py -m 100%

Removing all streams from port(s) [0, 1, 2, 3, 4, 5, 6, 7]: [SUCCESS]

Attaching 2 streams to port(s) [0]: [SUCCESS]

Attaching 2 streams to port(s) [1]: [SUCCESS]

Attaching 2 streams to port(s) [2]: [SUCCESS]

Attaching 2 streams to port(s) [3]: [SUCCESS]

Attaching 2 streams to port(s) [4]: [SUCCESS]

Attaching 2 streams to port(s) [5]: [SUCCESS]

Attaching 2 streams to port(s) [6]: [SUCCESS]

Attaching 2 streams to port(s) [7]: [SUCCESS]

Starting traffic on port(s) [0, 1, 2, 3, 4, 5, 6, 7]: [SUCCESS]

Thanks

santhosh

santhosh kumar

unread,

May 27, 2016, 7:17:17 PM5/27/16

to TRex Traffic Generator, santh...@gmail.com

Hi,

After updating the main_dpdk.cpp "port_to_show=m_num_of_ports;" we see stats for all 8 ports. you can ignore my previous mail.

main_dpdk.cpp:2624: fprintf (fd," per port - limited to 4 \n");

thanks

santhosh

santhosh kumar

unread,

May 27, 2016, 7:22:00 PM5/27/16

to TRex Traffic Generator, santh...@gmail.com

the tui is still showing stats for 4 ports alone..

hanoh haim

unread,

May 28, 2016, 2:08:13 PM5/28/16

to santhosh kumar, TRex Traffic Generator

Hi,

Console is limited to total of 4 concurrent ports. There is an option to choose.

You can test the WIP Stateless GUI (see release notes) there is no limit of number of ports .

Hanoh

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/a10927bf-233d-49d0-a3f3-abc9b8d16a22%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Hanoh
Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/2a05f1cf-ba1a-4aab-8c9b-952aeee02da2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

santhosh kumar

unread,

Jun 1, 2016, 1:02:05 AM6/1/16

to TRex Traffic Generator, santh...@gmail.com

Hi,

Do we have any default profile for 8Port configuration?

Thanks

santhosh

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/a10927bf-233d-49d0-a3f3-abc9b8d16a22%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Hanoh
Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/2a05f1cf-ba1a-4aab-8c9b-952aeee02da2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

hanoh haim

unread,

Jun 1, 2016, 1:50:37 AM6/1/16

to santhosh kumar, TRex Traffic Generator

Hi

Config profile or traffic profile?

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/a10927bf-233d-49d0-a3f3-abc9b8d16a22%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Hanoh
Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/2a05f1cf-ba1a-4aab-8c9b-952aeee02da2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Hanoh
Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/07a8599b-4872-4bef-8d93-1de482c2413d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

santhosh kumar

unread,

Jun 1, 2016, 2:15:02 AM6/1/16

to TRex Traffic Generator, santh...@gmail.com

Hi Hanoch,

Traffic profile for 8Ports.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/a10927bf-233d-49d0-a3f3-abc9b8d16a22%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Hanoh
Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/2a05f1cf-ba1a-4aab-8c9b-952aeee02da2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Hanoh
Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/07a8599b-4872-4bef-8d93-1de482c2413d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

hanoh haim

unread,

Jun 1, 2016, 3:39:44 AM6/1/16

to santhosh kumar, TRex Traffic Generator

have a look here:

http://trex-tgn.cisco.com/trex/doc/trex_stateless.html#_tutorial_advanced_traffic_profile

you can use port_id inside the profile to choose a different profile/val/tunable per port

def create_streams (self, direction = 0, **args):

    port_id = args.get('port_id')

Hi Hanoch,

Traffic profile for 8Ports.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/a10927bf-233d-49d0-a3f3-abc9b8d16a22%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Hanoh
Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/2a05f1cf-ba1a-4aab-8c9b-952aeee02da2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Hanoh
Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/07a8599b-4872-4bef-8d93-1de482c2413d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Hanoh
Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/d599ea3f-0ddf-4d2a-906e-b6898d613378%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Reply all

Reply to author

Forward