High Packet Drop Rate

[dheeraj...@gmail.com]

Hi all,

I am trying to run trex v2.20 in openstack.

Guest OS: Ubuntu 14.04 x86_64 with kernel 4.4.0-59-generic.
Host OS: Ubuntu 14.04 x86_64 with kernel 4.4.0-63-generic.
The host has Intel Corporation Ethernet 10G 2P X520 Adapter for SRIOV.
Guest VM has 3 interfaces connected to same network (20.0.0.0/24). Used 2 ports for binding dpdk.

Steps performed:
1. Generate MAC based config file using dpdk_setup_ports.py
2. Start TREX server stateless using ./t-rex-64 -i
3. Start TREX console in another terminal.
4. run stl/udp_1pkt_simple_mac_dst_src.py

I m generating bidirectional traffic between two ports using MAC address.
Using TREX, I m able to get high transmission rate but I m facing a heavy drop rate.

Following are observed statistics:

Global Statistics

connection : localhost, Port 4501 total_tx_L2 : 871.08 Mb/sec
version : v2.20 total_tx_L1 : 1.14 Gb/sec
cpu_util. : 96.4% @ 1 cores (1 per port) total_rx : 78.10 Mb/sec
rx_cpu_util. : 4.87% total_pps : 1.70 Mpkt/sec
async_util. : 1.27% / 18.47 KB/sec drop_rate : 792.98 Mb/sec
queue_full : 7,037,306 pkts

Port Statistics

port | 0 | 1 | total
----------------------------------------------------------------------
owner | root | root |
link | UP | UP |
state | TRANSMITTING | TRANSMITTING |
speed | 10 Gb/s | 10 Gb/s |
CPU util. | 96.4% | 96.4% |
-- | | |
Tx bps L2 | 435.54 Mbps | 435.55 Mbps | 871.08 Mbps
Tx bps L1 | 571.64 Mbps | 571.66 Mbps | 1.14 Gbps
Tx pps | 850.65 Kpps | 850.68 Kpps | 1.70 Mpps
Line Util. | 5.72 % | 5.72 % |
--- | | |
Rx bps | 43.70 Mbps | 34.40 Mbps | 78.10 Mbps
Rx pps | 91.05 Kpps | 71.66 Kpps | 162.71 Kpps
---- | | |
opackets | 44114765 | 44118030 | 88232795
ipackets | 4531425 | 4247973 | 8779398
obytes | 2823344952 | 2823553912 | 5646898864
ibytes | 271885644 | 254878500 | 526764144
tx-bytes | 2.82 GB | 2.82 GB | 5.65 GB
rx-bytes | 271.89 MB | 254.88 MB | 526.76 MB
tx-pkts | 44.11 Mpkts | 44.12 Mpkts | 88.23 Mpkts
rx-pkts | 4.53 Mpkts | 4.25 Mpkts | 8.78 Mpkts
----- | | |
oerrors | 0 | 0 | 0
ierrors | 0 | 0 | 0

Am I missing something, can you tell me is there any way to get less drop rate or improved performance.

Thanks,
Dheeraj

[hanoh haim]

Hi,

First it is a bad idea to connect the management to same network under test.

Second the Tx rate is low, you can get up to 10MPPS.

Another thing is the the profile you are using has one destination MAC so it can't work for both ports.

My suggestion is to configure IP to each port and try to ping the other port (see service mode in the manual)

Thanks,

Hanoh

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/958c6fb8-377d-4970-b3e2-9b41fa301e05%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Hanoh

Sent from my iPhone

[ido barnea]

Hi DHeeraj,

We noticed that env with x520 (82599) NICs using SRIOV is very tricky to get right.

You need to follow carefully the instructions from the manual. Very important to have latest ixgbe driver in kernel. Not sure if your Ubuntu version has it.

You can debug using script that send small amount of packets. For example, you can use:

scripts/automation/trex_control_plane/stl/examples/stl_flow_stats.py

See if all packets pass through OK.

We have seen an issue with old ixgbe drivers in which first packet in each burst we send was dropped.

You can also take TRex out of the loop, and try configuring the ports in Linux, and sending few packets between them. After Linux works well, go back to TRex.

Hope that helps,

Ido

To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/CA%2BYxBoJeg9632YA7H7YzBVGC_POKTYmH_2ptOBe3omx2Jd6%2BYw%40mail.gmail.com.