Stateless GUI 4.1 display: understand CPU, RxCPU and Queue Full

876 views
Skip to first unread message

Sheena Gu

unread,
Sep 25, 2017, 12:04:11 PM9/25/17
to TRex Traffic Generator
Hi, I am new to Trex. When trying to run traffic with stateless GUI 4.1 connect to Trex server V2.29, Sometime I see Rx CPU with high value in red display like 99.30% for RxCPU, Queue Full value like 3.73 Kpkts in red. Just wonder how the CPU and Rx CPU is calculated, What is Queue Full value mean ?

I am running server with 48 core ubuntu. the htop output seems okay.

sheena


htop output
Uptime: 3 days, 20:03:31

PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command
35073 root 20 0 40.5G 34256 7292 S 281. 0.0 204h ./_t-rex-64 -i
35076 root 20 0 40.5G 34256 7292 R 101. 0.0 67h59:02 ./_t-rex-64 -i
35077 root 20 0 40.5G 34256 7292 R 91.8 0.0 67h46:54 ./_t-rex-64 -i
35075 root 20 0 40.5G 34256 7292 R 87.9 0.0 67h41:52 ./_t-rex-64 -i
5938 labuser 20 0 29264 4672 3088 R 1.3 0.0 0:00.19 htop
35080 root 20 0 40.5G 34256 7292 S 0.7 0.0 2:13.99 ./_t-rex-64 -i
3314 root 20 0 20084 2888 1996 S 0.7 0.0 3:33.91 /usr/sbin/irqbalanc


server output on standard IO:=============


Per port stats table
ports | 0 | 1 | 2 | 3
-----------------------------------------------------------------------------------------
opackets | 164782003799 | 165737379356 | 150460569242 | 150320793577
obytes | 168736771888260 | 169715076617216 | 153921162334566 | 153928492623872
ipackets | 165745733000 | 164782003797 | 150320793556 | 150460569251
ibytes | 169652538393906 | 168736771888128 | 153928492602368 | 153921162343773
ierrors | 0 | 0 | 0 | 0
oerrors | 0 | 0 | 0 | 0
Tx Bw | 457.86 Mbps | 3.95 Gbps | 9.15 Gbps | 9.15 Gbps

-Global stats enabled
Cpu Utilization : 72.6 % 31.3 Gb/core
Platform_factor : 1.0
Total-Tx : 22.71 Gbps
Total-Rx : 22.71 Gbps
Total-PPS : 2.77 Mpps
Total-CPS : 0.00 cps

Expected-PPS : 0.00 pps
Expected-CPS : 0.00 cps
Expected-BPS : 0.00 bps

Active-flows : 0 Clients : 0 Socket-util : 0.0000 %
Open-flows : 0 Servers : 0 Socket : 0 Socket/Clients : -nan
Total_queue_full : 3728
drop-rate : 0.00 bps
current time : 245076.8 sec
test duration : 0.0 sec

hanoh haim

unread,
Sep 25, 2017, 12:23:09 PM9/25/17
to Sheena Gu, TRex Traffic Generator
See here

You probably overuse the latency/per stream stats (done by Rx core)

Thanks,
Hanoh

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/6a968a8e-b3ed-419e-94ce-0c0cd19753df%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Hanoh
Sent from my iPhone

Sheena Gu

unread,
Sep 25, 2017, 1:43:56 PM9/25/17
to TRex Traffic Generator
Thanks Hanoh for the quick response. I have 4 10g port running full speed traffic, I did not enable the Rx Stats on any stream. All the traffic has RX stats disabled by default.

hanoh haim

unread,
Sep 25, 2017, 2:01:39 PM9/25/17
to Sheena Gu, TRex Traffic Generator
Could you send the profile?
Another option that service mode is enabled?
Please send a snapshot of the GUI


Thanks,
Hanoh


For more options, visit https://groups.google.com/d/optout.
--

Sheena Gu

unread,
Sep 26, 2017, 10:20:50 AM9/26/17
to TRex Traffic Generator
On Monday, September 25, 2017 at 2:01:39 PM UTC-4, Hanoch Haim wrote: > Could you send the profile? > Another option that service mode is enabled? > Please send a snapshot of the GUI > > > > > Thanks, > Hanoh > > > On Mon, 25 Sep 2017 at 20:43 Sheena Gu <sheen...@gmail.com> wrote: > On Monday, September 25, 2017 at 12:23:09 PM UTC-4, Hanoch Haim wrote: > > > See here > > > https://trex-tgn.cisco.com/trex/doc/cp_stl_docs/api/client_code.html#trex_stl_lib.trex_stl_client.STLClient.get_stats > > > > > > > > > > > > You probably overuse the latency/per stream stats (done by Rx core) > > > > > > > > > Thanks, > > > Hanoh > > > > > > > > > On Mon, 25 Sep 2017 at 19:04 Sheena Gu <sheen...@gmail.com> wrote: > > > Hi, I am new to Trex. When trying to run traffic with stateless GUI 4.1 connect to Trex server V2.29, Sometime I see Rx CPU with high value in red display like 99.30% for RxCPU, Queue Full value like 3.73 Kpkts in red. Just wonder how the CPU and Rx CPU is calculated, What is Queue Full value mean ? > > > > > > > > > > > > I am running server with 48 core ubuntu. the htop output seems okay. > > > > > > > > > > > > sheena > > > > > > > > > > > > > > > > > > htop output > > > > > >                                                                      Uptime: 3 days, 20:03:31 > > > > > > > > > > > >   PID USER      PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  Command > > > > > > 35073 root       20   0 40.5G 34256  7292 S 281.  0.0     204h ./_t-rex-64 -i > > > > > > 35076 root       20   0 40.5G 34256  7292 R 101.  0.0 67h59:02 ./_t-rex-64 -i > > > > > > 35077 root       20   0 40.5G 34256  7292 R 91.8  0.0 67h46:54 ./_t-rex-64 -i > > > > > > 35075 root       20   0 40.5G 34256  7292 R 87.9  0.0 67h41:52 ./_t-rex-64 -i > > > > > >  5938 labuser    20   0 29264  4672  3088 R  1.3  0.0  0:00.19 htop > > > > > > 35080 root       20   0 40.5G 34256  7292 S  0.7  0.0  2:13.99 ./_t-rex-64 -i > > > > > >  3314 root       20   0 20084  2888  1996 S  0.7  0.0  3:33.91 /usr/sbin/irqbalanc > > > > > > > > > > > > > > > > > > server output on standard IO:============= > > > > > > > > > > > > > > > > > > Per port stats table > > > > > >       ports |               0 |               1 |               2 |               3 > > > > > >  ----------------------------------------------------------------------------------------- > > > > > >    opackets |    164782003799 |    165737379356 |    150460569242 |    150320793577 > > > > > >      obytes | 168736771888260 | 169715076617216 | 153921162334566 | 153928492623872 > > > > > >    ipackets |    165745733000 |    164782003797 |    150320793556 |    150460569251 > > > > > >      ibytes | 169652538393906 | 168736771888128 | 153928492602368 | 153921162343773 > > > > > >     ierrors |               0 |               0 |               0 |               0 > > > > > >     oerrors |               0 |               0 |               0 |               0 > > > > > >       Tx Bw |     457.86 Mbps |       3.95 Gbps |       9.15 Gbps |       9.15 Gbps > > > > > > > > > > > > -Global stats enabled > > > > > >  Cpu Utilization : 72.6  %  31.3 Gb/core > > > > > >  Platform_factor : 1.0 > > > > > >  Total-Tx        :      22.71 Gbps > > > > > >  Total-Rx        :      22.71 Gbps > > > > > >  Total-PPS       :       2.77 Mpps > > > > > >  Total-CPS       :       0.00  cps > > > > > > > > > > > >  Expected-PPS    :       0.00  pps > > > > > >  Expected-CPS    :       0.00  cps > > > > > >  Expected-BPS    :       0.00  bps > > > > > > > > > > > >  Active-flows    :        0  Clients :        0   Socket-util : 0.0000 % > > > > > >  Open-flows      :        0  Servers :        0   Socket :        0 Socket/Clients :  -nan > > > > > >  Total_queue_full : 3728 > > > > > >  drop-rate       :       0.00  bps > > > > > >  current time    : 245076.8 sec > > > > > >  test duration   : 0.0 sec > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com. > > > > > > To post to this group, send email to trex...@googlegroups.com. > > > > > > To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/6a968a8e-b3ed-419e-94ce-0c0cd19753df%40googlegroups.com. > > > > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > -- > > > > > > > > > > > > HanohSent from my iPhone > > > > Thanks Hanoh for the quick response.  I have 4 10g port running  full speed traffic, I did not enable the Rx Stats on any stream.  All the traffic has RX stats disabled by default. > > > > -- > > You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com. > > To post to this group, send email to trex...@googlegroups.com. > > To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/ec026a12-4d8c-4213-84e6-3ded02bbc4ee%40googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > > -- > > > > HanohSent from my iPhone
TrexPortstats.JPG
Port1 (Eth11).json
port2(Eth16).json
port4(Eth21).json
Port0 (Eth6).json

hanoh haim

unread,
Sep 26, 2017, 12:58:37 PM9/26/17
to Sheena Gu, TRex Traffic Generator
In the port configuration page look for service mode knob.It is probably enabled. This create the high Rx CPU.
Queue full is because you reached the NIC limitations 

Thanks,
Hanoh


For more options, visit https://groups.google.com/d/optout.
--
Reply all
Reply to author
Forward
0 new messages