Observing TreX traceback and core dump while sending EMIX traffic, how can we avoid the same !!

13 views

Skip to first unread message

Ravi Ranjan (raviran)

unread,

Jul 9, 2019, 9:15:45 PM7/9/19

to trex...@googlegroups.com, Lei Xia (leixia), hanoh haim

Hello Team,

I am new to Trex and using it to test EMIX and EFLOW profiles, can you please help with below issue.

This is kind of rare occurrences, I ran couple of times and then only see the issue.

The Trex are 1Gi ports connected to UUT through switch. Let me know if you need any more information.

I am running below commands:

sudo ./t-rex-64 --astf -f astf/sfr_full.py -d 360000 -c 1 -l 1000 --cfg /etc/trex_cfg.yaml -m 100 --learn-mode 1

Trex version: v2.53

-Per port stats table

ports | 0 | 1

-----------------------------------------------------------------------------------------

opackets | 21898137 | 1894768

obytes | 8764872270 | 665132596

ipackets | 1059786 | 1176607

ibytes | 409268496 | 708629867

ierrors | 650 | 483

oerrors | 0 | 0

Tx Bw | 420.98 Mbps | 39.36 Mbps

-Global stats enabled

Cpu Utilization : 72.4 % 1.3 Gb/core

Platform_factor : 1.0

Total-Tx : 460.33 Mbps

Total-Rx : 62.64 Mbps

Total-PPS : 126.69 Kpps

Total-CPS : 23.56 Kcps

Expected-PPS : 0.00 pps

Expected-CPS : 0.00 cps

Expected-L7-BPS : 0.00 bps

Active-flows : 262145 Clients : 255 Socket-util : 1.6324 %

Open-flows : 5417510 Servers : 65535 Socket : 262145 Socket/Clients : 1028.0

Total_queue_full : 3410772

drop-rate : 397.70 Mbps

current time : 93.4 sec

test duration : 359906.7 sec

-Latency stats enabled

Cpu Utilization : 0.0 %

if| tx_ok , rx_ok , rx check ,error, latency (usec) , Jitter max window

| , , , , average , max , (usec)

----------------------------------------------------------------------------------------------------------------

0 | 91240, 7395, 0, 1501, 61733 , 148286, 3845 | 69280 67902 69170 70938 63955 66906 68366 63340 67139 66425 70189 65118 68533

1 | 9592, 535, 0, 9380, 71096 , 290281, 3336 | 71486 71917 74756 77870 70374 71955 73549 79969 73232 71986 74825 73450 74196

WATCHDOG: task 'Trex DP core 1' has not responded for more than 1.00055 seconds - timeout is 1 seconds

*** traceback follows ***

1 0x7faf6b9569aa ./_t-rex-64(+0x15c9aa) [0x7faf6b9569aa]

2 0x7faf6acbd340 /lib/x86_64-linux-gnu/libpthread.so.0(+0x10340) [0x7faf6acbd340]

3 0x7faf6bba3a5a rte_delay_us_block + 106

4 0x7faf6b93adb6 CCoreEthIF::send_burst(CCorePerPort*, unsigned short, CVirtualIFPerSideStats*) + 198

5 0x7faf6b9466b3 CCoreEthIFTcp::send_node(CGenNode*) + 355

6 0x7faf6b904855 CTcpIOCb::on_tx(CTcpPerThreadCtx*, tcpcb*, rte_mbuf*) + 37

7 0x7faf6b8ff74c CEmulApp::next() + 572

8 0x7faf6b900d99 CEmulApp::on_tick() + 201

9 0x7faf6b8f3bbc ./_t-rex-64(+0xf9bbc) [0x7faf6b8f3bbc]

10 0x7faf6b9a6b7f CNATimerWheel::on_tick_level_count(int, void*, void (*)(void*, CHTimerObj*), unsigned short, unsigned int&) + 143

11 0x7faf6b8f51d4 CTcpPerThreadCtx::timer_w_on_tick() + 68

12 0x7faf6b905dee CFlowGenListPerThread::handle_tw(CGenNode*, bool) + 110

13 0x7faf6b92944f CNodeGenerator::handle_slow_messages(unsigned char, CGenNode*, CFlowGenListPerThread*, bool) + 415

14 0x7faf6b8f0d1f int CNodeGenerator::flush_file_realtime<24, false>(double, double, CFlowGenListPerThread*, double&) + 1103

15 0x7faf6bb20fed TrexDpCoreAstfBatch::start_astf() + 1021

16 0x7faf6bb21509 TrexDpCoreAstfBatch::start_scheduler() + 9

17 0x7faf6ba912b9 TrexDpCore::start() + 169

18 0x7faf6b91eff3 CFlowGenListPerThread::start(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, CPreviewMode&) + 115

19 0x7faf6b942b66 CGlobalTRex::run_in_core(unsigned char) + 326

20 0x7faf6b94a3fd ./_t-rex-64(+0x1503fd) [0x7faf6b94a3fd]

21 0x7faf6bbb65dd eal_thread_loop + 477

22 0x7faf6acb5182 /lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7faf6acb5182]

23 0x7faf69d2730d clone + 109

*** addr2line information follows ***

??:0

./t-rex-64: line 80: 13637 Aborted (core dumped) ./_$(basename $0) $INPUT_ARGS $EXTRA_INPUT_ARGS

Thanks,

Ravi.

Reply all

Reply to author

Forward

0 new messages