Crash observed while performing load testing

Vaibhav Sawke

unread,

Jul 1, 2021, 1:08:11 PM7/1/21

to UniMRCP

Hello All,

I am writing a custom Azure SR plugin for UniMRCP.

We used SIPp to perform load testing and the parameters used were r = 15 l =80 m=500000 (80 concurrent calls and hit rate is 15 calls per second.).

However, the UniMrcp server crashed after crossing 4Lac Calls in total.

Also, we executed the Valgrind but it did not report any invalid read or write.

Below is the crashed dump stack, the stack is nowhere near the plugin code.

Before the crash the channel destroy was called in the plugin.

RAM and CPU of system utilization were normal.

Please help me understand what possible reasons/mistakes in a plugin can cause the below stack.

crash core dump stack:

[Thread debugging using libthread_db enabled]

Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Core was generated by `./unimrcpserver'.

Program terminated with signal SIGSEGV, Segmentation fault.

#0 run_cleanups (cref=0x79) at memory/unix/apr_pools.c:2431

2431 cleanup_t *c = *cref;

[Current thread is 1 (Thread 0x7ffa5f105700 (LWP 24367))]

(gdb) bt

#0 run_cleanups (cref=0x79) at memory/unix/apr_pools.c:2431

#1 apr_pool_destroy (pool=0x1) at memory/unix/apr_pools.c:835

#2 0x00007ffa638a8d2d in apr_pool_destroy (pool=0x7ffa10106ab8) at memory/unix/apr_pools.c:842

#3 0x00007ffa63ae8728 in mrcp_session_destroy (session=session@entry=0x7ffa10106b88) at src/mrcp_sig_agent.c:131

#4 0x00007ffa63adb14b in mrcp_sofia_on_session_terminate (session=0x7ffa10106b88) at src/mrcp_sofiasip_server_agent.c:328

#5 0x00007ffa63ae612d in mrcp_session_terminate_response (session=0x7ffa10106b88) at ../../libs/mrcp-signaling/include/mrcp_session.h:155

#6 mrcp_server_session_terminate_send (session=0x7ffa10106b88) at src/mrcp_server_session.c:1048

#7 0x00007ffa63ae78a4 in mrcp_server_session_subrequest_remove (session=<optimized out>) at src/mrcp_server_session.c:309

#8 0x00007ffa63ae8454 in mrcp_server_session_subrequest_remove (session=<optimized out>) at src/mrcp_server_session.c:296

#9 mrcp_server_mpf_message_process (mpf_message_container=0x7ff9fc0c3510) at src/mrcp_server_session.c:1190

#10 0x00007ffa63ae44f6 in mrcp_server_msg_process (task=<optimized out>, msg=<optimized out>) at src/mrcp_server.c:858

#11 0x00007ffa63affe56 in apt_task_msg_process (task=0x561d3b66ab48, msg=0x7ff9fc0c3500) at src/apt_task.c:342

#12 0x00007ffa63b00201 in apt_consumer_task_run (task=<optimized out>) at src/apt_consumer_task.c:145

#13 0x00007ffa63aff410 in apt_task_run (thread_handle=0x561d3b69ee08, data=0x561d3b66ab48) at src/apt_task.c:589

#14 0x00007ffa627236db in start_thread (arg=0x7ffa5f105700) at pthread_create.c:463

#15 0x00007ffa635bc71f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Versions for UniMRCP:

****************************** REPORT ******************************

UniMRCP version............... : 1.7.0

APR version................... : 1.5.2

APR-util version.............. : 1.5.4

Sofia-SIP version............. : 1.12.11-239-g54ef3e2

Compiler...................... : gcc

Compiler flags................ : -g -O2 -pthread

Preprocessor definitions...... : -DLINUX -D_REENTRANT -D_GNU_SOURCE

Linker flags.................. :

UniMRCP client lib............ : yes

Sample UniMRCP client app..... : yes

Sample UMC C++ client app..... : yes

Misc ASR client lib and app... : yes

UniMRCP server lib............ : yes

UniMRCP server app............ : yes

Demo synthesizer plugin....... : yes

Demo recognizer plugin........ : yes

Demo verifier plugin.......... : yes

Recorder plugin............... : yes

Installation layout........... : classic

Installation directory........ : /usr/local/unimrcp

********************************************************************

Thanks,
Vaibhav Sawke

Vaibhav Sawke

unread,

Jul 8, 2021, 12:58:08 AM7/8/21

to UniMRCP

Can anyone help me with this issue?

Arsen Chaloyan

unread,

Aug 24, 2021, 9:26:36 PM8/24/21

to UniMRCP

The problem seems to be in the use of APR pools, which is commonly referenced by different entities, including the plugin.

--
You received this message because you are subscribed to the Google Groups "UniMRCP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unimrcp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/unimrcp/fa08a7f1-5d51-477f-a12f-564dbaac80e6n%40googlegroups.com.