Sofia SIP stack crashes on shutdown and reinitialisation of MRCP client stack
J.W.F. Thirion
I found a strange problem. I create an MRCP client and then shut it down
and destroy the stack. When I start it again, and allow synthesis to
take place, I get the crash (gdb dump below).
I can create and destroy the stack, no problem, but it appears that as
soon as the SIP interactions take place (maybe when the SIP stack gets
initialised the second time), my app. segfault.
I'm using Nuance as the speech server and it works very nicely if I
don't unload/load my module before using the MRCP client the second
time.
Anyone seen this?
Kind regards,
Derik Thirion
#0 0x004ebb27 in initstate_r () from /lib/libc.so.6
(gdb) bt
#0 0x004ebb27 in initstate_r () from /lib/libc.so.6
#1 0x004eb6f2 in initstate () from /lib/libc.so.6
#2 0x086c6600 in init () at su_uniqueid.c:169
#3 0x086c68b5 in su_randmem (mem=0xaf1bbd50, siz=8) at
su_uniqueid.c:312
#4 0x086a0f63 in soa_init_sdp_origin (ss=0x8bde308, o=0xaf1bbd48,
buffer=0xaf1bbcac "��h\bp��\b\020") at soa.c:2189
#5 0x086a1587 in soa_base_set_capability_sdp (ss=0x8bde308,
_sdp=0x5533a64,
str0=0x8bd45a4 "v=0\r\no=Test 0 0 IN IP4 192.168.1.2\r\ns=-\r\nc=IN
IP4 192.168.1.2\r\nt=0 0\r\nm=application 9 TCP/MRCPv2 1\r
\na=setup:active\r\na=connection:new\r\na=resource:speechsynth\r
\na=cmid:1\r\nm=audio 4000 RTP/AVP 0 8 96\r\na"..., len0=298) at
soa.c:910
#6 0x086a4cc2 in soa_static_set_capability_sdp (ss=0x8bde308,
sdp=0xaf1bbddc,
sdp_str=0x8bd45a4 "v=0\r\no=Test 0 0 IN IP4 192.168.1.2\r\ns=-\r
\nc=IN IP4 192.168.1.2\r\nt=0 0\r\nm=application 9 TCP/MRCPv2 1\r
\na=setup:active\r\na=connection:new\r\na=resource:speechsynth\r
\na=cmid:1\r\nm=audio 4000 RTP/AVP 0 8 96\r\na"..., sdp_len=298) at
soa_static.c:245
#7 0x086a1ec0 in soa_set_sdp (ss=0x8bde308,
what=soa_capability_sdp_kind, sdp0=0x8bd45a4,
sdp_str=0x8bd45a4 "v=0\r\no=Test 0 0 IN IP4 192.168.1.2\r\ns=-\r
\nc=IN IP4 192.168.1.2\r\nt=0 0\r\nm=application 9 TCP/MRCPv2 1\r
\na=setup:active\r\na=connection:new\r\na=resource:speechsynth\r
\na=cmid:1\r\nm=audio 4000 RTP/AVP 0 8 96\r\na"..., str_len=298) at
soa.c:2066
#8 0x086a38fd in soa_base_set_params (ss=0x8bde308, tags=0xaf1bbf98) at
soa.c:516
#9 0x086a4f4f in soa_static_set_params (ss=0x8bde308, tags=0xaf1bbf98)
at soa_static.c:190
#10 0x086a06b0 in soa_set_params (ss=0x8bde308, tag=0x87180b0,
value=146621844) at soa.c:408
#11 0x0867526d in nua_stack_set_params (nua=0x8bd4b78, nh=0x8be5620,
e=nua_i_error, tags=0x8bd4594) at nua_params.c:549
#12 0x08667510 in nua_stack_init_handle (nua=0x5533a64, nh=0x8be5620,
tags=0x8bd4594) at nua_stack.c:1001
#13 0x0866b144 in nua_client_init_request (cr=0x8bd46d8) at
nua_stack.c:2315
#14 0x0866db3f in nua_client_create (nh=0x8be5620, event=31,
methods=0x8711b00, tags=0x8bd4594) at nua_stack.c:2052
#15 0x0867a9c5 in nua_stack_invite (nua=0x8bd4b78, nh=0x8be5620,
e=nua_r_invite, tags=0x8bd4594) at nua_session.c:706
#16 0x0866d948 in nua_stack_signal (nua=0x8bd4b78, msg=0xaf1bc308,
ee=0x8bd457c) at nua_stack.c:594
#17 0x086bd89c in su_base_port_execute_msgs (queue=<value optimized
out>) at su_base_port.c:276
#18 0x086bdfdd in su_base_port_run (self=0x8df1788) at
su_base_port.c:331
#19 0x086b6029 in su_root_run (self=0x8bddb30) at su_port.h:310
#20 0x086be9da in su_pthread_port_clone_main (varg=0xb05be204) at
su_pthread_port.c:321
#21 0x0063a5ab in start_thread () from /lib/libpthread.so.0
#22 0x00590cfe in clone () from /lib/libc.so.6
(gdb) frame 1
#1 0x004eb6f2 in initstate () from /lib/libc.so.6
(gdb) frame 2
#2 0x086c6600 in init () at su_uniqueid.c:169
169 initstate(seed[0] ^ seed[1], (char *)&seed, sizeof(seed));
(gdb) frame 3
#3 0x086c68b5 in su_randmem (mem=0xaf1bbd50, siz=8) at
su_uniqueid.c:312
312 if (!initialized) init();
(gdb) frame 4
#4 0x086a0f63 in soa_init_sdp_origin (ss=0x8bde308, o=0xaf1bbd48,
buffer=0xaf1bbcac "��h\bp��\b\020") at soa.c:2189
2189 su_randmem(&o->o_id, sizeof o->o_id);
(gdb) frame 5
#5 0x086a1587 in soa_base_set_capability_sdp (ss=0x8bde308,
_sdp=0x5533a64,
str0=0x8bd45a4 "v=0\r\no=Test 0 0 IN IP4 192.168.1.2\r\ns=-\r\nc=IN
IP4 192.168.1.2\r\nt=0 0\r\nm=application 9 TCP/MRCPv2 1\r
\na=setup:active\r\na=connection:new\r\na=resource:speechsynth\r
\na=cmid:1\r\nm=audio 4000 RTP/AVP 0 8 96\r\na"..., len0=298) at
soa.c:910
910 if (soa_init_sdp_origin(ss, o, c_address) < 0)
(gdb) frame 6
#6 0x086a4cc2 in soa_static_set_capability_sdp (ss=0x8bde308,
sdp=0xaf1bbddc,
sdp_str=0x8bd45a4 "v=0\r\no=Test 0 0 IN IP4 192.168.1.2\r\ns=-\r
\nc=IN IP4 192.168.1.2\r\nt=0 0\r\nm=application 9 TCP/MRCPv2 1\r
\na=setup:active\r\na=connection:new\r\na=resource:speechsynth\r
\na=cmid:1\r\nm=audio 4000 RTP/AVP 0 8 96\r\na"..., sdp_len=298) at
soa_static.c:245
245 return soa_base_set_capability_sdp(ss, sdp, sdp_str, sdp_len);
Arsen Chaloyan
I've never met this problem before. Probably because I don't use the
client stack the way you do. However you should be able to create and
destroy client stack as many times as you need.
So using the backtrace and google search, it seems I know what is
going on and this is known problem.
The following patch should do the trick.
/libsofia-sip-ua/su/su_uniqueid.c 141
- uint32_t *seed = calloc(32, sizeof *seed);
+ uint32_t *seed = calloc(32, sizeof(uint32_t));
/libsofia-sip-ua/su/su_uniqueid.c 174
- initstate(seed[0] ^ seed[1], (char *)&seed, sizeof(seed));
+ initstate(seed[0] ^ seed[1], (char *)seed, 32 * sizeof(uint32_t));
More info
http://www.opensource-archive.org/archive/index.php/t-93732.html
http://fisheye.freeswitch.org/changelog/FreeSWITCH/libs/sofia-sip?cs=11872
Give it a try and let us if it helps.
Arsen Chaloyan
The author of UniMRCP
http://www.unimrcp.org
J.W.F. Thirion
Many thanks. I'm using unimrcp-0.8.0 and unimrcp-deps-0.8.0. Look at
unimrcp-deps-0.8.0/libs/sofia-sip/libsofia-sip-ua/su/su_uniqueid.c, I
could only see one possible way of patching it unimrcp-deps-0.8.0/libs/
sofia-sip/libsofia-sip-ua/su/su_uniqueid.c
169c169
(line 139 of su_uniqueid.c):
static uint32_t seed[32] = { 0 };
and this seems to be of uint32_t type instead of "seed" type.
I've recompiled libs/sofia-sip (make clean, make, make install), and
then recompiled unimrcp-0.8.0 as well as my application, using the
single line patch for line 169. Unfortunately, still no luck. I have
a .so with all of the code in, so the .so has a module load and unload
function, which should initialize and destroy all unimrcp stuff...
From the backtrace below, you can see the patch has been made:
#2 0x01275600 in init () at su_uniqueid.c:169
169 initstate(seed[0] ^ seed[1], (char *)seed, 32 * sizeof
(uint32_t));
PS: I've also noticed that destroying the client stack doesn't free
all memory, but that I can live with for now :-) ...
I'll start debugging a little more...
Many thanks for your expert advice and help!
Kind regards,
Derik
#0 0x004ebb27 in initstate_r () from /lib/libc.so.6
#1 0x004eb6f2 in initstate () from /lib/libc.so.6
#3 0x012758b5 in su_randmem (mem=0xb2cfdd50, siz=8) at su_uniqueid.c:
312
#4 0x0124ff63 in soa_init_sdp_origin (ss=0x8d5a7e0, o=0xb2cfdd48,
buffer=0xb2cfdcac "�\234#\001h\027�\b\020") at soa.c:2189
#5 0x01250587 in soa_base_set_capability_sdp (ss=0x8d5a7e0,
_sdp=0x622da64,
str0=0x8db9804 "v=0\r\no=Test 0 0 IN IP4 192.168.1.2\r\ns=-\r
\na=setup:active\r\na=connection:new\r\na=resource:speechsynth\r
\na=cmid:1\r\nm=audio 4000 RTP/AVP 0 8 96\r\na"..., len0=298) at soa.c:
910
sdp=0xb2cfdddc,
sdp_str=0x8db9804 "v=0\r\no=Test 0 0 IN IP4 192.168.1.2\r\ns=-\r
\na=setup:active\r\na=connection:new\r\na=resource:speechsynth\r
\na=cmid:1\r\nm=audio 4000 RTP/AVP 0 8 96\r\na"..., sdp_len=298) at
soa_static.c:245
what=soa_capability_sdp_kind, sdp0=0x8db9804,
sdp_str=0x8db9804 "v=0\r\no=Test 0 0 IN IP4 192.168.1.2\r\ns=-\r
\na=setup:active\r\na=connection:new\r\na=resource:speechsynth\r
\na=cmid:1\r\nm=audio 4000 RTP/AVP 0 8 96\r\na"..., str_len=298) at
soa.c:2066
at soa.c:516
#9 0x01253f4f in soa_static_set_params (ss=0x8d5a7e0,
tags=0xb2cfdf98) at soa_static.c:190
#10 0x0124f6b0 in soa_set_params (ss=0x8d5a7e0, tag=0x12c70b0,
value=148609012) at soa.c:408
#11 0x0122426d in nua_stack_set_params (nua=0x8d8ec48, nh=0x8daf0e8,
e=nua_i_error, tags=0x8db97f4) at nua_params.c:549
#12 0x01216510 in nua_stack_init_handle (nua=0x622da64, nh=0x8daf0e8,
tags=0x8db97f4) at nua_stack.c:1001
#13 0x0121a144 in nua_client_init_request (cr=0x8da2200) at
nua_stack.c:2315
#14 0x0121cb3f in nua_client_create (nh=0x8daf0e8, event=31,
methods=0x12c0b00, tags=0x8db97f4) at nua_stack.c:2052
#15 0x012299c5 in nua_stack_invite (nua=0x8d8ec48, nh=0x8daf0e8,
e=nua_r_invite, tags=0x8db97f4) at nua_session.c:706
#16 0x0121c948 in nua_stack_signal (nua=0x8d8ec48, msg=0xb2cfe308,
ee=0x8db97dc) at nua_stack.c:594
#17 0x0126c89c in su_base_port_execute_msgs (queue=<value optimized
331
#19 0x01265029 in su_root_run (self=0x8da3db8) at su_port.h:310
#20 0x0126d9da in su_pthread_port_clone_main (varg=0xb672d204) at
#21 0x0063a5ab in start_thread () from /lib/libpthread.so.0
#22 0x00590cfe in clone () from /lib/libc.so.6
#1 0x004eb6f2 in initstate () from /lib/libc.so.6
(gdb) frame 2
169 initstate(seed[0] ^ seed[1], (char *)seed, 32 * sizeof
(uint32_t));
(gdb) frame 3
#3 0x012758b5 in su_randmem (mem=0xb2cfdd50, siz=8) at su_uniqueid.c:
312 if (!initialized) init();
(gdb) frame 4
buffer=0xb2cfdcac "�\234#\001h\027�\b\020") at soa.c:2189
(gdb) frame 5
_sdp=0x622da64,
str0=0x8db9804 "v=0\r\no=Test 0 0 IN IP4 192.168.1.2\r\ns=-\r
J.W.F. Thirion
I have some success now. I've gone into unimrcp-deps-0.8.0/libs/sofia-
sip/m4/sac-su2.m4 and removed check for the initstate.h header (I just
deleted "initstate.h" inside AC_CHECK_HEADERS). I then ran aclocal and
autoreconf and recompile sofia-sip, unimrcp, and my application. For
unimrcp-deps-0.8.0/libs/sofia-sip/m4/sac-su2.m4, the patch is:
529c529
< AC_CHECK_FUNCS([gettimeofday strerror random initstate tcsetattr
flock \
---
> AC_CHECK_FUNCS([gettimeofday strerror random tcsetattr flock \
The application now works as expected. Looking at the su_uniqueid.c
around line 169:
...
#if HAVE_INITSTATE
srand(seed[0] ^ seed[1]);
#endif
...
this now obiously compiles "srand(seed[0] ^ seed[1]);" as
HAVE_INITSTATE is no longer defined in config.h due to the check for
the header not being done anymore. I'm sure there is a more elegant
way of "undefining" HAVE_INITSTATE (perhaps ./configure --without-
initstate) or something similar. Also, I suppose one has to wonder why
the initstate version doesn't work.
Kind regards,
Derik
Arsen Chaloyan
Thanks for your analysis.
Currently unimrcp deps package contains Sofia-SIP-1.12.10 plus one
small patch, which has already been included in Sofia's repository. I
believe the issue you have experienced should be fixed in recent
Sofia's tree as well. Hopefully new version of Sofia-SIP will be
released soon. Meantime, if you have some time for experiments, try to
use recent Sofia instead, without any patches.
BTW, if you noticed any issue, feel free to discuss it with us. All
the issues must be identified. It's another question when it will be
possible to address these issues. At least they should be logged and
eventually processed based on severity,majority...
J.W.F. Thirion
Ok. I've now tried the CVS HEAD of sofia-sip without any patches.
The same segmentation fault occurs when I generate TTS, unload my .so
and load it again, and then try to do a second TTS session.
Looks like it is the initstate issue again.
I am on CentOS 5.4, but I doubt if there's a bug in the lower level
libraries... Hmm.. strange one..
Kind regards,
Derik
unimrcp_log: Create Channel 0x9bbcfe8 <new>
unimrcp_log: Signal Application Task Message
unimrcp_log: Receive Application Task Message [0]
unimrcp_log: Receive App Request 0x9bbcfe8 <new> [2]
unimrcp_log: Add MRCP Handle 0x9bbcfe8
unimrcp_log: Dispatch Application Request 0x9bbcfe8 <new> [2]
unimrcp_log: Add Control Channel 0x9bbcfe8 <new@speechsynth>
unimrcp_log: Add RTP Termination 0x9bbcfe8 <new>
unimrcp_log: Process Control Message
unimrcp_log: Signal Connection Task Message
unimrcp_log: Receive Connection Task Message [0]
unimrcp_log: On Control Channel Add 0x9bbcfe8 <new>
unimrcp_log: Process MPF Message
unimrcp_log: Add Context
unimrcp_log: Add Termination
unimrcp_log: Add Termination
unimrcp_log: Receive Media Task Message
unimrcp_log: On Termination Add 0x9bbcfe8 <new>
unimrcp_log: On Termination Add 0x9bbcfe8 <new>
unimrcp_log: Send Offer 0x9bbcfe8 <new> [c:1 a:1 v:0]
unimrcp_log: Local SDP 0x9bbcfe8 <new>
v=0
c=IN IP4 192.168.1.2
t=0 0
m=application 9 TCP/MRCPv2 1
a=setup:active
a=connection:new
a=resource:speechsynth
a=cmid:1
a=rtpmap:8 PCMA/8000
a=rtpmap:96 L16/8000
a=recvonly
a=ptime:20
a=mid:1
Segmentation fault (core dumped)
#0 0x004ebb27 in initstate_r () from /lib/libc.so.6
#1 0x004eb6f2 in initstate () from /lib/libc.so.6
#3 0x0620a8b5 in su_randmem (mem=0xb0639d50, siz=8) at su_uniqueid.c:
312
#4 0x061e4f63 in soa_init_sdp_origin (ss=0x9b6eae8, o=0xb0639d48,
buffer=0xb0639cac "áì\034\006ØQ¶\t\020") at soa.c:2189
#5 0x061e5587 in soa_base_set_capability_sdp (ss=0x9b6eae8,
_sdp=0x11ffa44,
str0=0x9b4d594 "v=0\r\no=Test 0 0 IN IP4 192.168.1.2\r\ns=-\r
\na=setup:active\r\na=connection:new\r\na=resource:speechsynth\r
\na=cmid:1\r\nm=audio 4000 RTP/AVP 0 8 96\r\na"..., len0=298) at soa.c:
910
sdp=0xb0639ddc,
sdp_str=0x9b4d594 "v=0\r\no=Test 0 0 IN IP4 192.168.1.2\r\ns=-\r
\na=setup:active\r\na=connection:new\r\na=resource:speechsynth\r
\na=cmid:1\r\nm=audio 4000 RTP/AVP 0 8 96\r\na"..., sdp_len=298) at
soa_static.c:245
what=soa_capability_sdp_kind,
sdp0=0x9b4d594,
sdp_str=0x9b4d594 "v=0\r\no=Test 0 0 IN IP4 192.168.1.2\r\ns=-\r
\na=setup:active\r\na=connection:new\r\na=resource:speechsynth\r
\na=cmid:1\r\nm=audio 4000 RTP/AVP 0 8 96\r\na"..., str_len=298) at
soa.c:2066
at soa.c:516
#9 0x061e8f4f in soa_static_set_params (ss=0x9b6eae8,
tags=0xb0639f98)
at soa_static.c:190
#10 0x061e46b0 in soa_set_params (ss=0x9b6eae8, tag=0x625c090,
value=162846084)
at soa.c:408
#11 0x061b926d in nua_stack_set_params (nua=0x9b8cff8, nh=0x9902cf8,
e=nua_i_error, tags=0x9b4d584) at nua_params.c:549
#12 0x061ab510 in nua_stack_init_handle (nua=0x11ffa44, nh=0x9902cf8,
tags=0x9b4d584) at nua_stack.c:1001
#13 0x061af144 in nua_client_init_request (cr=0x9b4b1c0) at
nua_stack.c:2315
#14 0x061b1b3f in nua_client_create (nh=0x9902cf8, event=31,
methods=0x6255ae0, tags=0x9b4d584) at nua_stack.c:2052
#15 0x061be9c5 in nua_stack_invite (nua=0x9b8cff8, nh=0x9902cf8,
e=nua_r_invite, tags=0x9b4d584) at nua_session.c:706
#16 0x061b1948 in nua_stack_signal (nua=0x9b8cff8, msg=0xb063a308,
ee=0x9b4d56c) at nua_stack.c:594
#17 0x0620189c in su_base_port_execute_msgs (queue=<value optimized
at su_base_port.c:276
331
#19 0x061fa029 in su_root_run (self=0x9b631e8) at su_port.h:310
#20 0x062029da in su_pthread_port_clone_main (varg=0xb1a3c204)
#21 0x0063a5ab in start_thread () from /lib/libpthread.so.0
#22 0x00590cfe in clone () from /lib/libc.so.6
(gdb) frame 0
#0 0x004ebb27 in initstate_r () from /lib/libc.so.6
(gdb) frame 1
#1 0x004eb6f2 in initstate () from /lib/libc.so.6
(gdb) frame 2
169 initstate(seed[0] ^ seed[1], (char *)&seed, sizeof(seed));
(gdb) frame 3
#3 0x0620a8b5 in su_randmem (mem=0xb0639d50, siz=8) at su_uniqueid.c:
312 if (!initialized) init();
(gdb) frame 4
buffer=0xb0639cac "áì\034\006ØQ¶\t\020") at soa.c:2189
(gdb) frame 5
_sdp=0x11ffa44,
str0=0x9b4d594 "v=0\r\no=Test 0 0 IN IP4 192.168.1.2\r\ns=-\r
Arsen Chaloyan
If you couldn't solve this problem yet, please log an issue. I'll try
to look into it later.
If you could provide a small, minimal application to reproduce, it'll
save some time for me.
J.W.F. Thirion
I've managed to get a working sofia-sip where I do not get this
problem anymore. Basically, my setup uses APR and APR-Util from
unimrcp-deps-0.8.0 (so that I've got the patches you've supplied), and
then I checked out sofia-sip from the freeswitch SVN repository:
svn checkout http://svn.freeswitch.org/svn/freeswitch/trunk/libs/sofia-sip
sofia-sip
After checking this out, I had to modify the Makefile.am and libsofia-
sip-ua/Makefile.am so that libsofia-sip-ua.so, required by unimrcp,
can also get built, otherwise only libsofia-sip-ua-glib.so gets built
on my system. See the patch files below. It seems like the freeswitch
version of sofia-sip is more actively maintained. I am not sure what
the differences are with the CVS HEAD of the sofia-sip repository, but
the above works for me.
Kind regards,
Derik
Index: Makefile.am
===================================================================
--- Makefile.am (revision 15458)
+++ Makefile.am (working copy)
@@ -7,8 +7,10 @@
AUTOMAKE_OPTIONS = foreign 1.7
-SUBDIRS = libsofia-sip-ua $(GLIB_SUBDIRS) packages # tests s2check
utils
-DIST_SUBDIRS = s2check libsofia-sip-ua libsofia-sip-ua-glib utils
packages \
+# SUBDIRS = libsofia-sip-ua $(GLIB_SUBDIRS) packages # tests s2check
utils
+SUBDIRS = libsofia-sip-ua $(GLIB_SUBDIRS) utils packages tests
+# DIST_SUBDIRS = s2check libsofia-sip-ua libsofia-sip-ua-glib utils
packages \
+DIST_SUBDIRS = libsofia-sip-ua libsofia-sip-ua-glib utils packages \
tests win32 open_c
# note: when glib devel files are not available, make should not
Index: libsofia-sip-ua/Makefile.am
===================================================================
--- libsofia-sip-ua/Makefile.am (revision 15458)
+++ libsofia-sip-ua/Makefile.am (working copy)
@@ -35,7 +35,7 @@
DOXYGEN = doxygen
-noinst_LTLIBRARIES = libsofia-sip-ua.la
+lib_LTLIBRARIES = libsofia-sip-ua.la
libsofia_sip_ua_la_SOURCES =
libsofia_sip_ua_la_LIBADD = bnf/libbnf.la \
@@ -107,6 +107,4 @@
include $(top_srcdir)/rules/lcov.am
endif
-include $(top_srcdir)/rules/silent.am
-
.PHONY: $(PHONY)
Arsen Chaloyan
Thanks for the update. The provided information is very useful for us.
I'll address it with the next release of UniMRCP and I still hope,
that new version of Sofia-SIP will be released by that time. This
would allow us not to use patches, but the original release.