Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Re: DBD::Oracle forked client hangs with Oracle Client Libraries 19.20 and later.

14 views
Skip to first unread message

Justin Schoeman

unread,
Oct 28, 2023, 11:27:59 AM10/28/23
to dbi-...@perl.org
To follow up, with the hackaround in the previous post, and modify
test.pl to dispatch batches of 20 children, it eventually dies with a
segfault (below - coredump available on request). Although I can not be
sure if this is related to my hack, or the core library.

Thanks,

Justin

       Message: Process 1141691 (test.pl) of user 1000 dumped core.

                Module libnss_resolve.so.2 from rpm
systemd-253.10-1.fc38.x86_64
                Module libnss_mdns4_minimal.so.2 from rpm
nss-mdns-0.15.1-8.fc38.x86_64
                Module libcap.so.2 from rpm libcap-2.48-6.fc38.x86_64
                Module libnss_myhostname.so.2 from rpm
systemd-253.10-1.fc38.x86_64
                Module libnuma.so.1 from rpm numactl-2.0.16-2.fc38.x86_64
                Module libaio.so.1 from rpm libaio-0.3.111-15.fc38.x86_64
                Module libcrypt.so.2 from rpm
libxcrypt-4.4.36-1.fc38.x86_64
                Stack trace of thread 1141697:
                #0  0x00007f8bfe0b0884 __pthread_kill_implementation
(libc.so.6 + 0x8e884)
                #1  0x00007f8bfe05fafe raise (libc.so.6 + 0x3dafe)
                #2  0x00007f8beeccb73f skgesigOSCrash
(libclntsh.so.21.1 + 0x28cb73f)
                #3  0x00007f8bef3ca9ed kpeDbgSignalHandler
(libclntsh.so.21.1 + 0x2fca9ed)
                #4  0x00007f8beeccba52 skgesig_sigactionHandler
(libclntsh.so.21.1 + 0x28cba52)
                #5  0x00007f8bfe05fbb0 __restore_rt (libc.so.6 + 0x3dbb0)
                #6  0x00007f8bfe302e70 Perl_csighandler3
(libperl.so.5.36 + 0x102e70)
                #7  0x00007f8bfe05fbb0 __restore_rt (libc.so.6 + 0x3dbb0)
                #8  0x00007f8bfe0ab219 __futex_abstimed_wait_common
(libc.so.6 + 0x89219)
                #9  0x00007f8bfe0adf22
pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x8bf22)
                #10 0x00007f8beb6abfc3 sltspctimewait
(libclntshcore.so.21.1 + 0xabfc3)
                #11 0x00007f8bec8cf501 kpucpincrtime (libclntsh.so.21.1
+ 0x4cf501)
                #12 0x00007f8bfe0ae947 start_thread (libc.so.6 + 0x8c947)
                #13 0x00007f8bfe134860 __clone3 (libc.so.6 + 0x112860)

                Stack trace of thread 1141691:
                #0  0x00007f8bfe1230ea read (libc.so.6 + 0x1010ea)
                #1  0x00007f8bef8d02d0 snttread (libclntsh.so.21.1 +
0x34d02d0)
                #2  0x00007f8bef8ceb35 nttfprd (libclntsh.so.21.1 +
0x34ceb35)
                #3  0x00007f8bef8c3c95 nsbasic_brc (libclntsh.so.21.1 +
0x34c3c95)
                #4  0x00007f8bef8b7e7f nioqrc (libclntsh.so.21.1 +
0x34b7e7f)
                #5  0x00007f8bef8f7ecf ttcdrv (libclntsh.so.21.1 +
0x34f7ecf)
                #6  0x00007f8bef8bdccd nioqwa (libclntsh.so.21.1 +
0x34bdccd)
                #7  0x00007f8bef88b690 upirtrc (libclntsh.so.21.1 +
0x348b690)
                #8  0x00007f8bef8a0331 kpurcsc (libclntsh.so.21.1 +
0x34a0331)
                #9  0x00007f8bef890f40 kpuexec (libclntsh.so.21.1 +
0x3490f40)
                #10 0x00007f8bef88ad49 OCIStmtExecute
(libclntsh.so.21.1 + 0x348ad49)
                #11 0x00007f8bfe5e2377 ora_st_execute (Oracle.so + 0x24377)
                #12 0x00007f8bfe5cad7d XS_DBD__Oracle__st_execute
(Oracle.so + 0xcd7d)
                #13 0x00007f8bfe61b106 XS_DBI_dispatch (DBI.so + 0xf106)
                #14 0x00007f8bfe3265aa Perl_pp_entersub
(libperl.so.5.36 + 0x1265aa)
                #15 0x00007f8bfe317958 Perl_runops_standard
(libperl.so.5.36 + 0x117958)
                #16 0x00007f8bfe27c3e7 Perl_call_sv (libperl.so.5.36 +
0x7c3e7)
                #17 0x00007f8bfe61b164 XS_DBI_dispatch (DBI.so + 0xf164)
                #18 0x00007f8bfe3265aa Perl_pp_entersub
(libperl.so.5.36 + 0x1265aa)
                #19 0x00007f8bfe317958 Perl_runops_standard
(libperl.so.5.36 + 0x117958)
                #20 0x00007f8bfe28259d perl_run (libperl.so.5.36 + 0x8259d)
                #21 0x000056551ca1734a main (perl + 0x134a)
                #22 0x00007f8bfe049b8a __libc_start_call_main
(libc.so.6 + 0x27b8a)
                #23 0x00007f8bfe049c4b __libc_start_main@@GLIBC_2.34
(libc.so.6 + 0x27c4b)
                #24 0x000056551ca17385 _start (perl + 0x1385)
                ELF object binary architecture: AMD x86-64

On 2023/10/25 15:39, Justin Schoeman wrote:
> Good day,
>
> I am trying to dig into an issue after a recent Oracle update. The
> application has been running flawlessly from version 9.x to 18.5, and
> has only broken on the most recent client library updates.
>
> Attached is a cut down application which demonstrates the issue.
>
> The forked child processes never exit. Attaching to them with GDB gives:
>
> (gdb) t a a bt
>
> Thread 1 (Thread 0x7f0c48a62c40 (LWP 773676) "test.pl"):
> #0  futex_wait (private=<optimized out>, expected=14,
> futex_word=0x55c5ef1c96e4) at ../sysdeps/nptl/futex-internal.h:146
> #1  futex_wait_simple (private=<optimized out>, expected=14,
> futex_word=0x55c5ef1c96e4) at ../sysdeps/nptl/futex-internal.h:177
> #2  __GI___pthread_cond_destroy (cond=0x55c5ef1c96c0) at
> pthread_cond_destroy.c:53
> #3  0x00007f0c35cac072 in sltspcdestroy () from
> /usr/lib/oracle/21/client64/lib/libclntshcore.so.21.1
> #4  0x00007f0c36ecf68e in kpucpstopthr () from
> /usr/lib/oracle/21/client64/lib/libclntsh.so.21.1
> #5  0x00007f0c39e9a42e in kpufhndl0 () from
> /usr/lib/oracle/21/client64/lib/libclntsh.so.21.1
> #6  0x00007f0c48a1028a in ora_db_destroy
> (dbh=dbh@entry=0x55c5ef32c5b0, imp_dbh=imp_dbh@entry=0x55c5ef035390)
> at /root/.local/share/.cpan/build/DBD-Oracle-1.83-2/dbdimp.c:1263
> #7  0x00007f0c48a10752 in XS_DBD__Oracle__db_DESTROY
> (my_perl=0x55c5eee272a0, cv=<optimized out>) at ./Oracle.xsi:432
> #8  0x00007f0c48a4d106 in XS_DBI_dispatch (my_perl=0x55c5eee272a0,
> cv=0x55c5ef13b9c8) at
> /usr/src/debug/perl-DBI-1.643-15.fc38.x86_64/DBI.xs:3783
> #9  0x00007f0c487265aa in Perl_pp_entersub (my_perl=0x55c5eee272a0) at
> /usr/src/debug/perl-5.36.1-497.fc38.x86_64/pp_hot.c:5353
> #10 0x00007f0c4867c7a6 in Perl_call_sv (my_perl=<optimized out>,
> sv=<optimized out>, flags=<optimized out>) at
> /usr/src/debug/perl-5.36.1-497.fc38.x86_64/perl.c:3057
> #11 0x00007f0c4872c643 in S_curse
> (my_perl=my_perl@entry=0x55c5eee272a0, sv=sv@entry=0x55c5ef34af38,
> check_refcnt=check_refcnt@entry=true) at
> /usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:6973
> #12 0x00007f0c4872cd88 in Perl_sv_clear
> (my_perl=my_perl@entry=0x55c5eee272a0,
> orig_sv=orig_sv@entry=0x55c5ef079858) at
> /usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:6536
> #13 0x00007f0c4872d3a2 in Perl_sv_free2 (my_perl=0x55c5eee272a0,
> sv=0x55c5ef079858, rc=<optimized out>) at
> /usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:7073
> #14 0x00007f0c486fc878 in Perl_SvREFCNT_dec (sv=<optimized out>,
> my_perl=0x55c5eee272a0) at
> /usr/src/debug/perl-5.36.1-497.fc38.x86_64/inline.h:405
> #15 S_mg_free_struct (my_perl=0x55c5eee272a0, sv=<optimized out>,
> mg=0x55c5ef34bb70) at /usr/src/debug/perl-5.36.1-497.fc38.x86_64/mg.c:563
> #16 0x00007f0c486fc8c9 in Perl_mg_free (my_perl=0x55c5eee272a0,
> sv=0x55c5ef34aec0) at /usr/src/debug/perl-5.36.1-497.fc38.x86_64/mg.c:585
> #17 0x00007f0c4872cdc3 in Perl_sv_clear
> (my_perl=my_perl@entry=0x55c5eee272a0,
> orig_sv=orig_sv@entry=0x55c5ef34aec0) at
> /usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:6544
> #18 0x00007f0c4872d3a2 in Perl_sv_free2 (my_perl=0x55c5eee272a0,
> sv=0x55c5ef34aec0, rc=<optimized out>) at
> /usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:7073
> #19 0x00007f0c48722646 in Perl_SvREFCNT_dec_NN (sv=<optimized out>,
> my_perl=<optimized out>) at
> /usr/src/debug/perl-5.36.1-497.fc38.x86_64/inline.h:419
> #20 do_clean_objs (ref=0x55c5ef338ce0, my_perl=<optimized out>) at
> /usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:448
> #21 S_visit (mask=2048, flags=2048, f=<optimized out>,
> my_perl=0x55c5eee272a0) at
> /usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:391
> #22 Perl_sv_clean_objs (my_perl=0x55c5eee272a0) at
> /usr/src/debug/perl-5.36.1-497.fc38.x86_64/sv.c:542
> #23 0x00007f0c48680160 in perl_destruct (my_perl=0x55c5eee272a0) at
> /usr/src/debug/perl-5.36.1-497.fc38.x86_64/perl.c:880
> #24 0x000055c5eda0f324 in main (argc=<optimized out>, argv=<optimized
> out>, env=<optimized out>) at
> /usr/src/debug/perl-5.36.1-497.fc38.x86_64/perlmain.c:121
>
> I have instrumented dbdimpl.c as follows:
>
>                         /* free global environment handle during
> destruction of last connection */
>                         } else if ( (imp_dbh->envhp == imp_drh->envhp)
> && (SvTRUE(perl_get_sv("DBI::PERL_ENDING",0))) ) {
>                                 PerlIO_printf(PerlIO_stderr(),
> "EXIT1\n");
>                                 OCIHandleFree_log_stat(imp_dbh,
> imp_dbh->envhp, OCI_HTYPE_ENV, status);
>                                 PerlIO_printf(PerlIO_stderr(),
> "EXIT2\n");
>                                 if ( status == OCI_SUCCESS ) {
>                                 PerlIO_printf(PerlIO_stderr(),
> "EXIT3\n");
>                                         imp_dbh->envhp = NULL;
>                                         imp_drh->envhp = NULL;
>                                 }
>                         }
>
> And the child prints 'EXIT1', but never 'EXIT2'. If I comment out the
> call to OCIHandleFree_log_stat() then the child process exits without
> error (although there are random segfaults after long run times).
>
> 1. A complete log of all steps of the build, e.g.:
>
> Atatched (build.log)
>
> 2. Full details of which version of Oracle client and server you're using
>
> # rpm -qa | grep oracle
> oracle-instantclient-basic-21.11.0.0.0-1.x86_64
> oracle-instantclient-devel-21.11.0.0.0-1.x86_64
>
>
> 3. The output of perl -V       (that's a capital V, not lowercase)
>
> Attached (perl-V.log)
>
> Thanks,
> Justin
>
>
>
0 new messages