Segfault when calling weak symbol

[Iwan Vosloo]

Hi all.

I have been struggling to get a cython-built python extension to link properly using these:

Python 3.8.5

Cython version 0.29.23

g++ (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0

My extension segfaults when invoking functions in a C dynamic library loaded using dlopen.

This is code that used to work with a previous version of the library. Two things changed since then:

- The functions in the external library have been changed to be weak symbols (using __attribute__((weak)) ).

- With the old version I could link the shared library at compile time, instead of using dlopen (linking the external library at compile time breaks with the new version though, and its creators recommend loading at run time)

If I write a test program in C, and compile and link it using the same flags that setuptools uses when compiling the extension, it works fine.

When used in Python from a cython-compiled extension, it segfaults.

For the plain C program to work, I have to load the library using the RTLD_GLOBAL flag, else it segfaults too. Using the same flag in cython does not fix the problem.

I have to admit that I last touched C 3 decades ago... so there's probably a lot going on here that I do not understand.

How can I debug this issue?

Here is the stacktrace from gdb for my test program:

(the symbol that is called is GciTsEncrypt - which is not present here)

#0  0x0000000000000000 in ?? ()

#1  0x00007f0879e665c4 in __pyx_pf_10testcython_6TestMe_2encrypt_password (

    __pyx_v_self=<optimized out>, __pyx_v_unencrypted_password=<optimized out>)

    at testcython.cpp:1922

#2  __pyx_pw_10testcython_6TestMe_3encrypt_password (__pyx_v_self=<optimized out>, 

    __pyx_v_unencrypted_password=0x7f0879e746b0) at testcython.cpp:1817

#3  0x0000000000504186 in ?? ()

#4  0x000000000056b3fe in _PyEval_EvalFrameDefault ()

#5  0x00000000005f5ed6 in _PyFunction_Vectorcall ()

#6  0x000000000056b21e in _PyEval_EvalFrameDefault ()

#7  0x000000000056951a in _PyEval_EvalCodeWithName ()

#8  0x000000000068be87 in PyEval_EvalCode ()

#9  0x000000000067b671 in ?? ()

#10 0x000000000067b6ef in ?? ()

#11 0x000000000067b7ab in PyRun_FileExFlags ()

#12 0x000000000067d82e in PyRun_SimpleFileExFlags ()

#13 0x00000000006b5682 in Py_RunMain ()

#14 0x00000000006b5a0d in Py_BytesMain ()

#15 0x00007f087a2f80b3 in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6

#16 0x00000000005f9ece in _start ()

If I do:

 nm build/lib.linux-x86_64-3.8/testcython.cpython-38-x86_64-linux-gnu.so | grep GciTsEncrypt

I get:

                 w GciTsEncrypt

If I do this (on the external library used):                 

 nm $LD_LIBRARY_PATH/libgcits-3.6.1-64.so | grep GciTsEncrypt

I get:

 0000000000027330 W GciTsEncrypt

Thanks

Iwan

[Stefan Behnel]

Hi,

I don't think I can help much, but what you describe isn't enough for me to understand what is going wrong. I can only recommend digging into this more deeply with a debugger.

Your stack trace indicates that somehow it is jumping (or calling) through a NULL pointer. That may be due to a bug in your code, but might just as well be an uninitialised symbol due to the way you load your library. I can't say.

Try to print some symbols in gdb to see if they are meaningful.

Stefan