RTLD_GLOBAL and libcryptopp.so crash
Jeffrey Walton
I wanted to put some findings in play for those interested,
particularly David and Zooko. Previous threads have had problems with
obtaining desired results with EH and RTTI due to object/address
equality. This in turn forced folks to call dlopen() with RTLD_GLOBAL,
which had side effects on other Crypto++ objects in libcryptopp.so
(fortunately, there's not too many).
For those familiar with Windows, RTLD_GLOBAL has the effect of
creating a shared data segment, which is a big no-no in the Windows
world. If two EXEs can load a DLL (with a shared data segment) and if
the first EXE misbehaves, it can crash the second EXE through the DLL.
Shared data segments are so dangerous that hunting them is a
requirement to pass code security gates.
=======================
Minimum Crash Duplication
=======================
* dlopen() flags must include RTLD_GLOBAL
* single EXE
* two each [distinct] SOs
* two each threads in the EXE which load a single SO
* a global Crypto++ object in each SO with the *same name*
This configuration is available in Cryptopp-SO-Test-1.zip. The
relationship between the EXE and SOs:
EXE: no calls to Crypto++ (does not even link to libcryptopp)
EXE -> dlopen( ... RTLD_GLOBAL, so-1.so )
EXE -> dlopen( ... RTLD_GLOBAL, so-2.so )
SO1: links to libcryptopp.so
SO2: links to libcryptopp.so
The only thing that SO1 and SO2 offer is a global AutoSeededRandomPool
named g_rng. The EXE does not even call any methods on g_rng - it
simply exists. My test run is below.
$ ./exetest
Main thread, Crypto++ at 0
Thread 0 loaded dsotest-1.so at 0xb6d00520, Crypto++ at 0x525000
Thread 1 loaded dsotest-2.so at 0xb6100520, Crypto++ at 0x525000
Main thread exiting
Segmentation fault (core dumped)
Running exetest under GDB:
$ gdb ./exetest
GNU gdb (GDB) Fedora (7.0-13.fc12)
...
Reading symbols from /home/jeffrey/Desktop/Cryptopp-SO-Test-1/exetest...done.
(gdb) r
Starting program: /home/jeffrey/Desktop/Cryptopp-SO-Test-1/exetest
[Thread debugging using libthread_db enabled]
Main thread, Crypto++ at 0
[New Thread 0xb7fecb70 (LWP 3460)]
[New Thread 0xb75ebb70 (LWP 3461)]
Thread 0 loaded dsotest-1.so at 0xb6a00520, Crypto++ at 0x525000
Thread 1 loaded dsotest-2.so at 0xb6800520, Crypto++ at 0x525000
[Thread 0xb75ebb70 (LWP 3461) exited]
Main thread exiting
[Thread 0xb7fecb70 (LWP 3460) exited]
Program received signal SIGSEGV, Segmentation fault.
0x00000000 in ?? ()
(gdb) bt
#0 0x00000000 in ?? ()
#1 0x00b07875 in CryptoPP::member_ptr<CryptoPP::BlockCipher>::~member_ptr (
this=0xb09898, __in_chrg=<value optimized out>)
at /usr/include/cryptopp/smartptr.h:49
#2 0x00b076f5 in CryptoPP::RandomPool::~RandomPool (this=0xb09840,
__in_chrg=<value optimized out>) at /usr/include/cryptopp/randpool.h:13
#3 0x00b07a38 in CryptoPP::AutoSeededRandomPool::~AutoSeededRandomPool (
this=0xb09840, __in_chrg=<value optimized out>)
at /usr/include/cryptopp/osrng.h:85
...
#6 0x0036cbbe in __libc_start_main (main=<value optimized out>,
argc=<value optimized out>, ubp_av=<value optimized out>,
init=<value optimized out>, fini=<value optimized out>,
rtld_fini=<value optimized out>, stack_end=<value optimized out>)
at libc-start.c:252
#7 0x08048be1 in _start ()
Next, break on the dtor and restart:
(gdb) b CryptoPP::member_ptr<CryptoPP::BlockCipher>::~member_ptr
Breakpoint 1 at 0xb07856: file /usr/include/cryptopp/smartptr.h, line 49.
(gdb) r
Starting program: /home/jeffrey/Desktop/Cryptopp-SO-Test-1/exetest
[Thread debugging using libthread_db enabled]
Main thread, Crypto++ at 0
[New Thread 0xb7fecb70 (LWP 3463)]
[New Thread 0xb75ebb70 (LWP 3464)]
Thread 1 loaded dsotest-2.so at 0xb6800520, Crypto++ at 0xb38000
[Switching to Thread 0xb75ebb70 (LWP 3464)]
Breakpoint 1, CryptoPP::member_ptr<CryptoPP::BlockCipher>::~member_ptr (
this=0xae7898, __in_chrg=<value optimized out>)
at /usr/include/cryptopp/smartptr.h:49
49 template <class T> member_ptr<T>::~member_ptr() {delete m_p;}
(gdb) c
Continuing.
Thread 0 loaded dsotest-1.so at 0xb6a00520, Crypto++ at 0xb38000
[Thread 0xb75ebb70 (LWP 3464) exited]
Main thread exiting
[Thread 0xb7fecb70 (LWP 3463) exited]
[Switching to Thread 0xb7fed6d0 (LWP 3462)]
Breakpoint 1, CryptoPP::member_ptr<CryptoPP::BlockCipher>::~member_ptr (
this=0xae7898, __in_chrg=<value optimized out>)
at /usr/include/cryptopp/smartptr.h:49
49 template <class T> member_ptr<T>::~member_ptr() {delete m_p;}
(gdb) c
Continuing.
Program received signal SIGSEGV, Segmentation fault.
0x00000000 in ?? ()
From above, the crash occurred on the second call to the destructor.
This makes sense since there's only one g_rng.
=======================
Cleanup without Crash
=======================
Cryptopp-SO-Test-2.zip removes the global AutoSeededRandomPool from
both shared objects. Both SOs still offer the the object, but do so
from a function call:
extern "C" bool GenerateBytes(byte* buffer, size_t size);
...
AutoSeededRandomPool& GetRng()
{
static AutoSeededRandomPool s_rng;
return s_rng;
}
bool GenerateBytes(byte* buffer, size_t size)
{
// Start high, latch low
bool ret = true;
try
{
AutoSeededRandomPool& rng = GetRng();
rng.GenerateBlock(buffer,size);
}
catch(CryptoPP::Exception)
{
ret = false;
}
return ret;
}
Next, a run which calls GenerateBytes() through the loaded SOs (SO1
and SO2, and not libcryptopp.so):
$ ./exetest
Main thread, Crypto++ at 0
Thread 0 loaded dsotest-1.so at 0xb6200520, Crypto++ at 0x525000
Thread 0 generated bytes
Thread 1 loaded dsotest-2.so at 0xb6000520, Crypto++ at 0x525000
Thread 1 generated bytes
Main thread exiting
Examining under GDB shows no signals.
=======================
Global Crypto++ Objects
=======================
David pointed out a crash related to NameValuePairs. When I dumped
libcryptopp.so, I do see what *appears* to be a global NameValuePairs
object:
$ nm --demangle -D /usr/lib/libcryptopp.so | grep 'g_'
00427a23 B CryptoPP::g_hasSSSE3
00428344 B CryptoPP::g_actualMac
00426984 D CryptoPP::g_cacheLineSize
00428350 B CryptoPP::g_macFileLocation
00427a20 B CryptoPP::g_x86DetectionDone
00427b88 B CryptoPP::g_nullNameValuePairs
004271cc B CryptoPP::g_pAssignIntToInteger
00428340 B CryptoPP::g_powerUpSelfTestStatus
0036ace0 R CryptoPP::SAFER::Base::log_tab
00427a25 B CryptoPP::g_isP4
00427a24 B CryptoPP::g_hasMMX
00427a21 B CryptoPP::g_hasISSE
00427a22 B CryptoPP::g_hasSSE2
=======================
Conclusions
=======================
When using the Crypto++ library as a shared object and loading with
the RTLD_GLOBAL flag, do not use global objects. This also means that
those who are packaging for distros like Fedora should patch to remove
the global objects.
=======================
Sample Files
=======================
Cryptopp-SO-Test-1.zip (Crash) -
http://www.cryptopp.com/w/images/8/89/Cryptopp-SO-Test-1.zip
Cryptopp-SO-Test-2.zip (No Crash) -
http://www.cryptopp.com/w/images/a/af/Cryptopp-SO-Test-2.zip
Jeff
Jeffrey Walton
Here's an issue that should be raised, though I'm not sure what to
make of it. It is either a GDB bug, or the runtime linker is
channelling *all calls* to get the RNG (ie, SO1::GetRng and
SO2::GetRng) though a single function (SO1::GetRng) since s_rng is
static.
In the trace below, notice that GDB switches thread contexts. However,
also note that the address of s_rng (0x114d80) is the same across
threads, even though each thread makes its call through a different
shared object.
Jeff
$ gdb ./exetest
GNU gdb (GDB) Fedora (7.0-13.fc12)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/
gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show
copying"
and "show warranty" for details.
This GDB was configured as "i686-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/jeffrey/Desktop/Cryptopp-SO-Test-2/
exetest...done.
(gdb) b GetRng
Function "GetRng" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (GetRng) pending.
(gdb) r
Starting program: /home/jeffrey/Desktop/Cryptopp-SO-Test-2/exetest
[Thread debugging using libthread_db enabled]
Main thread, Crypto++ at 0
[New Thread 0xb7fecb70 (LWP 3892)]
[New Thread 0xb75ebb70 (LWP 3893)]
Thread 0 loaded dsotest-1.so at 0xb6a00520, Crypto++ at 0x6bd000
Thread 1 loaded dsotest-2.so at 0xb6800520, Crypto++ at 0x6bd000
[Switching to Thread 0xb75ebb70 (LWP 3893)]
Breakpoint 1, GetRng () at dsotest-2.cpp:46
46 static AutoSeededRandomPool s_rng;
(gdb) p &s_rng
$1 = (CryptoPP::AutoSeededRandomPool *) 0x114d80
(gdb) c
Continuing.
Thread 1 generated bytes
[Switching to Thread 0xb7fecb70 (LWP 3892)]
Breakpoint 1, GetRng () at dsotest-2.cpp:46
46 static AutoSeededRandomPool s_rng;
(gdb) p &s_rng
$2 = (CryptoPP::AutoSeededRandomPool *) 0x114d80
(gdb) c
Continuing.
Thread 0 generated bytes
[Thread 0xb75ebb70 (LWP 3893) exited]
Main thread exiting
[Thread 0xb7fecb70 (LWP 3892) exited]
Program exited normally.
(gdb)
alekcey
Do you mean that cryptopp needs patching?
If so, can you provide this patch for cryptopp?
Is this issue can be fixed in cryptopp upstream?
Alexey