libcrypto++ segfault on i386?
Jens Peter Secher
compiled with g++ 4.3.3 on i386.
[ This email is CC'ed to the Debian Bug Tracking system for reference. ]
I am trying to find the cause of a segfault in an application (amule)
liked against libcrypto++ 5.6.0. The segfault is only reported to
happen on i386, not amd64. On i386, I can reproduce the segfault every
time I run the program.
To locate the problem I have compiled a version of libcrypto++ with no
optimisation (-O0). The segfault seems to happen in the construction of
an AutoSeededRandomPool. More specifically, it seems to happen in the
initialisation of a SHA256 used by the RandomPool. The relevant lines
from sha.cpp is the following memcpy:
void SHA256::InitState(HashWordType *state)
{
static const word32 s[8] = {0x6a09e667, 0xbb67ae85, 0x3c6ef372,
0xa54ff53a, 0x510e527f, 0x9b05688c, 0x1f83d9ab, 0x5be0cd19};
memcpy(state, s, sizeof(s));
}
Here is part of the backtrace from gdb:
Program received signal SIGSEGV, Segmentation fault.
memcpy () at ../sysdeps/i386/i686/memcpy.S:75
(gdb) bt full
#0 memcpy () at ../sysdeps/i386/i686/memcpy.S:75
No locals.
#1 0x084f15d8 in ?? ()
No symbol table info available.
#2 0xb7df5e09 in CryptoPP::SHA256::InitState (state=0xb7f70980) at sha.cpp:96
s = {1779033703, 3144134277, 1013904242, 2773480762, 1359893119,
2600822924, 528734635, 1541459225}
#3 0x080c3784 in CryptoPP::IteratedHashWithStaticTransform<unsigned
int, CryptoPP::EnumToType<CryptoPP::ByteOrder, 1>, 64u, 32u,
CryptoPP::SHA256, 32u, true>::Init (this=0xbff79330) at
/usr/include/cryptopp/iterhash.h:90
No locals.
#4 0xb7cb94bc in IteratedHashWithStaticTransform (this=0xbff79330) at
iterhash.h:88
No locals.
#5 0xb7cb9525 in SHA256 (this=0xbff79330) at sha.h:21
No locals.
#6 0xb7dbf8c7 in CryptoPP::RandomPool::IncorporateEntropy (this=0x86e00e0,
input=0x97a0978
"Ç\veL°DÏM\037fLTyóÉ\036\2072D2VïýÒI\030µrÚôs\004", length=32) at
randpool.cpp:26
hash = {<CryptoPP::IteratedHashWithStaticTransform<unsigned int,
CryptoPP::EnumToType<CryptoPP::ByteOrder, 1>, 64u, 32u,
CryptoPP::SHA256, 32u, true>> =
{<CryptoPP::ClonableImpl<CryptoPP::SHA256,
CryptoPP::AlgorithmImpl<CryptoPP::IteratedHash<unsigned int,
CryptoPP::EnumToType<CryptoPP::ByteOrder, 1>, 64u,
CryptoPP::HashTransformation>, CryptoPP::SHA256> >> =
{<CryptoPP::AlgorithmImpl<CryptoPP::IteratedHash<unsigned int,
CryptoPP::EnumToType<CryptoPP::ByteOrder, 1>, 64u,
CryptoPP::HashTransformation>, CryptoPP::SHA256>> =
{<CryptoPP::IteratedHash<unsigned int,
CryptoPP::EnumToType<CryptoPP::ByteOrder, 1>, 64u,
CryptoPP::HashTransformation>> = {<CryptoPP::IteratedHashBase<unsigned
int, CryptoPP::HashTransformation>> = {<CryptoPP::HashTransformation>
= {<CryptoPP::Algorithm> = {<CryptoPP::Clonable> = {
_vptr.Clonable = 0x836ea88}, <No data fields>}, <No
data fields>}, m_countLo = 0, m_countHi = 0},
static BLOCKSIZE = <optimized out>, static
cryptopp_assert_63 = <optimized out>,
m_data = {<CryptoPP::SecBlock<unsigned int,
CryptoPP::FixedSizeAllocatorWithCleanup<unsigned int, 16u,
CryptoPP::NullAllocator<unsigned int>, false> >> = {m_alloc =
{<CryptoPP::AllocatorBase<unsigned int>> = {<No data fields>}, m_array
= {3080954372, 3086029504,
137769968, 139400664, 3220673400, 3086420352,
3220673500, 3083089170, 3220673500, 3086029504, 3220673400,
3083089259,
3220673500, 3086029504, 3220673416, 3083108035},
m_fallbackAllocator =
{<CryptoPP::AllocatorBase<unsigned int>> = {<No data fields>}, <No
data fields>},
m_allocated = true}, m_size = 16, m_ptr = 0xbff79340},
<No data fields>}}, <No data fields>}, <No data fields>},
static DIGESTSIZE = <optimized out>,
m_state = {<CryptoPP::FixedSizeSecBlock<unsigned int, 16u,
CryptoPP::FixedSizeAllocatorWithCleanup<unsigned int, 16u,
CryptoPP::NullAllocator<unsigned int>, true> >> =
{<CryptoPP::SecBlock<unsigned int,
CryptoPP::FixedSizeAllocatorWithCleanup<unsigned int, 16u,
CryptoPP::NullAllocator<unsigned int>, true> >> = {m_alloc =
{<CryptoPP::AllocatorBase<unsigned int>> = {<No data fields>}, m_array
= {4294967295,
3086458868, 134580384, 3086460528, 3220673504,
3086397003, 3086460968, 0, 1, 1, 0, 134629935, 2114, 0, 139192488, 1,
3080978708, 3086029504},
m_fallbackAllocator = {<CryptoPP::AllocatorBase<unsigned
int>> = {<No data fields>}, <No data fields>},
m_allocated = false}, m_size = 0, m_ptr = 0x0}, <No data
fields>}, <No data fields>}}, <No data fields>}
key = (const byte *) 0xb7f70980 "Z\213\f$\211\004$\213D$\004Â\f"
#7 0xb7db11f2 in CryptoPP::AutoSeededRandomPool::Reseed
(this=0x86e00e0, blocking=false, seedSize=32) at osrng.cpp:164
seed = {m_alloc = {<CryptoPP::AllocatorBase<unsigned char>> = {<No
data fields>}, <No data fields>}, m_size = 32,
m_ptr = 0x97a0978 "Ç\veL°DÏM\037fLTyóÉ\036\2072D2VïýÒI\030µrÚôs\004"}
#8 0x082b81ae in AutoSeededRandomPool (this=0x86e00e0,
blocking=false, seedSize=32) at /usr/include/cryptopp/osrng.h:89
Stepping through the program on the way to this segfault, I noticed the
following strange thing during construction of the SHA256:
(gdb) bt full
#0 SecBlock (this=0xbfd200f0, size=16) at secblock.h:283
No locals.
#1 0xb7c1f051 in FixedSizeSecBlock (this=0xbfd200f0) at secblock.h:458
No locals.
#2 0xb7c22b2f in IteratedHash (this=0xbfd200e0) at iterhash.h:56
No locals.
#3 0xb7c60767 in AlgorithmImpl (this=0xbfd200e0) at simple.h:25
No locals.
#4 0xb7c60799 in ClonableImpl (this=0xbfd200e0) at simple.h:17
No locals.
#5 0xb7c62495 in IteratedHashWithStaticTransform (this=0xbfd200e0) at
iterhash.h:88
No locals.
#6 0xb7c62525 in SHA256 (this=0xbfd200e0) at sha.h:21
No locals.
Am I right in thinking that SecBlock(size=16) looks odd considering that
everything else in the trace use sizes of 32?
Cheers,
--
Jens Peter Secher.
_DD6A 05B0 174E BFB2 D4D9 B52E 0EE5 978A FE63 E8A1 jpsecher gmail com_.
A. Because it breaks the logical sequence of discussion.
Q. Why is top posting bad?
Wei Dai
> Am I right in thinking that SecBlock(size=16) looks odd considering that
> everything else in the trace use sizes of 32?
I don't think that's it. 16 is the number of word32, and should be correct.
One problem is here:
m_state = {<CryptoPP::FixedSizeSecBlock<unsigned int, 16u,
CryptoPP::FixedSizeAllocatorWithCleanup<unsigned int, 16u,
CryptoPP::NullAllocator<unsigned int>, true> >> =
{<CryptoPP::SecBlock<unsigned int,
CryptoPP::FixedSizeAllocatorWithCleanup<unsigned int, 16u,
CryptoPP::NullAllocator<unsigned int>, true> >> = {m_alloc =
{<CryptoPP::AllocatorBase<unsigned int>> = {<No data fields>}, m_array
= {4294967295,
3086458868, 134580384, 3086460528, 3220673504,
3086397003, 3086460968, 0, 1, 1, 0, 134629935, 2114, 0, 139192488, 1,
3080978708, 3086029504},
m_fallbackAllocator = {<CryptoPP::AllocatorBase<unsigned
int>> = {<No data fields>}, <No data fields>},
m_allocated = false}, m_size = 0, m_ptr = 0x0}
Note that m_state.m_ptr = 0x0, which means m_state hasn't been constructed
when the segfault happened. Why? On the segfault stack it's going through
line 88 of iterhash.h:
IteratedHashWithStaticTransform() {this->Init();}
m_state belongs to IteratedHashWithStaticTransform, and the segfault is
happening inside Init(), so m_state should have been constructed already.
Also, look at this part:
#2 0xb7df5e09 in CryptoPP::SHA256::InitState (state=0xb7f70980) at
sha.cpp:96
The state parameter should equal m_state.m_ptr. But m_state.m_ptr is shown
as 0x0, and state is shown as 0xb7f70980. 0xb7f70980 certainly isn't correct
either, since m_state should be near m_data, which is at 0xbff79340. I see
this:
key = (const byte *) 0xb7f70980 "Z\213\f$\211\004$\213D$\004Â\f"
This variable isn't in my copy of randpool.cpp. Is it something you added to
help debugging? In any case, I don't understand how that value got passed to
InitState().
If you can't figure this out, please send me instructions to reproduce the
bug.
Jens Peter Secher
>
> I don't think that's it. 16 is the number of word32, and should be correct.
> One problem is here:
>
> Note that m_state.m_ptr = 0x0, which means m_state hasn't been constructed
> when the segfault happened.
Ahh, thanks.
[...]
> I see this:
>
> key = (const byte *) 0xb7f70980 "Z\213\f$\211\004$\213D$\004Â\f"
>
> This variable isn't in my copy of randpool.cpp. Is it something you added to
> help debugging?
Yes, sorry I didn't mention this, but I modified randpool.cpp:
void RandomPool::IncorporateEntropy(const byte *input, size_t length)
{
SHA256 hash;
const byte *key = m_key;
hash.Update(key, 32);
hash.Update(input, length);
hash.Final(m_key);
m_keySet = false;
}
because it first seemed that the segfault happened in the cast operator.
> If you can't figure this out, please send me instructions to reproduce the
> bug.
Thanks, I will try some more now that I know what to watch. But just
in case; are you by any chance able to run on a Debian / Ubuntu
system? Because if you are, I could upload some unoptimised packages
with debugging symbols included which makes it easier to follow what
is going on.