Debug assertion when exporting public RSA key to std::string

[dreijer]

So, I've been testing out the latest version of Crypto++ for the past
few days and got really happy once I noticed that all the problems I
had had previously had now gone away.

However, today I stumbled across a problem. I'm generating a pair of
1024 bit RSA keys and am exporting the public part to a string. It's
not a problem in Crypto++ per se, as the key is correctly exported.
However, the std::string holding the encoded key is broken somehow.

Here is the code I'm using to reproduce the error:

AutoSeededRandomPool rnd;

RSAES_OAEP_SHA_Decryptor privateKey(rnd, 1024);
RSAES_OAEP_SHA_Encryptor publicKey(privateKey);

// Once pubEncoded goes out of scope, VS2005 throws a debug assertion:
// _BLOCK_TYPE_IS_VALID(pHead->nBlockUse)
// in dbgdel.cpp, line 52
{
string pubEncoded;
HexEncoder pubFile(new StringSink(pubEncoded));
publicKey.DEREncode(pubFile);
pubFile.MessageEnd();
}

[Jeffrey Walton]

Hi Wei,

I confirmed this. No asserts for me - the memory arena didn't appeare
trashed. The following ends up in a catch block. I had to put the
HexEncoder on the stack... And I reused it with a call to MessageEnd().
Is this correct usage?

Before I put up sample code, I want to make sure it is broken, and not
something on my end.

Jeff

But I got BER Encoding and Decoding working.

[Wei Dai]

This code works fine for me, and I don't see any errors. Dreijer, can
you please create a ZIP file with test code that reproduces the
problem, along with all of the project files and Crypto++ source code
needed to compile the test code, and email that to me directly? That
would really help me narrow down where the problem might be.

> }- Hide quoted text -- Show quoted text -