Memory leak in RSA 5.6.3?
120 views
Skip to first unread message
Gavin Lambert
Jun 27, 2016, 4:29:27 AM6/27/16
to Crypto++ Users
Hi, I'm using Crypto++ for the first time, to do some decryption and verification. I've downloaded the 5.6.3 release zip.
The resulting decrypted data is correct. but after the function containing the above exits (so all objects should have been destroyed) I get the following leaks reported:
Is this a known issue? Is there something else I need to call to clean up the memory?
I've written some test code using the RSA example on the wiki as a basis, but I'm getting memory leaks reported; unfortunately the leak detector I'm using only reports the location of the "new", not a full call stack, and due to Crypto++'s code structure this doesn't help me find the root cause.
The code is fairly straightfoward:
using namespace CryptoPP;
RSA::PrivateKey privateKey;
privateKey.Load(...); // verified this doesn't leak; it's a 3072-bit key
std::string ciphertext(raw_ciphertext, sizeof(raw_ciphertext)); // 384 bytes
RSAES_OAEP_SHA_Decryptor decryptor(privateKey);
AutoSeededRandomPool rng;
SecByteBlock decrypted(decryptor.MaxPlaintextLength(ciphertext.size());
auto result = decryptor.Decrypt(rng, (const uint8_t*)ciphertext.data(), ciphertext.size(), decrypted);
assert(result.isValidCoding);
decrypted.resize(result.messageLength);
assert(0x20 == result.messageLength);The resulting decrypted data is correct. but after the function containing the above exits (so all objects should have been destroyed) I get the following leaks reported:
Memory leak(s) found.Alloc num (658157) Leak size: 4 Allocated at: misc.h and line: 202. Type: "new" Memory: <01A698E8> Content: 0000: 28 63 a6 01 |(c..|Alloc num (658163) Leak size: 4 Allocated at: misc.h and line: 202. Type: "new" Memory: <01A695D0> Content: 0000: 50 3c 3c 01 |P<<.|Alloc num (658165) Leak size: 20 Allocated at: integer.cpp and line: 2922. Type: "new" Memory: <01A43798> Content: 0000: 48 0d 43 01 cd cd cd cd 02 00 00 00 d8 c2 9f 01 |H.C.............| 0010: 00 00 00 00 |....|FWIW, the first two leaks are in NewObject and the last one is in NewInteger (as I said, doesn't seem particularly helpful on its own).
Is this a known issue? Is there something else I need to call to clean up the memory?
Gavin Lambert
Jun 27, 2016, 6:08:05 PM6/27/16
to Crypto++ Users
The leaks still appear to be present with current master code.
Gavin Lambert
Jun 28, 2016, 4:56:37 AM6/28/16
to Crypto++ Users
Curiouser and curiouser. The prior tests were on Windows (VS2013) using the static library. I tried testing it on Linux with AddressSanitizer (Clang 3.4) but this doesn't report any leaks -- I even tried explicitly leaking some memory with a stray malloc and it didn't report that, so it's obviously not working. I'm not sure if something special is required to make it do something.
The above code runs and passes on both Windows and Linux, and I've verified that the final plaintext is correct (which you would hope is the case when the signature verifies). But keeping the preamble and replacing the sources with this:
This runs and passes on Windows, but on Linux while it runs and appears to generate the correct plaintext (I only verified the first hundred bytes, as the test data was fairly large), the assertion fails. Isn't this how you're supposed to chain filters? Why does it work on Windows but not Linux?
Trying Valgrind, it reported no definite CryptoPP leaks, but there were some singletons still allocated. I'm not sure if they match the ones found on Windows (there seem to be more of them) but for now I'm assuming this was a false positive.
But another weird behaviour surfaced when I was testing it on Linux. Following the prior RSA decrypt I added a more complex transformation:
CBC_Mode<AES>::Decryption aes(key.data(), key.size(), iv.data());
RSASS<PKCS1v15, SHA256>::Verifier verifier(publicKey);
auto snf = new SignatureVerificationFilter(verifier, new StringSink(plaintext),
SignatureVerificationFilter::PUT_MESSAGE | SignatureVerificationFilter::SIGNATURE_AT_BEGIN);
StringSource source1(ciphertext, true, new StreamTransformationFilter(aes, new StringSink(decrypted_data)));
StringSource source2(decrypted_data, true, new Gunzip(new StringSink(uncompressed_data)));
StringSource source3(uncompressed_data, true, snf);
assert(snf->GetLastResult());StringSource source(ciphertext, true, new StreamTransformationFilter(aes, new Gunzip(snf)));
assert(snf->GetLastResult());This runs and passes on Windows, but on Linux while it runs and appears to generate the correct plaintext (I only verified the first hundred bytes, as the test data was fairly large), the assertion fails. Isn't this how you're supposed to chain filters? Why does it work on Windows but not Linux?
Possibly of note is that merging source1 and source2 seems to be ok, but merging source2 and source3 is not. Perhaps Gunzip and SignatureVerificationFilter don't like each other (but only on Linux)?
Jeffrey Walton
Jun 29, 2016, 7:58:38 PM6/29/16
to Crypto++ Users
Still an issue in Windows, as far as I know: http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=106937
Also see http://www.cryptopp.com/wiki/Profiling. It discusses some (most?) of the issues/irregularities we are aware of.
Jeff
Gavin Lambert
Jul 7, 2016, 8:42:42 PM7/7/16
to Crypto++ Users
On Thursday, June 30, 2016 at 11:58:38 AM UTC+12, Jeffrey Walton wrote:
Still an issue in Windows, as far as I know: http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=106937
Connect issues seem to be private now, so I can't actually view that. I'm happy to treat it as a false positive though, as I said before.
What about the issue with Gunzip and SignatureVerificationFilter though? (For now I'm just working around it by doing them in separate stages, so I'm not pining for a fix, it's just an FYI.)
Jeffrey Walton
Jul 7, 2016, 9:10:23 PM7/7/16
to Crypto++ Users
On Thursday, July 7, 2016 at 8:42:42 PM UTC-4, Gavin Lambert wrote:
On Thursday, June 30, 2016 at 11:58:38 AM UTC+12, Jeffrey Walton wrote:
Still an issue in Windows, as far as I know: http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=106937Connect issues seem to be private now, so I can't actually view that. I'm happy to treat it as a false positive though, as I said before.What about the issue with Gunzip and SignatureVerificationFilter though?
If the library that tracks and reports leaks is unloaded before the memory is freed, then it will report the leak.
The best I can tell, the references to misc.h and integer.cpp are static objects. They apparently outlive the DLL reporting them as leaks.
If interested, you might try to force the load order of libraries. In this case, your leak checking DLL should be loaded before the Crypto++ library. You can usually control library load order by using `#pragma comment(lib, "xxx")`. You may have to build from the command line if the pragma does not work.
Jeff
Gavin Lambert
Jul 7, 2016, 9:27:59 PM7/7/16
to Crypto++ Users
On Friday, July 8, 2016 at 1:10:23 PM UTC+12, Jeffrey Walton wrote:
If the library that tracks and reports leaks is unloaded before the memory is freed, then it will report the leak.
The best I can tell, the references to misc.h and integer.cpp are static objects. They apparently outlive the DLL reporting them as leaks.
The leak tester I was using is unit-test based -- it reports anything created during the test and not destroyed by the end of the test case teardown as a leak. So yes, it's vulnerable to singletons, and as I've already said several times I'm satisfied that this is what is happening here and I'm no longer interested in it.
The Gunzip / SignatureVerificationFilter bug is entirely unrelated (and only occurs in Linux). I'm reasonably certain that this is a real bug.
Jeffrey Walton
Jul 7, 2016, 9:36:40 PM7/7/16
to Crypto++ Users
The Gunzip / SignatureVerificationFilter bug is entirely unrelated (and only occurs in Linux). I'm reasonably certain that this is a real bug.
OK, thanks.
Please check it under Valgrind. You may need to build Valgrind by hand because many distros still provide the down level 3.10 version.
To test the library and achieve accurate results, you must use -O0 or -O1 (http://valgrind.org/docs/manual/quick-start.html):
cd cryptopp
export CXXFLAGS="-DNDEBUG -g3 -O1"
make
Then, compile your program with the same CXXFLAGS. Finally, run your test against the library in that configuration under Valgrind.
If Valgrind reports the leak, then please open a bug report.
Jeff
Gavin Lambert
Jul 7, 2016, 10:29:07 PM7/7/16
to Crypto++ Users
On Friday, July 8, 2016 at 1:36:40 PM UTC+12, Jeffrey Walton wrote:
The Gunzip / SignatureVerificationFilter bug is entirely unrelated (and only occurs in Linux). I'm reasonably certain that this is a real bug.
OK, thanks.
Please check it under Valgrind. You may need to build Valgrind by hand because many distros still provide the down level 3.10 version.
No, by unrelated I mean it's not even a memory leak. (My fault for posting an unrelated issue in the same thread I guess.) It simply returns the wrong answer, presumably because it's accessing the wrong data somewhere.
Have another look at https://groups.google.com/d/msg/cryptopp-users/1q0r9zvDrdU/u2uQzJm7BwAJ.
Jeffrey Walton
Jul 9, 2016, 11:36:43 PM7/9/16
to Crypto++ Users
On Thursday, July 7, 2016 at 8:42:42 PM UTC-4, Gavin Lambert wrote:
On Thursday, June 30, 2016 at 11:58:38 AM UTC+12, Jeffrey Walton wrote:
Still an issue in Windows, as far as I know: http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=106937Connect issues seem to be private now, so I can't actually view that. I'm happy to treat it as a false positive though, as I said before.
Here's a related KB: "FIX: Using Run-Time Type Info May Cause Memory Leak Report" (https://support.microsoft.com/en-us/kb/140670).
Jeff
Reply all
Reply to author
Forward
0 new messages