Android5.0 with ART(Android Runtime) segfaults on Crypto++'s RSA::PublicKey Load function

[ja...@peekabu.com]

I posted the same question on StackOverflow but it might be Cryptopp specific issue:Enter code here.

http://stackoverflow.com/questions/28900187/android5-0-with-artandroid-runtime-segfaults-on-cryptos-rsapublickey-load

We have implemented a small wrapper for Cryptopp to exchange keys between iOS and Android(JNI). Shared code works on iOS and pre-ART android devices. Supposedly ART and its garbage collector is a lot stricter now.

It might be important to point out that set private key function succeeds on both Dalvik and ART.

Code:

--> setting public key :

   

 jboolean *isCopy;
    //get bytes from jbytearray
    jbyte* ba = (jbyte *)env->GetByteArrayElements( byteArray, isCopy);
   
    //load bytearray to crypto bytequeue
    ByteQueue queue2;
   
    queue2.Put2((byte*)ba, 1000, 0, true);
   
    //build public key
    AutoSeededRandomPool rnd;
    RSA::PublicKey publicKey;
    publicKey.Load(*queue2);  //<-------- CRASH

    

--> setting private key

   

 jboolean *isCopy;
    jbyte* ba = (jbyte *)env->GetByteArrayElements( byteArray, isCopy);
   
    //load bytearray to bytequeue
    ByteQueue queue2;
    queue2.Put2((byte*)ba, 3072, 0, true);
   
    //fill up the key
    RSA::PrivateKey privateKey;
    privateKey.Load(queue2);

Stack trace:

     

   

 backtrace:
         #00 pc 00027e6c  <project_name>/lib/arm/libstlport_shared.so
         #01 pc 00027e79  <project_name>/lib/arm/libstlport_shared.so
         #02 pc 00027efb  <project_name>/lib/arm/libstlport_shared.so (std::terminate()+6)
         #03 pc 000273d3  <project_name>/lib/arm/libstlport_shared.so
         #04 pc 000268c9  <project_name>/lib/arm/libstlport_shared.so
         #05 pc 0002698b  <project_name>/lib/arm/libstlport_shared.so (__cxa_throw+34)
         #06 pc 001b3ce4  <project_name>/lib/arm/libcryptopp.so (CryptoPP::BERDecodeError()+128)
         #07 pc 001b1598  <project_name>/lib/arm/libcryptopp.so (CryptoPP::BERGeneralDecoder::Init(unsigned char)+56)
         #08 pc 001b1638  <project_name>/lib/arm/libcryptopp.so (CryptoPP::BERGeneralDecoder::BERGeneralDecoder(CryptoPP::BufferedTransformation&, unsigned char)+104)
         #09 pc 0027697c  <project_name>/lib/arm/libcryptopp.so (CryptoPP::Integer::BERDecode(CryptoPP::BufferedTransformation&)+20)
         #10 pc 002aec7c  <project_name>/lib/arm/libcryptopp.so (CryptoPP::RSAFunction::BERDecodePublicKey(CryptoPP::BufferedTransformation&, bool, unsigned int)+64)
         #11 pc 001b20e0  <project_name>/lib/arm/libcryptopp.so (CryptoPP::X509PublicKey::BERDecode(CryptoPP::BufferedTransformation&)+264)
         #12 pc 00014a0b  <project_name>/lib/arm/libsecurity.so (CryptoPP::ASN1CryptoMaterial<CryptoPP::PublicKey>::Load(CryptoPP::BufferedTransformation&)+6)

Might be worth mentioning that new (mostly) Google devices (Nexus 4,5,7) use ART by default now.

Please advice!

[Jeffrey Walton]

Ho did you make out on this?