BER decode error on loading EC private key.

[Vasily Demidenok]

Hello list,

I'm trying to use cryptopp for implementing small lib which will allow me to integrate with Android Pay.

I have difficulties with loading private key into Crypto++ wich is mentioned in https://developers.google.com/android-pay/integration/gateway-processor-integration#using-openssl-to-generate-and-format-a-key

 private static final String MERCHANT_PRIVATE_KEY_PKCS8_BASE64 = "MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgCPSuFr4iSIaQprjj" + "chHPyDu2NXFe0vDBoTpPkYaK9dehRANCAATnaFz/vQKuO90pxsINyVNWojabHfbx" + "9qIJ6uD7Q7ZSxmtyo/Ez3/o2kDT8g0pIdyVIYktCsq65VoQIDWSh2Bdm";

I tried it this way, using PEM_Load:

 string ecPriv = 
 "-----BEGIN EC PRIVATE KEY-----\n"
 "MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgCPSuFr4iSIaQprjj\n"
 "chHPyDu2NXFe0vDBoTpPkYaK9dehRANCAATnaFz/vQKuO90pxsINyVNWojabHfbx\n"
 "9qIJ6uD7Q7ZSxmtyo/Ez3/o2kDT8g0pIdyVIYktCsq65VoQIDWSh2Bdm\n"
 "-----END EC PRIVATE KEY-----\n";
 
 try
 {
 AutoSeededRandomPool prng2;
 CryptoPP::StringSource ss(ecPriv, true);
 
 ECIES<ECP>::PrivateKey privKey;
 
 PEM_Load(ss, privKey);
 
 bool valid = privKey.Validate(prng2, 3);
 if(!valid)
 std::cerr << "EC private key is not valid" << endl;

 cout << "SUCCESS 2222" << endl;
 
 }
 catch (const CryptoPP::Exception& ex)
 {
 std::cerr << ex.what() << endl;
 exit (1);
 }

And also manually:

 string PRIV_KEY =
 "-----BEGIN EC PRIVATE KEY-----\n"
 "MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgCPSuFr4iSIaQprjj\n"
 "chHPyDu2NXFe0vDBoTpPkYaK9dehRANCAATnaFz/vQKuO90pxsINyVNWojabHfbx\n"
 "9qIJ6uD7Q7ZSxmtyo/Ez3/o2kDT8g0pIdyVIYktCsq65VoQIDWSh2Bdm\n"
 "-----END EC PRIVATE KEY-----\n";

 static string HEADER = "-----BEGIN EC PRIVATE KEY-----";
 static string FOOTER = "-----END EC PRIVATE KEY-----";
 
 size_t pos1, pos2;
 pos1 = PRIV_KEY.find(HEADER);
 if(pos1 == string::npos)
 throw runtime_error("PEM header not found");
 
 pos2 = PRIV_KEY.find(FOOTER, pos1+1);
 if(pos2 == string::npos)
 throw runtime_error("PEM footer not found");
 
 // Start position and length
 pos1 = pos1 + HEADER.length();
 pos2 = pos2 - pos1;
 string keystr = PRIV_KEY.substr(pos1, pos2);

 // Base64 decode, place in a ByteQueue 
 ByteQueue queue;
 Base64Decoder decoder;

 decoder.Attach(new Redirector(queue));
 decoder.Put((const byte*)keystr.data(), keystr.length());
 decoder.MessageEnd();

 try
 {
 //CryptoPP::RSA::PrivateKey rsaPrivate;
 ECIES<ECP>::PrivateKey ePrivate;
 ePrivate.BERDecodePrivateKey(queue, false /*paramsPresent*/, queue.MaxRetrievable());

 AutoSeededRandomPool prng2;
 bool valid = ePrivate.Validate(prng2, 3);
 if(!valid)
 cerr << "RSA private key is not valid" << endl;

 cout << "SUCCESS" << endl;
 }
 catch (const CryptoPP::Exception& ex)
 {
 cerr << ex.what() << " BLA_BLA" << endl;
 exit (1);
 }

But with no luck, always getting:
BER decode error

Rather strange, but another EC key from pem library worked fine for me:
-----BEGIN EC PRIVATE KEY-----
MHQCAQEEIMcH2yHJcahaqSB0XdPKr+TotYB38wfV9caILqg9Jj7uoAcGBSuBBAAK
oUQDQgAEX7J/7ZwW1rO3HpoGxpYRv0PyvfeGKAmcXDVz3qJefS3ToGHYZhPVIyBY
i0xlRDP/EwFMilLCA9Rl/bD0E1hzGg==
-----END EC PRIVATE KEY-----

What I'm doing wrong here? 
I'm confused, that openssl successfully parse both of the keys.

[Jeffrey Walton]

On Wednesday, March 2, 2016 at 10:52:44 AM UTC-5, Vasily Demidenok wrote:

Hello list,

I'm trying to use cryptopp for implementing small lib which will allow me to integrate with Android Pay.

I have difficulties with loading private key into Crypto++ wich is mentioned in https://developers.google.com/android-pay/integration/gateway-processor-integration#using-openssl-to-generate-and-format-a-key

 private static final String MERCHANT_PRIVATE_KEY_PKCS8_BASE64 = "MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgCPSuFr4iSIaQprjj" + "chHPyDu2NXFe0vDBoTpPkYaK9dehRANCAATnaFz/vQKuO90pxsINyVNWojabHfbx" + "9qIJ6uD7Q7ZSxmtyo/Ez3/o2kDT8g0pIdyVIYktCsq65VoQIDWSh2Bdm";

I tried it this way, using PEM_Load:

 string ecPriv = 
 "-----BEGIN EC PRIVATE KEY-----\n"
 "MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgCPSuFr4iSIaQprjj\n"
 "chHPyDu2NXFe0vDBoTpPkYaK9dehRANCAATnaFz/vQKuO90pxsINyVNWojabHfbx\n"
 "9qIJ6uD7Q7ZSxmtyo/Ez3/o2kDT8g0pIdyVIYktCsq65VoQIDWSh2Bdm\n"
 "-----END EC PRIVATE KEY-----\n"; 

Ah, that's a PEM encoded PKCS #8 private key. The library only supports BER and DER out-of-the-box.

For PEM, you need a patch called the PEM Pack. You can find it at http://www.cryptopp.com/wiki/PEM_Pack.

Unzip it, and then recompile the library. Everything you need is picked up automatically under Linux. Under Windows, you need to add the CPP files to the cryptlib project.

Jeff

[Jeffrey Walton]

The website is down at the moment....

Email me off-list and I'll send you the ZIP file. noloader, gmail address.

Jeff

[Jeffrey Walton]

I have difficulties with loading private key into Crypto++ wich is mentioned in https://developers.google.com/android-pay/integration/gateway-processor-integration#using-openssl-to-generate-and-format-a-key

 private static final String MERCHANT_PRIVATE_KEY_PKCS8_BASE64 = "MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgCPSuFr4iSIaQprjj" + "chHPyDu2NXFe0vDBoTpPkYaK9dehRANCAATnaFz/vQKuO90pxsINyVNWojabHfbx" + "9qIJ6uD7Q7ZSxmtyo/Ez3/o2kDT8g0pIdyVIYktCsq65VoQIDWSh2Bdm";

My bad... I missed that you were already using the PEM Pack.

Try this, it works for me:

#include <iostream>
#include <string>
using namespace std;

#include "files.h"
#include "filters.h"
#include "base64.h"
#include "osrng.h"
#include "eccrypto.h"
using namespace CryptoPP;

string encoded =
    "MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgCPSuFr4iSIaQprjj"
    "chHPyDu2NXFe0vDBoTpPkYaK9dehRANCAATnaFz/vQKuO90pxsINyVNWojabHfbx"
    "9qIJ6uD7Q7ZSxmtyo/Ez3/o2kDT8g0pIdyVIYktCsq65VoQIDWSh2Bdm";

int main(int argc, char* argv[])
{
    try
    {
        string decoded;
        StringSource ss1(encoded.c_str(), true, new Base64Decoder(new StringSink(decoded)));       
        StringSource ss2((const byte*)decoded.data(), decoded.size(), true, new FileSink("key.der"));

        DL_PrivateKey_EC<ECP> key;
        key.Load(StringStore((const byte*)decoded.data(), decoded.size()).Ref());

        AutoSeededRandomPool prng;
        key.ThrowIfInvalid(prng, 3);
       
        cout << "Private key appears to be valid" << endl;
    }
    catch(Exception& ex)
    {
        cerr << ex.what() << endl;
        return 1;
    }
   
    return 0;
}

And:

$ g++ -DNDEBUG -g2 -O2 -I . test.cxx -o test.exe ./libcryptopp.a
$ ./test.exe
Private key appears to be valid

[Jeffrey Walton]

On Wednesday, March 2, 2016 at 10:12:01 PM UTC-5, Jeffrey Walton wrote:

I have difficulties with loading private key into Crypto++ wich is mentioned in https://developers.google.com/android-pay/integration/gateway-processor-integration#using-openssl-to-generate-and-format-a-key

 private static final String MERCHANT_PRIVATE_KEY_PKCS8_BASE64 = "MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgCPSuFr4iSIaQprjj" + "chHPyDu2NXFe0vDBoTpPkYaK9dehRANCAATnaFz/vQKuO90pxsINyVNWojabHfbx" + "9qIJ6uD7Q7ZSxmtyo/Ez3/o2kDT8g0pIdyVIYktCsq65VoQIDWSh2Bdm";

My bad... I missed that you were already using the PEM Pack.

Try this, it works for me...

I probably should have given you this, too. It uses ECIES<ECP> rather than the private key.

string encoded =
    "MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgCPSuFr4iSIaQprjj"
    "chHPyDu2NXFe0vDBoTpPkYaK9dehRANCAATnaFz/vQKuO90pxsINyVNWojabHfbx"
    "9qIJ6uD7Q7ZSxmtyo/Ez3/o2kDT8g0pIdyVIYktCsq65VoQIDWSh2Bdm";

int main(int argc, char* argv[])
{
    try
    {
        string decoded;
        StringSource ss1(encoded.c_str(), true, new Base64Decoder(new StringSink(decoded)));       

        ECIES<ECP>::Decryptor decryptor;
        decryptor.AccessKey().Load(StringStore((const byte*)decoded.data(), decoded.size()).Ref());
        // AccessKey() provides a non-const reference

        AutoSeededRandomPool prng;
        decryptor.GetKey().ThrowIfInvalid(prng, 3);
        // GetKey() provides a const reference

        
        cout << "Private key appears to be valid" << endl;
    }
    catch(Exception& ex)
    {
        cerr << ex.what() << endl;
        return 1;
    }
   
    return 0;
}

It produces the same result:

[Jeffrey Walton]

I'm trying to use cryptopp for implementing small lib which will allow me to integrate with Android Pay.

As you work through issues, I'm going to be adding it to our ECIES page at http://cryptopp.com/wiki/Elliptic_Curve_Integrated_Encryption_Scheme#Android_Pay_Example

This is the sort of thing I'd like to see well documented.

I also reached out to the Bouncy Castle folks. We are on the same page ECIES and interop, so they can ensure their library works with the Android Pay example, too.

Jeff

[Vasily Demidenok]

Hello Jeffrey,

Thank you for your quick answers, now it works fine, although I do not understand what I was doing wrong.

So now I stuck on trying to load ephemeralPublicKey in the Decryptor. While it's mentioned in the Android docs, that they provide it, it's seems to me that it
is not actually the key, because of the following code:

ECParameterSpec aasymmetricKeyParams = generateECParameterSpec(); 
KeyFactory asymmetricKeyFactory = KeyFactory.getInstance(ASYMMETRIC_KEY_TYPE, SECURITY_PROVIDER); 
PublicKey ephemeralPublicKey = asymmetricKeyFactory.generatePublic( 
 new ECPublicKeySpec( ECPointUtil.decodePoint(asymmetricKeyParams.getCurve(), ephemeralPublicKeyBytes), asymmetricKeyParams));

But even if I will succeed with generating it out of EC type (I should use CryptoPP::ASN1::secp256r1() I guess)
and the provided ephemeralPublicKeyBytes, it's still not clear for me, how to provide this information to the Decryptor.

It seems to me that we get ephemeralPublicKey here.
https://github.com/weidai11/cryptopp/blob/master/pubkey.h#L1668

But I have no idea how to store it there.. also rather confusing that we call params.DecodeElement(ciphertext, true).

It would be great if you could point me to the right direction.

четверг, 3 марта 2016 г., 4:48:06 UTC+1 пользователь Jeffrey Walton написал:

[Vasily Demidenok]

Ok, so I have passed two stages now - generating the ephemeralPubKey and parsing the Private key using your code.

Just in case someone would be interested it should be smth like this:

string ephemeralPubKey64 =
            "BPhVspn70Zj2Kkgu9t8+ApEuUWsI/zos5whGCQBlgOkuYagOis7"
            "qsrcbQrcprjvTZO3XOU+Qbcc28FSgsRtcgQE=";
        StringSource ssECPoint(ephemeralPubKey64, true, new Base64Decoder);
       
        DL_GroupParameters_EC<ECP> params(CryptoPP::ASN1::secp256r1());
        ECP::Point point;
       
        if (!params.GetCurve().DecodePoint(point, ssECPoint, (size_t)ssECPoint.MaxRetrievable()))
            std::cerr << "Invalid decoding" << endl;
       
        ECIES<ECP>::PublicKey ephemeralPubKey;
        ephemeralPubKey.Initialize(params, point);
 
        ephemeralPubKey.ThrowIfInvalid(prng, 3);
       
        cout << "Public key appears to be valid" << endl;

But still need to figure out how to pass it to Decryptor.

четверг, 3 марта 2016 г., 13:33:58 UTC+1 пользователь Vasily Demidenok написал:

[Jeffrey Walton]

string ephemeralPubKey64 =
            "BPhVspn70Zj2Kkgu9t8+ApEuUWsI/zos5whGCQBlgOkuYagOis7"
            "qsrcbQrcprjvTZO3XOU+Qbcc28FSgsRtcgQE=";
        StringSource ssECPoint(ephemeralPubKey64, true, new Base64Decoder);
       
        DL_GroupParameters_EC<ECP> params(CryptoPP::ASN1::secp256r1());
        ECP::Point point;
       
        if (!params.GetCurve().DecodePoint(point, ssECPoint, (size_t)ssECPoint.MaxRetrievable()))
            std::cerr << "Invalid decoding" << endl;
       
        ECIES<ECP>::PublicKey ephemeralPubKey;
        ephemeralPubKey.Initialize(params, point);
 
        ephemeralPubKey.ThrowIfInvalid(prng, 3);
       
        cout << "Public key appears to be valid" << endl;

But still need to figure out how to pass it to Decryptor

Decryptors use the private key; Encryptor's use the public key.

Its not readily apparent... the way to work with an ephemeral public key is detailed at "Ephemeral Key as (x,y)" under the ECDH page, http://www.cryptopp.com/wiki/Elliptic_Curve_Diffie-Hellman#Ephemeral_Key_as_.28x.2Cy.29 . It applies to ECIES as well.

The pain point in using ephemeral keys is they are meant for one run of the protocol, so they don't lend themselves to manipulation outside of the scheme objects. The static or long term keys are a different story.

An ephemeral key is meant to be used by, for example, SimpleKeyAgreementDomain, which culminates in a call to Agree(...). Since the SimpleKeyAgreementDomain does all the work, there's usually less support for importing and exporting the ephemeral key. It can be done, but you have to understand the details.

Jeff

[Vasily Demidenok]

Thank you Jeff, 

I managed to write the code related to the key agreement and validated that at least I got the same sharedSecret as in the example with Bouncy Castle. 

So now I've got the questions related to KeyDerivationAlgo and EncryptionAlgo. 

1.  According to Bouncy Castle compatability patch and java example as I understand I need to instantiate:

DL_KeyDerivationAlgorithm_P1363<ECP::Point, true /*DHAES_MODE*/, P1363_KDF2<SHA256> > derieveAlgo;
DL_EncryptionAlgorithm_Xor<HMAC<SHA256>, true /*DHAES_MODE*/, true /*BC_COMPAT*/> encAlgo;

In the documentation they mention DEM2 from ISO 18033-2. 

Is the encAlgo definition above fully compatible with that?

2. One more thing that I'm missing right now - which params should I store in NameValuePairs?

I guess for:

 - deriviation they should be at least: null sault, info string Android encoded in Ascii.

 - encryption algo: zero IV and no padding

But I have no idea how to specify them properly or if I missed any of them. Documentation for NameValuePairs didn't help.

Also I failed to find any examples/tests which specify KeyDerivationParameters or EncodingParameters.

Any help would be appreciated.

четверг, 3 марта 2016 г., 18:46:40 UTC+1 пользователь Jeffrey Walton написал:

[Jeffrey Walton]

So now I've got the questions related to KeyDerivationAlgo and EncryptionAlgo. 

1.  According to Bouncy Castle compatability patch and java example as I understand I need to instantiate:

DL_KeyDerivationAlgorithm_P1363<ECP::Point, true /*DHAES_MODE*/, P1363_KDF2<SHA256> > derieveAlgo;
DL_EncryptionAlgorithm_Xor<HMAC<SHA256>, true /*DHAES_MODE*/, true /*BC_COMPAT*/> encAlgo;

In the documentation they mention DEM2 from ISO 18033-2. 

Is the encAlgo definition above fully compatible with that?

I have a copy of ISO 18033-2. I should probably copy that section and get you a copy for interop/academic/research purposes since you are actively working the problem. Can you email me off list at noloader, gmail account?

Eventually, I think we should provide something like the following. Then all these interop problems go away.

    ECIES_ANDROID_PAY<...>

We can provide it in the library proper, on the ECIES page under Android Pay, or on the Patch Page (http://www.cryptopp.com/wiki/Category:Patch). The point is, its available and "it just works" so users don't suffer the interop.

I really like ECIES, but I really despise all of the interop problems with it. No wonder it has never gained popularity.

Jeff

[Jeffrey Walton]

2. One more thing that I'm missing right now - which params should I store in NameValuePairs?

I guess for:

 - deriviation they should be at least: null sault, info string Android encoded in Ascii.

This looks like what you need for the KDF:

template <class H>
class AndroidPay_KDF
{
public:
    static void CRYPTOPP_API DeriveKey(byte *output, size_t outputLength, const byte *input, size_t inputLength,
            const byte *derivationParams, size_t derivationParamsLength)
    {
        CRYPTOPP_UNUSED(derivationParams), CRYPTOPP_UNUSED(derivationParamsLength);

        static const byte INFO[] = "Android";
        static const size_t INFO_SIZE = 7;
        
        HKDF<H> hkdf;
        hkdf.DeriveKey(output, outputLength, input, inputLength, NULL, 0, INFO, INFO_SIZE);
    }
};
 

 - encryption algo: zero IV and no padding

This will probably be specified in the Encryption Algorithm (EA) to the Discrete Log Encryption Scheme (DL_ES). For ECIES (https://www.cryptopp.com/docs/ref/struct_e_c_i_e_s.html), that is:

    DL_EncryptionAlgorithm_Xor

Here's the full view of ECIES (http://www.cryptopp.com/docs/ref/eccrypto_8h_source.html#l00284):

289 template <class EC, class COFACTOR_OPTION = NoCofactorMultiplication, bool DHAES_MODE = false>
290 struct ECIES
291  : public DL_ES<
292  DL_Keys_EC<EC>,
293  DL_KeyAgreementAlgorithm_DH<typename EC::Point, COFACTOR_OPTION>,
294  DL_KeyDerivationAlgorithm_P1363<typename EC::Point, DHAES_MODE, P1363_KDF2<SHA1> >,
295  DL_EncryptionAlgorithm_Xor<HMAC<SHA1>, DHAES_MODE>,
296  ECIES<EC> >
297 {
298  static std::string CRYPTOPP_API StaticAlgorithmName() {return "ECIES";}
        ...
305 };

We'll still need to map Android Pay specs to ECIES terminology, like Cofactor Options and DHAES_MODE. Some of this has been documented in Crypto++, like COFACTOR_OPTIONS (http://www.cryptopp.com/docs/ref/pubkey_8h.html#ae4b59f7b9d3c7e03bb739f0584905ff1). See, for example, "Methods for Avoiding the "Small-Subgroup" Attacks on the Diffie-Hellman Key Agreement Method", http://tools.ietf.org/html/rfc2785.

I'm not familiar with other terminology like CheckMode and SingleHashMode, but I believe the answer lies in the implementation of the algorithm. Once you get there, it will be obvious what they meant.

Jeff