PyCrypto Errors with .p12 file from Google Developer Console when using gitkitv3 with GAE Python

[gaurav sinha]

I am trying to implement Google Identity Toolkit (gitkitv3) in GAE Python. After a user signs in on the website, I get the following errors:

'PKCS12 format is not supported by the PyCrpto library. '
NotImplementedError: PKCS12 format is not supported by the PyCrpto library. Try converting to a "PEM" (openssl pkcs12 -in xxxxx.p12 -nodes -nocerts > privatekey.pem) or using PyOpenSSL if native code is an option.

Based on a StackOverflow reply, I ran the following commands on my x.p12 file and used the generated privatekey.pem file instead:

openssl pkcs12 -passin pass:notasecret -in x.p12 -nocerts -passout pass:notasecret -out key.pem 
openssl pkcs8 -nocrypt -in key.pem -passin pass:notasecret -topk8 -out privatekey.pem 

Now, I am getting the following error:

'X509 certs are not supported by the PyCrypto library. '
NotImplementedError: X509 certs are not supported by the PyCrypto library. Try using PyOpenSSL if native code is an option. 

I had downloaded the x.p12 from Google Developer Console. How to fix this error? Please help

 I had downloaded the required supporting libraries (httplib2, oauth2client) from https://developers.google.com/api-client-library/python/start/installation#appengine

[Mengcheng Duan]

try only running the pkcs12 command:

openssl pkcs12 -in <key.p12> -nocerts -passin pass:notasecret -nodes -out <key.pem>

- Mengcheng

--
You received this message because you are subscribed to the Google Groups "Google Identity Toolkit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-identity-t...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[gaurav sinha]

Thanks Mengcheng. I tried the following command.

openssl pkcs12 -in myapp-1234.p12 -nocerts -passin pass:notasecret -nodes -out key.pem

I got the below error:

Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 1049, in add_context_wrapper
    return synctaskletfunc(*args, **kwds)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 1030, in synctasklet_wrapper
    return taskletfunc(*args, **kwds).get_result()
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 1003, in tasklet_wrapper
    result = func(*args, **kwds)
  File "/base/data/home/apps/s~myapp/1.378327257984780792/main_v3.py", line 1350, in get
    gitkit_user = gitkit_instance.VerifyGitkitToken (self.request.cookies['gtoken'])
  File "/base/data/home/apps/s~myapp/1.378327257984780792/gitkitclient.py", line 216, in VerifyGitkitToken
    certs = self.rpc_helper.GetPublicCert()
  File "/base/data/home/apps/s~myapp/1.378327257984780792/rpchelper.py", line 157, in GetPublicCert
    headers = {'Authorization': 'Bearer ' + self._GetAccessToken()}
  File "/base/data/home/apps/s~myapp/1.378327257984780792/rpchelper.py", line 196, in _GetAccessToken
    'assertion': self._GenerateAssertion(),
  File "/base/data/home/apps/s~myapp/1.378327257984780792/rpchelper.py", line 219, in _GenerateAssertion
    crypt.Signer.from_string(self.service_account_key),
  File "/base/data/home/apps/s~myapp/1.378327257984780792/oauth2client/crypt.py", line 237, in from_string
    'PKCS12 format is not supported by the PyCrpto library. '

NotImplementedError: PKCS12 format is not supported by the PyCrpto library. Try converting to a "PEM" (openssl pkcs12 -in xxxxx.p12 -nodes -nocerts > privatekey.pem) or using PyOpenSSL if native

Am I doing something wrong.

Relevant part of app.yaml:

libraries:
- name: pycrypto
  version: latest

- Mengcheng

To unsubscribe from this group and stop receiving emails from it, send an email to google-identity-toolkit+unsub...@googlegroups.com.

[gaurav sinha]

The contents of generated key.pem file:

Bag Attributes
    friendlyName: privatekey
    localKeyID: 54 69 6D 65 20 31 34 30 31 32 33 34 35 36 37 38 39 30
Key Attributes: <No Attributes>
-----BEGIN PRIVATE KEY-----
lots_of_characters_are_present_here_which_i_have_replaced_for_this_post=
-----END PRIVATE KEY-----

I have changed the value of "localKeyID:".

[Jin Liu]

The pem file should start with "-----BEGIN ". Could you try to delete the first 4 lines of your key.pem?

To unsubscribe from this group and stop receiving emails from it, send an email to google-identity-t...@googlegroups.com.

[gaurav sinha]

Thanks Jin. I changed the key.pem file to as shown below:

-----BEGIN PRIVATE KEY-----
lots_of_characters_are_present_here_which_i_have_replaced_for_this_post=
-----END PRIVATE KEY-----

I get the following error below:

Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 1049, in add_context_wrapper
    return synctaskletfunc(*args, **kwds)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 1030, in synctasklet_wrapper
    return taskletfunc(*args, **kwds).get_result()
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 1003, in tasklet_wrapper
    result = func(*args, **kwds)

  File "/base/data/home/apps/s~myapp/1.378327768548187440/main_v3.py", line 1350, in get

    gitkit_user = gitkit_instance.VerifyGitkitToken (self.request.cookies['gtoken'])

  File "/base/data/home/apps/s~myapp/1.378327768548187440/gitkitclient.py", line 219, in VerifyGitkitToken
    parsed = crypt.verify_signed_jwt_with_certs(jwt, certs, self.client_id)
  File "/base/data/home/apps/s~myapp/1.378327768548187440/oauth2client/crypt.py", line 338, in verify_signed_jwt_with_certs
    verifier = Verifier.from_string(pem, True)
  File "/base/data/home/apps/s~myapp/1.378327768548187440/oauth2client/crypt.py", line 190, in from_string
    'X509 certs are not supported by the PyCrypto library. '

NotImplementedError: X509 certs are not supported by the PyCrypto library. Try using PyOpenSSL if native code is an option.

- Mengcheng

To unsubscribe from this group and stop receiving emails from it, send an email to google-identity-toolkit+unsubscr...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[gaurav sinha]

Do I necessarily need this file .p12 file or can I copy its contents to a python global variable and use it?
Could someone please explain me the actual use of this file?

I am thinking of the above as workarounds since I am totally blocked in this part.

Appreciate any help

[Jin Liu]

According to the latest trace, your app has successfully read and parsed the pem file converted from the .p12 file. The problem now is that the underlying PyCrypto lib on AppEngine is quite limited and does not support X509 cert which is needed to verify the Google Identity Toolkit token.

There might be some possible workarounds. We will post the findings here.

FYI - the .p12 file contains the RSA private key, so that when you send a signed API request to Google service, the Google service can verify the request is from you, not anybody else.

--
You received this message because you are subscribed to the Google Groups "Google Identity Toolkit" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-identity-t...@googlegroups.com.

[gaurav sinha]

Thanks Jin for the confirmation that the pem (from .p12) file has been successfully parsed by the app. Now the next problem of reading X509 certificate exists. Thanks for explaining what this .p12 file does.

I am very eagerly awaiting a solution. Thanks for your support.

To unsubscribe from this group and stop receiving emails from it, send an email to google-identity-toolkit+unsub...@googlegroups.com.

[gaurav sinha]

Could this StackOverflow answer be used to find a workaround? I am not able to figure out what to change myself since I am not comfortable with code in oauth2client/crypt.py.
.

On Saturday, 30 August 2014 10:49:31 UTC+5:30, gaurav sinha wrote:

[ad...@bookhallonline.com]

Please help fix this issue. Unable to implement it.

[gaurav sinha]

I tried a workaround based on this StackOverflow reply by modifying oauth2client/crypt.py but got new errors:

Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 1049, in add_context_wrapper
    return synctaskletfunc(*args, **kwds)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 1030, in synctasklet_wrapper
    return taskletfunc(*args, **kwds).get_result()
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 1003, in tasklet_wrapper
    result = func(*args, **kwds)

  File "/base/data/home/apps/s~myapp/1.378382704328166956/main_v3.py", line 1350, in get

    gitkit_user = gitkit_instance.VerifyGitkitToken (self.request.cookies['gtoken'])

  File "/base/data/home/apps/s~myapp/1.378382704328166956/gitkitclient.py", line 219, in VerifyGitkitToken

    parsed = crypt.verify_signed_jwt_with_certs(jwt, certs, self.client_id)

  File "/base/data/home/apps/s~myapp/1.378382704328166956/oauth2client/crypt.py", line 363, in verify_signed_jwt_with_certs
    verifier = Verifier.from_string(pem, True)
  File "/base/data/home/apps/s~myapp/1.378382704328166956/oauth2client/crypt.py", line 209, in from_string
    tbsCertificate.decode(cert[0])
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/pycrypto-2.6/Crypto/Util/asn1.py", line 244, in decode
    idx += newInteger.decode(self.payload[idx:])
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/pycrypto-2.6/Crypto/Util/asn1.py", line 147, in decode
    raise ValueError ("Negative INTEGER.")
ValueError: Negative INTEGER.

[gaurav sinha]

I tried some more things:

I generated the new certificate using

openssl pkcs12 -clcerts -nokeys -in myapp-36.p12 -out usercert.pem

The generated certificate (replaced sensitive data):

Bag Attributes
    friendlyName: privatekey
    localKeyID: 66 77 88 99 22 33 44 55 66 77 88 99 20 35 56 65 88 68
subject=/CN=some_character_sequence_here_1.apps.googleusercontent.com
issuer=/CN=some_character_sequence_here_1.apps.googleusercontent.com
-----BEGIN CERTIFICATE-----
some_character_sequence_here_2
-----END CERTIFICATE-----

I tried and got the following error:

Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 1049, in add_context_wrapper
    return synctaskletfunc(*args, **kwds)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 1030, in synctasklet_wrapper
    return taskletfunc(*args, **kwds).get_result()
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 1003, in tasklet_wrapper
    result = func(*args, **kwds)

  File "/base/data/home/apps/s~myapp/1.378405227381861331/main_v3.py", line 1350, in get

    gitkit_user = gitkit_instance.VerifyGitkitToken (self.request.cookies['gtoken'])

  File "/base/data/home/apps/s~myapp/1.378405227381861331/gitkitclient.py", line 216, in VerifyGitkitToken
    certs = self.rpc_helper.GetPublicCert()
  File "/base/data/home/apps/s~myapp/1.378405227381861331/rpchelper.py", line 157, in GetPublicCert

    headers = {'Authorization': 'Bearer ' + self._GetAccessToken()}

  File "/base/data/home/apps/s~myapp/1.378405227381861331/rpchelper.py", line 196, in _GetAccessToken
    'assertion': self._GenerateAssertion(),
  File "/base/data/home/apps/s~myapp/1.378405227381861331/rpchelper.py", line 219, in _GenerateAssertion
    crypt.Signer.from_string(self.service_account_key),
  File "/base/data/home/apps/s~myapp/1.378405227381861331/oauth2client/crypt.py", line 262, in from_string
    'PKCS12 format is not supported by the PyCrpto library. '

NotImplementedError: PKCS12 format is not supported by the PyCrpto library. Try converting to a "PEM" (openssl pkcs12 -in xxxxx.p12 -nodes -nocerts > privatekey.pem) or using PyOpenSSL if native

Then I removed everything before "-----BEGIN PRIVATE KEY-----" and tried again. The new certificate:

-----BEGIN CERTIFICATE-----
some_character_sequence_here_2
-----END CERTIFICATE-----

This time, I got the following error:

Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 1049, in add_context_wrapper
    return synctaskletfunc(*args, **kwds)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 1030, in synctasklet_wrapper
    return taskletfunc(*args, **kwds).get_result()
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 1003, in tasklet_wrapper
    result = func(*args, **kwds)

  File "/base/data/home/apps/s~myapp/1.378404742263836203/main_v3.py", line 1350, in get

    gitkit_user = gitkit_instance.VerifyGitkitToken (self.request.cookies['gtoken'])

  File "/base/data/home/apps/s~myapp/1.378404742263836203/gitkitclient.py", line 216, in VerifyGitkitToken
    certs = self.rpc_helper.GetPublicCert()
  File "/base/data/home/apps/s~myapp/1.378404742263836203/rpchelper.py", line 157, in GetPublicCert

    headers = {'Authorization': 'Bearer ' + self._GetAccessToken()}

  File "/base/data/home/apps/s~myapp/1.378404742263836203/rpchelper.py", line 196, in _GetAccessToken
    'assertion': self._GenerateAssertion(),
  File "/base/data/home/apps/s~myapp/1.378404742263836203/rpchelper.py", line 219, in _GenerateAssertion
    crypt.Signer.from_string(self.service_account_key),
  File "/base/data/home/apps/s~myapp/1.378404742263836203/oauth2client/crypt.py", line 259, in from_string
    pkey = RSA.importKey(key)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/pycrypto-2.6/Crypto/PublicKey/RSA.py", line 665, in importKey
    return self._importKeyDER(der)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/pycrypto-2.6/Crypto/PublicKey/RSA.py", line 588, in _importKeyDER
    raise ValueError("RSA key format is not supported")
ValueError: RSA key format is not supported

On Saturday, 30 August 2014 10:49:31 UTC+5:30, gaurav sinha wrote:

[gaurav sinha]

Hi Jin Liu,
    Did you get a chance to figure out a solution / workaround for this problem? I have tried a few attempts at a solution based on StackOverflow answers but have been completely unsuccessful. It would be very helpful if you could share your findings too.

Thanks

Regards,
Gaurav

On Saturday, 30 August 2014 10:49:31 UTC+5:30, gaurav sinha wrote:

[Jin Liu]

The latest experimental version PyCrypto 2.7a1 can parse the Google Identity Toolkit X509 cert using Crypto.Util.asn1.DerSequence. We are still evaluating how Google Identity Toolkit python library can use that version on AppEngine.

[gaurav sinha]

Thanks a lot Jin for the good news. Could you please provide an expected time line (even a tentative one) when that experimental version would be available in production? This would help us better plan our launch schedule. Thanks.

[Jin Liu]

The PyCrypto is not part of AppEngine SDK and you need to include the PyCrypto lib in your own app. In addition to that, Google Identity Toolkit or oauth2client python library needs to be updated, which may take a couple of weeks.

I will contact you offline to see how we can help you before the formal rollout.

[Jin Liu]

Thanks for your patience. The fix has been rolled out. You can download the updated oauth2client from its project home page https://github.com/google/oauth2client.

In your GAE Python app, add the following lines to app.yaml:

libraries:

- name: pycrypto

  version: 2.6

And include the oauth2client lib along with your app.