ECDSA signing problem in openssl-1.0.0 (was ok in 0.9.8)

[Brian Warner]

Hey folks, I'm tearing out some hair trying to figure out how to make
progress on a problem I'm having. I've got a pure-python ECDSA library
(http://github.com/warner/python-ecdsa) that includes some
interoperability tests against OpenSSL. The first test uses OpenSSL to
generate a keypair and sign a message, then uses the python library to
load the public key and verify the signature. The setup phase for this
creates a 'data.txt' and then runs the following pair of openssl
commands:

openssl ecparam -name secp384r1 -genkey -out privkey.pem
openssl dgst -sign privkey.pem -ecdsa-with-SHA1 -out data.sig data.txt

These commands worked fine on openssl-0.9.8, but now when I run them
against openssl-1.0.0a, the second one gives me the following error:

Error setting context
140735084903676:error:100C508A:elliptic curve
routines:PKEY_EC_CTRL:invalid digest type:ec_pmeth.c:229:

Some fussing with the source tells me that the ec_pmeth.c:229 code looks
like this:

case EVP_PKEY_CTRL_MD:
if (EVP_MD_type((const EVP_MD *)p2) != NID_sha1 &&
EVP_MD_type((const EVP_MD *)p2) != NID_sha224 &&
EVP_MD_type((const EVP_MD *)p2) != NID_sha256 &&
EVP_MD_type((const EVP_MD *)p2) != NID_sha384 &&
EVP_MD_type((const EVP_MD *)p2) != NID_sha512)
{
ECerr(EC_F_PKEY_EC_CTRL, EC_R_INVALID_DIGEST_TYPE);

and that EVP_MD_type(p2) is equal to 416, which in
include/openssl/obj_mac.h is #defined as NID_ecdsa_with_SHA1.

I'm assuming that the "ecparam -genkey" command I'm using to generate
the keypair is not correct, and that 0.9.8 was tolerating my mistake but
1.0.0a is not. That, or there's some funky bug that I don't understand.

I believe that this EVP_PKEY_CTRL_MD is a way for the private key to
tell a subsequent user (in this case the "dgst -sign" pass) how it wants
the message-being-signed to get hashed, but it is asking for a scheme
which the signing code doesn't know how to handle.

Is there anything obvious that I'm doing wrong here? I've found very few
examples of how these commands ought to look, so I'm mostly working by
experimentation. Has anyone else run into this?

thanks much,
-Brian
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openss...@openssl.org
Automated List Manager majo...@openssl.org

[Brian Warner]

> openssl ecparam -name secp384r1 -genkey -out privkey.pem
> openssl dgst -sign privkey.pem -ecdsa-with-SHA1 -out data.sig data.txt
>
> These commands worked fine on openssl-0.9.8, but now when I run them
> against openssl-1.0.0a, the second one gives me the following error:

Ah, figured it out. The necessary arguments changed from 0.9.8 to 1.0.0
. In newer versions of openssl, you need to use just "-SHA1" or leave
that argument out entirely: apparently it now deduces the necessary
message-digest type from the key itself:

openssl ecparam -name secp384r1 -genkey -out privkey.pem

openssl dgst -sign privkey.pem -SHA1 -out data.sig data.txt
openssl dgst -sign privkey.pem -out data.sig data.txt
# both work

To write code that can use either new or old versions of openssl, you'll
need to probe "openssl version" and switch on the output.

thanks,