RSA-PSS image signing: invalid salt length
Oliver Kaestner
I tried enabling RSA-PSS-signed images, but I received errors regarding the
salt length:
> [ERROR] : SWUPDATE failed [0] ERROR : EVP_PKEY_CTX_set_rsa_pss_saltlen failed, error error:1C800070:Provider routines::invalid salt length
I generated an RSA-PSS key pair explicitly, because it allows parameters such as
the hashing and MGF algorithm (here: SHA256) and the minimum salt length to be
specified. As far as I understand, keys generated this way are more secure
because they can only be used for signatures [1].
$ openssl genpkey -algorithm rsa-pss \
-pkeyopt rsa_keygen_bits:4096 \
-pkeyopt rsa_pss_keygen_md:sha256 \
-pkeyopt rsa_pss_keygen_mgf1_md:sha256 \
-pkeyopt rsa_pss_keygen_saltlen:32 \
-out swupdate_rsapss.pem
$ openssl rsa -in swupdate_rsapss.pem -pubout -out swupdate_rsapss.pem.pub
As described in RFC4055 [2] section 3.1, the parameters are encoded in the keys.
Section 3.3 of the same RFC states that if the any parameters are specified in
the public key, they must be validated.
The parameters can be read using `openssl asn1parse`. Note that the salt length
is displayed in hex (here: 0x20).
$ openssl asn1parse -in swupdate_rsapss.pem.pub
0:d=0 hl=4 l= 598 cons: SEQUENCE
4:d=1 hl=2 l= 65 cons: SEQUENCE
6:d=2 hl=2 l= 9 prim: OBJECT :rsassaPss
17:d=2 hl=2 l= 52 cons: SEQUENCE
19:d=3 hl=2 l= 15 cons: cont [ 0 ]
21:d=4 hl=2 l= 13 cons: SEQUENCE
23:d=5 hl=2 l= 9 prim: OBJECT :sha256
34:d=5 hl=2 l= 0 prim: NULL
36:d=3 hl=2 l= 28 cons: cont [ 1 ]
38:d=4 hl=2 l= 26 cons: SEQUENCE
40:d=5 hl=2 l= 9 prim: OBJECT :mgf1
51:d=5 hl=2 l= 13 cons: SEQUENCE
53:d=6 hl=2 l= 9 prim: OBJECT :sha256
64:d=6 hl=2 l= 0 prim: NULL
66:d=3 hl=2 l= 3 cons: cont [ 2 ]
68:d=4 hl=2 l= 1 prim: INTEGER :20
71:d=1 hl=4 l= 527 prim: BIT STRING
Currently, I cannot install .swu images signed with these keys using swupdate
due to the error above, when any key parameter is set. Skipping the argument
`-pkeyopt rsa_pss_keygen_saltlen:32` is not sufficient.
The reason seems to be that swupdate's verification implementation for openssl
explicitly sets the salt length to the default value -2 (RSA_PSS_SALTLEN_AUTO)
for some reason:
rc = EVP_PKEY_CTX_set_rsa_pss_saltlen(dgst->ckey, -2);
The OpenSSL docs [3] state that it's not necessary to call this function:
> EVP_PKEY_CTX_set_rsa_pss_saltlen() sets the RSA PSS salt length to saltlen.
> As its name implies it is only supported for PSS padding. If this function is
> not called then the salt length is maximized up to the digest length when
> signing and auto detection when verifying.
When compiling with CONFIG_SSL_IMPL_WOLFSSL, I even get build errors due to
calling this function:
crypto/swupdate_rsa_verify_openssl.c:78:22: warning: implicit declaration of function ‘EVP_PKEY_CTX_set_rsa_pss_saltlen’; did you mean ‘EVP_PKEY_CTX_set_rsa_padding’? [-Wimplicit-function-declaration]
78 | rc = EVP_PKEY_CTX_set_rsa_pss_saltlen(dgst->ckey, -2);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| EVP_PKEY_CTX_set_rsa_padding
[...]
==========
/usr/bin/ld: crypto/built-in.o: in function `openssl_rsa_verify_file':
swupdate_rsa_verify_wolfssl.c:(.text.openssl_rsa_verify_file+0x345): undefined reference to `EVP_PKEY_CTX_set_rsa_pss_saltlen'
collect2: error: ld returned 1 exit status
Another OpenSSL doc [4] also warns (the last part is especially important here):
> The EVP_PKEY_CTX_set_rsa_pss_saltlen() macro is used to set the salt length.
> If the key has usage restrictions then an error is returned if an attempt is
> made to set the salt length below the minimum value. It is otherwise similar
> to the RSA operation except detection of the salt length
> (using RSA_PSS_SALTLEN_AUTO) is not supported for verification if the key has
> usage restrictions.
So, in my understanding, the call to `EVP_PKEY_CTX_set_rsa_pss_saltlen()` can
and should be removed. However, I'm by far not a cryptography specialist, so
please correct me if I'm wrong.
Thanks for your continuous work
Oliver
---
Link: https://docs.openssl.org/3.5/man7/RSA-PSS/ [1]
Link: https://www.rfc-editor.org/rfc/rfc4055 [2]
Link: https://docs.openssl.org/3.5/man3/EVP_PKEY_CTX_ctrl/#rsa-parameters [3]
Link: https://docs.openssl.org/3.5/man3/EVP_PKEY_CTX_set_rsa_pss_keygen_md [4]