RSA-PSS image signing: invalid salt length
Oliver Kaestner
I tried enabling RSA-PSS-signed images, but I received errors regarding the
salt length:
> [ERROR] : SWUPDATE failed [0] ERROR : EVP_PKEY_CTX_set_rsa_pss_saltlen failed, error error:1C800070:Provider routines::invalid salt length
I generated an RSA-PSS key pair explicitly, because it allows parameters such as
the hashing and MGF algorithm (here: SHA256) and the minimum salt length to be
specified. As far as I understand, keys generated this way are more secure
because they can only be used for signatures [1].
$ openssl genpkey -algorithm rsa-pss \
-pkeyopt rsa_keygen_bits:4096 \
-pkeyopt rsa_pss_keygen_md:sha256 \
-pkeyopt rsa_pss_keygen_mgf1_md:sha256 \
-pkeyopt rsa_pss_keygen_saltlen:32 \
-out swupdate_rsapss.pem
$ openssl rsa -in swupdate_rsapss.pem -pubout -out swupdate_rsapss.pem.pub
As described in RFC4055 [2] section 3.1, the parameters are encoded in the keys.
Section 3.3 of the same RFC states that if the any parameters are specified in
the public key, they must be validated.
The parameters can be read using `openssl asn1parse`. Note that the salt length
is displayed in hex (here: 0x20).
$ openssl asn1parse -in swupdate_rsapss.pem.pub
0:d=0 hl=4 l= 598 cons: SEQUENCE
4:d=1 hl=2 l= 65 cons: SEQUENCE
6:d=2 hl=2 l= 9 prim: OBJECT :rsassaPss
17:d=2 hl=2 l= 52 cons: SEQUENCE
19:d=3 hl=2 l= 15 cons: cont [ 0 ]
21:d=4 hl=2 l= 13 cons: SEQUENCE
23:d=5 hl=2 l= 9 prim: OBJECT :sha256
34:d=5 hl=2 l= 0 prim: NULL
36:d=3 hl=2 l= 28 cons: cont [ 1 ]
38:d=4 hl=2 l= 26 cons: SEQUENCE
40:d=5 hl=2 l= 9 prim: OBJECT :mgf1
51:d=5 hl=2 l= 13 cons: SEQUENCE
53:d=6 hl=2 l= 9 prim: OBJECT :sha256
64:d=6 hl=2 l= 0 prim: NULL
66:d=3 hl=2 l= 3 cons: cont [ 2 ]
68:d=4 hl=2 l= 1 prim: INTEGER :20
71:d=1 hl=4 l= 527 prim: BIT STRING
Currently, I cannot install .swu images signed with these keys using swupdate
due to the error above, when any key parameter is set. Skipping the argument
`-pkeyopt rsa_pss_keygen_saltlen:32` is not sufficient.
The reason seems to be that swupdate's verification implementation for openssl
explicitly sets the salt length to the default value -2 (RSA_PSS_SALTLEN_AUTO)
for some reason:
rc = EVP_PKEY_CTX_set_rsa_pss_saltlen(dgst->ckey, -2);
The OpenSSL docs [3] state that it's not necessary to call this function:
> EVP_PKEY_CTX_set_rsa_pss_saltlen() sets the RSA PSS salt length to saltlen.
> As its name implies it is only supported for PSS padding. If this function is
> not called then the salt length is maximized up to the digest length when
> signing and auto detection when verifying.
When compiling with CONFIG_SSL_IMPL_WOLFSSL, I even get build errors due to
calling this function:
crypto/swupdate_rsa_verify_openssl.c:78:22: warning: implicit declaration of function ‘EVP_PKEY_CTX_set_rsa_pss_saltlen’; did you mean ‘EVP_PKEY_CTX_set_rsa_padding’? [-Wimplicit-function-declaration]
78 | rc = EVP_PKEY_CTX_set_rsa_pss_saltlen(dgst->ckey, -2);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| EVP_PKEY_CTX_set_rsa_padding
[...]
==========
/usr/bin/ld: crypto/built-in.o: in function `openssl_rsa_verify_file':
swupdate_rsa_verify_wolfssl.c:(.text.openssl_rsa_verify_file+0x345): undefined reference to `EVP_PKEY_CTX_set_rsa_pss_saltlen'
collect2: error: ld returned 1 exit status
Another OpenSSL doc [4] also warns (the last part is especially important here):
> The EVP_PKEY_CTX_set_rsa_pss_saltlen() macro is used to set the salt length.
> If the key has usage restrictions then an error is returned if an attempt is
> made to set the salt length below the minimum value. It is otherwise similar
> to the RSA operation except detection of the salt length
> (using RSA_PSS_SALTLEN_AUTO) is not supported for verification if the key has
> usage restrictions.
So, in my understanding, the call to `EVP_PKEY_CTX_set_rsa_pss_saltlen()` can
and should be removed. However, I'm by far not a cryptography specialist, so
please correct me if I'm wrong.
Thanks for your continuous work
Oliver
---
Link: https://docs.openssl.org/3.5/man7/RSA-PSS/ [1]
Link: https://www.rfc-editor.org/rfc/rfc4055 [2]
Link: https://docs.openssl.org/3.5/man3/EVP_PKEY_CTX_ctrl/#rsa-parameters [3]
Link: https://docs.openssl.org/3.5/man3/EVP_PKEY_CTX_set_rsa_pss_keygen_md [4]
Stefano Babic
Last but not least, have you tested removing the call to the function
and is your SWU then verified ?
Best regards,
Stefano Babic
--
_______________________________________________________________________
Nabla Software Engineering GmbH
Hirschstr. 111A | 86156 Augsburg | Tel: +49 821 45592596
Geschäftsführer : Stefano Babic | HRB 40522 Augsburg
E-Mail: sba...@nabladev.com
Oliver Kästner
Mar 24 12:23:56 qemux86-64 swupdate.sh[275]: [TRACE] : SWUPDATE running : [extract_file_to_tmp] : filename sw-description
Mar 24 12:23:56 qemux86-64 swupdate.sh[275]: [TRACE] : SWUPDATE running : [extract_file_to_tmp] : size 1872
Mar 24 12:23:56 qemux86-64 swupdate.sh[275]: [TRACE] : SWUPDATE running : [extract_file_to_tmp] : Found file
Mar 24 12:23:56 qemux86-64 swupdate.sh[275]: [TRACE] : SWUPDATE running : [extract_file_to_tmp] : filename sw-description.sig
Mar 24 12:23:56 qemux86-64 swupdate.sh[275]: [TRACE] : SWUPDATE running : [extract_file_to_tmp] : size 512
Mar 24 12:23:56 qemux86-64 swupdate.sh[275]: [TRACE] : SWUPDATE running : [openssl_rsa_verify_file] : Verify signed image: Read 1872 bytes
Mar 24 12:23:56 qemux86-64 swupdate.sh[275]: [TRACE] : SWUPDATE running : [openssl_rsa_verify_file] : Verified OK
Stefano Babic
On 3/24/26 13:43, 'Oliver Kästner' via swupdate wrote:
> Hi Stefano,
>
> Yes, removing the five lines causes the signature verification to
> succeed without error:
>
> Mar 24 12:23:56 qemux86-64 swupdate.sh[275]: [TRACE] : SWUPDATE
> running : [extract_file_to_tmp] : Found file
> Mar 24 12:23:56 qemux86-64 swupdate.sh[275]: [TRACE] : SWUPDATE
> running : [extract_file_to_tmp] : filename sw-description
> Mar 24 12:23:56 qemux86-64 swupdate.sh[275]: [TRACE] : SWUPDATE
> running : [extract_file_to_tmp] : size 1872
> Mar 24 12:23:56 qemux86-64 swupdate.sh[275]: [TRACE] : SWUPDATE
> running : [extract_file_to_tmp] : Found file
> Mar 24 12:23:56 qemux86-64 swupdate.sh[275]: [TRACE] : SWUPDATE
> running : [extract_file_to_tmp] : filename sw-description.sig
> Mar 24 12:23:56 qemux86-64 swupdate.sh[275]: [TRACE] : SWUPDATE
> running : [extract_file_to_tmp] : size 512
> Mar 24 12:23:56 qemux86-64 swupdate.sh[275]: [TRACE] : SWUPDATE
> running : [openssl_rsa_verify_file] : Verify signed image: Read 1872 bytes
> Mar 24 12:23:56 qemux86-64 swupdate.sh[275]: [TRACE] : SWUPDATE
> running : [openssl_rsa_verify_file] : Verified OK
>
> I could prepare a patch for that, if it's okay for you.
>
Best regards,
Stefano Babic
> +49%20821%2045592596>
> --
> You received this message because you are subscribed to the Google
> Groups "swupdate" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to swupdate+u...@googlegroups.com
> <mailto:swupdate+u...@googlegroups.com>.
> To view this discussion visit https://groups.google.com/d/msgid/
> swupdate/fc9d2ee7-29b6-43fd-be69-3f2f9f6a14dfn%40googlegroups.com
> <https://groups.google.com/d/msgid/swupdate/fc9d2ee7-29b6-43fd-
> be69-3f2f9f6a14dfn%40googlegroups.com?utm_medium=email&utm_source=footer>.