Malformed Trustwave certificates in Mozilla's ca cert collection

[Jeffrey Walton]

Hi Everyone,

We use cURL's cacert.pem to validate our implementation. cURL's
cacert.pem uses Mozilla's certificates as a source.[1] When testing
our code with cURL's cacert.pem we are seeing errors. The errors are
relatively new. The code has been solid for the last 4 or 5 years.

It looks like Trustwave is distributing malformed certificates. Below
Gutmann's dumpasn1 is having trouble, too. Notice the ''Error:
Spurious zero bits in bitstring":

$ openssl x509 -in trustwave-1.pem -inform PEM -out trustwave-1.der -outform DER
$ dumpasn1 trustwave-1.der
0 608: SEQUENCE {
4 519: SEQUENCE {
8 3: [0] {
10 1: INTEGER 2
: }
13 12: INTEGER 0D 6A 5F 08 3F 28 5C 3E 51 95 DF 5D
27 10: SEQUENCE {
29 8: OBJECT IDENTIFIER ecdsaWithSHA256 (1 2 840 10045 4 3 2)
: }
39 145: SEQUENCE {
42 11: SET {
44 9: SEQUENCE {
46 3: OBJECT IDENTIFIER countryName (2 5 4 6)
51 2: PrintableString 'US'
: }
: }
55 17: SET {
57 15: SEQUENCE {
59 3: OBJECT IDENTIFIER stateOrProvinceName (2 5 4 8)
64 8: PrintableString 'Illinois'
: }
: }
74 16: SET {
76 14: SEQUENCE {
78 3: OBJECT IDENTIFIER localityName (2 5 4 7)
83 7: PrintableString 'Chicago'
: }
: }
92 33: SET {
94 31: SEQUENCE {
96 3: OBJECT IDENTIFIER organizationName (2 5 4 10)
101 24: PrintableString 'Trustwave Holdings, Inc.'
: }
: }
127 58: SET {
129 56: SEQUENCE {
131 3: OBJECT IDENTIFIER commonName (2 5 4 3)
136 49: PrintableString
: 'Trustwave Global ECC P256 Certification Authorit'
: 'y'
: }
: }
: }
187 30: SEQUENCE {
189 13: UTCTime 23/08/2017 19:35:10 GMT
204 13: UTCTime 23/08/2042 19:35:10 GMT
: }
219 145: SEQUENCE {
222 11: SET {
224 9: SEQUENCE {
226 3: OBJECT IDENTIFIER countryName (2 5 4 6)
231 2: PrintableString 'US'
: }
: }
235 17: SET {
237 15: SEQUENCE {
239 3: OBJECT IDENTIFIER stateOrProvinceName (2 5 4 8)
244 8: PrintableString 'Illinois'
: }
: }
254 16: SET {
256 14: SEQUENCE {
258 3: OBJECT IDENTIFIER localityName (2 5 4 7)
263 7: PrintableString 'Chicago'
: }
: }
272 33: SET {
274 31: SEQUENCE {
276 3: OBJECT IDENTIFIER organizationName (2 5 4 10)
281 24: PrintableString 'Trustwave Holdings, Inc.'
: }
: }
307 58: SET {
309 56: SEQUENCE {
311 3: OBJECT IDENTIFIER commonName (2 5 4 3)
316 49: PrintableString
: 'Trustwave Global ECC P256 Certification Authorit'
: 'y'
: }
: }
: }
367 89: SEQUENCE {
369 19: SEQUENCE {
371 7: OBJECT IDENTIFIER ecPublicKey (1 2 840 10045 2 1)
380 8: OBJECT IDENTIFIER prime256v1 (1 2 840 10045 3 1 7)
: }
390 66: BIT STRING
: 04 7E FB 6C E6 23 E3 73 32 08 CA 60 E6 53 9C BA
: 74 8D 18 B0 78 90 52 80 DD 38 C0 4A 1D D1 A8 CC
: 93 A4 97 06 38 CA 0D 15 62 C6 8E 01 2A 65 9D AA
: DF 34 91 2E 81 C1 E4 33 92 31 C4 FD 09 3A A6 3F
: AD
: }
458 67: [3] {
460 65: SEQUENCE {
462 15: SEQUENCE {
464 3: OBJECT IDENTIFIER basicConstraints (2 5 29 19)
469 1: BOOLEAN TRUE
472 5: OCTET STRING, encapsulates {
474 3: SEQUENCE {
476 1: BOOLEAN TRUE
: }
: }
: }
479 15: SEQUENCE {
481 3: OBJECT IDENTIFIER keyUsage (2 5 29 15)
486 1: BOOLEAN TRUE
489 5: OCTET STRING, encapsulates {
491 3: BIT STRING 7 unused bits
: '001100000'B
: Error: Spurious zero bits in bitstring.
: }
: }
496 29: SEQUENCE {
498 3: OBJECT IDENTIFIER subjectKeyIdentifier (2 5 29 14)
503 22: OCTET STRING, encapsulates {
505 20: OCTET STRING
: A3 41 06 AC 90 6D D1 4A EB 75 A5 4A 10 99 B3 B1
: A1 8B 4A F7
: }
: }
: }
: }
: }
527 10: SEQUENCE {
529 8: OBJECT IDENTIFIER ecdsaWithSHA256 (1 2 840 10045 4 3 2)
: }
539 71: BIT STRING, encapsulates {
542 68: SEQUENCE {
544 32: INTEGER
: 07 E6 54 DA 0E A0 5A B2 AE 11 9F 87 C5 B6 FF 69
: DE 25 BE F8 A0 B7 08 F3 44 CE 2A DF 08 21 0C 37
578 32: INTEGER
: 2D 26 03 A0 05 BD 6B D1 F6 5C F8 65 CC 86 6D B3
: 9C 34 48 63 84 09 C5 8D 77 1A E2 CC 9C E1 74 7B
: }
: }
: }

0 warnings, 1 error.

Here are the certificates:

Trustwave Global ECC P256 Certification Authority
=================================================
-----BEGIN CERTIFICATE-----
MIICYDCCAgegAwIBAgIMDWpfCD8oXD5Rld9dMAoGCCqGSM49BAMCMIGRMQswCQYDVQQGEwJVUzER
MA8GA1UECBMISWxsaW5vaXMxEDAOBgNVBAcTB0NoaWNhZ28xITAfBgNVBAoTGFRydXN0d2F2ZSBI
b2xkaW5ncywgSW5jLjE6MDgGA1UEAxMxVHJ1c3R3YXZlIEdsb2JhbCBFQ0MgUDI1NiBDZXJ0aWZp
Y2F0aW9uIEF1dGhvcml0eTAeFw0xNzA4MjMxOTM1MTBaFw00MjA4MjMxOTM1MTBaMIGRMQswCQYD
VQQGEwJVUzERMA8GA1UECBMISWxsaW5vaXMxEDAOBgNVBAcTB0NoaWNhZ28xITAfBgNVBAoTGFRy
dXN0d2F2ZSBIb2xkaW5ncywgSW5jLjE6MDgGA1UEAxMxVHJ1c3R3YXZlIEdsb2JhbCBFQ0MgUDI1
NiBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABH77bOYj
43MyCMpg5lOcunSNGLB4kFKA3TjASh3RqMyTpJcGOMoNFWLGjgEqZZ2q3zSRLoHB5DOSMcT9CTqm
P62jQzBBMA8GA1UdEwEB/wQFMAMBAf8wDwYDVR0PAQH/BAUDAwcGADAdBgNVHQ4EFgQUo0EGrJBt
0UrrdaVKEJmzsaGLSvcwCgYIKoZIzj0EAwIDRwAwRAIgB+ZU2g6gWrKuEZ+Hxbb/ad4lvvigtwjz
RM4q3wghDDcCIC0mA6AFvWvR9lz4ZcyGbbOcNEhjhAnFjXca4syc4XR7
-----END CERTIFICATE-----

Trustwave Global ECC P384 Certification Authority
=================================================
-----BEGIN CERTIFICATE-----
MIICnTCCAiSgAwIBAgIMCL2Fl2yZJ6SAaEc7MAoGCCqGSM49BAMDMIGRMQswCQYDVQQGEwJVUzER
MA8GA1UECBMISWxsaW5vaXMxEDAOBgNVBAcTB0NoaWNhZ28xITAfBgNVBAoTGFRydXN0d2F2ZSBI
b2xkaW5ncywgSW5jLjE6MDgGA1UEAxMxVHJ1c3R3YXZlIEdsb2JhbCBFQ0MgUDM4NCBDZXJ0aWZp
Y2F0aW9uIEF1dGhvcml0eTAeFw0xNzA4MjMxOTM2NDNaFw00MjA4MjMxOTM2NDNaMIGRMQswCQYD
VQQGEwJVUzERMA8GA1UECBMISWxsaW5vaXMxEDAOBgNVBAcTB0NoaWNhZ28xITAfBgNVBAoTGFRy
dXN0d2F2ZSBIb2xkaW5ncywgSW5jLjE6MDgGA1UEAxMxVHJ1c3R3YXZlIEdsb2JhbCBFQ0MgUDM4
NCBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTB2MBAGByqGSM49AgEGBSuBBAAiA2IABGvaDXU1CDFH
Ba5FmVXxERMuSvgQMSOjfoPTfygIOiYaOs+Xgh+AtycJj9GOMMQKmw6sWASr9zZ9lCOkmwqKi6vr
/TklZvFe/oyujUF5nQlgziip04pt89ZF1PKYhDhloKNDMEEwDwYDVR0TAQH/BAUwAwEB/zAPBgNV
HQ8BAf8EBQMDBwYAMB0GA1UdDgQWBBRVqYSJ0sEyvRjLbKYHTsjnnb6CkDAKBggqhkjOPQQDAwNn
ADBkAjA3AZKXRRJ+oPM+rRk6ct30UJMDEr5E0k9BpIycnR+j9sKS50gU/k6bpZFXrsY3crsCMGcl
CrEMXu6pY5Jv5ZAL/mYiykf9ijH3g/56vxC+GCsej/YpHpRZ744hN8tRmKVuSw==
-----END CERTIFICATE-----

[1] https://curl.se/docs/caextract.html

Jeff

[Aaron Gable]

I just had to do a bunch of digging to fully understand what's going on here, so I figured I'd share for the benefit of everyone else.

The keyUsage extension bitstring in these certificates is encoded as 03 03 07 06 00. Those bytes are interpreted as follows:
03: This item is a BIT STRING

03: This item has a length of 3 bytes
07: The least significant 7 bits of the remaining bytes should be ignored
06 00: The bits 0000 0110 0XXX XXXX (with Xs representing ignored bits as indicated by the 07 byte)

This means that the final keyUsage bitstring is 000001100. This has a length of 9 bits, which makes some sense given that there are 9 possible bits (indexed 0 through 8) that can be set in the keyUsage extension specification. This bit string indicates the keyCertSign (5) and crlSign (6) key usages.

However, there is another (shorter!) way to encode the exact same value. There are two zero-bits at the end of that nine-bit string, so it could instead be encoded as 03 02 01 06:
03: This item is a BIT STRING

02: This item has a length of 2 bytes

01: The least significant 1 bit of the remaining byte should be ignored

06: The bits 0000 011X

This results in a keyUsage bit string of 0000011, which also equivalently indicates the keyCertSign (5) and crlSign (6) key usages.

Finally, the Distinguished Encoding Rules (DER, X.690) requires that this second shorter encoding be used:

> 11.2.2 Where Rec. ITU-T X.680 | ISO/IEC 8824-1, 22.7, applies, the bitstring shall have all trailing 0 bits removed before it is encoded.

And the cited section of X.680 says:

> 22.7 When a "NamedBitList" is used in defining a bitstring type ASN.1 encoding rules are free to add (or remove) arbitrarily any trailing 0 bits to (or from) values that are being encoded or decoded. Application designers should therefore ensure that different semantics are not associated with such values which differ only in the number of trailing 0 bits.

The keyUsage extension is specified using the "NamedBitList" syntax, so BER says arbitrarily many trailing 0 bits may be dropped, so DER says they must be dropped. Therefore the encoding used in the two certificates in this thread is incorrect.

Aaron

--
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.
To view this discussion on the web visit https://groups.google.com/a/mozilla.org/d/msgid/dev-security-policy/CAH8yC8m7c3Dg8DEt%3DTT0Hh1icRXOE356aEu6iEUegTwxvNtWRQ%40mail.gmail.com.

[Corey Bonnell]

I’d also like to add that amongst the linters that are available publicly, only x509lint (https://github.com/kroeckx/x509lint) flags this error. Neither zlint or certlint flag this.

 

It would probably be useful for Mozilla to add running x509lint alongside zlint and cablint as part of its evaluation process for new root certificate inclusions.

 

Thanks,

Corey

To view this discussion on the web visit https://groups.google.com/a/mozilla.org/d/msgid/dev-security-policy/CAEmnErfQ64AzUUN-syGL%2B%2BxxZFn-UXs9uD7NaeTEygEMhvc3ow%40mail.gmail.com.

[Peter Gutmann]

'Corey Bonnell' via dev-secur...@mozilla.org writes:

>I’d also like to add that amongst the linters that are available publicly,
>only x509lint (https://github.com/kroeckx/x509lint) flags this error. Neither
>zlint or certlint flag this.

Or run them through dumpasn1, that's been flagging this, and many other
issues, for years...

Peter.

[Corey Bonnell]

> Or run them through dumpasn1, that's been flagging this, and many other
issues, for years...

I think that's another good check. Automatable too since you return the
error count as the process return code.

-----Original Message-----
From: Peter Gutmann <pgu...@cs.auckland.ac.nz>
Sent: Tuesday, June 21, 2022 2:25 PM
To: Aaron Gable <aa...@letsencrypt.org>; nolo...@gmail.com; Corey Bonnell
<Corey....@digicert.com>
Cc: dev-secur...@mozilla.org
Subject: Re: Malformed Trustwave certificates in Mozilla's ca cert
collection

[Peter Gutmann]

Corey Bonnell writes:

>I think that's another good check. Automatable too since you return the error
>count as the process return code.

One thing to be aware of if you're using it to check large numbers of certs
automatically rather than just a one-off check of a few certs is that with
enough certs you'll eventually run into FPs because it uses heuristics to dig
into X-stuffed-inside-Y encodings. It's fine for the hole encodings in
extensions since it knows about those, but there are other things where you
can occasionally get FPs.

Peter.

[Jeffrey Walton]

On Tue, Jun 21, 2022 at 1:59 PM Aaron Gable <aa...@letsencrypt.org> wrote:
>
> I just had to do a bunch of digging to fully understand what's going on here, so I figured I'd share for the benefit of everyone else.
>
> The keyUsage extension bitstring in these certificates is encoded as 03 03 07 06 00. Those bytes are interpreted as follows:
> 03: This item is a BIT STRING
> 03: This item has a length of 3 bytes
> 07: The least significant 7 bits of the remaining bytes should be ignored
> 06 00: The bits 0000 0110 0XXX XXXX (with Xs representing ignored bits as indicated by the 07 byte)
> This means that the final keyUsage bitstring is 000001100. This has a length of 9 bits, which makes some sense given that there are 9 possible bits (indexed 0 through 8) that can be set in the keyUsage extension specification. This bit string indicates the keyCertSign (5) and crlSign (6) key usages.
>
> However, there is another (shorter!) way to encode the exact same value. There are two zero-bits at the end of that nine-bit string, so it could instead be encoded as 03 02 01 06:
> 03: This item is a BIT STRING
> 02: This item has a length of 2 bytes
> 01: The least significant 1 bit of the remaining byte should be ignored
> 06: The bits 0000 011X
> This results in a keyUsage bit string of 0000011, which also equivalently indicates the keyCertSign (5) and crlSign (6) key usages.

A small nit about this... The ITU standards says (from X.690, Section 6.2):

For the purposes of this Recommendation | International Standard only,
the bits of an octet are numbered from 8 to 1, where bit 8 is the
"most significant bit", and bit 1 is the "least significant bit"

RFC 5280, Section 4.2.1.3, calls out bits but does not specify where
the msb is located, or where the lsb is located. I think that means
according to X.690, 0000011 would assert digitalSignature (bit 0) and
nonRepudiation (bit 1).

It would be helpful if RFC 5280 said what the bit positions are when
they say "bit 0", "bit 1", etc.

Jeff

[Aaron Gable]

On Tuesday, June 21, 2022 at 6:38:27 PM UTC-7 nolo...@gmail.com wrote:

A small nit about this... The ITU standards says (from X.690, Section 6.2):

For the purposes of this Recommendation | International Standard only,
the bits of an octet are numbered from 8 to 1, where bit 8 is the
"most significant bit", and bit 1 is the "least significant bit"

RFC 5280, Section 4.2.1.3, calls out bits but does not specify where
the msb is located, or where the lsb is located. I think that means
according to X.690, 0000011 would assert digitalSignature (bit 0) and
nonRepudiation (bit 1).

A good observation, but I think irrelevant in this case. The quoted paragraph from X.690 Section 6.2 says "the bits of an octet are numbered..." (emphasis added). The value in question here is of type BIT STRING, which doesn't care about octet boundaries, so once we've assembled the bits of that bit string and need to interpret it, that paragraph no longer applies. Additionally, if this observation were correct, every certificate parser I'm aware of would have to change how they interpret this field.

As a additional note, I did some further exploration into the certs in question, just for my own edification. In particular, I performed the following operations:

1) Extract the raw bytes of the tbsCert

2) Compute the appropriate (SHA256 or SHA384) hash over those bytes

3) Verify that the public key in the cert validates the signature in the cert over that hash

In both cases, the signature validates.

This means that the signature in the certificate was originally computed over the same bytes as we currently see in the certificate. From the earlier investigation, we know that the keyUsage extension bytes are actually BER encoded, not DER encoded. Therefore the certificates are in violation of RFC5280, Section 4.1.1.3, which states that "The signatureValue field contains a digital signature computed upon the ASN.1 DER encoded tbsCertificate" (emphasis added).

Of course, it may be that it doesn't matter. Because these are self-signed roots, they're essentially just wrappers around a public key -- most certificate validation routines will not end up checking the self-signature on these certs because they're inherently trusted. I just thought it was interesting to dig into this and learn more myself.

Aaron

[Peter Gutmann]

'Aaron Gable' via dev-secur...@mozilla.org <dev-secur...@mozilla.org> writes:

>Therefore the certificates are in violation of RFC5280, Section 4.1.1.3,
>which states that "The signatureValue field contains a digital signature
>computed upon the ASN.1 DER encoded tbsCertificate" (emphasis added).

Technically they are, but I would assume that every implementer on the planet
knows that that's one of the numerous parts of the spec that you need to
ignore if you want things to work:

* There is only one encoding rule and that is memcpy()
* There is only one comparison rule and that is memcmp()

In other words you take what the other side sends you and either memcpy() or
memcmp() it as required without ever trying to do any re-coding or
canonicalisation or whatever, because all that'll ever do is break things.
This is illustrated by the fact that in the five years since those certs were
issued and presumably billions of times they've been used, nothing even
noticed the non-DER encoding apart from a tool specially written to check for
such violations.

So it's really a minor encoding issue, not a significant problem.

Peter.

[Aaron Gable]

Just to close the loop on this:

Christopher Henderson has done some great work to get two new checks added to zlint testing for this circumstance:

- https://github.com/zmap/zlint/pull/682 ensures that there is not a whole byte of zeros at the end of an encoded BIT STRING; and

- https://github.com/zmap/zlint/pull/684 ensures that there are no trailing zeroes which are not counted in the "unused" bits of the BIT STRING

Thanks so much to Christopher and the ZMap team for working to add these lints!

Aaron