Hunting for intermediates that still haven't been disclosed to CCADB

[Rob Stradling]

Yesterday I knocked together a script that: scrapes a URL (or a list of
URLs) for certificate files; then attempts to build a trust chain (using
https://crt.sh/gen-add-chain) for each certificate found; then submits
to some CT logs any trust chains that crt.sh hasn't previously seen.

I've thrown the code up on GitHub [1].

Overnight I left certscraper to scrape the following lists of URLs:
- the disclosure URLs that CAs provided in response to Mozilla's May
2014 CA Communication [2].
- the CP/CPS URLs currently listed in the CCADB (some of which appear
to be repository pages).
- the Belgian Government eID repository pages [3].

certscraper found...

- 8 DigiCert intermediates (found on [4]) that should've been already
disclosed to CCADB, but weren't:
https://crt.sh/?id=135966905
https://crt.sh/?id=135966906
https://crt.sh/?id=135966907
https://crt.sh/?id=135966908
https://crt.sh/?id=135970325
https://crt.sh/?id=135970327
https://crt.sh/?id=135970329
https://crt.sh/?id=135970332

- 10 Belgian eID intermediates:
https://crt.sh/?id=135626971
https://crt.sh/?id=135626972
https://crt.sh/?id=135626973
https://crt.sh/?id=135626974
https://crt.sh/?id=135626975
https://crt.sh/?id=135626980
https://crt.sh/?id=135626981
https://crt.sh/?id=135626982
https://crt.sh/?id=135626983
https://crt.sh/?id=135626984

Another 1 undisclosed Belgian eID intermediate
(https://crt.sh/?id=135002620) had appeared in crt.sh a couple of days
earlier.

It would seem that DigiCert noticed these 19 intermediates appear on
https://crt.sh/mozilla-disclosures#undisclosed whilst I was asleep,
because they've all now been disclosed to the CCADB.

They should've been disclosed some time ago, however.


[1] https://github.com/robstradling/certscraper

[2]
https://docs.google.com/spreadsheets/d/1v-Lrxo6mYlyrEli_wSpLsHZvV5dJ_vvSzLTAMfxI5n8/pubhtml

[3]
https://github.com/robstradling/certscraper/blob/master/url_lists/belgian_eid.txt

[4] https://www.digicert.com/digicert-root-certificates.htm?show=all

--
Rob Stradling
Senior Research & Development Scientist
COMODO - Creating Trust Online

[Kurt Roeckx]

On 2017-05-11 13:07, Rob Stradling wrote:
> It would seem that DigiCert noticed these 19 intermediates appear on
> https://crt.sh/mozilla-disclosures#undisclosed whilst I was asleep,
> because they've all now been disclosed to the CCADB.
>
> They should've been disclosed some time ago, however.

Does the CCADB keep track of when something was disclosed? A history?


Kurt

[Rob Stradling]

On 11/05/17 12:28, Kurt Roeckx via dev-security-policy wrote:
> On 2017-05-11 13:07, Rob Stradling wrote:

>> It would seem that DigiCert noticed these 19 intermediates appear on
>> https://crt.sh/mozilla-disclosures#undisclosed whilst I was asleep,
>> because they've all now been disclosed to the CCADB.
>>
>> They should've been disclosed some time ago, however.
>

> Does the CCADB keep track of when something was disclosed? A history?

There's a "Created by" field (Username, Timestamp) and a "Last Modified
By" field (Username, Timestamp) in the CCADB, but neither of these
fields are currently provided in the public CSV reports that Mozilla
publishes.

[Ben Wilson]

Both sets had been publicly disclosed through affirmative publishing in the
repositories of the respective CAs--that's probably how your crawler found
them, because I don't believe they are issuing SSL/TLS certificates. I
thought I had disclosed the ones chaining to the DigiCert Orion Health
intermediate a while ago (when I first populated the CCADB with our
certificates), but apparently I missed those. The Belgian ones were posted
just recently, I believe, because I do try to keep the CCADB up to date.

-----Original Message-----
From: dev-security-policy
[mailto:dev-security-policy-bounces+ben=digice...@lists.mozilla.org] On
Behalf Of Rob Stradling via dev-security-policy
Sent: Thursday, May 11, 2017 5:47 AM
To: Kurt Roeckx <ku...@roeckx.be>;
mozilla-dev-s...@lists.mozilla.org
Subject: Re: Hunting for intermediates that still haven't been disclosed to
CCADB

On 11/05/17 12:28, Kurt Roeckx via dev-security-policy wrote:
> On 2017-05-11 13:07, Rob Stradling wrote:

>> It would seem that DigiCert noticed these 19 intermediates appear on
>> https://crt.sh/mozilla-disclosures#undisclosed whilst I was asleep,
>> because they've all now been disclosed to the CCADB.
>>
>> They should've been disclosed some time ago, however.
>

> Does the CCADB keep track of when something was disclosed? A history?

There's a "Created by" field (Username, Timestamp) and a "Last Modified By"
field (Username, Timestamp) in the CCADB, but neither of these fields are
currently provided in the public CSV reports that Mozilla publishes.

--
Rob Stradling
Senior Research & Development Scientist
COMODO - Creating Trust Online

_______________________________________________
dev-security-policy mailing list
dev-secur...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

[Gervase Markham]

On 11/05/17 12:46, Rob Stradling wrote:
> There's a "Created by" field (Username, Timestamp) and a "Last Modified
> By" field (Username, Timestamp) in the CCADB, but neither of these
> fields are currently provided in the public CSV reports that Mozilla
> publishes.

Rob: do you think you could enhance
https://crt.sh/mozilla-disclosures#undisclosed to give the number of
days a certificate has been on the list?

Gerv

[Kurt Roeckx]

Rob,

Could you also split the "Technically Constrained", into those that are
really technically constrained, and those that are out of scope for the
CCADB?

For instance https://crt.sh/?id=12729339 is out of scope because it's
code signing. https://crt.sh/?id=8951039 because it's client / email.

Or maybe it just needs to be renamed?


Kurt

[Rob Stradling]

On 11/05/17 18:05, Gervase Markham via dev-security-policy wrote:
> On 11/05/17 12:46, Rob Stradling wrote:

>> There's a "Created by" field (Username, Timestamp) and a "Last Modified
>> By" field (Username, Timestamp) in the CCADB, but neither of these
>> fields are currently provided in the public CSV reports that Mozilla
>> publishes.
>

> Rob: do you think you could enhance
> https://crt.sh/mozilla-disclosures#undisclosed to give the number of
> days a certificate has been on the list?

Hi Gerv.

I've just added two columns to
https://crt.sh/mozilla-disclosures#undisclosed:
- "Earliest SCT".
- "Listed Here Since".

Note that if an intermediate cert goes out of scope for disclosure
(i.e., all trust paths become revoked) but then comes back into scope
for disclosure (i.e., a new trust path is discovered), then its "Listed
Here Since" timestamp will be reset.

[Rob Stradling]

On 12/05/17 16:37, Kurt Roeckx via dev-security-policy wrote:
> On 2017-05-11 19:05, Gervase Markham wrote:

> Rob,

Hi Kurt.

> Could you also split the "Technically Constrained", into those that are
> really technically constrained, and those that are out of scope for the
> CCADB?

What's your definition of "really technically constrained"?

> For instance https://crt.sh/?id=12729339 is out of scope because it's
> code signing. https://crt.sh/?id=8951039 because it's client / email.

They're out of scope because they're Technically Constrained in
accordance with Section 5.3.1 of the Mozilla Root Store Policy v2.4.1 [1]

> Or maybe it just needs to be renamed?

I'm really not sure what "it" you'd like to see categorized differently.


[1]
https://www.mozilla.org/en-US/about/governance/policies/security-group/certs/policy/

[Kurt Roeckx]

On 2017-05-17 14:40, Rob Stradling wrote:
> On 12/05/17 16:37, Kurt Roeckx via dev-security-policy wrote:
>> On 2017-05-11 19:05, Gervase Markham wrote:
>>> On 11/05/17 12:46, Rob Stradling wrote:
>>>> There's a "Created by" field (Username, Timestamp) and a "Last Modified
>>>> By" field (Username, Timestamp) in the CCADB, but neither of these
>>>> fields are currently provided in the public CSV reports that Mozilla
>>>> publishes.
>>>
>>> Rob: do you think you could enhance
>>> https://crt.sh/mozilla-disclosures#undisclosed to give the number of
>>> days a certificate has been on the list?
>>
>> Rob,
>
> Hi Kurt.
>
>> Could you also split the "Technically Constrained", into those that
>> are really technically constrained, and those that are out of scope
>> for the CCADB?
>
> What's your definition of "really technically constrained"?
>
>> For instance https://crt.sh/?id=12729339 is out of scope because it's
>> code signing. https://crt.sh/?id=8951039 because it's client / email.
>
> They're out of scope because they're Technically Constrained in
> accordance with Section 5.3.1 of the Mozilla Root Store Policy v2.4.1 [1]
>
>> Or maybe it just needs to be renamed?
>
> I'm really not sure what "it" you'd like to see categorized differently.

I seem to have been confused. For some reason I was under the impression
that only those that can be used for server auth were required to be
disclosed when I was reading it last week. It didn't really make sense
to me at the time, and now I can't find anything that suggests that.

And it seem that only an EKU is needed with the current policy for email
and code signing to be considered constrained. But you could argue that
for email you can't say for sure that it's constrained or not, which I
guess is why we have https://github.com/mozilla/pkipolicy/issues/69

I guess with code signing we have the situation that Mozilla doesn't
care about it, but requires you to disclose them anyway.


Kurt

[Rob Stradling]

On 17/05/17 14:43, Kurt Roeckx via dev-security-policy wrote:
> On 2017-05-17 14:40, Rob Stradling wrote:

>> On 12/05/17 16:37, Kurt Roeckx via dev-security-policy wrote:
>>> On 2017-05-11 19:05, Gervase Markham wrote:
>>>> On 11/05/17 12:46, Rob Stradling wrote:
>>>>> There's a "Created by" field (Username, Timestamp) and a "Last
>>>>> Modified
>>>>> By" field (Username, Timestamp) in the CCADB, but neither of these
>>>>> fields are currently provided in the public CSV reports that Mozilla
>>>>> publishes.
>>>>
>>>> Rob: do you think you could enhance
>>>> https://crt.sh/mozilla-disclosures#undisclosed to give the number of
>>>> days a certificate has been on the list?
>>>
>>> Rob,
>>
>> Hi Kurt.
>>
>>> Could you also split the "Technically Constrained", into those that
>>> are really technically constrained, and those that are out of scope
>>> for the CCADB?
>>
>> What's your definition of "really technically constrained"?
>>
>>> For instance https://crt.sh/?id=12729339 is out of scope because it's
>>> code signing. https://crt.sh/?id=8951039 because it's client / email.
>>
>> They're out of scope because they're Technically Constrained in
>> accordance with Section 5.3.1 of the Mozilla Root Store Policy v2.4.1 [1]
>>
>>> Or maybe it just needs to be renamed?
>>
>> I'm really not sure what "it" you'd like to see categorized differently.
>

> I seem to have been confused. For some reason I was under the impression
> that only those that can be used for server auth were required to be
> disclosed when I was reading it last week. It didn't really make sense
> to me at the time, and now I can't find anything that suggests that.

The impression you were under is correct.

Any intermediate that is EKU-constrained to not permit serverAuth, or
that permits serverAuth but is appropriately name-constrained, is
considered to be "Technically Constrained" and is therefore not subject
to the current disclosure requirement.

> And it seem that only an EKU is needed with the current policy for email
> and code signing to be considered constrained.

Correct.

> But you could argue that
> for email you can't say for sure that it's constrained or not, which I
> guess is why we have https://github.com/mozilla/pkipolicy/issues/69

Correct.

See also https://github.com/mozilla/pkipolicy/issues/73, which proposes
that even Technically Constrained intermediates should fall under the
disclosure requirement.

> I guess with code signing we have the situation that Mozilla doesn't
> care about it, but requires you to disclose them anyway.

Where does it say that code signing intermediates need to be disclosed?

That's not my understanding of section 5.3.1 of
https://www.mozilla.org/en-US/about/governance/policies/security-group/certs/policy/

Incidentally, it's true that Mozilla have said that they don't care
about the Code Signing trust bit any more, but the
CKA_TRUST_CODE_SIGNING haven't yet been removed from certdata.txt. Bug?

[Gervase Markham]

On 17/05/17 15:08, Rob Stradling wrote:
> Incidentally, it's true that Mozilla have said that they don't care
> about the Code Signing trust bit any more, but the
> CKA_TRUST_CODE_SIGNING haven't yet been removed from certdata.txt. Bug?

Yes, but a low priority one. Feel free to file :-)

Gerv

[Gervase Markham]

On 17/05/17 13:32, Rob Stradling wrote:
> I've just added two columns to
> https://crt.sh/mozilla-disclosures#undisclosed:
> - "Earliest SCT".
> - "Listed Here Since".

Lovely! Now we just need a cert to be on the list so we can see what it
looks like ;-)

Gerv

[Rob Stradling]

On 17/05/17 15:12, Gervase Markham via dev-security-policy wrote:
> On 17/05/17 13:32, Rob Stradling wrote:

>> I've just added two columns to
>> https://crt.sh/mozilla-disclosures#undisclosed:
>> - "Earliest SCT".
>> - "Listed Here Since".
>

> Lovely! Now we just need a cert to be on the list so we can see what it
> looks like ;-)

Shall I add the same two fields to
https://crt.sh/mozilla-disclosures#disclosureincomplete as well?

[Gervase Markham]

On 17/05/17 15:15, Rob Stradling wrote:
> Shall I add the same two fields to
> https://crt.sh/mozilla-disclosures#disclosureincomplete as well?

Yes, why not? :-)

Gerv

[Rob Stradling]

On 17/05/17 15:21, Gervase Markham via dev-security-policy wrote:
> On 17/05/17 15:15, Rob Stradling wrote:

>> Shall I add the same two fields to
>> https://crt.sh/mozilla-disclosures#disclosureincomplete as well?
>

> Yes, why not? :-)
>
> Gerv

Done.

The "Listed Here Since" timestamps for the 24 intermediates currently in
this category are set to today, because I don't have a time machine to
go back and find out how long they've actually been listed in this
category. ;-)

[Gervase Markham]

On 17/05/17 15:49, Rob Stradling wrote:
> The "Listed Here Since" timestamps for the 24 intermediates currently in
> this category are set to today, because I don't have a time machine to
> go back and find out how long they've actually been listed in this
> category. ;-)

Lack of time machine is unfortunate but forgiveable.

Gerv

[Rob Stradling]

On 17/05/17 15:12, Gervase Markham wrote:
> On 17/05/17 15:08, Rob Stradling wrote:

>> Incidentally, it's true that Mozilla have said that they don't care
>> about the Code Signing trust bit any more, but the
>> CKA_TRUST_CODE_SIGNING haven't yet been removed from certdata.txt. Bug?
>

> Yes, but a low priority one. Feel free to file :-)

Filed.

https://bugzilla.mozilla.org/show_bug.cgi?id=1366243