Advice sought on pragmatic (and policy-compliant) ways to divide a root set between two logs

185 views
Skip to first unread message

Paul Hadfield

unread,
Jul 29, 2016, 9:12:06 AM7/29/16
to Certificate Transparency Policy
Hi,

I'm looking for suggestions on how one might seek to operate a pair of CT Logs that, between them, operate an open acceptance policy, in that a cert trusted by the roots sets of major OSs/browsers will be accepted by one of the two logs [crudely, I refer to this below as the 'open root set'].

Background to this question is that growing certificate issuance rates mean that log size is also growing - perhaps exponentially - and that operators may already face the questions of: what is the maximum practical size their log can reach, and when will it get there?

Sharding the root set is one way of spreading the load - in fact, I believe it is the only practical way of doing so in a manner compliant with RFC6962.

An issue that arises as one goes down this path is how to deal with new CAs whose CA certs have been cross-signed by an established CA.

To give a concrete example; say we have a pair of CT Logs where (conceptually) Log A accepts any cert chaining to the 'open root set' _except_ for certs issued by Let's Encrypt, and Log B accepts only certs that chain to one of the Let's Encrypt roots X{1, 2, 3, 4}.

Log B's root set is in turn cross-signed by IdenTrust DST CA X3, which in my first approximation is present in the root set of Log A.  Now, A will end up accepting all the certs that B accepts, which has not achieved the sharding goal.

A way to circumvent this is for Log A not to include the IdenTrust root, but instead to include all known subordinate CA's to that root _except_ the Let's Encrypt set.  This approach would work and is compliant with RFC6962.  I believe it should be compliant also with Chromium policy.

But it feels brittle, and places an onus on the operator to keep close tabs on the issuance of any new subordinate CAs by IdenTrust DST CA X3.

Are there better ways of achieving this?   [aside: this could be addressed in 6962-bis if the Log can be permitted to have both included and excluded trust anchors - but that's a topic for a different maillist].

regards
Paul



Fabrice Gautier

unread,
Jul 29, 2016, 1:18:27 PM7/29/16
to Paul Hadfield, Certificate Transparency Policy
Hi,

It seems the simplest solution to deal with Log getting too big, would
be to just stop accepting new certificates when you reach a certain
size, and start another instance. The old log would become read-only
at that point.

The only requirement in the Chrome policy that I see might be a
problem would be this:
"If requested, accept certificates issued by a test CA run by Google
to enable Google to monitor the log’s compliance to these policies."

So Chrome might not consider the log "Qualified at time of check"
anymore, which could affect SCTs that are presented via extensions. So
server/ocsp responders operators that use extensions to send SCTs may
have to re-submit their certs in the new log and get new SCTs.
The frozen log would still be considered "Qualified at time of
issuance" for any logged certificate, so for certs using embedded SCTs
it should still work.

You could also use freeze the log from new certs, except those certs
issued by the Google test CA. So that technically, the frozen log
would still be qualified.
> --
> You received this message because you are subscribed to the Google Groups
> "Certificate Transparency Policy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ct-policy+...@chromium.org.
> To post to this group, send email to ct-p...@chromium.org.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CAGDCdM704hyvbsVhK0faPOh5r%3DFLw1EzPrF7spF0hQCLsbe%3D1A%40mail.gmail.com.

Ryan Sleevi

unread,
Jul 29, 2016, 2:46:29 PM7/29/16
to Fabrice Gautier, Paul Hadfield, Certificate Transparency Policy
Right, it feels like freezing logs when they're at the size that growth becomes difficult is the right approach.

How often a log needs freezing is a different question. Ideally, much like you want log additions/removals measured on O(years), you wouldn't need to freeze a log until it'd been operational for O(years).

Ryan Hurst

unread,
Jul 29, 2016, 3:00:17 PM7/29/16
to Ryan Sleevi, Fabrice Gautier, Paul Hadfield, Certificate Transparency Policy
I personally like this as it also forces periodic key rotation for logs.

Ryan

Rob Stradling

unread,
Jul 29, 2016, 3:23:42 PM7/29/16
to Paul Hadfield, Ryan Hurst, Ryan Sleevi, Fabrice Gautier, Certificate Transparency Policy
Sharding based on the hash of the certificate would ensure a more even
distribution (compared to sharding by issuer). However, I agree that
freezing / rotating is the right approach.
> <mailto:ct-policy%2Bunsu...@chromium.org>.
> > To post to this group, send email to ct-p...@chromium.org
> <mailto:ct-p...@chromium.org>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CAGDCdM704hyvbsVhK0faPOh5r%3DFLw1EzPrF7spF0hQCLsbe%3D1A%40mail.gmail.com.
>
> --
> You received this message because you are subscribed to the
> Google Groups "Certificate Transparency Policy" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to ct-policy+...@chromium.org
> <mailto:ct-policy%2Bunsu...@chromium.org>.
> To post to this group, send email to ct-p...@chromium.org
> <mailto:ct-p...@chromium.org>.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CANOyrg8PD2Yp%3DqKWz8mPdNzPYgi57kr1GdXga7-nfoZtrz5wRg%40mail.gmail.com.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Certificate Transparency Policy" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to ct-policy+...@chromium.org
> <mailto:ct-policy+...@chromium.org>.
> To post to this group, send email to ct-p...@chromium.org
> <mailto:ct-p...@chromium.org>.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CACvaWvZpz2TKxhZr56145izk-scvQOsKnzuPy-KuL7%3DwD9ymYg%40mail.gmail.com
> <https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CACvaWvZpz2TKxhZr56145izk-scvQOsKnzuPy-KuL7%3DwD9ymYg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Certificate Transparency Policy" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to ct-policy+...@chromium.org
> <mailto:ct-policy+...@chromium.org>.
> To post to this group, send email to ct-p...@chromium.org
> <mailto:ct-p...@chromium.org>.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CAKexNNPioNdZwHF_-Au-ORGPXjj%2BSV57rO%3DXYm70F8NzAFgRPw%40mail.gmail.com
> <https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CAKexNNPioNdZwHF_-Au-ORGPXjj%2BSV57rO%3DXYm70F8NzAFgRPw%40mail.gmail.com?utm_medium=email&utm_source=footer>.

--
Rob Stradling
Senior Research & Development Scientist
COMODO - Creating Trust Online
Office Tel: +44.(0)1274.730505
Office Fax: +44.(0)1274.730909
www.comodo.com

COMODO CA Limited, Registered in England No. 04058690
Registered Office:
3rd Floor, 26 Office Village, Exchange Quay,
Trafford Road, Salford, Manchester M5 3EQ

This e-mail and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they are
addressed. If you have received this email in error please notify the
sender by replying to the e-mail containing this attachment. Replies to
this email may be monitored by COMODO for operational or business
reasons. Whilst every endeavour is taken to ensure that e-mails are free
from viruses, no liability can be accepted and the recipient is
requested to use their own virus checking software.

Paul Hadfield

unread,
Jul 29, 2016, 4:19:56 PM7/29/16
to Rob Stradling, Fabrice Gautier, Ryan Hurst, Certificate Transparency Policy, Ryan Sleevi

On 29 Jul 2016 20:23, "Rob Stradling" <rob.st...@comodo.com> wrote:
>
> Sharding based on the hash of the certificate would ensure a more even distribution (compared to sharding by issuer).  However, I agree that freezing / rotating is the right approach.

I agree that technically this seems an appealing approach, but I don't think sharding based on certificate hash would be compliant with the RFC. I believe a log is required to accept submissions that chain to one of its accepted roots.

Rob Stradling

unread,
Jul 29, 2016, 4:26:43 PM7/29/16
to Paul Hadfield, Fabrice Gautier, Ryan Hurst, Certificate Transparency Policy, Ryan Sleevi
On 29/07/16 21:19, 'Paul Hadfield' via Certificate Transparency Policy
wrote:
> On 29 Jul 2016 20:23, "Rob Stradling" <rob.st...@comodo.com
> <mailto:rob.st...@comodo.com>> wrote:
>>
>> Sharding based on the hash of the certificate would ensure a more even
> distribution (compared to sharding by issuer). However, I agree that
> freezing / rotating is the right approach.
>
> I agree that technically this seems an appealing approach, but I don't
> think sharding based on certificate hash would be compliant with the
> RFC.

I agree it wouldn't be compliant with 6962.

(Sorry, I have my 6962-bis head on, so I'm thinking about what we could
conceivably permit in the future ;-) ).

> I believe a log is required to accept submissions that chain to one
> of its accepted roots.
>
>>
>> On 29/07/16 20:00, 'Ryan Hurst' via Certificate Transparency Policy wrote:
>>>
>>> I personally like this as it also forces periodic key rotation for logs.
>>>
>>> Ryan
>>>
>>> On Fri, Jul 29, 2016 at 11:45 AM, Ryan Sleevi <rsl...@chromium.org
> <mailto:rsl...@chromium.org>
>>> <mailto:rsl...@chromium.org <mailto:rsl...@chromium.org>>> wrote:
>>>
>>> Right, it feels like freezing logs when they're at the size that
>>> growth becomes difficult is the right approach.
>>>
>>> How often a log needs freezing is a different question. Ideally,
>>> much like you want log additions/removals measured on O(years), you
>>> wouldn't need to freeze a log until it'd been operational for
> O(years).
>>>
>>> On Fri, Jul 29, 2016 at 10:18 AM, Fabrice Gautier
>>> <fabrice...@gmail.com <mailto:fabrice...@gmail.com>
> <mailto:fabrice...@gmail.com <mailto:fabrice...@gmail.com>>>
>>> <mailto:ct-p...@chromium.org
>>> <mailto:ct-policy%2Bunsu...@chromium.org
> <mailto:ct-policy%252Buns...@chromium.org>>.
>>>
>>> > To post to this group, send email to ct-p...@chromium.org
> <mailto:ct-p...@chromium.org>
>>> <mailto:ct-p...@chromium.org <mailto:ct-p...@chromium.org>>.
>>>
>>> > To view this discussion on the web visit
>>> >
>>>
> https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CAGDCdM704hyvbsVhK0faPOh5r%3DFLw1EzPrF7spF0hQCLsbe%3D1A%40mail.gmail.com.
>>>
>>> --
>>> You received this message because you are subscribed to the
>>> Google Groups "Certificate Transparency Policy" group.
>>> To unsubscribe from this group and stop receiving emails from
>>> it, send an email to ct-policy+...@chromium.org
> <mailto:ct-policy%2Bunsu...@chromium.org>
>>> <mailto:ct-policy%2Bunsu...@chromium.org
> <mailto:ct-policy%252Buns...@chromium.org>>.
>>>
>>> To post to this group, send email to ct-p...@chromium.org
> <mailto:ct-p...@chromium.org>
>>> <mailto:ct-p...@chromium.org <mailto:ct-p...@chromium.org>>.
>>>
>>> To view this discussion on the web visit
>>>
> https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CANOyrg8PD2Yp%3DqKWz8mPdNzPYgi57kr1GdXga7-nfoZtrz5wRg%40mail.gmail.com.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Certificate Transparency Policy" group.
>>> To unsubscribe from this group and stop receiving emails from it,
>>> send an email to ct-policy+...@chromium.org
> <mailto:ct-policy%2Bunsu...@chromium.org>
>>> <mailto:ct-policy+...@chromium.org
> <mailto:ct-policy%2Bunsu...@chromium.org>>.
>>>
>>> To post to this group, send email to ct-p...@chromium.org
> <mailto:ct-p...@chromium.org>
>>> <mailto:ct-p...@chromium.org <mailto:ct-p...@chromium.org>>.
>>>
>>> To view this discussion on the web visit
>>>
> https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CACvaWvZpz2TKxhZr56145izk-scvQOsKnzuPy-KuL7%3DwD9ymYg%40mail.gmail.com
>>>
> <https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CACvaWvZpz2TKxhZr56145izk-scvQOsKnzuPy-KuL7%3DwD9ymYg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>>>
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Certificate Transparency Policy" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to ct-policy+...@chromium.org
> <mailto:ct-policy%2Bunsu...@chromium.org>
>>> <mailto:ct-policy+...@chromium.org
> <mailto:ct-policy%2Bunsu...@chromium.org>>.
>>>
>>> To post to this group, send email to ct-p...@chromium.org
> <mailto:ct-p...@chromium.org>
>>> <mailto:ct-p...@chromium.org <mailto:ct-p...@chromium.org>>.
>> www.comodo.com <http://www.comodo.com>
>>
>> COMODO CA Limited, Registered in England No. 04058690
>> Registered Office:
>> 3rd Floor, 26 Office Village, Exchange Quay,
>> Trafford Road, Salford, Manchester M5 3EQ
>>
>> This e-mail and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they are
> addressed. If you have received this email in error please notify the
> sender by replying to the e-mail containing this attachment. Replies to
> this email may be monitored by COMODO for operational or business
> reasons. Whilst every endeavour is taken to ensure that e-mails are free
> from viruses, no liability can be accepted and the recipient is
> requested to use their own virus checking software.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Certificate Transparency Policy" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to ct-policy+...@chromium.org
> <mailto:ct-policy+...@chromium.org>.
> To post to this group, send email to ct-p...@chromium.org
> <mailto:ct-p...@chromium.org>.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CAGDCdM5pkY3n6aDcz5tiEQL%2B2-_HaodoEtdBHKSE_ebUT8n5ew%40mail.gmail.com
> <https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CAGDCdM5pkY3n6aDcz5tiEQL%2B2-_HaodoEtdBHKSE_ebUT8n5ew%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Richard Salz

unread,
Jul 29, 2016, 6:30:12 PM7/29/16
to Rob Stradling, Paul Hadfield, ct-p...@chromium.org, Ryan Hurst, Fabrice Gautier, Ryan Sleevi

Is a log required to accept all certs on its root list? Can you point me to where the rfc says that?

Rob Stradling

unread,
Jul 29, 2016, 6:42:41 PM7/29/16
to Richard Salz, Paul Hadfield, ct-p...@chromium.org, Ryan Hurst, Fabrice Gautier, Ryan Sleevi
Hmmm, I don't see the requirement in 6962. This must be one of the
things we've clarified in 6962-bis (see section 5.1).

On 29/07/16 23:30, Richard Salz wrote:
> Is a log required to accept all certs on its root list? Can you point me
> to where the rfc says that?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Certificate Transparency Policy" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to ct-policy+...@chromium.org
> <mailto:ct-policy+...@chromium.org>.
> To post to this group, send email to ct-p...@chromium.org
> <mailto:ct-p...@chromium.org>.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CAFH29topri1cFoWw%2ByiyBFM3eBhTJMqPOjxS-7kZmKS4GUL8qw%40mail.gmail.com
> <https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CAFH29topri1cFoWw%2ByiyBFM3eBhTJMqPOjxS-7kZmKS4GUL8qw%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Richard Salz

unread,
Jul 30, 2016, 5:26:04 PM7/30/16
to Rob Stradling, Paul Hadfield, ct-p...@chromium.org, Ryan Hurst, Fabrice Gautier, Ryan Sleevi
Been awhile since I looked at the latest draft.  "
Log operators MUST NOT impose any conditions on retrieving or sharing
   data from the log."

So I a log can't do https-only? 

 

Ryan Sleevi

unread,
Aug 1, 2016, 2:37:06 AM8/1/16
to Richard Salz, Rob Stradling, Paul Hadfield, ct-p...@chromium.org, Ryan Hurst, Fabrice Gautier, Ryan Sleevi
I suspect you're probably wanting the IETF group for feedback on the IETF draft, but to the point of your question about HTTPS-only:

It seems clear to me from 6962-bis that logs MUST support HTTPS, but MAY support others.

That said, if you do want to split hairs on the implications of https://tools.ietf.org/html/draft-ietf-trans-rfc6962-bis-18#section-6 , it could be useful to consider whether or not QUIC-only (accessed via an HTTPS URL) meets the policy constraint that's in the text, or whether or not an ALPN restriction (e.g. only returns the logs contents if ALPN string "Foo" is negotiated). Is that a condition?

And now we know why policy in IETF drafts is difficult :)

Paul Hadfield

unread,
Aug 1, 2016, 4:45:30 AM8/1/16
to Fabrice Gautier, Certificate Transparency Policy
On Fri, Jul 29, 2016 at 6:18 PM, Fabrice Gautier <fabrice...@gmail.com> wrote:
Hi,

It seems the simplest solution to deal with Log getting too big, would
be to just stop accepting new certificates when you reach a certain
size, and start another instance. The old log would become read-only
at that point.

Certainly, that's the right option for a Log that is nearing its operational
capacity.

The motivation for my question though was whether splitting the root set
is a practical approach to try to slow the growth of new logs.  One reason
to want to do this is that, if the operator has an open acceptance policy,
then there's nothing to prevent a third party from copying the entire
content of an existing open acceptance policy Log into the new one.  Thus
the new Log may approach operational capacity much sooner than the
operator had intended!

There are other tactics one might use to avoid this happening but some
of them are prohibited by the existing RFC and others if not explicitly
prohibited seem to fall in a gray area.

For example; an approach that seems desirable from the operational
perspective is for the new Log to apply a cut-off date to the validity
period of logged certificates, with the goal of not accepting chains that
have expired in the last 'n' days.  We believe this is prohibited by the RFC.
Also, it doesn't really help in a world where issuance rates are increasing.

Another approach is to limit the availability of the log to clients that seek to
bulk load certs into the Log, by rate-limiting the add-chain API.
[note that this would not be applied to add-pre-chain].  I think this might
be wiggled through but seems against the spirit of the RFC.



The only requirement in the Chrome policy that I see might be a
problem would be this:
"If requested, accept certificates issued by a test CA run by Google
to enable Google to monitor the log’s compliance to these policies."

So Chrome might not consider the log "Qualified at time of check"
anymore, which could affect SCTs that are presented via extensions. So
server/ocsp responders operators that use extensions to send SCTs may
have to re-submit their certs in the new log and get new SCTs.
The frozen log would still be considered "Qualified at time of
issuance" for any logged certificate, so for certs using embedded SCTs
it should still work.

I'm not sure there's any point in the the Google monitor continuing to issue
probes that are intended to verify that logs stay within MMD when the log has
been switched to a read-only mode.

The Google monitor should certainly continue to check log availability.

Paul Hadfield

unread,
Aug 1, 2016, 4:56:31 AM8/1/16
to Richard Salz, Rob Stradling, Certificate Transparency Policy, Ryan Hurst, Fabrice Gautier, Ryan Sleevi
On Fri, Jul 29, 2016 at 11:30 PM, Richard Salz <rich...@gmail.com> wrote:

Is a log required to accept all certs on its root list? Can you point me to where the rfc says that?

My reading is that this is a consequence of wording in RFC 6962 section 3:

section 3:  "When a valid certificate is submitted to a log, the log MUST immediately return a Signed Certificate Timestamp (SCT)"
  -- it comes down to the meaning of 'valid'.  The interpretation of this by Google for our logs is that, provided the signatures in the submitted chain are good, and the chain terminates at one of the log's accepted roots then the submitted chain cert must is considered 'valid' and must be logged.  Well-formedness and other aspects of cert content are not part of the check for validity.

This interpretation prevents the our logs from adopting policies such as rejecting expired chains; sharding based on a hash of the leaf cert DER, etc.

Ben Laurie

unread,
Aug 1, 2016, 6:18:48 AM8/1/16
to Paul Hadfield, Fabrice Gautier, Certificate Transparency Policy
On 1 August 2016 at 01:45, 'Paul Hadfield' via Certificate
Transparency Policy <ct-p...@chromium.org> wrote:
> On Fri, Jul 29, 2016 at 6:18 PM, Fabrice Gautier <fabrice...@gmail.com>
> wrote:
>>
>> Hi,
>>
>> It seems the simplest solution to deal with Log getting too big, would
>> be to just stop accepting new certificates when you reach a certain
>> size, and start another instance. The old log would become read-only
>> at that point.
>
>
> Certainly, that's the right option for a Log that is nearing its operational
> capacity.
>
> The motivation for my question though was whether splitting the root set
> is a practical approach to try to slow the growth of new logs. One reason
> to want to do this is that, if the operator has an open acceptance policy,
> then there's nothing to prevent a third party from copying the entire
> content of an existing open acceptance policy Log into the new one. Thus
> the new Log may approach operational capacity much sooner than the
> operator had intended!

Another reason is that it would make sense to have different logs for
short and long lifetime certs.

One thing I thought might be feasible is to invent the idea of a "log
group", which between them must accept all qualifying certs, but which
are allowed to apply arbitrary criteria for which of the group accepts
any particular cert.
> https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CAGDCdM5DPbnMH4bKVTqwrJxP2zCbcU%3DqY0ThR3NxZ78pU6odcQ%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages