Possible Issue with Domain Validation Method 9 in a shared hosting environment

[Doug Beattie]

Based on reported issues with TLS-SNI-01, we started investigation of our systems late yesterday regarding the use of "Test Certificate" validation, BR section 3.2.2.4.9.

We found that this method may be vulnerable to the some of the same underlying issue as the ACME TLS-SNI-01 so we disabled it at 10:51 AM today EST, January 11th.

While TLS-SNI-01 uses a host name like 773c7d.13445a.acme.invalid, GlobalSign uses the actual host name, www.example.com<http://www.example.com> which limits abuse, but we believe that the process might be vulnerable in some cases.

We're continuing to research this and will let you know what we find.

Doug


Doug Beattie
Vice President of Product Management
GlobalSign
Two International Drive | Suite 150 | Portsmouth, NH 03801
Email: doug.b...@globalsign.com<mailto:doug.b...@globalsign.com>
www.globalsign.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.globalsign.com_&d=AwMFAg&c=qRq7a-87GiVVW7v8KD1gdQ&r=yL2kJgSsccUq5VcaUHiaiErHSMoqqBV4kmZtle8pI0U&m=7LSnl4Q_Qu_BEe5I_P8WSvWs0evmNYHNhThvhJlrvzE&s=8HjQZHbWrcD_ik5cm6C2gK7iPzU_KT9tF7RSZfrF1c0&e=>

[Ryan Sleevi]

On Thu, Jan 11, 2018 at 4:50 PM, Doug Beattie via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

>
> Based on reported issues with TLS-SNI-01, we started investigation of our
> systems late yesterday regarding the use of "Test Certificate" validation,
> BR section 3.2.2.4.9.
>
> We found that this method may be vulnerable to the some of the same
> underlying issue as the ACME TLS-SNI-01 so we disabled it at 10:51 AM today
> EST, January 11th.
>
> While TLS-SNI-01 uses a host name like 773c7d.13445a.acme.invalid,
> GlobalSign uses the actual host name, www.example.com<http://www.
> example.com> which limits abuse, but we believe that the process might be
> vulnerable in some cases.
>
> We're continuing to research this and will let you know what we find.
>
> Doug
>

(Wearing a Chrome Hat, again)

Doug,

Thanks for the update. That seems consistent with Chrome's evaluation, as
shared at
https://groups.google.com/d/msg/mozilla.dev.security.policy/RHsIInIjJA0/HACyY9tMAAAJ


We'd like to ask that you work with the community and browsers on this
prior to re-enabling this validation method, for the reasons outlined in
that thread.

In particular, unlike the ACME methods that are specified, it's unclear the
validation process used by GlobalSign in applying 3.2.2.4.9. This makes it
particularly more challenging to evaluate the potential risks and/or
mitigations that may exist, and so sharing full and complete details about
the method you use publicly can assist browsers and the broader community
to evaluate and assess, much the same as TLS-SNI for ACME.

Further, as called out in that other thread, there's a risk calculus based
on the lifetime of the certificates issued that directly plays into whether
or not the method can be re-enabled and for how long. We'd love to work
with you to better understand these tradeoffs, as applied to .9, so that we
can make informed decisions about the risk to sites and users.

Thanks for your report, for disabling the validation, and for continued
collaboration to best protect users.

[Doug Beattie]

Ryan,

I’d like to follow up on our investigation and provide the community with some more information about how we use Method 9.

We use a process that we refer to as OneClick to automate the domain validation and issuance of certificates by issuing a test certificate to an FQDN and then verifying that the certificate is present on that FQDN. This is different from ACME method TLS-SNI-01, regardless of what some GlobalSign tweets may have mentioned. Where dedicated IP addresses are used, we believe this method is safe and secure. So, I’ll focus this discussion on when there are shared IP addresses and SNI is used. This is how the OneClick validation works:

1) Client requests a test certificate for a domain (only one FQDN)

2) We issue a test certificate valid for 7 days

3) Client places the test certificate on their server

4) We connect to the server (DNS look-up and then use SNI to ask for the certificate)

5) If the certificate is returned, the validation passes and we issue a production certificate which is downloaded and installed. The issued certificate can have validity up to 39 months (soon 825 days)

For shared IP address environments, it may be possible to receive a certificate for a domain you don’t actually control, but a number of things need to happen in order for this to be successful. What can go wrong?

1) A user could request a test certificate for a domain they don’t own within a shared IP address environment. In order for this to be successful:

a. User must know which other sites are hosted on the same IP address (the attack is limited to the set of customers on that shared IP address)

b. For this case, I’m assuming that sites don’t have TLS enabled (if they did when we went to validate them, we’d receive their certificate – more on this below in case 2)

c. The hosting company must allow you to manually create and upload a CSR for a site you don’t own

d. The user must be able to trick the hosting provider to enable SNI for this domain and link it to the certificate they uploaded

2) In the event that the target site does have TLS enabled and the attacker wants to override the account settings to provide this test certificate, they would need the provider to allow multiple accounts to claim the SNI traffic for that site. This scenario seems unlikely (and if they did, it would be generally insecure)

Our typical hosting customers have integrated certificate provisioning into their account/service set-up so a certificate can be provisioned quickly and easily. Normally, there is no user involvement in key generation and the backend systems take care of this provisioning and would not allow test certificates to be uploaded other than for the purpose they are intended (to secure a specific site). In this case, we don’t believe that there is a security issue since the system would be creating and validating domains/certificates as expected.

If users are able to initiate the domain validation process and if they are permitted to upload certificates for sites they don’t control, then there is a possibility that they could get a certificate for that domain. We can’t control what every provider does, so this risk remains.

While the vulnerabilities and risks are different between ACME TLS-SNI-01 and OneClick, we’d like to propose a risk mitigation approach similar to Let’s Encrypt with the use of a whitelist. We’ll verify that certain providers have secure practices in place to prevent users from requesting certificates outside of their permitted domains and then whitelist them.

If this is acceptable, we’d like to resume issuance today if possible.

Doug


From: Ryan Sleevi [mailto:ry...@sleevi.com]
Sent: Thursday, January 11, 2018 5:19 PM
To: Doug Beattie <doug.b...@globalsign.com>
Cc: mozilla-dev-s...@lists.mozilla.org
Subject: Re: Possible Issue with Domain Validation Method 9 in a shared hosting environment

On Thu, Jan 11, 2018 at 4:50 PM, Doug Beattie via dev-security-policy <dev-secur...@lists.mozilla.org<mailto:dev-secur...@lists.mozilla.org>> wrote:

Based on reported issues with TLS-SNI-01, we started investigation of our systems late yesterday regarding the use of "Test Certificate" validation, BR section 3.2.2.4.9.

We found that this method may be vulnerable to the some of the same underlying issue as the ACME TLS-SNI-01 so we disabled it at 10:51 AM today EST, January 11th.

While TLS-SNI-01 uses a host name like 773c7d.13445a.acme.invalid, GlobalSign uses the actual host name, www.example.com<http://www.example.com><http://www.example.com> which limits abuse, but we believe that the process might be vulnerable in some cases.

[Wayne Thayer]

Doug,

I have some questions:

>
> c. The hosting company must allow you to manually create and upload
> a CSR for a site you don’t own
>

> Did you mean to say 'certificate' here instead of 'CSR'?

d. The user must be able to trick the hosting provider to enable SNI
> for this domain and link it to the certificate they uploaded
>

> Is 'trick' the right term here? Isn't this just a default configuration
for vulnerable hosting providers?

While the vulnerabilities and risks are different between ACME TLS-SNI-01
> and OneClick,

Can you explain this statement? My impression is that the same
vulnerability affects both methods.

we’d like to propose a risk mitigation approach similar to Let’s Encrypt
> with the use of a whitelist. We’ll verify that certain providers have
> secure practices in place to prevent users from requesting certificates
> outside of their permitted domains and then whitelist them.
>

> Let's Encrypt has stated that this is a short- to medium-term mitigation.
Is your plan to continue to use this method indefinitely? Or are you
ultimately planning to fix or deprecate the method?

If this is acceptable, we’d like to resume issuance today if possible.
>

> If my understanding of the 3.2.2.4.9 vulnerability being essentially the
same as the 3.2.2.4.10 vulnerability, then this seems reasonable to me, at
least in the short term.

Wayne

[Gervase Markham]

On 12/01/18 14:52, Doug Beattie wrote:
> For shared IP address environments, it may be possible to receive a
> certificate for a domain you don’t actually control, but a number of
> things need to happen in order for this to be successful. What can
> go wrong?

Doug: what do you see as the exact differences between your setup and
the TLS-SNI-01 configuration? It seems to me that both are vulnerable in
the same circumstances (i.e., hosting provider has many users hosted on
the same IP address, and users have the ability to upload certificates
for arbitrary names without proving domain control).

Gerv

[Doug Beattie]

Wayne and Gerv,

I’ll try to answer both of your questions here.

From: Wayne Thayer [mailto:wth...@mozilla.com]
Sent: Friday, January 12, 2018 11:03 AM
To: Doug Beattie <doug.b...@globalsign.com>
Cc: ry...@sleevi.com; mozilla-dev-s...@lists.mozilla.org
Subject: Re: Possible Issue with Domain Validation Method 9 in a shared hosting environment

Doug,

I have some questions:

c. The hosting company must allow you to manually create and upload a CSR for a site you don’t own
Did you mean to say 'certificate' here instead of 'CSR'?

Yes, I meant to say certificate.

d. The user must be able to trick the hosting provider to enable SNI for this domain and link it to the certificate they uploaded
Is 'trick' the right term here? Isn't this just a default configuration for vulnerable hosting providers?

From Gerv: Doug: what do you see as the exact differences between your setup and the TLS-SNI-01 configuration? It seems to me that both are vulnerable in the same circumstances (i.e., hosting provider has many users hosted on the same IP address, and users have the ability to upload certificates for arbitrary names without proving domain control).

Normally a web hosting provider should not let you set SNI for a domain someone else is using, especially on that IP address. I think this is where method 9 deviates from method 10.

For method 10, you set up SNI on your server and add the acme.invalid string associated with your request/cert. Since nobody owns that invalid domain, the provider probably doesn’t care that you set up that SNI name and are using a certificate for that fqdn on their shared IP address.

For method 10 we look explicitly for the FQDN in the cert and there is no special SNI reconfiguration required (the site is there before, during and after the validation and issuance). Do hosting providers allow you to set SNI for domains you don’t own on a shared IP addresses? That sounds bad, but I defer to the experts here. Method 9 does not require that.

Also, the ACME client actively support the process of allowing this random acme.invalid value to be tied to the real FQDN and looks for requests based on that SNI name. All of the OneClick plugins (which btw, support similar features like client like key generation, cert installation and apache configuration), require that the FQDN being validated match the value in the certificate and the SNI server name. Validation will fail when the SNI does not match what is expected. The vast majority of OneClick endpoints are not vulnerable (yes, bad guys can modify the plugins and subvert the security we built in). Yes, there is a vulnerability, but I think it’s a smaller scale than what’s in TLS-SNI-01.

While the vulnerabilities and risks are different between ACME TLS-SNI-01 and OneClick,

Can you explain this statement? My impression is that the same vulnerability affects both methods.

Listed above.

we’d like to propose a risk mitigation approach similar to Let’s Encrypt with the use of a whitelist. We’ll verify that certain providers have secure practices in place to prevent users from requesting certificates outside of their permitted domains and then whitelist them.
Let's Encrypt has stated that this is a short- to medium-term mitigation. Is your plan to continue to use this method indefinitely? Or are you ultimately planning to fix or deprecate the method?

If we’re required to deprecate this because of similar security concerns, we can do that.

If this is acceptable, we’d like to resume issuance today if possible.
If my understanding of the 3.2.2.4.9 vulnerability being essentially the same as the 3.2.2.4.10 vulnerability, then this seems reasonable to me, at least in the short term.

Thanks Wayne.
Wayne

[Wayne Thayer]

On Fri, Jan 12, 2018 at 11:21 AM, Doug Beattie <doug.b...@globalsign.com>
wrote:

>
>
> Normally a web hosting provider should not let you set SNI for a domain
> someone else is using, especially on that IP address. I think this is
> where method 9 deviates from method 10.
>
>
>

I agree, it seems somewhat less likely that a hosting provider would allow
someone to create a site for abc.example.com if one already exists on the
same server. Are you aware of any hosting providers that do allow this?
Also, did you consider wildcard DNS records in your analysis of the
vulnerability? (see below)

For method 10, you set up SNI on your server and add the acme.invalid
> string associated with your request/cert. Since nobody owns that invalid
> domain, the provider probably doesn’t care that you set up that SNI name
> and are using a certificate for that fqdn on their shared IP address.
>
>
>

It's also possible that the only thing the hosting provider checks is if
there is already an SNI entry for that FQDN, in which case sites that
aren't configured for TLS would be vulnerable.

For method 10 we look explicitly for the FQDN in the cert and there is no
> special SNI reconfiguration required (the site is there before, during and
> after the validation and issuance).
>

Are you confusing method 9 and method 10 in this sentence and the one
below?

Do hosting providers allow you to set SNI for domains you don’t own on a
> shared IP addresses?
>

I think that is exactly what has been found to be true.

That sounds bad, but I defer to the experts here. Method 9 does not
> require that.
>
>
>


> Also, the ACME client actively support the process of allowing this random
> acme.invalid value to be tied to the real FQDN and looks for requests based
> on that SNI name. All of the OneClick plugins (which btw, support similar
> features like client like key generation, cert installation and apache
> configuration), require that the FQDN being validated match the value in
> the certificate and the SNI server name. Validation will fail when the SNI
> does not match what is expected. The vast majority of OneClick endpoints
> are not vulnerable (yes, bad guys can modify the plugins and subvert the
> security we built in). Yes, there is a vulnerability, but I think it’s a
> smaller scale than what’s in TLS-SNI-01.
>
>
>

Do you perform wildcard certificate validation with this method? If so,
could someone create a site for evil.example.com on the same server as
www.example.com and then get a cert for *.example.com by relying on a
wildcard DNS record in the example.com zone (i.e. DNS responds to a query
for evil.example.com with the IP for www.example.com)?

[Doug Beattie]

Wayne,

We didn’t really investigate wildcard issuance yet, but we can.

Given the discuss so far, we’re planning to proceed with a whitelisting approach tomorrow and we will plan to end the use of Method 9 (schedule TBD) which follows Let’s Encrypt handling of Method 10. If there are any additional security concerns that we need to be made aware of, please let me know and we can adjust the plan accordingly.

Doug

From: Wayne Thayer [mailto:wth...@mozilla.com]
Sent: Friday, January 12, 2018 3:43 PM
To: Doug Beattie <doug.b...@globalsign.com>
Cc: Gervase Markham <ge...@mozilla.org>; ry...@sleevi.com; mozilla-dev-s...@lists.mozilla.org
Subject: Re: Possible Issue with Domain Validation Method 9 in a shared hosting environment

On Fri, Jan 12, 2018 at 11:21 AM, Doug Beattie <doug.b...@globalsign.com<mailto:doug.b...@globalsign.com>> wrote:

Normally a web hosting provider should not let you set SNI for a domain someone else is using, especially on that IP address. I think this is where method 9 deviates from method 10.

I agree, it seems somewhat less likely that a hosting provider would allow someone to create a site for abc.example.com<http://abc.example.com> if one already exists on the same server. Are you aware of any hosting providers that do allow this? Also, did you consider wildcard DNS records in your analysis of the vulnerability? (see below)

For method 10, you set up SNI on your server and add the acme.invalid string associated with your request/cert. Since nobody owns that invalid domain, the provider probably doesn’t care that you set up that SNI name and are using a certificate for that fqdn on their shared IP address.

It's also possible that the only thing the hosting provider checks is if there is already an SNI entry for that FQDN, in which case sites that aren't configured for TLS would be vulnerable.

For method 10 we look explicitly for the FQDN in the cert and there is no special SNI reconfiguration required (the site is there before, during and after the validation and issuance).

Are you confusing method 9 and method 10 in this sentence and the one below?

Yes, Method 9.

Do hosting providers allow you to set SNI for domains you don’t own on a shared IP addresses?

I think that is exactly what has been found to be true.

That sounds bad, but I defer to the experts here. Method 9 does not require that.


Also, the ACME client actively support the process of allowing this random acme.invalid value to be tied to the real FQDN and looks for requests based on that SNI name. All of the OneClick plugins (which btw, support similar features like client like key generation, cert installation and apache configuration), require that the FQDN being validated match the value in the certificate and the SNI server name. Validation will fail when the SNI does not match what is expected. The vast majority of OneClick endpoints are not vulnerable (yes, bad guys can modify the plugins and subvert the security we built in). Yes, there is a vulnerability, but I think it’s a smaller scale than what’s in TLS-SNI-01.

Do you perform wildcard certificate validation with this method? If so, could someone create a site for evil.example.com<http://evil.example.com> on the same server as www.example.com<http://www.example.com> and then get a cert for *.example.com<http://example.com> by relying on a wildcard DNS record in the example.com<http://example.com> zone (i.e. DNS responds to a query for evil.example.com<http://evil.example.com> with the IP for www.example.com<http://www.example.com>)?

[Ryan Sleevi]

On Fri, Jan 12, 2018 at 4:24 PM, Doug Beattie <doug.b...@globalsign.com>
wrote:

> Wayne,
>
>
>
> We didn’t really investigate wildcard issuance yet, but we can.
>
>
>
> Given the discuss so far, we’re planning to proceed with a whitelisting
> approach tomorrow and we will plan to end the use of Method 9 (schedule
> TBD) which follows Let’s Encrypt handling of Method 10. If there are any
> additional security concerns that we need to be made aware of, please let
> me know and we can adjust the plan accordingly.
>

(Wearing a Google Hat)

Doug,

Thanks for sharing additional details. On the basis of what you've shared
so far, we do not believe this results in an appropriate level of security
for the ecosystem, and request that you do not re-enable issuance at this
time. This applies for any CA using methods similar to what you're using.

Broadly speaking,
https://groups.google.com/d/msg/mozilla.dev.security.policy/RHsIInIjJA0/HACyY9tMAAAJ
has shared the some of the principles we've used in this consideration. If
there is additional details that GlobalSign can share, related to those
principles, this would be invaluable.

This assessment is based on a number of factors, but includes:
- The validity period of certificates issued via this method means that
there is an unacceptably large window for certificates improperly issued to
be used.
- Based on the available information of expiration times and the potential
difficulty in renewing certificates using this method, the ecosystem risk
of disallowing this method is much less.
- The subtleties regarding Authorization Domain Names means that the risk
analysis provided is not sufficient - namely, it is unclear, as described,
whether it is possible to obtain a certificate for "www.example.com", on a
host that has a customer already configured on that domain (and
checking/enforcing certificates), by first applying for a certificate for "
example.com" as an attacker, providing and provisioning a test certificate
using that method (which is not configured to serve a certificate by the
'victim'), and then using that subsequent authorization of the
Authorization Domain Name to then apply for "www.example.com". Mitigating
this as a site operator would necessitate blocking not just on existant
domains, but also by the notion of Authorization Domain Name, and thus
represents a significant greater complexity in both assessing compliance
(for those on the whitelist) and for minimizing risk.
- The potential risk in maintaining this whitelist, given both the
statements provided by plans to move to deprecate this method post-haste
(e.g. no such plans) and the validity period of issued certificates (up to
39 months or, soon, 825 days).
- The lack of preexisting review and documentation of the specific protocol
being employed
- The potential risk of both domain name wildcards and of wildcard
issuance, which remains

While it is possible that you may be correct that the underlying root cause
of TLS-SNI presents greater risk, compared to this method, the many
mitigating factors that influenced our decision are not applicable here.

As this thread shows, we anticipate we will continue to find variants of
risk, and thus the whitelist approach, combined with the validity periods
caused by the risk (both of issued certificates and "completed
validations"), poses a long-term risk, even if we catch issues 'within
days'.

We'd like to continue discussing with GlobalSign and the community as to
the risk posed by immediately and permanently disabling this method, as
well as possible mitigations to the risk - both through issuance policies
of GlobalSign and technical measures in the usage - that may permit its
usage for a short-time to transition this method away. This is a
conversation we look forward to having over the next week. In the interim,
we'd ask you not re-enable this method.

[Matt Palmer]

On Fri, Jan 12, 2018 at 02:52:54PM +0000, Doug Beattie via dev-security-policy wrote:
> I’d like to follow up on our investigation and provide the community with some more information about how we use Method 9.
>

> 1) Client requests a test certificate for a domain (only one FQDN)

Does this test certificate chain to a publicly-trusted root? If so, on what
basis are you issuing a publicly-trusted certificate for a name which
doesn't appear to have been domain-control validated? If not, doesn't this
test certificate break the customer's SSL validation for the period the
certificate is installed, while you do the validation?

- Matt

[Ryan Hurst]

The certificate comes from a private PKI, not public one.

[Ryan Sleevi]

On Sat, Jan 13, 2018 at 8:46 PM, Ryan Hurst via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> On Friday, January 12, 2018 at 6:10:00 PM UTC-8, Matt Palmer wrote:

> The certificate comes from a private PKI, not public one.

Matt: The Baseline Requirements provide a definition of Test Certificate
that applies to 3.2.2.4.9 that already addresses your concerns:

Test Certificate: A Certificate with a maximum validity period of 30 days
and which: (i) includes a critical
extension with the specified Test Certificate CABF OID (2.23.140.2.1), or
(ii) is issued under a CA where there
are no certificate paths/chains to a root certificate subject to these
Requirements.

Ryan: I think it'd be good to let GlobalSign answer, or, if the answer is
available publicly, to point them out. This hopefully helps avoid confusion
:)

[Ryan Hurst]

Sleevi,

Valid point, no intention to confuse, I have no current affiliation with
GlobalSign, though I once did.

The documentation that described the protocol seems to no longer be online,
the behavior is observable and has been discussed in the validation working
group within the CABFORUM so it is not a secret.

Ryan

On Sun, Jan 14, 2018 at 7:10 AM, Ryan Sleevi <ry...@sleevi.com> wrote:

>
>
> On Sat, Jan 13, 2018 at 8:46 PM, Ryan Hurst via dev-security-policy <
> dev-secur...@lists.mozilla.org> wrote:
>
>> On Friday, January 12, 2018 at 6:10:00 PM UTC-8, Matt Palmer wrote:

[Doug Beattie]

From: Ryan Sleevi [mailto:ry...@sleevi.com]

Sent: Friday, January 12, 2018 5:53 PM
To: Doug Beattie <doug.b...@globalsign.com>
Cc: Wayne Thayer <wth...@mozilla.com>; Gervase Markham <ge...@mozilla.org>; ry...@sleevi.com; mozilla-dev-s...@lists.mozilla.org
Subject: Re: Possible Issue with Domain Validation Method 9 in a shared hosting environment

(Wearing a Google Hat)

Doug,

Thanks for sharing additional details. On the basis of what you've shared so far, we do not believe this results in an appropriate level of security for the ecosystem, and request that you do not re-enable issuance at this time. This applies for any CA using methods similar to what you're using.

Broadly speaking, https://groups.google.com/d/msg/mozilla.dev.security.policy/RHsIInIjJA0/HACyY9tMAAAJ has shared the some of the principles we've used in this consideration. If there is additional details that GlobalSign can share, related to those principles, this would be invaluable.

Ryan,

I had a hard time digesting that email because it compared so many different items, many of which aren’t directly applicable to the OneClick vs. method 10 that I want to focus on. The key points I took away from your email are:

“weak” manual method comparison with methods 9 and 10 (not applicable to the methods 9-10 comparison since we’re not comparing them to manual methods).

Short validity certificates represent more risk to ecosystem (expiration) and less risk (certs issued under the exploit will expire within 90 days – badness lasts for only 90 days).
I’ll address this point below, but given LE will allow renewals of possibly bad validations and attackers generally only operate with short periods of attacks before moving on, I don’t see the value of short lived certificates having meaningful reduction in risk within this context.

Ease of which an alternate method exists and can be used (discussion of manual vs. automated methods): Not applicable to the methods 9-10 comparison since they are both automated and have the same characteristics.

Risk is applicable to shared service providers and an accepted risk mitigation is to block SNI negotiations that contain “.invalid”. We also propose working with our customers on an account by account basis to assure they comply with the guidelines for use of method 9 until such time it’s re-affirmed, improved or deprecated from the BRs.

Perhaps I missed some other key points from that email.

This assessment is based on a number of factors, but includes:
- The validity period of certificates issued via this method means that there is an unacceptably large window for certificates improperly issued to be used.

Risk should not be based so heavily on the validity period, which seems to be one of your consistent points. The number of certificates issued along with the probability of a failure should both be used in the ecosystem risk computation. Given LE issues orders of magnitude more certificates to unique endpoints, I think the risk to the eco system at large with the GlobalSign issuance is lower of that with LE (when it comes to the topic of validity periods).

Risk = impact x probability: With the number of LE endpoints (or anyone using Method 10 in high volumes), the probability of a successful attack is vastly higher due to the sheer number of servers, and the impact for both methods is the same (a certificate issued to a successful attacker)

- Based on the available information of expiration times and the potential difficulty in renewing certificates using this method, the ecosystem risk of disallowing this method is much less.

How did you come to the conclusion that validity periods and renewal challenges substantially increase the risk of method 9?
1) While a GlobalSign certificate would be valid for a longer period than LE (typically 1 year, but up to 3), typical attacks are done, detected, resolved within days or weeks I don’t believe that the validity period of certificates significantly increases the risk when exploited in the way as described (the target site would typically notice they were compromised and it would be reported and the certified revoked within days or weeks). A more important factor is the number of certificates that may be issued, not their validity period.
2) While LE’s validity period is shorter, they re-use the validation for subsequent issuance thus the time between validation and expiration is longer than 90 days (I believe the domain validations can be cached for 60 days). This equates to 5 months vs. generally 12 months for GlobalSign. And since LE will permit domain renewal of possibly bad authentications, the 5 months could average out to be substantially higher.
3) While the renewal process is currently not optimal, it’s been working for 5 years without significant pushback from our customers. I fail to see how this factors into risk in a meaningful way. I may have missed your point.


- The subtleties regarding Authorization Domain Names means that the risk analysis provided is not sufficient - namely, it is unclear, as described, whether it is possible to obtain a certificate for "www.example.com<http://www.example.com>", on a host that has a customer already configured on that domain (and checking/enforcing certificates), by first applying for a certificate for "example.com<http://example.com>" as an attacker, providing and provisioning a test certificate using that method (which is not configured to serve a certificate by the 'victim'), and then using that subsequent authorization of the Authorization Domain Name to then apply for "www.example.com<http://www.example.com>".
Each and every certificate undergoes its own validation – there is no re-use of validation data when issuing subsequent certificates. I should have stated this earlier. I hope this answers this question.

Mitigating this as a site operator would necessitate blocking not just on existant domains, but also by the notion of Authorization Domain Name, and thus represents a significant greater complexity in both assessing compliance (for those on the whitelist) and for minimizing risk.

Given the statement above, this isn’t a risk.

- The potential risk in maintaining this whitelist, given both the statements provided by plans to move to deprecate this method post-haste (e.g. no such plans) and the validity period of issued certificates (up to 39 months or, soon, 825 days).

Since LE can continue to renew certificates issued under this method prior to this change, doesn’t that effectively allow longer effective validity periods? I recognize there is a difference between renewing and long validity certs, but allowing renewal of certs issued under the flawed method seems to reduce value of your argument here.

- The lack of preexisting review and documentation of the specific protocol being employed

The process was discussed as part of the BR update and it’s documented in the BRs, and I hope the supplemental information provided here helps. Also, the OneClick plugins and intended use is not susceptible to this attack because the plugin does not permit upload/use of certificates for domains other than the one the plug-in is managing (yes, clients can change the code or develop their own, so the protocol is susceptible in the end).

- The potential risk of both domain name wildcards and of wildcard issuance, which remains

I hope I addressed this above. We don’t re-use domain validation data for subsequent requests.

While it is possible that you may be correct that the underlying root cause of TLS-SNI presents greater risk, compared to this method, the many mitigating factors that influenced our decision are not applicable here.

I attempted (likely unsuccessfully) to reiterate the risk topics you raised with both methods and hope this makes the risks associated with method 9 similar to those of method 10.

As this thread shows, we anticipate we will continue to find variants of risk, and thus the whitelist approach, combined with the validity periods caused by the risk (both of issued certificates and "completed validations"), poses a long-term risk, even if we catch issues 'within days'.

Validity period discussions above hopefully helps to mitigate the risk of OneClick as compared with method 10 @ LE

We'd like to continue discussing with GlobalSign and the community as to the risk posed by immediately and permanently disabling this method, as well as possible mitigations to the risk - both through issuance policies of GlobalSign and technical measures in the usage - that may permit its usage for a short-time to transition this method away. This is a conversation we look forward to having over the next week. In the interim, we'd ask you not re-enable this method.

Thanks for keeping the dialog open. The domain validation method remains disabled, but we want to continue the discussion so the community better understands the risks an how we can re-enable this for some specific customer accounts who have demonstrated the proper controls are in place.

Doug’s summary:

- Implement mitigations to prevent exploitations can be coordinated with our major customers to reduce or eliminate the risk (similar to LE)

- GlobalSign performs domain validation for every issuance, vs caching validation info.

- Total active OneClick certificates <100K

- Total number of active OneClick customers: < 10

- Risk to Ecosystem with GlobalSign is lower based on active certificates (100K vs. tens of millions). Perhaps the more important factor is not active certificates, but certificates issued which increases the difference between GlobalSign and LE.

- Time between domain validation and certificate expiration is 12 months (typical GlobalSign cert) vs. 5-8 or more months with LE (since domains can be renewed with method 10 without mitigations, even if prior and current validation methods are vulnerable)

I hope the information provided above will help advance the discussion.

[Ryan Sleevi]

On Mon, Jan 15, 2018 at 1:18 PM, Doug Beattie <doug.b...@globalsign.com>
wrote:

>
>
>
>
> *From:* Ryan Sleevi [mailto:ry...@sleevi.com]
> *Sent:* Friday, January 12, 2018 5:53 PM
> *To:* Doug Beattie <doug.b...@globalsign.com>
> *Cc:* Wayne Thayer <wth...@mozilla.com>; Gervase Markham <
> ge...@mozilla.org>; ry...@sleevi.com; mozilla-dev-security-policy@
> lists.mozilla.org
> *Subject:* Re: Possible Issue with Domain Validation Method 9 in a shared

I think these points may not have been fully appreciated. I don't see
evidence from this mail, or from the ecosystem, that the OneClick method
poses both the same risk and the same level of review as ACME's TLS-SNI,
and I think we may fundamentally disagree about the risk profile of
certificates with long validity periods, and both the detrimental effect
they have on reasoning about ecosystem security AND the ways in which they
mitigate the need to 'quickly re-enable this'

> This assessment is based on a number of factors, but includes:
>
> - The validity period of certificates issued via this method means that
> there is an unacceptably large window for certificates improperly issued to
> be used.
>
> Risk should not be based so heavily on the validity period, which seems to
> be one of your consistent points. The number of certificates issued along
> with the probability of a failure should both be used in the ecosystem risk
> computation.
>

We must disagree then. Risk is profoundly dependent on the validity period
- one of the key mitigations to making an improper evaluation is the
knowledge that the risk is bounded, whereas greater than 90 days represents
significantly greater risk.

> Given LE issues orders of magnitude more certificates to unique
> endpoints, I think the risk to the eco system at large with the GlobalSign
> issuance is lower of that with LE (when it comes to the topic of validity
> periods).
>

As I tried to explained in our considerations, we believe quite the
opposite. The large number of ACME (since this is, to be clear, not Let's
Encrypt specific) endpoints, combined with the shorter-lived validity
period, presents significant risk to immediately turning it off, while
GlobalSign's longer period, combined with lesser issuance, reduces that
risk of impact to the ecosystem from taking the defensibly conservative
choice.

> Risk = impact x probability: With the number of LE endpoints (or anyone
> using Method 10 in high volumes), the probability of a successful attack is
> vastly higher due to the sheer number of servers, and the impact for both
> methods is the same (a certificate issued to a successful attacker)
>

Your assessment of the impact I believe is inverse. An impact of a
misissued 3-year certificate is profoundly worse than that of a 90 day
certificate.

This should be rather self-evident, given the ample discussion about
certificate lifetimes in the Forum, but consider a CA that issued 100 3
year certificates versus a CA that issued 1000 30 day certificates. Let's
further presume that all turn out to be problematic. The size of the CRLs,
the impact to clients, and the cost to mitigate that are substantially
greater for those 3 year certs, even though there are an order of magnitude
fewer of them than the 30 day certificates. The risk exposure, to the
overall ecosystem (e.g. including those outside of the browser space) is
equally profoundly greater.

- Based on the available information of expiration times and the potential
> difficulty in renewing certificates using this method, the ecosystem risk
> of disallowing this method is much less.
>
> How did you come to the conclusion that validity periods and renewal
> challenges substantially increase the risk of method 9?
>
> 1) While a GlobalSign certificate would be valid for a longer period than
> LE (typically 1 year, but up to 3), typical attacks are done, detected,
> resolved within days or weeks I don’t believe that the validity period of
> certificates significantly increases the risk when exploited in the way as
> described (the target site would typically notice they were compromised and
> it would be reported and the certified revoked within days or weeks). A
> more important factor is the number of certificates that may be issued, not
> their validity period.
>

We disagree on this point, for the reasons explained above. This is a
long-standing position regarding risk in the ecosystem, as reflected in our
past discussions regarding validity periods, and thus is entirely
consistent with past policies and actions.

> 2) While LE’s validity period is shorter, they re-use the validation for
> subsequent issuance thus the time between validation and expiration is
> longer than 90 days (I believe the domain validations can be cached for 60
> days). This equates to 5 months vs. generally 12 months for GlobalSign.
> And since LE will permit domain renewal of possibly bad authentications,
> the 5 months could average out to be substantially higher.
>
> 3) While the renewal process is currently not optimal, it’s been working
> for 5 years without significant pushback from our customers. I fail to see
> how this factors into risk in a meaningful way. I may have missed your
> point.
>

To be honest, I'm not quite sure what you mean by "domain renewal" in this
respect.

However, as explained, when factoring in risks to the ecosystem, we take a
holistic view about the impact of making a security-negative decision (such
as improperly allowing something that should be rejected) and a
security-positive decision (such as rejecting something that might
otherwise be acceptable). This holistic calculus informed our position
here, and while it's clear that we disagree with respect to the 'improperly
allow' risk (and for which I tried to explain why that may be), it's also
unclear that there's any new information about the impact from taking a
security-positive decision here.

> - The subtleties regarding Authorization Domain Names means that the risk
> analysis provided is not sufficient - namely, it is unclear, as described,

> whether it is possible to obtain a certificate for "www.example.com", on

> a host that has a customer already configured on that domain (and
> checking/enforcing certificates), by first applying for a certificate for "

> example.com" as an attacker, providing and provisioning a test

> certificate using that method (which is not configured to serve a
> certificate by the 'victim'), and then using that subsequent authorization

> of the Authorization Domain Name to then apply for "www.example.com".

>
> Each and every certificate undergoes its own validation – there is no
> re-use of validation data when issuing subsequent certificates. I should
> have stated this earlier. I hope this answers this question.
>

It doesn't. A reuse of validation is distinct from the validation of the
Authorization Domain Name. I was highlighting the scoping issues of ADN vs
FQDN, and your response doesn't seem to address that.

> - The potential risk in maintaining this whitelist, given both the
> statements provided by plans to move to deprecate this method post-haste
> (e.g. no such plans) and the validity period of issued certificates (up to
> 39 months or, soon, 825 days).
>
> Since LE can continue to renew certificates issued under this method prior
> to this change, doesn’t that effectively allow longer effective validity
> periods? I recognize there is a difference between renewing and long
> validity certs, but allowing renewal of certs issued under the flawed
> method seems to reduce value of your argument here.
>

No, it doesn't, because in the event of misissuance, the attacker's ability
is not the full duration (or 5 months, as you suggest), but bounded by the
lifetime of the certificate. These are fundamentally different risks - and
that's why the validity period of the certificate itself is far more
important than the reuse period of the information. A victim can contact an
ACME using CA to invalidate the information, thus preventing renewal, and
the attacker is still bound to the lifetime of the existing certificate.

Compare this with a certificate issued by 1-3 years by GlobalSign, in which
even if a victim contacts GlobalSign, the most that GlobalSign can do is to
revoke that certificate, which is ineffective at scale. This permits the
attacker a far greater 'attack' window, even though GS might have revoked
it, and is a key and fundamental difference.

> - The lack of preexisting review and documentation of the specific
> protocol being employed
>
> The process was discussed as part of the BR update and it’s documented in
> the BRs, and I hope the supplemental information provided here helps.
> Also, the OneClick plugins and intended use is not susceptible to this
> attack because the plugin does not permit upload/use of certificates for
> domains other than the one the plug-in is managing (yes, clients can change
> the code or develop their own, so the protocol is susceptible in the end).
>

Right, I'm focusing on the protocol, and I don't believe that the protocol
implemented by OneClick is equivalent to what's described in 3.2.2.4.9. I
believe it is compatible with/an implementation of, but just as 3.2.2.4.10
does not describe ACME TLS-SNI (nor does 3.2.2.4.6 describe ACME HTTP-01),
despite being compatible, we're treating this based on the available
information of and public review and discussion of the specific protocol.

> As this thread shows, we anticipate we will continue to find variants of
> risk, and thus the whitelist approach, combined with the validity periods
> caused by the risk (both of issued certificates and "completed
> validations"), poses a long-term risk, even if we catch issues 'within
> days'.
>
> Validity period discussions above hopefully helps to mitigate the risk of
> OneClick as compared with method 10 @ LE
>

As I hope you can see, the validity period conversation again highlights
that we believe it presents significant risk to the ecosystem to allow such
certificates to be valid for greater than 90 days, at the most. This
applies to any user of ACME's TLS-SNI, and more than likely, any (existing)
validation method under the 3.2.2.4.9 / 3.2.2.4.10 aegis.

> Doug’s summary:
>
> - Implement mitigations to prevent exploitations can be
> coordinated with our major customers to reduce or eliminate the risk
> (similar to LE)
>

We don't believe the proposal meaningfully does so, for the reasons
outlined.

> - GlobalSign performs domain validation for every issuance, vs
> caching validation info.
>

The issue is not with revalidation at issuance, but about the permitted use
of the Authorization Domain Name.

> - Total active OneClick certificates <100K
>
> - Total number of active OneClick customers: < 10
>

This highlights that the risk (to not allowing issuance) is potentially
small, and can meaningfully be addressed by engaging with those 10
customers.

As you no doubt know, our goal is to ensure a consistent response, and we
see significant danger in posing 'whitelisting' as a solution, as it
fundamentally devolves into a question about how the CA manages and
maintains that whitelist, which exists outside the remit of the BRs. This
is where steps, such as those proposed by Let's Encrypt, to move to disable
and/or prevent the usage of TLS-SNI (particularly in new deployments),
poses a way to both mitigate the immediate risk and to move towards
deprecation overall.

GlobalSign's in a unique position, given the small number of deployments,
to be able to work to devise technical solutions that may mitigate the
risk, and effective deployment may be a better solution for the ecosystem
both medium- and long-term. Inevitably, however, validity periods must play
a part as any short-term mitigation (in addition to exploring technical
solutions), so that we can arrive at something that is consistent across
CAs and consistent among risk factors.

> - Risk to Ecosystem with GlobalSign is lower based on active
> certificates (100K vs. tens of millions). Perhaps the more important
> factor is not active certificates, but certificates issued which increases
> the difference between GlobalSign and LE.
>

Agreed that the risk to disallowing issuance is lower. I believe you were
trying to highlight something different, but hopefully the responses above
highlight why we may disagree that the GlobalSign solution is less risky.

> - Time between domain validation and certificate expiration is
> 12 months (typical GlobalSign cert) vs. 5-8 or more months with LE (since
> domains can be renewed with method 10 without mitigations, even if prior
> and current validation methods are vulnerable)
>

As noted above, the time period noted here is incorrect in its assessment.

[Doug Beattie]

Ryan,

I’m not sure where we go from here. We have customers that need certificates and they have demonstrated they can comply with not permitting the creation and use of certificates for domains other than those that the hosting company is hosting for that customer. All certificates will continue to be posted to CT logs.

As far as the wildcard question, when someone asks for a wildcard cert for a domain like *.us.example.com, we validate on that minus the * (so, us.example.com in this case).

We’d like to move forward with issuing certificates with controls in place. If there are any other controls you need us to implement to resume issuance, let us know. For example, if we limit validity to 1 year (possibly up to 15 months) and if we put a firm end date for OneClick for July 1, 2018, would that suffice?

Doug

From: Ryan Sleevi [mailto:ry...@sleevi.com]
Sent: Monday, January 15, 2018 2:31 PM
To: Doug Beattie <doug.b...@globalsign.com>

Cc: ry...@sleevi.com; Wayne Thayer <wth...@mozilla.com>; Gervase Markham <ge...@mozilla.org>; mozilla-dev-s...@lists.mozilla.org
Subject: Re: Possible Issue with Domain Validation Method 9 in a shared hosting environment



On Mon, Jan 15, 2018 at 1:18 PM, Doug Beattie <doug.b...@globalsign.com<mailto:doug.b...@globalsign.com>> wrote:

From: Ryan Sleevi [mailto:ry...@sleevi.com<mailto:ry...@sleevi.com>]
Sent: Friday, January 12, 2018 5:53 PM
To: Doug Beattie <doug.b...@globalsign.com<mailto:doug.b...@globalsign.com>>
Cc: Wayne Thayer <wth...@mozilla.com<mailto:wth...@mozilla.com>>; Gervase Markham <ge...@mozilla.org<mailto:ge...@mozilla.org>>; ry...@sleevi.com<mailto:ry...@sleevi.com>; mozilla-dev-s...@lists.mozilla.org<mailto:mozilla-dev-s...@lists.mozilla.org>
Subject: Re: Possible Issue with Domain Validation Method 9 in a shared hosting environment

(Wearing a Google Hat)

Doug,

Thanks for sharing additional details. On the basis of what you've shared so far, we do not believe this results in an appropriate level of security for the ecosystem, and request that you do not re-enable issuance at this time. This applies for any CA using methods similar to what you're using.

Broadly speaking, https://groups.google.com/d/msg/mozilla.dev.security.policy/RHsIInIjJA0/HACyY9tMAAAJ has shared the some of the principles we've used in this consideration. If there is additional details that GlobalSign can share, related to those principles, this would be invaluable.

- The subtleties regarding Authorization Domain Names means that the risk analysis provided is not sufficient - namely, it is unclear, as described, whether it is possible to obtain a certificate for "www.example.com<http://www.example.com>", on a host that has a customer already configured on that domain (and checking/enforcing certificates), by first applying for a certificate for "example.com<http://example.com>" as an attacker, providing and provisioning a test certificate using that method (which is not configured to serve a certificate by the 'victim'), and then using that subsequent authorization of the Authorization Domain Name to then apply for "www.example.com<http://www.example.com>".

[Doug Beattie]

> -----Original Message-----
> From: Nick Lamb [mailto:n...@tlrmx.org]
> Sent: Monday, January 15, 2018 2:39 PM
>
> > - Total number of active OneClick customers: < 10
>

> What constitutes a OneClick customer in this sense?

These are web hosting companies that receive certificates for their users. We used to focus this on cPanel and similar control panels, but have largely moved away from them. These are customers that want an automated method to issue certificates and where HTTP and DNS methods are not suitable, or where they haven't wanted to re-work their APIs to use them. We believe all of these customers can be migrated over to HTTP or DNS methods (there are basically no other automated options if both 9 and 10 have security vulnerabilities).

Each customer has an account with us so we know where the requests are coming from.

> The focus of concern for tls-sni-01 was service providers who present an
> HTTPS endpoint for many independent entities, most commonly a bulk web
> host or a CDN. These function as essentially a "Confused Deputy" in the
> discovered attack on tls-sni-01. For those providers there would undoubtedly
> be a temptation to pretend all is well (to keep things
> working) even if in fact they aren't able to defeat this attack or some trivial
> mutation of it, and that's coloured Let's Encrypt's response, because there's
> just no way to realistically police whitelisting of thousands or tens of
> thousands of such service providers.

> From the volumes versus numbers of customers, it seems as though OneClick
> must be targeting the same type of service providers, is that right?

Yes.

> The small number of such customers suggests that, unlike Let's Encrypt, it
> could be possible for GlobalSign to diligently affirm that each of the customers
> has technical countermeasures in place to protect their clients from each
> other.

Yes, for those customers that want to continue with this method, we would confirm they meet the criteria.

> In my opinion such an approach ought to be adequate to continue using
> OneClick in the short term, say for 12-18 months with the understanding that
> this validation method will either be replaced by something less problematic or
> the OneClick service will go away in that time.

We can do with an even shorter period - 6 months should be sufficient.

Thanks for the support!

> But of course I do not speak for Google, Mozilla or any major trust store.

[Doug Beattie]

From: Ryan Sleevi [mailto:ry...@sleevi.com]
Sent: Monday, January 15, 2018 4:14 PM
To: Doug Beattie <doug.b...@globalsign.com>
Cc: ry...@sleevi.com; mozilla-dev-s...@lists.mozilla.org; Gervase Markham <ge...@mozilla.org>; Wayne Thayer <wth...@mozilla.com>
Subject: Re: Possible Issue with Domain Validation Method 9 in a shared hosting environment

On Mon, Jan 15, 2018 at 3:36 PM, Doug Beattie via dev-security-policy <dev-secur...@lists.mozilla.org<mailto:dev-secur...@lists.mozilla.org>> wrote:
Ryan,

I’m not sure where we go from here.

As suggested, we encourage you to work on devising technical mitigations or alternative methods of validating such certificates that can meet the use case. We don't think that, as described, the OneClick method meets the necessary level of assurance, nor do the necessary level of mitigating factors exist, to consider such certificates trustworthy.

Ryan – I’m at a loss. The security threat is that a user can request a certificate for a domain they don’t own from hosting companies that permit SNI mappings to domains the user doesn’t own or control. This permits them to pass validation for a domain they don’t control that is on the same IP address as their legitimate site (or at least to which they have this level of SNI control). We will verify that our OneClick customers will request certificates for domains the hosting company is actively managing for their users and not permit malicious actions (much like LE verifying that their hosting companies do not permit “acme.invalid” domains to be used). This eliminates the problems of SNI being used as an avenue for domain validation for malicious actors. Is this not sufficient for some reason?

Surely you agree that within non-shared hosting environments OneClick is not vulnerable and can be used.

We have customers that need certificates and they have demonstrated they can comply with not permitting the creation and use of certificates for domains other than those that the hosting company is hosting for that customer. All certificates will continue to be posted to CT logs.

While understanding and sensitive to this, what you're asking is that, on the basis of an abstract need, that known-insecure methods be used, with the assurance that the CA has taken steps (which are fundamentally non-interoperable) to mitigate, and for which an improper decision by a CA has no further mitigating factors. This does not provide a sufficient level of assurance to permit its continued use.

As far as the wildcard question, when someone asks for a wildcard cert for a domain like *.us.example.com<http://us.example.com>, we validate on that minus the * (so, us.example.com<http://us.example.com> in this case).

I'm afraid you're still missing the point of FQDN versus Authorization Domain Name. This further does not instill confidence that it's fully been described.
We’re using us.example.com as the ADN for validation in this example. We always use the FQDN minus “*.” For the ADN.

We’d like to move forward with issuing certificates with controls in place.

I'm sorry, but at present, we do not feel it is in the appropriate interests of users, sites, or the ecosystem to permit this.

If there are any other controls you need us to implement to resume issuance, let us know. For example, if we limit validity to 1 year (possibly up to 15 months) and if we put a firm end date for OneClick for July 1, 2018, would that suffice?

As stated, we believe 90 days is an appropriate and necessary upper-bound for such certificates.

[Eric Mill]

On Mon, Jan 15, 2018 at 4:22 PM, Ryan Sleevi <ry...@sleevi.com> wrote:

>
>
> On Mon, Jan 15, 2018 at 4:11 PM, Eric Mill via dev-security-policy <
> dev-secur...@lists.mozilla.org> wrote:
>
>> That said, GlobalSign's offer to cut certificate lifetimes down to X
>> months
>> during the short-term, and to make sure OneClick is disabled within Y
>> months from now, seems like a reasonable compromise that doesn't undercut
>> the incentive for GlobalSign or their customers to rapidly transition to
>> more secure methods. It seems like there should be a value of X and Y that
>> are acceptable.
>>
>
> There are a lot of factors to consider, as I tried to highlight, that
> contribute to whether or not X can be > 0 and whether Y can be > 0 (e.g. no
> issuance, immediate discontinuance) at all. That is, these additional
> factors beyond the protocol itself inherently contribute to whether or not
> there is a generalizable answer. Factors such as ecosystem impact versus
> ecosystem risk, existing practices that can be used as mitigating factors,
> the level of trust necessary to ascribe to the issuing CA (and the steps
> that are taken to mitigate failures of that trust) - all influence that
> calculus.
>

I can only go on what's on the public list, but if it is as it appears and
GS proactively researched their offering, identified a similar weakness via
a separate BR method, and voluntarily turned off their implementation right
away, that is the kind of activity I would think Mozilla and Google would
want to encourage (and not accidentally penalize). An X of 3 months (90
days) and a Y that resembled Let's Encrypt's deprecation timeline, might
help offer a lifeline without introducing unacceptable risk.

I understand that there are other factors, including the level of review
the protocol has been subject to and a holistic consideration of
GlobalSign's infrastructure and history, including non-public information,
and I'm not saying it would be necessarily unfair to keep GS' OneClick
offline for shared hosts. But I do think that incentivizing proactive
security interventions on the part of CAs is another factor worth
considering.

-- Eric

--
konklone.com | @konklone <https://twitter.com/konklone>

[Ryan Sleevi]

On Mon, Jan 15, 2018 at 4:40 PM, Doug Beattie <doug.b...@globalsign.com>

wrote:

>
>
>
>
> *From:* Ryan Sleevi [mailto:ry...@sleevi.com]

> *Sent:* Monday, January 15, 2018 4:14 PM

> *To:* Doug Beattie <doug.b...@globalsign.com>

> *Cc:* ry...@sleevi.com; mozilla-dev-s...@lists.mozilla.org;

> Gervase Markham <ge...@mozilla.org>; Wayne Thayer <wth...@mozilla.com>

> *Subject:* Re: Possible Issue with Domain Validation Method 9 in a shared

> hosting environment
>
>
>
>
>
>
>
> On Mon, Jan 15, 2018 at 3:36 PM, Doug Beattie via dev-security-policy <

> dev-secur...@lists.mozilla.org> wrote:
>
> Ryan,
>
> I’m not sure where we go from here.
>
>
>
> As suggested, we encourage you to work on devising technical mitigations
> or alternative methods of validating such certificates that can meet the
> use case. We don't think that, as described, the OneClick method meets the
> necessary level of assurance, nor do the necessary level of mitigating
> factors exist, to consider such certificates trustworthy.
>
>
>
> Ryan – I’m at a loss. The security threat is that a user can request a
> certificate for a domain they don’t own from hosting companies that permit
> SNI mappings to domains the user doesn’t own or control. This permits them
> to pass validation for a domain they don’t control that is on the same IP
> address as their legitimate site (or at least to which they have this level
> of SNI control). We will verify that our OneClick customers will request
> certificates for domains the hosting company is actively managing for their
> users and not permit malicious actions (much like LE verifying that their
> hosting companies do not permit “acme.invalid” domains to be used). This
> eliminates the problems of SNI being used as an avenue for domain
> validation for malicious actors. Is this not sufficient for some reason?
>
>
>
> Surely you agree that within non-shared hosting environments OneClick is
> not vulnerable and can be used.
>

No, it's not sufficient.

The failure mode unfortunately necessarily includes a failure by GlobalSign
process and/or personnel, and in that failure mode, there are further no
mitigating factors.

- If GlobalSign adds a vulnerable entity to their whitelist
- The certificates issued will be valid for 1-3 years, leaving only the
(broken) revocation system as recourse
- There is no step organizations can take to pre-emptively mitigate the
risk of GlobalSign adding to the whitelist (compared to, say, blocking
.invalid)
- There is no ability for site operators to detect such situations. A
consideration, not listed within the full set when discussing Let's Encrypt
and the ACME TLS-SNI method, is that we have at least public commitment by
Let's Encrypt and demonstrated evidence of sustained/long-term compliance
with publicly disclosing all issued certificates ( as noted in
https://letsencrypt.org/certificates/ ). While I realize you've offered to
do so, I can find no evidence of GlobalSign doing so by default, and so
this further adds to the risk calculus of a commitment to do something not
yet practice and thus not yet consistently, reliably delivered on.

There is not, in our view, reason to accept this significantly greater
(holistically considered) risk.

We're open to understanding whether GlobalSign has additional proposals how
to mitigate this risk, given the set of concerns expressed - technical
measures and policy measures. These may provide a path to allowing such
issuance in the future, but we don't think that, given the holistic risk
assessment, it's appropriate to allow it to immediately resume. We are keen
to find solutions that work, as we understand that these can enable
powerful new use cases, but we want to balance that with the risk posed.

I would encourage GlobalSign to consult Sections 3.2.2.4 and 3.2.2.5 to see
if there are any other alternative methods to validating that might
represent an appropriate balance. Given that these are posed as automated
certificates, there seem that other methods may be suitable for the
issuance of Domain Validated certificates - or, with care, Organizational
Validated. If it truly is that none of these other methods (outside
3.2.2.4.9 and .10, and understandably the in-discussion-for-removal .1 and
.5), are there steps that can be taken that provide concrete technical
mitigations (ideally, through an opt-in technical signal by the site
operator) that can reduce or eliminate this risk? Are there steps that can
be taken with policy to similarly address the concerns?

It's not that this is an unsolvable problem, but it's necessary to make
further changes to mitigate and minimize the risk, and some of the major
factors that contribute to that assessment have been shared.

[Ryan Sleevi]

On Mon, Jan 15, 2018 at 4:54 PM, Eric Mill <er...@konklone.com> wrote:

> I can only go on what's on the public list, but if it is as it appears and
> GS proactively researched their offering, identified a similar weakness via
> a separate BR method, and voluntarily turned off their implementation right
> away, that is the kind of activity I would think Mozilla and Google would
> want to encourage (and not accidentally penalize).
>

It is certainly the minimum we (Google) would expect when CAs are notified
of potential issues with validation methods :)

> An X of 3 months (90 days) and a Y that resembled Let's Encrypt's
> deprecation timeline, might help offer a lifeline without introducing
> unacceptable risk.
>

Hopefully it's clear that finding a solution within that space of variables
would indeed help resolve a number of the concerns.

I understand that there are other factors, including the level of review
> the protocol has been subject to and a holistic consideration of
> GlobalSign's infrastructure and history, including non-public information,
> and I'm not saying it would be necessarily unfair to keep GS' OneClick
> offline for shared hosts. But I do think that incentivizing proactive
> security interventions on the part of CAs is another factor worth
> considering.
>

Indeed, it's the absence of these other details that make it considerably
more difficult to find a solution to enable the issuance, and why we remain
committed to working with GlobalSign to understand whether there can be a
solution that meets the needs by reducing the risks.

As you've highlighted, shorter lived certificates with an explicit sunset
of either (correct or deprecate), are a great way to move that forward.
Unlike Let's Encrypt's usage of ACME's TLS-SNI, GlobalSign is in an
advantageous position of having a limited set of customers, for which they
can quickly explore and roll out technical solutions - making it easier to
move to the 'correct' side of the equation, rather than 'deprecate'.
Similarly, it's possible that such customers may be viable under the
3.2.2.4.8 method of validation, which, while prohibiting the issuance of
wildcard certificates, does represent an alternative method for validating
the cloud/hosting provider meets those requirements.

Unlike the discussion within the ACME WG of IETF, it may be that moving
OneClick to an ALPN (RFC 7301) solution may not be viable, due to the
considerations for the registry expressed within Section 6. However, it may
be that alternative technical solutions exist that signal the 'opt-in'
nature exist, such as the CAA list that Let's Encrypt decided against.

This is the discussion, and which we welcome all participants to help us
understand. Given the set of concerns, what steps can be taken to reduce or
mitigate those concerns. Hopefully they're reasonably highlighted, and
while there represents a bare minimum expectation of security that is
unfortunately uncompromising, we are hopeful we can find a solution that
works best for the ecosystem.

[Doug Beattie]

Ryan,

Here is some more information to continue the discussion.

- We will continue to post all certificates to CT logs so issuance can be monitored.

- We will reduce validity period of OneClick certificates to 6 months.

- We will work with the hosting providers (on a case by case basis) to implement processes and procedures which prevent the uploading and use of test certificates on user controlled shared IP addresses (similar to how LE worked with their larger customers to blocking acme.invalid from being used)

More below.

From: Ryan Sleevi [mailto:ry...@sleevi.com]
Sent: Monday, January 15, 2018 4:56 PM
To: Doug Beattie <doug.b...@globalsign.com>

Cc: ry...@sleevi.com; mozilla-dev-s...@lists.mozilla.org; Gervase Markham <ge...@mozilla.org>; Wayne Thayer <wth...@mozilla.com>
Subject: Re: Possible Issue with Domain Validation Method 9 in a shared hosting environment
As suggested, we encourage you to work on devising technical mitigations or alternative methods of validating such certificates that can meet the use case. We don't think that, as described, the OneClick method meets the necessary level of assurance, nor do the necessary level of mitigating factors exist, to consider such certificates trustworthy.

Ryan – I’m at a loss. The security threat is that a user can request a certificate for a domain they don’t own from hosting companies that permit SNI mappings to domains the user doesn’t own or control. This permits them to pass validation for a domain they don’t control that is on the same IP address as their legitimate site (or at least to which they have this level of SNI control). We will verify that our OneClick customers will request certificates for domains the hosting company is actively managing for their users and not permit malicious actions (much like LE verifying that their hosting companies do not permit “acme.invalid” domains to be used). This eliminates the problems of SNI being used as an avenue for domain validation for malicious actors. Is this not sufficient for some reason?

Surely you agree that within non-shared hosting environments OneClick is not vulnerable and can be used.

No, it's not sufficient.

The failure mode unfortunately necessarily includes a failure by GlobalSign process and/or personnel, and in that failure mode, there are further no mitigating factors.

- If GlobalSign adds a vulnerable entity to their whitelist
- The certificates issued will be valid for 1-3 years, leaving only the (broken) revocation system as recourse

We can and will reduce max validity to 6 months as a standard configuration option within our system.

- There is no step organizations can take to pre-emptively mitigate the risk of GlobalSign adding to the whitelist (compared to, say, blocking .invalid)

Actually, there is and I apologize for not making this more clear before. We have site operators that manage the issuance of certificates for their users. End users have no access to the issuance process, in uploading test certificates to their sites, or any involvement in the issuance process as this is automated by the site operators. Given this approach is verified with the provider, we would propose whitelisting the account.

- There is no ability for site operators to detect such situations. A consideration, not listed within the full set when discussing Let's Encrypt and the ACME TLS-SNI method, is that we have at least public commitment by Let's Encrypt and demonstrated evidence of sustained/long-term compliance with publicly disclosing all issued certificates ( as noted in https://letsencrypt.org/certificates/ ). While I realize you've offered to do so, I can find no evidence of GlobalSign doing so by default, and so this further adds to the risk calculus of a commitment to do something not yet practice and thus not yet consistently, reliably delivered on.

We currently include SCTs in all certificates we issue with the possible exception of some Enterprise customers that prefer to keep their OV certificates private (at least for now). This has been configured since mid-November for all GlobalSign SSL products.

There is not, in our view, reason to accept this significantly greater (holistically considered) risk.

We're open to understanding whether GlobalSign has additional proposals how to mitigate this risk, given the set of concerns expressed - technical measures and policy measures. These may provide a path to allowing such issuance in the future, but we don't think that, given the holistic risk assessment, it's appropriate to allow it to immediately resume. We are keen to find solutions that work, as we understand that these can enable powerful new use cases, but we want to balance that with the risk posed.

I would encourage GlobalSign to consult Sections 3.2.2.4 and 3.2.2.5 to see if there are any other alternative methods to validating that might represent an appropriate balance. Given that these are posed as automated certificates, there seem that other methods may be suitable for the issuance of Domain Validated certificates - or, with care, Organizational Validated. If it truly is that none of these other methods (outside 3.2.2.4.9 and .10, and understandably the in-discussion-for-removal .1 and .5), are there steps that can be taken that provide concrete technical mitigations (ideally, through an opt-in technical signal by the site operator) that can reduce or eliminate this risk? Are there steps that can be taken with policy to similarly address the concerns?

There are some use cases where the provider does not control DNS or web site content, so moving to another method is not automated. Certainly with the issues with methods 9 and 10 for shared IP address providers, we hope an alternate approach is identified and rolled out as a new BR Domain Validation method.

[Ryan Sleevi]

On Tue, Jan 16, 2018 at 3:31 PM, Doug Beattie <doug.b...@globalsign.com>
wrote:

> Ryan,
>
>
>
> Here is some more information to continue the discussion.
>
> - We will continue to post all certificates to CT logs so
> issuance can be monitored.
>
> - We will reduce validity period of OneClick certificates to 6
> months.
>
> - We will work with the hosting providers (on a case by case
> basis) to implement processes and procedures which prevent the uploading
> and use of test certificates on user controlled shared IP addresses
> (similar to how LE worked with their larger customers to blocking
> acme.invalid from being used)
>
>
>

Hi Doug,

For the reasons explained below, this set of proposed mitigations aren't
equivalent, and unfortunately don't meet the necessary level. Hopefully, by
pointing to past discussion with additional details, it may inspire
additional ideas to meet that mitigation. Certainly, on the validity
period, there's not much opportunity to budge here, given our risk
assessment to the ecosystem and our users.


> More below.

>
>
>
> *From:* Ryan Sleevi [mailto:ry...@sleevi.com]

> *Sent:* Monday, January 15, 2018 4:56 PM

> *To:* Doug Beattie <doug.b...@globalsign.com>

> *Cc:* ry...@sleevi.com; mozilla-dev-s...@lists.mozilla.org;

> Gervase Markham <ge...@mozilla.org>; Wayne Thayer <wth...@mozilla.com>

> *Subject:* Re: Possible Issue with Domain Validation Method 9 in a shared

I'm glad to here - but I hope you can see this is still over twice the
upper-bound for the acceptable safety margin of 90 days. On this point, we
really don't think there's room to be flexible here. 90 days represents the
tolerable maximum, assuming a perfect set of mitigations.

> - There is no step organizations can take to pre-emptively mitigate the
> risk of GlobalSign adding to the whitelist (compared to, say, blocking
> .invalid)
>
> Actually, there is and I apologize for not making this more clear before.
> We have site operators that manage the issuance of certificates for their
> users. End users have no access to the issuance process, in uploading test
> certificates to their sites, or any involvement in the issuance process as
> this is automated by the site operators. Given this approach is verified
> with the provider, we would propose whitelisting the account.
>

I'm not sure that really addresses the concern. There's nothing a site
provider who is a GlobalSign customer can do to prevent GlobalSign from
making an improper whitelisting mistake here - no technical ability to
thwart users from using the method if said users can convince GlobalSign to
add to the whitelist.

Put more generically - this method is entirely reliant on the maintenance
of that whitelist as a mitigation, and that entirely rests upon the CA.
Much for the same reason that 3.2.2.4.11 was unacceptable, a method of
validation which cannot be mitigated against is problematic, both in the
case of pre-existing customers and new customers.

>
>
> - There is no ability for site operators to detect such situations. A
> consideration, not listed within the full set when discussing Let's Encrypt
> and the ACME TLS-SNI method, is that we have at least public commitment by
> Let's Encrypt and demonstrated evidence of sustained/long-term compliance
> with publicly disclosing all issued certificates ( as noted in
> https://letsencrypt.org/certificates/ ). While I realize you've offered
> to do so, I can find no evidence of GlobalSign doing so by default, and so
> this further adds to the risk calculus of a commitment to do something not
> yet practice and thus not yet consistently, reliably delivered on.
>
> We currently include SCTs in all certificates we issue with the possible
> exception of some Enterprise customers that prefer to keep their OV
> certificates private (at least for now). This has been configured since
> mid-November for all GlobalSign SSL products.
>

Thanks for clarifying! I had missed
https://support.globalsign.com/customer/en/portal/articles/2534583-rolling-out-certificate-transparency-to-domain-validated-ssl
was completed

> I would encourage GlobalSign to consult Sections 3.2.2.4 and 3.2.2.5 to
> see if there are any other alternative methods to validating that might
> represent an appropriate balance. Given that these are posed as automated
> certificates, there seem that other methods may be suitable for the
> issuance of Domain Validated certificates - or, with care, Organizational
> Validated. If it truly is that none of these other methods (outside
> 3.2.2.4.9 and .10, and understandably the in-discussion-for-removal .1 and
> .5), are there steps that can be taken that provide concrete technical
> mitigations (ideally, through an opt-in technical signal by the site
> operator) that can reduce or eliminate this risk? Are there steps that can
> be taken with policy to similarly address the concerns?
>
> There are some use cases where the provider does not control DNS or web
> site content, so moving to another method is not automated. Certainly with
> the issues with methods 9 and 10 for shared IP address providers, we hope
> an alternate approach is identified and rolled out as a new BR Domain
> Validation method.
>

Did you have a chance to review
https://groups.google.com/d/msg/mozilla.dev.security.policy/PiOiGCyuxCU/ISvCL4GJAQAJ
?