Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Possible DigiCert in-addr.arpa Mis-issuance

7,237 views
Skip to first unread message

Cynthia Revström

unread,
Feb 26, 2019, 2:02:34 PM2/26/19
to dev-secur...@lists.mozilla.org, b...@benjojo.co.uk
Hello dev.security.policy


Apologies if I have made any mistakes in how I post, this is my first
time posting here. Anyway:


I have managed to issue a certificate with a FQDN in the SAN that I do
not have control of via Digicert.


The precert is here: https://crt.sh/?id=1231411316

SHA256: 651B68C520492A44A5E99A1D6C99099573E8B53DEDBC69166F60685863B390D1


I have notified Digicert who responded back with a generic response
followed by the certificate being revoked through OCSP. However I
believe that this should be wider investigated, since this cert was
issued by me adding 69.168.110.79.in-addr.arpa to my SAN, a DNS area
that I do control though reverse DNS.


When I verified 5.168.110.79.in-addr.arpa (same subdomain), I noticed
that the whole of in-addr.arpa became validated on my account, instead
of just my small section of it (168.110.79.in-addr.arpa at best).


To test if digicert had just in fact mis-validated a FQDN, I tested with
the reverse DNS address of 192.168.1.1, and it worked and Digicert
issued me a certificate with 1.1.168.192.in-addr.arpa on it.


Is there anything else dev.security.policy needs to do with this? This
seems like a clear case of mis issuance. It's also not clear if
in-addr.arpa should even be issuable.


I would like to take a moment to thank Ben Cartwright-Cox and igloo22225
in pointing out this violation.


Regards

Cynthia Revström

Jeremy Rowley

unread,
Feb 26, 2019, 2:05:06 PM2/26/19
to dev-secur...@lists.mozilla.org, Cynthia Revström, b...@benjojo.co.uk
Thanks Cynthia. We are investigating and will report back shortly.
________________________________
From: dev-security-policy <dev-security-...@lists.mozilla.org> on behalf of Cynthia Revström via dev-security-policy <dev-secur...@lists.mozilla.org>
Sent: Tuesday, February 26, 2019 12:02:20 PM
To: dev-secur...@lists.mozilla.org
Cc: b...@benjojo.co.uk
Subject: Possible DigiCert in-addr.arpa Mis-issuance
_______________________________________________
dev-security-policy mailing list
dev-secur...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Matthew Hardeman

unread,
Feb 26, 2019, 6:11:11 PM2/26/19
to Jeremy Rowley, dev-secur...@lists.mozilla.org, Cynthia Revström, b...@benjojo.co.uk
Is it even proper to have a SAN dnsName in in-addr.arpa ever?

While in-addr.arpa IS a real DNS heirarchy under the .arpa TLD, it rarely
has anything other than PTR and NS records defined.

Here this was clearly achieved by creating a CNAME record for
69.168.110.79.in-addr.arpa pointed to cynthia.re.

I've never seen any software or documentation anywhere attempting to
utilize a reverse-IP formatted in-addr.arpa address as though it were a
normal host name for resolution. I wonder whether this isn't a case that
should just be treated as an invalid domain for purposes of SAN dnsName
(like .local).

Cynthia Revström

unread,
Feb 26, 2019, 6:17:33 PM2/26/19
to Matthew Hardeman, dev-secur...@lists.mozilla.org, b...@benjojo.co.uk
I am not so sure that is proper to have .arpa domains in the SANs.

But I think the larger issue I think is that this might allow for non
in-addr.arpa domains to be used as well.

It was just that I just tried to get a cert for my domain for a test to
see if that would be issued.

And upon verifying the domain via email, I saw that on the page linked
in the email it had an option along the lines of "Authorize in-addr.arpa
for all orders on account #123456" (not that exact text but the same
meaning).

Now I am not sure if this would work, but I suspect if for example I got
DNS control over cynthia.site.google.com, I could get a cert for
google.com if this is a systemic issue and not just a one off.

- Cynthia

On 2019-02-27 00:10, Matthew Hardeman wrote:
> Is it even proper to have a SAN dnsName in in-addr.arpa ever?
>
> While in-addr.arpa IS a real DNS heirarchy under the .arpa TLD, it
> rarely has anything other than PTR and NS records defined.
>
> Here this was clearly achieved by creating a CNAME record for
> 69.168.110.79.in-addr.arpa pointed to cynthia.re <http://cynthia.re>.
>
> I've never seen any software or documentation anywhere attempting to
> utilize a reverse-IP formatted in-addr.arpa address as though it were
> a normal host name for resolution.  I wonder whether this isn't a case
> that should just be treated as an invalid domain for purposes of SAN
> dnsName (like .local).
>
> On Tue, Feb 26, 2019 at 1:05 PM Jeremy Rowley via dev-security-policy
> <dev-secur...@lists.mozilla.org
> <mailto:dev-secur...@lists.mozilla.org>> wrote:
>
> Thanks Cynthia. We are investigating and will report back shortly.
> ________________________________
> From: dev-security-policy
> <dev-security-...@lists.mozilla.org
> <mailto:dev-security-...@lists.mozilla.org>> on behalf
> of Cynthia Revström via dev-security-policy
> <dev-secur...@lists.mozilla.org
> <mailto:dev-secur...@lists.mozilla.org>>
> Sent: Tuesday, February 26, 2019 12:02:20 PM
> To: dev-secur...@lists.mozilla.org
> <mailto:dev-secur...@lists.mozilla.org>
> Cc: b...@benjojo.co.uk <mailto:b...@benjojo.co.uk>
> <mailto:dev-secur...@lists.mozilla.org>
> https://lists.mozilla.org/listinfo/dev-security-policy
> _______________________________________________
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org
> <mailto:dev-secur...@lists.mozilla.org>
> https://lists.mozilla.org/listinfo/dev-security-policy
>

Jeremy Rowley

unread,
Feb 26, 2019, 7:55:44 PM2/26/19
to mozilla-dev-security-policy
>From our side, a validation agent weirdly scoped the domain, saying that the domain was approved using an email to ad...@in-addr.arpa. However, the email clearly went to ad...@5.168.110.79.in-addr.arpa. We're looking to see 1) how did the validation staff override the domain approval scope and 2) did anyone in validation ever do this before. Once we complete that search, we'll be able to file a bug report and give you more information on what exactly went wrong.

.arpa is in IANA's root zone per https://www.iana.org/domains/root/db.

Jeremy
https://lists.mozilla.org/listinfo/dev-security-policy

Jakob Bohm

unread,
Feb 27, 2019, 9:54:35 AM2/27/19
to mozilla-dev-s...@lists.mozilla.org
On 27/02/2019 00:10, Matthew Hardeman wrote:
> Is it even proper to have a SAN dnsName in in-addr.arpa ever?
>
> While in-addr.arpa IS a real DNS heirarchy under the .arpa TLD, it rarely
> has anything other than PTR and NS records defined.
>

While there is no current use, and the test below was obviously
somewhat contrived (and seems to have triggered a different issue),
one cannot rule out the possibility of a need appearing in the
future.

One hypothetical use would be to secure BGP traffic, as certificates
with IpAddress SANs are less commonly supported.
Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

Nick Lamb

unread,
Feb 27, 2019, 10:04:49 AM2/27/19
to dev-secur...@lists.mozilla.org, Matthew Hardeman
On Tue, 26 Feb 2019 17:10:49 -0600
Matthew Hardeman via dev-security-policy
<dev-secur...@lists.mozilla.org> wrote:

> Is it even proper to have a SAN dnsName in in-addr.arpa ever?

It does feel as though ARPA should consider adding a CAA record to
in-addr.arpa and similar hierarchies that don't want certificates,
denying all CAs, as a defence in depth measure.

Nick.

Ryan Sleevi

unread,
Feb 27, 2019, 10:34:11 AM2/27/19
to Nick Lamb, Matthew Hardeman, dev-secur...@lists.mozilla.org
Alternatively, and perhaps more comprehensively, it may be better to ensure
that those Special Use Domains that are either delegated to or reserved by
IANA or the IESG can only have certificates issued by those respective
organizations.

These are enumerated prosaically at
https://www.iana.org/domains/reserved for those reserved by policy, and
https://www.iana.org/assignments/special-use-domain-names/special-use-domain-names.xhtml
is a registry of those reserved by relevant standards.

This approach is already taken with regards to IP addresses, and more
comprehensively avoids ambiguity. It has the benefit of defaulting secure -
by not requiring a domain holder (including IANA) to somehow take special
action to protect existing practice. Should concrete use cases present
themselves - of which BGP is not one (see BGPsec for more details) - then
those can be relaxed on a case by case basis. The .onion Domain is an
example of a pre-existing relaxation.

This would still permit .arpa certificates - specific language would be
needed (and should be done) to either prohibit or apply the same
consistency to as IP certificates - but would otherwise close a class of
“obvious” errors. The suggestion was intentionally not a blanket ban, as
IANA/ICANN does and has obtained legitimate certificates for the example
domains in the past.

Tim Hollebeek

unread,
Feb 27, 2019, 10:43:15 AM2/27/19
to Jakob Bohm, mozilla-dev-s...@lists.mozilla.org

> On 27/02/2019 00:10, Matthew Hardeman wrote:
> > Is it even proper to have a SAN dnsName in in-addr.arpa ever?
> >
> > While in-addr.arpa IS a real DNS heirarchy under the .arpa TLD, it
> > rarely has anything other than PTR and NS records defined.
> >
>
> While there is no current use, and the test below was obviously somewhat
> contrived (and seems to have triggered a different issue), one cannot rule
> out
> the possibility of a need appearing in the future.

At least the last time this came up a few years ago, there were actually a
significant number of webservers running under in-addr.arpa, with Comodo and
LE certificates (as well as a handful of others). I believe Corey posted a
list.

Exactly what they were doing there is an open question, and when I asked, no
one responded. I'm still very curious as to why some people seem to actually
be running servers there, or if it's just a side-effect of misconfigured
CNAMEs causing them to appear to be there, or similar.

-Tim

Cynthia Revström

unread,
Feb 27, 2019, 10:45:24 AM2/27/19
to dev-secur...@lists.mozilla.org
Okay that seems like an issue as to me that says that this could have
happened to any domain and not just in-addr.arpa?

- Cynthia

On 2019-02-27 01:55, Jeremy Rowley via dev-security-policy wrote:
> From our side, a validation agent weirdly scoped the domain, saying that the domain was approved using an email to ad...@in-addr.arpa. However, the email clearly went to ad...@5.168.110.79.in-addr.arpa. We're looking to see 1) how did the validation staff override the domain approval scope and 2) did anyone in validation ever do this before. Once we complete that search, we'll be able to file a bug report and give you more information on what exactly went wrong.
>
> .arpa is in IANA's root zone per https://www.iana.org/domains/root/db.
>
> Jeremy
>
> -----Original Message-----
> From: dev-security-policy <dev-security-...@lists.mozilla.org> On Behalf Of Cynthia Revström via dev-security-policy
> Sent: Tuesday, February 26, 2019 4:17 PM
> To: Matthew Hardeman <mhar...@gmail.com>
> Cc: dev-secur...@lists.mozilla.org; b...@benjojo.co.uk
> Subject: Re: Possible DigiCert in-addr.arpa Mis-issuance
>
> I am not so sure that is proper to have .arpa domains in the SANs.
>
> But I think the larger issue I think is that this might allow for non in-addr.arpa domains to be used as well.
>
> It was just that I just tried to get a cert for my domain for a test to see if that would be issued.
>
> And upon verifying the domain via email, I saw that on the page linked in the email it had an option along the lines of "Authorize in-addr.arpa for all orders on account #123456" (not that exact text but the same meaning).
>
> Now I am not sure if this would work, but I suspect if for example I got DNS control over cynthia.site.google.com, I could get a cert for google.com if this is a systemic issue and not just a one off.
>
> - Cynthia
>
> On 2019-02-27 00:10, Matthew Hardeman wrote:
>> Is it even proper to have a SAN dnsName in in-addr.arpa ever?
>>
>> While in-addr.arpa IS a real DNS heirarchy under the .arpa TLD, it
>> rarely has anything other than PTR and NS records defined.
>>
>> Here this was clearly achieved by creating a CNAME record for
>> 69.168.110.79.in-addr.arpa pointed to cynthia.re <http://cynthia.re>.
>>
>> I've never seen any software or documentation anywhere attempting to
>> utilize a reverse-IP formatted in-addr.arpa address as though it were
>> a normal host name for resolution. I wonder whether this isn't a case
>> that should just be treated as an invalid domain for purposes of SAN
>> dnsName (like .local).
>>
>> On Tue, Feb 26, 2019 at 1:05 PM Jeremy Rowley via dev-security-policy
>> <dev-secur...@lists.mozilla.org
>> <mailto:dev-secur...@lists.mozilla.org>> wrote:
>>
>> Thanks Cynthia. We are investigating and will report back shortly.
>> ________________________________
>> From: dev-security-policy
>> <dev-security-...@lists.mozilla.org
>> <mailto:dev-security-...@lists.mozilla.org>> on behalf
>> of Cynthia Revström via dev-security-policy
>> <dev-secur...@lists.mozilla.org
>> <mailto:dev-secur...@lists.mozilla.org>>
>> Sent: Tuesday, February 26, 2019 12:02:20 PM
>> To: dev-secur...@lists.mozilla.org
>> <mailto:dev-secur...@lists.mozilla.org>
>> Cc: b...@benjojo.co.uk <mailto:b...@benjojo.co.uk>

Jeremy Rowley

unread,
Feb 27, 2019, 10:48:44 AM2/27/19
to mozilla-dev-security-policy
Well yes, assuming the validation person overrides the safeguards and randomly changes the scope of the domain validation to something other than what the system specifies. We're still looking into how this happened and if the validation agent did this in any other case.

Matthew Hardeman

unread,
Feb 27, 2019, 11:53:55 AM2/27/19
to Nick Lamb, dev-secur...@lists.mozilla.org
On Wed, Feb 27, 2019 at 9:04 AM Nick Lamb <n...@tlrmx.org> wrote:

>
> It does feel as though ARPA should consider adding a CAA record to
> in-addr.arpa and similar hierarchies that don't want certificates,
> denying all CAs, as a defence in depth measure.
>

Unless I significantly misunderstand CAA, this mechanism would not
necessarily be effective.

The normal mode of operation is that at the in-addr.arpa zone delegates sub
zones, for example 199.in-addr.arpa to the relevant RIR via an NS record.
Further, the relevant RIR would delegate sub zones of that zone via NS
records to an IP space holder, for example 88.99.199.in-addr.arpa would
have NS records configured on the RIR name servers which would refer to the
authoritative DNS servers serving the IP space holder for 199.99.88.0/24.

As such, superseding CAA records which would allow issuance could be added
back into those hierarchies by the DNS admins of those zones.

Corey Bonnell

unread,
Feb 27, 2019, 10:05:43 PM2/27/19
to mozilla-dev-s...@lists.mozilla.org
Hi Tim,
As you said, I vaguely recall this coming up in some discussion (perhaps in the CAB Forum Validation Subcommittee?) but nothing was concluded. I believe the list was merely a crt.sh query of all unexpired certificates with a dNSName ending in "in-addr.arpa": https://crt.sh/?dNSName=%25.in-addr.arpa&exclude=expired

The query results are definitely worth a look as there are some unexpected findings, such as wildcards (such as "*.0.195.206.in-addr.arpa") and host nodes (such as "www.175.232.77.in-addr.arpa", etc.) under in-addr.arpa. Several of the domain names starting with "www" actually appear to resolve to an IP address with a web server running. Definitely an interesting (ab)use of the reverse zones.

Thanks,
Corey

Jeremy Rowley

unread,
Feb 28, 2019, 12:52:32 AM2/28/19
to mozilla-dev-security-policy
Hi Cynthia,

We've figured out what happened with your certificate but are still looking at whether other certificates were issued using the same process. I don't have enough information to file an incident report yet, but I wanted to give you the update I promised earlier.

Basically, here's what happened:
1. A validation agent took an email address provided during a chat session with you and set that email as a WHOIS admin email address. This email qualified as a constructed email (admin@domain)
2. The system marked the WHOIS as unavailable for automated parsing (generally, this happens if we are being throttled or the WHOIS info is behind a CAPTCHA), which allows a validation agent to manually upload a WHOIS document
3. The WHOIS document uploaded was a screen capture of WHOIS information that did not match the domain being requested but was close enough that the mis-match went unnoticed.
4. The validation agent specified the approval scope as id-addr.arpa which is normal for a domain approved by the admin listed in WHOIS. As a constructed email, the approval scope should have been limited to the scope set by the constructed address.
5. The validation agent specified that the email address listed in WHOIS matched the email address provided by you (admin@domain) and sent the email to authorize the legit cert
6 . When the validation completed successfully, the validation authorized your account for all certificates with the in-addr.arpa domain. This was because the scope was improperly set based on the validation agent's improper specification of the source of the email address.
7. During the review, no one noticed that the WHOIS document did not match the verification email nor did anyone notice that the email used for verification was actually a constructed email instead of the WHOIS admin email.
8. The mis-issued certificate essentially "piggy-backed" on the previous verification because of the erroneous approval scope set by the validation staff.

Make sense?

We shut down manual WHOIS verification while we figure out how to improve the process and eliminate validation mistakes like this. We'll post another update after we figure out how to improve the process and after the data review is complete.

Jeremy

-----Original Message-----
From: dev-security-policy <dev-security-...@lists.mozilla.org> On Behalf Of Cynthia Revström via dev-security-policy
Sent: Tuesday, February 26, 2019 4:17 PM
To: Matthew Hardeman <mhar...@gmail.com>
Cc: dev-secur...@lists.mozilla.org; b...@benjojo.co.uk
Subject: Re: Possible DigiCert in-addr.arpa Mis-issuance

I am not so sure that is proper to have .arpa domains in the SANs.

But I think the larger issue I think is that this might allow for non in-addr.arpa domains to be used as well.

It was just that I just tried to get a cert for my domain for a test to see if that would be issued.

And upon verifying the domain via email, I saw that on the page linked in the email it had an option along the lines of "Authorize in-addr.arpa for all orders on account #123456" (not that exact text but the same meaning).

Now I am not sure if this would work, but I suspect if for example I got DNS control over cynthia.site.google.com, I could get a cert for google.com if this is a systemic issue and not just a one off.

- Cynthia

On 2019-02-27 00:10, Matthew Hardeman wrote:
> Is it even proper to have a SAN dnsName in in-addr.arpa ever?
>
> While in-addr.arpa IS a real DNS heirarchy under the .arpa TLD, it
> rarely has anything other than PTR and NS records defined.
>

Nick Lamb

unread,
Feb 28, 2019, 6:21:38 AM2/28/19
to dev-secur...@lists.mozilla.org, Jeremy Rowley
On Thu, 28 Feb 2019 05:52:14 +0000
Jeremy Rowley via dev-security-policy
<dev-secur...@lists.mozilla.org> wrote:

Hi Jeremy,

> 4. The validation agent specified the approval scope as id-addr.arpa

I assume this is a typo by you not the agent, for in-addr.arpa ?

Meanwhile, and without prejudice to the report itself once made:

> 2. The system marked the WHOIS as unavailable for automated parsing
> (generally, this happens if we are being throttled or the WHOIS info
> is behind a CAPTCHA), which allows a validation agent to manually
> upload a WHOIS document

This is a potentially large hole in issuance checks based on WHOIS.

Operationally the approach taken ("We can't get it to work, press on")
makes sense, but if we take a step back there's obvious potential for
nasty security surprises like this one.

There has to be something we can do here, I will spitball something in
a next paragraph just to have something to start with, but to me if it
turns out we can't improve on basically "sometimes it doesn't work so
we just shrug and move on" we need to start thinking about deprecating
this approach altogether. Not just for DigiCert, for everybody.

- Spitball: What if the CA/B went to the registries, at least the big
ones, and said we need this, strictly for this defined purpose, give
us either reliable WHOIS, or RDAP, or direct database access or
_something_ we can automate to do these checks ? The nature of CA/B
may mean that it's not appropriate to negotiate paying for this
(pressuring suppliers to all agree to offer members the same rates is
just as much a problem as all agreeing what you'll charge customers)
but it should be able to co-ordinate making sure members get access,
and that it isn't opened up to dubious data resellers that the
registries don't want rifling through their database.

My argument to the registries would be that this is a service for their
customers. Unlike the data resellers, either the registry customer, or
some agent of theirs is asking you to authenticate their registration,
so giving you access makes sense as part of what the registry does for
its customers anyway.

> 7. During the review, no one noticed that the WHOIS document did not
> match the verification email nor did anyone notice that the email
> used for verification was actually a constructed email instead of the
> WHOIS admin email

So, reviews are good, but this review was not very effective. Valuable
to consider in the final report why not and how that can be improved.

Just to be clear though, are you sure "no one noticed" ? It can happen
that in review processes somebody does notice the issue, but they
are persuaded or persuade themselves that it's fine. A British railway
incident occurred when the person transcribing a document effectively
"moved" a railway crossing. Manual reviewers did see it, and so did the
controllers responsible for managing the crossing, but both persuaded
themselves that the movement must be a correction and approved it.

With the crossing now shown in the wrong place, instructions authorising
use of the crossing were no longer protected by the controller's view
of the movement of trains, this resulted in a "near miss" and thanks to
the victim's persistence in demanding it be properly investigated
fortunately accident investigators visited the crossing, found the
mistake and had things corrected before anyone died.

Nick.

Ryan Sleevi

unread,
Feb 28, 2019, 6:43:44 AM2/28/19
to Nick Lamb, MDSP, Jeremy Rowley
Unfortunately, this is not really viable. The CA/Browser Forum maintains
relationships with ICANN, as do individual members. While this, on its
face, seems reasonable, there are practical, business, and legal concerns
that prevent this from being viable. Further, proposals which would require
membership in the CA/Browser Forum should, on their face, be rejected - a
CA should not have to join the Forum in order to be a CA.

I do agree, however, that the use of WHOIS data continues to show
problematic incidents - whether it's with OCR issues or manual entry - and
suspect a more meaningful solution is to move away from this model
entirely. The recently approved methods to the BRs for expressing contact
information via the DNS directly is one such approach. The GDPR issues
surrounding WHOIS and RDAP have already led it to be compelling in its own
right.

Most importantly, you are on the right path of questions, though - which is
we should examine such incidents systemically and look for improvements,
and not convince ourselves that the status quo is the best possible
solution :)

Daniel McCarney

unread,
Feb 28, 2019, 8:22:59 AM2/28/19
to Corey Bonnell, mozilla-dev-s...@lists.mozilla.org
>
> I believe the list was merely a crt.sh query of all unexpired certificates
> with a dNSName ending in "in-addr.arpa":
> https://crt.sh/?dNSName=%25.in-addr.arpa&exclude=expired


Any list for this general issue should also consider unexpired certificates
with a dNSName ending in "ip6.arpa" to cover the IPv6 reverse zone in
addition to the IPv4 one. I noticed there are similar interesting
wildcards/host nodes under the ip6.arpa zone when I was writing a linter[0]
for this.

[0] - https://github.com/zmap/zlint/pull/260

On Wed, Feb 27, 2019 at 10:05 PM Corey Bonnell via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> On Wednesday, February 27, 2019 at 10:43:15 AM UTC-5, Tim Hollebeek wrote:
> Hi Tim,
> As you said, I vaguely recall this coming up in some discussion (perhaps
> in the CAB Forum Validation Subcommittee?) but nothing was concluded. I
> believe the list was merely a crt.sh query of all unexpired certificates
> with a dNSName ending in "in-addr.arpa":
> https://crt.sh/?dNSName=%25.in-addr.arpa&exclude=expired
>
> The query results are definitely worth a look as there are some unexpected
> findings, such as wildcards (such as "*.0.195.206.in-addr.arpa") and host
> nodes (such as "www.175.232.77.in-addr.arpa", etc.) under in-addr.arpa.
> Several of the domain names starting with "www" actually appear to resolve
> to an IP address with a web server running. Definitely an interesting
> (ab)use of the reverse zones.
>
> Thanks,
> Corey

Matthew Hardeman

unread,
Feb 28, 2019, 7:04:53 PM2/28/19
to Ryan Sleevi, Nick Lamb, MDSP, Jeremy Rowley
In addition to the GDPR concerns over WHOIS and RDAP data, reliance upon
these data sources has a crucial differentiation from other domain
validation methods.

Specifically, the WHOIS/RDAP data sources are entirely "off-path" with
respect to how a browser will locate and access a given site. To my way of
thinking, this renders these mechanisms functionally inferior to an
"on-path" mechanism, such as reliances upon demonstrated change control
over an authoritative DNS record or even demonstration content change
control over a website.

Since domain validation is, in theory, about validating that the party to
whom a certificate is to be issued has demonstrated control over the
subject of the desired name(s) or the name space of the desired name(s), it
seems clear that "off-path" validation is less valuable as a security
measure.

Although I'm aware that the BRs bless a number of methods, it's also clear
that methods have been excluded by the Mozilla root program before. Is it
time to consider further winnowing down the accepted methods?

On Thu, Feb 28, 2019 at 5:43 AM Ryan Sleevi via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

Matthew Hardeman

unread,
Feb 28, 2019, 8:28:01 PM2/28/19
to mozilla-dev-s...@lists.mozilla.org
On Wednesday, February 27, 2019 at 8:54:35 AM UTC-6, Jakob Bohm wrote:

> One hypothetical use would be to secure BGP traffic, as certificates
> with IpAddress SANs are less commonly supported.

The networking / interconnection world has already worked out the trust hierarchy for the RPKI scheme. As there are a number of global RIRs who are the authoritative source of ASN and IP space information, they've elected to themselves be the Root CAs involved. Its an interesting infrastructure. You can learn more about it here:

https://www.arin.net/resources/rpki/index.html

Jakob Bohm

unread,
Mar 1, 2019, 7:40:28 AM3/1/19
to mozilla-dev-s...@lists.mozilla.org
On 01/03/2019 01:04, Matthew Hardeman wrote:
> In addition to the GDPR concerns over WHOIS and RDAP data, reliance upon
> these data sources has a crucial differentiation from other domain
> validation methods.
>
> Specifically, the WHOIS/RDAP data sources are entirely "off-path" with
> respect to how a browser will locate and access a given site. To my way of
> thinking, this renders these mechanisms functionally inferior to an
> "on-path" mechanism, such as reliances upon demonstrated change control
> over an authoritative DNS record or even demonstration content change
> control over a website.
>
> Since domain validation is, in theory, about validating that the party to
> whom a certificate is to be issued has demonstrated control over the
> subject of the desired name(s) or the name space of the desired name(s), it
> seems clear that "off-path" validation is less valuable as a security
> measure.
>

And there, you completely misunderstood the purpose. The purpose of
domain validation is to determine if the certificate application is
(directly or indirectly) authorized by the domain registrant.
Technical control (via the various automated methods) is a proxy to
this information, but not as close to the truth as WHOIS validations
(provided either is done securely).

The panic mishandling of GDPR concerns by ICANN (something that
continues in the current process to make a "permanent" solution) has
essentially crippled all useful purposes of the WHOIS/RDAP database.
Including making it near impossible to do domain ownership (as opposed
to mere control) validation impossible except for a few national
TLDs that continue to provide open WHOIS.

I sincerely hope that ICANN cleans up its misunderstandings and
reopens WHOIS, at least for domain owners that request this (it
currently can't be done because registrars are vary of implementing
the temporary specification of how to do that, as a new spec is in
the works).

Therefore WHOIS validation should remain valid, but only if the
actual registry/registrar provides authoritative computer readable
data for the domain in question. (Thus having to do OCR or similar
of a picture of text is not acceptable, and neither are manually
entered entries).

In particular "substitute WHOIS" lookups via manual steps that
don't result in the validation computers directly reading the data
from the registrar/registry server should not be allowed.

There are many ways this can be done technically, ranging from a
straightforward WHOIS lookup from multiple network vantage points
to a process where a special web browser is used to authenticate
through CAPTCHAs etc. until the validation system automatically
captures and parses the "known correct" textual web response
from a known registrar/registry url.

This in turn means that IV, OV and EV certs will often need to use
other means of associating the domain control with the certificate
applicant entity. For example one or more challenge values/pins
can be securely provided to the real world entity validated, then
used in the protocol for validating domain control. But this still
only validates domain control, not legitimacy of domain authority.

Wayne Thayer

unread,
Mar 1, 2019, 12:00:01 PM3/1/19
to Jeremy Rowley, mozilla-dev-security-policy
https://bugzilla.mozilla.org/show_bug.cgi?id=1531817 has been created to
track this issue.

On Wed, Feb 27, 2019 at 10:52 PM Jeremy Rowley via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> Hi Cynthia,
>
> We've figured out what happened with your certificate but are still
> looking at whether other certificates were issued using the same process. I
> don't have enough information to file an incident report yet, but I wanted
> to give you the update I promised earlier.
>
> Basically, here's what happened:
> 1. A validation agent took an email address provided during a chat session
> with you and set that email as a WHOIS admin email address. This email
> qualified as a constructed email (admin@domain)
> 2. The system marked the WHOIS as unavailable for automated parsing
> (generally, this happens if we are being throttled or the WHOIS info is
> behind a CAPTCHA), which allows a validation agent to manually upload a
> WHOIS document
> 3. The WHOIS document uploaded was a screen capture of WHOIS information
> that did not match the domain being requested but was close enough that the
> mis-match went unnoticed.
> 4. The validation agent specified the approval scope as id-addr.arpa which
> is normal for a domain approved by the admin listed in WHOIS. As a
> constructed email, the approval scope should have been limited to the scope
> set by the constructed address.
> 5. The validation agent specified that the email address listed in WHOIS
> matched the email address provided by you (admin@domain) and sent the
> email to authorize the legit cert
> 6 . When the validation completed successfully, the validation authorized
> your account for all certificates with the in-addr.arpa domain. This was
> because the scope was improperly set based on the validation agent's
> improper specification of the source of the email address.
> 7. During the review, no one noticed that the WHOIS document did not match
> the verification email nor did anyone notice that the email used for
> verification was actually a constructed email instead of the WHOIS admin
> On 2019-02-27 00:10, Matthew Hardeman wrote:
> > Is it even proper to have a SAN dnsName in in-addr.arpa ever?
> >
> > While in-addr.arpa IS a real DNS heirarchy under the .arpa TLD, it
> > rarely has anything other than PTR and NS records defined.
> >
> > Here this was clearly achieved by creating a CNAME record for
> > 69.168.110.79.in-addr.arpa pointed to cynthia.re <http://cynthia.re>.
> >
> > I've never seen any software or documentation anywhere attempting to
> > utilize a reverse-IP formatted in-addr.arpa address as though it were
> > a normal host name for resolution. I wonder whether this isn't a case
> > that should just be treated as an invalid domain for purposes of SAN
> > dnsName (like .local).
> >
> > On Tue, Feb 26, 2019 at 1:05 PM Jeremy Rowley via dev-security-policy
> > <dev-secur...@lists.mozilla.org
> > _______________________________________________
> > dev-security-policy mailing list
> > dev-secur...@lists.mozilla.org
> > <mailto:dev-secur...@lists.mozilla.org>
> > https://lists.mozilla.org/listinfo/dev-security-policy
> > _______________________________________________
> > dev-security-policy mailing list
> > dev-secur...@lists.mozilla.org
> > <mailto:dev-secur...@lists.mozilla.org>
> > https://lists.mozilla.org/listinfo/dev-security-policy

Jeremy Rowley

unread,
Mar 1, 2019, 12:07:59 PM3/1/19
to Wayne Thayer, mozilla-dev-security-policy
Thanks Wayne



From: Wayne Thayer <wth...@mozilla.com>
Sent: Friday, March 1, 2019 10:00 AM
To: Jeremy Rowley <jeremy...@digicert.com>
Cc: mozilla-dev-security-policy <mozilla-dev-s...@lists.mozilla.org>
Subject: Re: Possible DigiCert in-addr.arpa Mis-issuance



https://bugzilla.mozilla.org/show_bug.cgi?id=1531817 has been created to track this issue.



Now I am not sure if this would work, but I suspect if for example I got DNS control over cynthia.site.google.com <http://cynthia.site.google.com> , I could get a cert for google.com <http://google.com> if this is a systemic issue and not just a one off.

- Cynthia

On 2019-02-27 00:10, Matthew Hardeman wrote:
> Is it even proper to have a SAN dnsName in in-addr.arpa ever?
>
> While in-addr.arpa IS a real DNS heirarchy under the .arpa TLD, it
> rarely has anything other than PTR and NS records defined.
>
> Here this was clearly achieved by creating a CNAME record for
> 69.168.110.79.in-addr.arpa pointed to cynthia.re <http://cynthia.re> <http://cynthia.re>.
>
> I've never seen any software or documentation anywhere attempting to
> utilize a reverse-IP formatted in-addr.arpa address as though it were
> a normal host name for resolution. I wonder whether this isn't a case
> that should just be treated as an invalid domain for purposes of SAN
> dnsName (like .local).
>
> On Tue, Feb 26, 2019 at 1:05 PM Jeremy Rowley via dev-security-policy
> <dev-secur...@lists.mozilla.org <mailto:dev-secur...@lists.mozilla.org>
> <mailto:dev-secur...@lists.mozilla.org <mailto:dev-secur...@lists.mozilla.org> >> wrote:
>
> Thanks Cynthia. We are investigating and will report back shortly.
> ________________________________
> From: dev-security-policy
> <dev-security-...@lists.mozilla.org <mailto:dev-security-...@lists.mozilla.org>
> <mailto:dev-security-...@lists.mozilla.org <mailto:dev-security-...@lists.mozilla.org> >> on behalf
> of Cynthia Revström via dev-security-policy
> <dev-secur...@lists.mozilla.org <mailto:dev-secur...@lists.mozilla.org>
> <mailto:dev-secur...@lists.mozilla.org <mailto:dev-secur...@lists.mozilla.org> >>
> Sent: Tuesday, February 26, 2019 12:02:20 PM
> To: dev-secur...@lists.mozilla.org <mailto:dev-secur...@lists.mozilla.org>
> <mailto:dev-secur...@lists.mozilla.org <mailto:dev-secur...@lists.mozilla.org> >
> Cc: b...@benjojo.co.uk <mailto:b...@benjojo.co.uk> <mailto:b...@benjojo.co.uk <mailto:b...@benjojo.co.uk> >
> <mailto:dev-secur...@lists.mozilla.org <mailto:dev-secur...@lists.mozilla.org> >
> https://lists.mozilla.org/listinfo/dev-security-policy
> _______________________________________________
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org <mailto:dev-secur...@lists.mozilla.org>
> <mailto:dev-secur...@lists.mozilla.org <mailto:dev-secur...@lists.mozilla.org> >
> https://lists.mozilla.org/listinfo/dev-security-policy
>
_______________________________________________
dev-security-policy mailing list
dev-secur...@lists.mozilla.org <mailto:dev-secur...@lists.mozilla.org>
https://lists.mozilla.org/listinfo/dev-security-policy
_______________________________________________
dev-security-policy mailing list
dev-secur...@lists.mozilla.org <mailto:dev-secur...@lists.mozilla.org>
https://lists.mozilla.org/listinfo/dev-security-policy

George Macon

unread,
Mar 1, 2019, 7:49:11 PM3/1/19
to mozilla-dev-s...@lists.mozilla.org
On 2/28/19 12:52 AM, Jeremy Rowley wrote:
> 4. The validation agent specified the approval scope as id-addr.arpa which is normal for a domain approved by the admin listed in WHOIS. As a constructed email, the approval scope should have been limited to the scope set by the constructed address.

One specific question on this point: Why did the software permit setting
the approval scope to a public suffix (as defined by inclusion on the
public suffix list)? Could validation agent action set the approval
scope to some other two-label public suffix like co.uk?

-George Macon

Cynthia Revström

unread,
Mar 2, 2019, 3:45:52 AM3/2/19
to George Macon, mozilla-dev-s...@lists.mozilla.org
On 2019-03-02 01:49, George Macon via dev-security-policy wrote:

> One specific question on this point: Why did the software permit setting
> the approval scope to a public suffix (as defined by inclusion on the
> public suffix list)? Could validation agent action set the approval
> scope to some other two-label public suffix like co.uk?

I think this is highly unlikely seeing as this was a human error and
unlike in-addr.arpa, people might know about .co.uk.

- Cynthia

Gijs Kruitbosch

unread,
Mar 2, 2019, 6:38:42 PM3/2/19
to mozilla-dev-s...@lists.mozilla.org
But the PSL is very large (by human, not machine, standards) and most
humans will not be familiar with most/all of the entries on the list.
Note for instance that (most/all of?) AWS is represented in one way or
another, as are other hosting services that are much less well-known. It
seems worth checking the PSL automatically, and it's curious that such
checks were not present or did not prevent/discourage the agent from
acting as they did.

(Note that I'm not overly familiar with the BR and various other
guidelines, and under what circumstances issuance to entries in the PSL
is/isn't permitted, but intuitively it seems like a red flag once we're
talking about manual (rather than automatically verified) issuance.)

~ Gijs

Jeremy Rowley

unread,
Mar 4, 2019, 2:23:59 PM3/4/19
to mozilla-dev-s...@lists.mozilla.org
Technically, the same issue could exist on the system. However, co.uk is
actually blocked as a valid approval address by our system. In-addr.arpa was
not blocked.

Here's a status update:
1) We identified 3000 certificates where the scope was changed by validation
staff based on a WHOIS document.
2) Of the 3,000, the only certificate we found where the scope was not set
to be the scope of the WHOIS document was the one reported by Cynthia.

The next step is to look at why we didn't block in-addr.arpa as an eligible
scope. We generally pull from the PSL, so I need to find out why
in-addr.arpa was not blocked.

Thanks!
Jeremy

-----Original Message-----
From: dev-security-policy <dev-security-...@lists.mozilla.org> On
Behalf Of Cynthia Revström via dev-security-policy
Sent: Saturday, March 2, 2019 1:46 AM
To: George Macon <george...@gmail.com>
Cc: mozilla-dev-s...@lists.mozilla.org
Subject: Re: Possible DigiCert in-addr.arpa Mis-issuance

On 2019-03-02 01:49, George Macon via dev-security-policy wrote:

> One specific question on this point: Why did the software permit
> setting the approval scope to a public suffix (as defined by inclusion
> on the public suffix list)? Could validation agent action set the
> approval scope to some other two-label public suffix like co.uk?

I think this is highly unlikely seeing as this was a human error and unlike
in-addr.arpa, people might know about .co.uk.

- Cynthia

Cynthia Revström

unread,
Mar 4, 2019, 2:25:47 PM3/4/19
to dev-secur...@lists.mozilla.org
On 2019-03-04 20:23, Jeremy Rowley via dev-security-policy wrote:

> 2) Of the 3,000, the only certificate we found where the scope was not set
> to be the scope of the WHOIS document was the one reported by Cynthia.

That is good to hear :)

- Cynthia

0 new messages