Investigating validations & issuances - The high value IP space BGP Hijacks on 2017-12-12

[Matthew Hardeman]

Has anyone started looking into CA issuances -- or even more importantly -- CA domain validations performed successfully and yet without issuing a certificate (say, wanting to cache the validation) for the brief periods in which much of the internet saw alternative target destinations for a great deal of high value organization IP space?

For those CAs with workflows which allow for expressly requesting a domain validation but not necessarily requiring that it be immediately utilized (say, for example LetsEncrypt or another CA running ACME protocol or similar) it might be of interest to review the validations performed successfully during those time windows.

Additionally, it may be of value for various CAs to check their issuances upon domain validation for those periods.

You can find the time periods and details about some of the IP space hijacked at bgpmon.net

[Tom Ritter]

This is an extremely good point. I wonder:

1. If Mozilla should ask/require CAs to perform this check.
2. If Mozilla should ask/require CAs to invest in the capability to
make this check for future requests in the future (where we would
require responses within a certain time period.)

-tom

On 14 December 2017 at 22:16, Matthew Hardeman via dev-security-policy

> _______________________________________________
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy

[Matthew Hardeman]

(reposting as I originally accidentally sent to Tom Ritter only)

Both are great questions.

My short answers on those are

#1 - yes, but as a courtesy with the implication being that those who don't
take the time or trouble might catch a sideways glare if a certificate
issuance is later discovered arising from a hijacked validation during that
period ultimately is found by third parties.

#2 - This is more complicated. Short version: it's not impossible but
difficult and I think it might be better to focus the labor on mechanisms
which would reduce the original risk more directly:

BGP hijacks are not exactly rare. If CA processes would involve lot of
manual work then that may require more intercession than either side would
want.

The specific type of BGP hijack which occurred on the 12th is rare. As
mentioned, bgpmon.net covers some of whose space was hijacked. It's a
who's who of a nature hard to codify -- but we'd all, on review, pretty
much agree -- of IP blocks of prominent and/or important companies,
organizations, and infrastructure. There can be no reasonable conclusion
upon review that the list of IP space briefly hijacked twice for several
minutes was anything other than a human engineered list.

If building infrastructure to build for those checks later, a lot of work
has to be defined. Ultimately you need to collect the set of IP networks
for which your validation infrastructure's view of the world would have
been potentially altered. That alone is a non-trivial but possible problem
to solve.

Then you have to analyze every possible point of your automated domain
validations for risk surfaces exposed by those IP address changes. Did you
rely upon recipient receiving an email to quote a random value? If so, the
IPs of not only the authoritative DNS but also the full set of the various
MX hosts' various IPs become in-scope for analysis of impact.

As automated domain validation evolves you would have to update this
infrastructure every time, re-assessing the full set of mappings of "this
IP address is no longer where it belongs" to all the ways that that IP may
have been a part of the path of reliances upon which the validation was
formed.

I believe it's theoretically possible to achieve a framework that largely
to fully automates posthumous checking of all of this.

Having said that, I also believe that a not dissimilar amount of effort
could be built which would not hold perfect but would harden against the
attack in the first place.

Some of those mechanisms have costs, financial and otherwise.

For example, imagine a domain validation check scheme which requires
constancy over a series of randomly checked times across a (computer time
wise) lengthy period.

Still fully able to be automated, and becomes immaterial time delay at
renewal resistance.

Imagine a scheme where initiating validation versus certificate request to
delivery time cycle are separate matters.

Ex: (Validation to issuance for a new domain label)

1. Turning up a completely new and novel domain, www.r0flrapt0rs.com, I
establish the domain zone in my DNS servers, etc.
2. I request validation via DNS mechanisms. Lets call this time point
initial_validation_request.
3. Within seconds or even less, the first validation requests are made.
Time point initial_validation_request+10_seconds.
4. Over the course of the next several days (say, from the prior time
point to initial_validation_request+36_hours), a number of randomly spaced
revalidations are performed. A reasonable number of immediate retries are
performed to allow grace for intermittent packet loss, system availability,
etc. Ultimately, any final failure of any one of the randomly times
validation process instances would yield an overall failure to validate.
Let's call the time point at the end of this period
validation_issuance_embargo_end.
5. Upon validation_issuance_embargo_end, the client polls for the
certificate which would now be available to the authenticated party.

Ex: (Renewal of existing, non-expired validation)

1. Request new certificate on strength of prior validation. (Clearly this
is only useful if the requestor is cryptographically authenticated to be
the same requestor, as is accomplished in ACME or similar schemes).
2. Certificate issues directly on basis of prior validation.

At any time, an automated client could keep fresh validations available,
ideally with overlapping or renewable validation such that the multi-day
validation process can revalidate the domain label before previous
validation expires. In that case, the renewal for a label with an expired
validation need not be considered. In fact, if it ever did occur, you're
back to the first process - issuance to a new domain label.

A scheme similar to the rough sketch laid out above would levy a new very
significant and hard to achieve burden upon those who seek to secure a
domain validation by way of network manipulation.

Specifically, brief hijacks of even prominent IP space are really easy to
pull off. Sustained hijacks -- beyond (for example) 12 hours -- become
really unlikely to survive. The other networks and network operators will
fence and mitigate -- generally by hand. (In most network operators' shops
that's a manual process because every time they try to automate it the
results are even worse than briefly accepting the hijacks. That's a whole
different discussion.)

It merits acknowledgement that a scheme such as I have illuminated here
relies upon the notion that a new and novel requestor for a given domain
label will inevitably be denied access to a certificate for days, versus a
process today that allows for such issuances within seconds. On the other
hand, this process still provides for 100% automation and would, broadly
speaking, allow for a given domain label at interest to be periodically
refreshed as to validation by that automation such that for subsequent
issuances there would at all times be an already satisfied validation.
(Subsequent issuances to same subscriber for any cause _can_ be automated
such as to allow for immediate delivery.)

A potential upside of the embargo period imposed on the novel domain label
by new requestor is that the intervening period from initial request of
validation to end of the embargo period allows for CAs to do all kinds of
data mining and cross-checking whether or not automatic prior to that first
issuance of a cert with a given domain label to a first time requestor of
said cert with said label.

Thanks,

Matt Hardeman

[CCSFN Postmaster]

The Microsoft Volume Licensing Service Center (VLSC) is definitely affected, at
least from my recent experience - i've been struggling with their service for
the past week because the email address validations from Microsoft VLSC seem to
be intercepted/blocked somewhere - i'm having difficulties validating the
postmaster address as business address for VLSC access because the email from
their servers is simply not even delivered to me - from my mail server's point
of view it looks like a classic hijack - my mail server doesn't see even the
slightest attempt at a TCP/25 IPv4 connection from their mail servers.

If anyone from Microsoft is following this thread, please look into support issue
"VLS Ref: C 6495613 / VLSC access" - most of it is unrelated to the BGP routing
issue, but the email address validation part might be related.

~~~~
Adrian R.