Intent To Deprecate And Remove Support For Invalid DNS Names

[Chris Palmer]

Primary eng (and PM) emails

pal...@chromium.org

Summary

This I2D&R is to restrict the set of valid hostnames that the network name resolver will attempt to resolve. We will restrict names to those that follow the “preferred name syntax” established for hostnames (A/AAAA) — in particular, those containing only letters, digits, and hyphens (the “LDH rule”). In addition to supporting these characters, for web-compatibility reasons, we will also accept underscores (‘_’).

This is an attempt to resolve known interoperability issues with spec non-compliance, but also security issues that arise from the current, liberal approach.

Because internationalized domain names (IDNs) are encoded as Punycode before being sent over the network, and because Punycode follows the LDH rule, this will not change support for IDNs in Chrome.

Motivation

DNS is a complex protocol. On the wire, RFC 1034 (and related RFCs) establish that a valid DNS record can include any character (“8-bit clean”), other than the NUL byte (‘\0’, used to separate labels). However, hostnames in particular (A/AAAA) are expected to observe the preferred name syntax, which is letters, digits, and hyphens. While other record types (such as SRV) use additional characters, like underscores, these are not part of the legal syntax for hostnames. Further, names using the preferred name syntax are expected to match in a case-insensitive manner. The confusion is broadly covered in RFC 7719 in its discussion of terminology.

Many DNS libraries do not validate that the input follows the preferred name syntax, and thus send whatever the caller requests over the wire. This can create security vulnerabilities, both in clients and in parsers (such as URL parsers) that attempt to determine the hostname portion, since the expected syntax rules are not followed.

Interoperability and Compatibility Risk

The current Chromium code attempts to resolve any names that callers pass, even when they do not conform to the LDH rule. This includes any arbitrary 8-bit sequence, including both spaces and control characters, provided that there are no embedded NUL bytes (‘\0’).

We expect limited interoperability or compatibility problems, since these names do not conform to the relevant RFCs for valid hostnames.

To measure this, we added the UMA historgram Net.ValidDNSName, which measures resolution attempts for invalid characters. As of 22 May 2017, Chrome Stable (58) shows that approximately 0.02% of resolutions include invalid characters.

While this means there is some interoperability risk for DNS, these names are invalid for use in other contexts. For example, hosts using such names cannot use QUIC (which strictly enforces the proposed rule), and they are not supposed to use TLS SNI (which Chrome always includes, except for IP addresses), because SNI is specified to follow the LDH rule.

Alternative implementation suggestion for web developers

Web developers whose servers have names that currently resolve should experience no change. :)

Usage information from UseCounter

The UMA histogram measures name resolution attempts, rather than successes, and shows a low number of attempts (0.02% on stable). This is higher than we expect, but we are still confident that this will not cause real web breakage — as explained above, such hostnames will already fail in a variety of web contexts. We expect that the invalid hostnames are due to errors that are already causing breakage, such as malformed URLs and HTML markup.

OWP launch tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=716626

Entry on the feature dashboard

https://www.chromestatus.com/feature/5747040885669888

Requesting approval to remove too?

Yes.

[mark a. foltz]

I assume this I2R doesn't affect Chromium's mDNS resolver for browser features (Cloud Print, Cast, etc.)?

DNS-SD hostnames always include underscores.  However, I don't believe there's a way to trigger resolution of them from navigation or Web content.

m.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOuvq20r56rsxKQKYxJjhLR6jxtzkQ4WZvoYYEAQzMQYvjGL2A%40mail.gmail.com.

[Chris Palmer]

On Mon, May 22, 2017 at 2:55 PM, mark a. foltz <mfo...@chromium.org> wrote:

I assume this I2R doesn't affect Chromium's mDNS resolver for browser features (Cloud Print, Cast, etc.)?

It affects anything calls https://cs.chromium.org/chromium/src/net/dns/dns_util.cc?type=cs&l=110, which is currently only https://cs.chromium.org/chromium/src/net/dns/dns_util.cc?type=cs&q=net.valid&l=46.

DNS-SD hostnames always include underscores.  However, I don't believe there's a way to trigger resolution of them from navigation or Web content.

We are allowing underscores unconditionally anyway.

[Peter Kasting]

On Mon, May 22, 2017 at 1:00 PM, Chris Palmer <pal...@chromium.org> wrote:

Summary

This I2D&R is to restrict the set of valid hostnames that the network name resolver will attempt to resolve. We will restrict names to those that follow the “preferred name syntax” established for hostnames (A/AAAA) — in particular, those containing only letters, digits, and hyphens (the “LDH rule”). In addition to supporting these characters, for web-compatibility reasons, we will also accept underscores (‘_’).

For reference, the omnibox won't navigate by default on hostnames that fail the check in IsCanonicalizedHostCompliant ( https://cs.chromium.org/chromium/src/net/base/url_util.cc?type=cs&l=280 ), which is basically a slightly stricter version of the above (it rejects hostnames whose last component does not begin with an alphanumeric character).

We had more restrictive policies in the past (based on RFC 1738), but found that real-world websites violated the restrictions.  However, the policy above has been in place for some time, so I imagine that the proposed less-stringent rule for Blink will work in practice.

You may want to consider adopting the "last component must begin with an alphanumeric character" rule for consistency.

PK

[Mike West]

Non-OWNERS LGTM. Even aside from security issues, tightening restrictions on the data we send over the wire is a good thing (Postel was wrong!), and aligning our DNS lookup behavior with our Omnibox behavior as pkasting@ suggests sounds like the right thing to do. Would you be willing to change the restrictions you're proposing to match that behavior?

-mike

--

[Chris Palmer]

On Wed, May 24, 2017 at 4:54 AM, Mike West <mk...@chromium.org> wrote:

Non-OWNERS LGTM. Even aside from security issues, tightening restrictions on the data we send over the wire is a good thing (Postel was wrong!),

Well, he was right... for his time. :)

and aligning our DNS lookup behavior with our Omnibox behavior as pkasting@ suggests sounds like the right thing to do. Would you be willing to change the restrictions you're proposing to match that behavior?

Hey, I'm not going to argue with "slightly stricter". :) Thanks Peter!

[Peter Kasting]

You might want to try shipping the relaxed behavior with some kind of monitoring code to check how often it would have violated the stricter behavior, before just going to that.  It's possible that this case does come up, but only in intranets, and we haven't seen bugs from those folks filed against the omnibox?  I'm willing to believe that the omnibox needs to get looser to match your behavior rather than the other way around.

PK

[Rick Byers]

What do the other major browsers do here?  Breaking 2 in 10,000 resolutions sounds substantial enough that there's certainly some risk of causing developer and user confusion if we're deviating from something that is currently interoperable behavior between browsers.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAAHOzFBQ37DkAadHSQskwDBtgykN9Ai2GAMeQHyyzJztZQ%3DcWA%40mail.gmail.com.

[sle...@google.com]

I think the mistake is assuming those 2 in 10,000 resolutions are _successful_ resolutions, versus _attempted_ resolutions.

Other browsers today just hand the parsed value to the network stack/resolver stack. It may be that the higher layer enforces restrictions (e.g. from the URL spec), it may be that the resolution succeeds but the certificate is rejected (e.g. the certificate hostname is restricted to the characters palmer@ mentioned, so there's no way TLS will work for those hosts), or it may be that the hosts themselves are typoed within the source and just failing.

In terms of "On the public Internet, what will happen" - well, as palmer@ mentioned, many (most?) DNS server stacks won't allow you to configure such hostnames, certainly not in the 'exotic' character space (e.g. control characters). You could find one that does, but then you're going to have trouble with every intermediate caching resolver that enforces these rules.

So that's why it's useful to be aware of that 2 in 10,000 resolutions are attempts, not necessarily successes. All the RFCs are relatively aligned on that - it is the APIs that evolved around these RFCs that ended up liberal, and so clients themselves haven't implemented the checks (yet). This is fairly similar in concept to https://tools.ietf.org/html/draft-west-let-localhost-be-localhost-03 :)

[Chris Palmer]

On Wed, May 24, 2017 at 5:35 PM, Peter Kasting <pkas...@chromium.org> wrote:

You might want to try shipping the relaxed behavior with some kind of monitoring code to check how often it would have violated the stricter behavior, before just going to that.  It's possible that this case does come up, but only in intranets, and we haven't seen bugs from those folks filed against the omnibox?  I'm willing to believe that the omnibox needs to get looser to match your behavior rather than the other way around.

 Sure, I can do that. I'll put up a CL for review and people can see what they think.

[Chris Palmer]

 Sure, I can do that. I'll put up a CL for review and people can see what they think.

https://codereview.chromium.org/2906723002

[Rick Byers]

If this doesn't affect any _succecssful_ resolutions (eg. because DNS servers or our upstream DNS stack already prevent these cases), then I don't think there's really any web exposure and so no need for an intent at all.

But if even half of those cases turn out to be some special case that does actually work today and we break a bunch of page load, then we absolutely need to consider the compat and interop risks here.

Why are we measuring attempts at all?  Could we instead just be measuring resolution successes of this form?  If dev channel data suggests that's near zero, then I think you can safely treat this as a bug fix and not a web platform change at all.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/9362ba3b-109c-4fed-b183-d230a7ed6325%40chromium.org.

[Chris Palmer]

On Thu, May 25, 2017 at 2:36 PM, Rick Byers <rby...@chromium.org> wrote:

If this doesn't affect any _succecssful_ resolutions (eg. because DNS servers or our upstream DNS stack already prevent these cases), then I don't think there's really any web exposure and so no need for an intent at all.

That was my instinct as well :) but I was convinced to send an intent.

But if even half of those cases turn out to be some special case that does actually work today and we break a bunch of page load, then we absolutely need to consider the compat and interop risks here.

Again, I'd be really surprised if half of these result in 'successful' resolutions, because such names would fail for a variety of reasons at other layers/in other contexts/in other applications.

Why are we measuring attempts at all?  Could we instead just be measuring resolution successes of this form?  If dev channel data suggests that's near zero, then I think you can safely treat this as a bug fix and not a web platform change at all.

I just wanted to get a feel for how much of this is going on at all. If nothing else, it provides an upper bound on the rate of exploitation of the security vulnerability.

(Which, again, is fundamentally due to errors in higher layers: "We expect that the invalid hostnames are due to errors that are already causing breakage, such as malformed URLs and HTML markup." If any Chromium committer wants access to the security bug for more detail, I can add you. Bottom line: a lot of Chromium/Blink code obeys Postel's Law, and should stop doing so. :) )

I will try to write and land a CL to count 'successful' resolutions too. But keep in mind we're talking about things that would be hard for DNS server administrators to type into their zone files, even if the DNS server will accept such zone files.

[Chris Palmer]

On Thu, May 25, 2017 at 3:13 PM, Chris Palmer <pal...@google.com> wrote:

I will try to write and land a CL to count 'successful' resolutions too. But keep in mind we're talking about things that would be hard for DNS server administrators to type into their zone files, even if the DNS server will accept such zone files.

Here's a rather nasty piece of work:

https://codereview.chromium.org/2921553002

As you can see, once this issue is resolved, we're going to be removing this ugly stuff. And that will be a happy day. :)

[Rick Byers]

Nice, thanks.

To be clear, if the opinion of you and your code reviewers is that this should really never (or only exceedingly rarely) trigger, then you're free to treat it as a bug fix and skip the intent process ("trivial" web platform change). But this is definitely the data we'd want to see to evaluate the compat risk as an "intent to remove".

Assuming you want to proceed with this as an intent, since there's an active security issue I assume there's some time pressure?  Perhaps it's not too late to merge this CL back into M60 and then we can make a tentative decision based on dev-channel data for M61 (and just double-check that beta/stable isn't wildly different sometime before 61 hits stable)?

Rick

[Ryan Sleevi]

On Thu, Jun 1, 2017 at 10:27 AM, Rick Byers <rby...@chromium.org> wrote:

Nice, thanks.

To be clear, if the opinion of you and your code reviewers is that this should really never (or only exceedingly rarely) trigger, then you're free to treat it as a bug fix and skip the intent process ("trivial" web platform change). But this is definitely the data we'd want to see to evaluate the compat risk as an "intent to remove".

Yeah, I was the one who suggested the Intent process because this would be a (potentially) web-observable change in interop - and arguably, something we'd want all UAs to adopt. Thus, even if it doesn't break something (and it shouldn't), it helps provide greater awareness of the issue, and the whole chain of issues that result from being liberal and not enforcing RFC1034/1123 rules, in a way that can help reduce interop challenges and surprises :)

So somewhere between "PSA" and "Intent", with a public double-check of our numbers and reasoning.

[Chris Palmer]

On Thu, Jun 1, 2017 at 7:27 AM, Rick Byers <rby...@chromium.org> wrote:

To be clear, if the opinion of you and your code reviewers is that this should really never (or only exceedingly rarely) trigger, then you're free to treat it as a bug fix and skip the intent process ("trivial" web platform change).

I've decided to take that tack:

https://chromium-review.googlesource.com/c/569298/

I'd really like a Blink engineer to take on the Blink aspect of https://bugs.chromium.org/p/chromium/issues/detail?id=695474&desc=2 as well.

[Mike West]

On Thu, Jul 13, 2017 at 3:31 AM, Chris Palmer <pal...@google.com> wrote:

On Thu, Jun 1, 2017 at 7:27 AM, Rick Byers <rby...@chromium.org> wrote:

To be clear, if the opinion of you and your code reviewers is that this should really never (or only exceedingly rarely) trigger, then you're free to treat it as a bug fix and skip the intent process ("trivial" web platform change).

I've decided to take that tack:

https://chromium-review.googlesource.com/c/569298/

This approach still LGTM, FWIW. :)

 

I'd really like a Blink engineer to take on the Blink aspect of https://bugs.chromium.org/p/chromium/issues/detail?id=695474&desc=2 as well.

I think this is somewhat mitigated by blocking requests that contain both raw newlines and `<` characters (https://groups.google.com/a/chromium.org/d/msg/blink-dev/KaA_YNOlTPk/VmmoV88xBgAJ). Is there more work that you think ought to be done?

-mike

[Jenn Cook]

please make these people leave me alone

jenn cook 661.717.4980

> --
> You received this message because you are subscribed to the Google Groups
> "blink-dev" group.

> To unsubscribe from this group and stop receiving emails from it, send an
> email to blink-dev+...@chromium.org.

> To view this discussion on the web visit

> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAKXHy%3DdRAByRo6KWCVAwM7XvOjbE%2Bm3GV%2B8VciWAfYb9VzCf0g%40mail.gmail.com.
>

[Matthew Menke]

Note that we're talking about a slightly different approach - making DNSDomainFromDot itself fail to resolve invalid names, as opposed to those only made through the SystemResolverProc.  This covers the built-in stub resolver (And presumably mdns as well), as well as a number of other consumers.