Public Suffix Query Service / case sensitivity

[Peter Thomassen]

Hi,

While operating the authoritative name server at www.desec.io (which has several tenants), I realized that we need to prevent users from registering zones corresponding to public suffixes. Otherwise, users would be able to register e.g. "co.uk" and block other legitimate users from setting up their zones under co.uk.

This is because registered zones generally extend over all subdomains, which implies that other users should not be allowed to arbitrarily register zones that are a subzone of an existing one. (Not thinking this through made some DNS providers vulnerable to subzone hijacking. In case you're interested, you can find some research on this here: https://www.ronpub.com/ojwt/OJWT_2018v5i1n02_Thomassen.html)

The registration system thus needs to be aware of what's a public suffix and what's not. However, maintaining an up-to-date copy of the PSL was something that we deemed out of scope of the registration system. Instead, we were looking for some way for our systems to do a "PSL lookup" and find a domain's public suffix without having to load and parse the whole list.

As a consequence, we came up with an easy-to-use DNS-based Public Suffix Query Service. You can take a look here: https://publicsuffix.zone/

(As the service is DNS-based, resolvers obviously learn about the domains that get queried, so using the service may not be up to your privacy standards, depending of the use case. I'll add a note on the page about this later.)

While writing a Python library for convenient usage of the lookup service (https://pypi.org/project/psl-dns/), I noticed that the test cases at https://github.com/publicsuffix/list/blob/master/tests/test_psl.txt are case-insensitive. However, the algorithm under https://publicsuffix.org/list/ specifies that a "domain is said to match a rule" iff one finds that all label pairs from the rule and from the domain "are identical" or wildcard.

In my understanding, COM and com are equal, but not identical. The above library therefore is currently case-aware, and as a consequence does not pass the standard PSL tests. Have I misunderstood the algorithm definition, or do the tests ignore lettercase in violation of the specs?

Cheers,

Peter

-- 

Dr. Peter Thomassen

Senior Security Expert

Mobile +49 176 63159879

peter.t...@securesystems.de

SSE Secure Systems Engineering GmbH | Am Sandwerder 21 | 14109 Berlin

[Dave Crocker]

On 5/24/2019 5:41 PM, Peter Thomassen wrote:
> The registration system thus needs to be aware of what's a public suffix
> and what's not.

Doesn't dnssec support obviate this kind of problem? (If the DNS data
are validated by it, it doesn't matter what server provides the data.
If the data are not validated, then the rogue server is useless.)

d/
--
Dave Crocker
Brandenburg InternetWorking
bbiw.net

[Dr. Peter Thomassen | SSE]

Hi Dave,

On 5/25/19 3:23 AM, Dave Crocker wrote:
> On 5/24/2019 5:41 PM, Peter Thomassen wrote:
>> The registration system thus needs to be aware of what's a public
>> suffix and what's not.
>
>
> Doesn't dnssec support obviate this kind of problem?  (If the DNS data
> are validated by it, it doesn't matter what server provides the data. If
> the data are not validated, then the rogue server is useless.)

DNSSEC ensure that an attacker who manages to create and control a
subzone cannot inject malicious records, because the signing key is not
available to the attacker. The attacker also can't come up with a
different key because that would require creating a corresponding DS
record in the parent zone, but that requires collaboration of the parent
zone operator and is out of the attacker's control.

However, even with DNSSEC, queries will still be answered from the more
specific zone, producing invalid responses. Thus, unauthorized creation
of subzones leads to DoS on all names covered by the subzone. That is,
DNSSEC does not remove the problem, it just turns the integrity breach
into a DoS scenario. Depending on what subdomain you knock off this way,
that's already a pretty bad scenario.

Nevertheless, you're completely right that DNSSEC helps a lot. That was
precisely my thinking when I chose the title for the paper I referenced.

[Ryan Sleevi]

On Fri, May 24, 2019 at 8:41 PM Peter Thomassen <ma...@peter-thomassen.de> wrote:

While writing a Python library for convenient usage of the lookup service (https://pypi.org/project/psl-dns/), I noticed that the test cases at https://github.com/publicsuffix/list/blob/master/tests/test_psl.txt are case-insensitive. However, the algorithm under https://publicsuffix.org/list/ specifies that a "domain is said to match a rule" iff one finds that all label pairs from the rule and from the domain "are identical" or wildcard.

In my understanding, COM and com are equal, but not identical. The above library therefore is currently case-aware, and as a consequence does not pass the standard PSL tests. Have I misunderstood the algorithm definition, or do the tests ignore lettercase in violation of the specs?

The Formal Algorithm definition on that page makes note of a pre-processing step, to avoid any ambiguity about the case-sensitivity, namely:

The domain and all rules must be canonicalized in the normal way for hostnames - lower-case, Punycode (RFC 3492).

The language is left like this to allow for different canonicalization processes (such as IDNA-2003 vs IDNA-2008 vs UTS-46) by implementations, provided that both the input hostnames and the rule list have been canonicalized according to the same process. The further presumption here, with respect to DNS terminology, is that you're dealing with hostnames, and per RFC 1034, Section 3.1, "By convention, domain names can be stored with arbitrary case, but domain name comparisons for all present domain functions are done in a case-insensitive manner, assuming an ASCII character set, and a high order zero bit."

[Dr. Peter Thomassen | SSE]

Hi Ryan,

On 5/28/19 9:46 PM, 'Ryan Sleevi' via psl-discuss wrote:
> The Formal Algorithm definition on that page makes note of a
> pre-processing step, to avoid any ambiguity about the case-sensitivity,
> namely:
>
> The domain and all rules must be canonicalized in the normal way for
> hostnames - lower-case, Punycode (RFC 3492).

Thank you! I had somehow overlooked this sentence and jumped right to
the section where the algorithm steps are listed.

I just published a new release of psl-dns fixing this issue.

It may be worth adding the normalization requirement as the 0-th step to
the algorithm. Do you welcome pull requests at
https://github.com/publicsuffix/publicsuffix.org/?

Best,