Exception TLDs?

[csaa...@igalia.com]

Hi,

I'm looking forward to replace libsoup's internal handling of the public suffix list [1] with this library, but I'm currently stuck at making our API still be able to find out when a TLD is not public. I see in the source code the following comment:

/* TLD, this is the prevailing '*' match. 

* We don't currently support exception TLDs (TLDs that are not a public suffix)

*/

 return 1;

So stuff like example, example.example, etc are considered public whereas we don't, which breaks a few of our tests [2]. Are there any plans to change this?

Best regards,

Claudio

[1] https://git.gnome.org/browse/libsoup/tree/libsoup/soup-tld.c

[2] https://git.gnome.org/browse/libsoup/tree/tests/tld-test.c#n113

[Tim Ruehsen]

Am Dienstag, den 20.02.2018, 09:42 -0800 schrieb csaa...@igalia.com:
> Hi,
>
> I'm looking forward to replace libsoup's internal handling of the
> public suffix list [1] with this library, but I'm currently stuck at
> making our API still be able to find out when a TLD is not public. I
> see in the source code the following comment:
>
> /* TLD, this is the prevailing '*' match.
> * We don't currently support exception TLDs (TLDs that are not a
> public suffix)
> */
> return 1;
>
> So stuff like example, example.example, etc are considered public
> whereas we don't, which breaks a few of our tests [2]. Are there any
> plans to change this?

Hi Claudio,

the above comment references rules as
!example

Such a rule is not covered by the Mozilla Public Suffix List. But
perhaps I am wrong - sometimes they change rules or add/remove things.

So what standard/rule are you referring to ?

But besides of this you can of course always keep your own private
exception list. Just check for it before you use libpsl API and you are
fine. Out of interest, how many entries does your list have ?

I would accept a patch as long as it is 'backwards compatible' (doesn't
change API behavior for anything else but those exceptions). Of course
I try to help if there are questions or blocker.

Regards, Tim

[Claudio Saavedra]

Hi Tim,

On Tue, 2018-02-20 at 22:28 +0100, Tim Ruehsen wrote:

> > /* TLD, this is the prevailing '*' match.
> > * We don't currently support exception TLDs (TLDs that are not a
> > public suffix)
> > */
> > return 1;
> >
> > So stuff like example, example.example, etc are considered public
> > whereas we don't, which breaks a few of our tests [2]. Are there
> > any
> > plans to change this?
>
> Hi Claudio,
>
> the above comment references rules as
> !example
>
> Such a rule is not covered by the Mozilla Public Suffix List. But
> perhaps I am wrong - sometimes they change rules or add/remove
> things.
>
> So what standard/rule are you referring to ?

I think I didn't explain myself well. Please bear with me as I'm
relatively new to TLDs and how they are dealt with.

In libsoup's API, soup_is_public_suffix() checks against its own copy
of the PSL and if the given domain is not found in the list (say,
.thisdomaindoesntexist), it returns FALSE. We only return TRUE for
domains that are known and are in the list.

However, psl_is_public_suffix() works differently, from what I see in
the code. Whenever I pass a suffix that has only one label, the list is
not checked and the function returns 1. So for example, when called for
.thisdomandoesntexist, the function returns 1. This is because of the
code I quoted above. The internal _psl_is_public_suffix() first counts
the number of labels and if it finds only one, it always returns 1,
regardless of what the label says.

> But besides of this you can of course always keep your own private
> exception list. Just check for it before you use libpsl API and you
> are fine. Out of interest, how many entries does your list have ?

We don't have exceptions. We just don't consider as public suffixes
elements that are not in the list. So we wouldn't be able to solve this
by keeping an exception list.

>
> I would accept a patch as long as it is 'backwards compatible'
> (doesn't change API behavior for anything else but those exceptions).
> Of course I try to help if there are questions or blocker.

Thanks for your help! Much appreciated :)

Claudio