IDN TLD whitelist, and .com

[Gervase Markham]

As some participants may recall, our IDN TLD whitelist was created in
response to the "payp-cyrillic-a-l.com" incident of 2005.

http://www.shmoo.com/idn/

Since that time, we have whitelisted over 50 TLDs after having inspected
their anti-spoofing policies.

http://www.mozilla.org/projects/security/tld-idn-policy-list.html

Recently, it was decided that a whitelist was not scalable in the face
of hundreds of new TLDs, and that we had to come up with a new approach.
We did, based on some suggestions from the Unicode Consortium:

https://wiki.mozilla.org/IDN_Display_Algorithm

The new criteria are not as strict as the old (for example, they can't
spot whole-script homographs (All-Latin "scope.tld" vs all-Cyrillic
"ѕсоре.tld"), but are the best we can do programmatically without a
manually-maintained whitelist, and without compromising other principles
(like "works somewhere => works everywhere").

Up until now, Verisign have not formally applied for inclusion in the
TLD whitelist, although preliminary discussions have occurred on more
than one occasion. Now, they have applied (for .com, .net and
..name), and their current policies do meet the new criteria:
https://bugzilla.mozilla.org/show_bug.cgi?id=770877

However, given that it was a .com domain which started all this fuss, I
thought it was worth posting publicly in case anyone had any comments.

Gerv

[Daniel Veditz]

On 7/5/12 1:37 AM, Gervase Markham wrote:
> Recently, it was decided that a whitelist was not scalable in the face
> of hundreds of new TLDs, and that we had to come up with a new approach.
> We did, based on some suggestions from the Unicode Consortium:
>
> https://wiki.mozilla.org/IDN_Display_Algorithm

Big thanks to you and Simon Montagu for driving this forward!

Given that the new criteria are not as strict as our old policy, why
would we want to preserve the old whitelist system in parallel? The
big flaw in the whitelist policy was that a registrar's policy
applied to the domain labels directly issued by that registrar but
not to any sub-domains created by the domain owner. Those
sub-domains could be as many levels deep and as spoofy as they'd
like. The new algorithm, in contrast, applies to each label
separately and will prevent spoofy sub-domains.

If the stated policies of the currently whitelisted TLDs fall within
the new algorithm let's just scrap the whitelist. Even if some small
percentage of edge-case domains end up being flipped to punycode the
code and policy simplification on our end will be worth it.

If there were any such edge-case domains would they be shown as IDN
in any of the other browsers (besides Opera who uses the same
whitelist mechanism)?

> Now, they have applied (for .com, .net and .name), and

> their current policies do meet the new criteria:
> https://bugzilla.mozilla.org/show_bug.cgi?id=770877

What's the time-frame on the new IDN algorithm? Sounds relatively
close so why not let them just start working when that lands instead
of whitelisting them?

> However, given that it was a .com domain which started all this fuss, I
> thought it was worth posting publicly in case anyone had any comments.

Have they revoked all the previously spoofing domains? Have they
audited all their existing domains to make sure there aren't
additional ones in there that violate their new rules? What is their
transition plan for the domains that do exist?

Their new rules going forward sound fine, it's any grand-fathered
mess I'm worried about. I'm especially worried if you proceed with
your currently stated plan of preserving the whitelist even after
the new algorithm lands.

-Dan Veditz

[hill...@gmail.com]

Gerv,

May I suggest the following additions?

Hostname U-labels to be displayed as Unicode SHALL NOT include confusable bidirectional text. [http://unicode.org/reports/tr36/#Bidirectional_Text_Spoofing] and [http://www.ietf.org/rfc/rfc3987.txt]

a. Hostname labels SHALL NOT include left-to-right override characters. U+200E, U+202E
b. Hostname labels SHALL NOT include both left-to-right and right-to-left characters
c. Hostname labels using a right-to-left character must start and end with right-to-left characters, with the exception that labels using right-to-left characters may end with combining marks or numbers


-Brad Hill

[hill...@gmail.com]

Gerv,

May I suggest the following additions?

Hostname U-labels to be displayed as Unicode SHALL NOT include confusable bidirectional text. [http://unicode.org/reports/tr36/#Bidirectional_Text_Spoofing] and [http://www.ietf.org/rfc/rfc3987.txt]

a. Hostname labels SHALL NOT include left-to-right override characters. U+200E, U+202E
b. Hostname labels SHALL NOT include both left-to-right and right-to-left characters
c. Hostname labels using a right-to-left character must start and end with right-to-left characters, with the exception that labels using right-to-left characters may end with combining marks or numbers


-Brad Hill

On Thursday, July 5, 2012 1:37:54 AM UTC-7, Gervase Markham wrote:

[Gervase Markham]

Hi Dan,

Sorry for the delay here.

On 05/07/12 16:39, Daniel Veditz wrote:
> On 7/5/12 1:37 AM, Gervase Markham wrote:
>> Recently, it was decided that a whitelist was not scalable in the face
>> of hundreds of new TLDs, and that we had to come up with a new approach.
>> We did, based on some suggestions from the Unicode Consortium:
>>
>> https://wiki.mozilla.org/IDN_Display_Algorithm
>
> Big thanks to you and Simon Montagu for driving this forward!
>
> Given that the new criteria are not as strict as our old policy, why
> would we want to preserve the old whitelist system in parallel?

The new policy is tighter in some ways; as you say, it applies to all
levels. We also wanted to avoid any nasty surprises. I'm not ruling out
removing the old system later, but it's simple (only a few lines of
code) and I wanted to make sure this change didn't break any
previously-working sites.

> If there were any such edge-case domains would they be shown as IDN
> in any of the other browsers (besides Opera who uses the same
> whitelist mechanism)?

Everyone does it differently, even Opera (different whitelist, plus it
also has some heuristics as well).

> What's the time-frame on the new IDN algorithm? Sounds relatively
> close so why not let them just start working when that lands instead
> of whitelisting them?

Because I don't want to gate making things better for IDN-in-.com users
on the completion of a patch I'm not writing.

Also, we can check in a TLD whitelist change on beta pretty easily; we
can't port a patch forward that far.

> Have they revoked all the previously spoofing domains?

The Paypal one now belongs to paypal.

> Have they
> audited all their existing domains to make sure there aren't
> additional ones in there that violate their new rules? What is their
> transition plan for the domains that do exist?

Good questions; I will ask.

> Their new rules going forward sound fine, it's any grand-fathered
> mess I'm worried about. I'm especially worried if you proceed with
> your currently stated plan of preserving the whitelist even after
> the new algorithm lands.

I could revert the whitelist to its state pre-new-plan once the new
algorithm lands, if that would be better. (I.e. remove the ones included
under transitional arrangements.)

Gerv

[Gervase Markham]

On 05/07/12 21:12, hill...@gmail.com wrote:
> May I suggest the following additions?

Hi Brad,

We are basing our algorithm on that in the UTR. Perhaps your feedback is
better directed to Mark Davis, who maintains it?

Or are you saying these restrictions are already in that document, and
we missed them?

Gerv

[John Nagle]

On 7/18/2012 3:13 AM, Gervase Markham wrote:
> Hi Dan,

>
>> What's the time-frame on the new IDN algorithm? Sounds relatively
>> close so why not let them just start working when that lands instead
>> of whitelisting them?
>
> Because I don't want to gate making things better for IDN-in-.com users
> on the completion of a patch I'm not writing.

Related to this, does someone have a list of interesting non-test
sites that actually use IDN TLDs? I need a list of IDN domains for
test purposes to exercise some code, and some live web pages that
link to them.

After all the implementation effort, there don't seem to be all
that many IDN domains.

John Nagle

[Gervase Markham]

On 05/07/12 16:39, Daniel Veditz wrote:

>> However, given that it was a .com domain which started all this fuss, I
>> thought it was worth posting publicly in case anyone had any comments.
>
> Have they revoked all the previously spoofing domains? Have they
> audited all their existing domains to make sure there aren't
> additional ones in there that violate their new rules? What is their
> transition plan for the domains that do exist?
>
> Their new rules going forward sound fine, it's any grand-fathered
> mess I'm worried about. I'm especially worried if you proceed with
> your currently stated plan of preserving the whitelist even after
> the new algorithm lands.

Sorry for the delay here; Verisign said:

"You are correct that in conjunction with our agreements and
consultation with ICANN we grandfather domain names that no longer meet
updated IDN standards and policies.

As with the introduction of IDNA2008 and previous updates Verisign goes
through a process of communication with the registrar channel to
identify specific domain names that are no longer compliant, and
potential implications of allowing those domain names to continue in the
DNS. Verisign has provided a wholesale registration fee refund for the
remaining term left on such disallowed registrations when the registrar
works with the registrant to replace the disallowed registration with
one that is allowed.

Separately, Verisign has an internal process place to address reported
malicious use of domain names in cases such as malware, phishing, etc..."

I personally am happy with this "grandfathering plus proactive
observation" method. I suggest that requiring wholesale revocation of
registrations is unrealistic, and disruptive to domain owners who
registered their domains in good faith.

So, unless there are further objections, I plan to take this addition
forward.

(This is bug https://bugzilla.mozilla.org/show_bug.cgi?id=770877 )

Gerv

[John Nagle]

On 10/2/2012 6:21 AM, Gervase Markham wrote:
> On 05/07/12 16:39, Daniel Veditz wrote:
>>> However, given that it was a .com domain which started all this fuss, I
>>> thought it was worth posting publicly in case anyone had any comments.
>>
>> Have they revoked all the previously spoofing domains? Have they
>> audited all their existing domains to make sure there aren't
>> additional ones in there that violate their new rules? What is their
>> transition plan for the domains that do exist?
>>
>> Their new rules going forward sound fine, it's any grand-fathered
>> mess I'm worried about. I'm especially worried if you proceed with
>> your currently stated plan of preserving the whitelist even after
>> the new algorithm lands.

I would argue against this exception for ".com" and ".net".
If someone is mounting an attack, it would probably be in those TLDs.

If Network Solutions wants an exception for "grandfathered"
domain names, let them publish a list of those domains for public
comment. Is the problem big enough to worry about?

John Nagle

[Gervase Markham]

On 11/10/12 22:05, John Nagle wrote:
> I would argue against this exception for ".com" and ".net".
> If someone is mounting an attack, it would probably be in those TLDs.

Not if they aren't included.

Discriminating against domain owners in .com or .net because of what
some criminals may or may not do seems unreasonable to me.

> If Network Solutions wants an exception for "grandfathered"
> domain names, let them publish a list of those domains for public
> comment. Is the problem big enough to worry about?

The fact that a name is no longer permitted doesn't make it
automatically problematic from a "can be used to commit crime" point of
view.

Gerv

[Justin Dolske]

On 10/12/12 6:26 AM, Gervase Markham wrote:

>> If Network Solutions wants an exception for "grandfathered"
>> domain names, let them publish a list of those domains for public
>> comment. Is the problem big enough to worry about?
>
> The fact that a name is no longer permitted doesn't make it
> automatically problematic from a "can be used to commit crime" point of
> view.

Though neither does it make it automatically make it safe. :)

Grandfathering makes me a bit wary, but I'd agree that it's also hard to
tell how much of a problem it really is.

Seems like it should be fairly simple for NetSol to generate a list of
grandfathered domains. Have they done so and reviewed it? I wouldn't be
surprised if there are difficulties in making said list _public_ (unless
it can already be generated by a 3rd party?), but even a 3rd party
private/NDA'd review would make me feel a better.

Justin

[Dan Veditz]

On 10/12/12 12:59 PM, Justin Dolske wrote:
> Though neither does it make it automatically make it safe. :)
>
> Grandfathering makes me a bit wary, but I'd agree that it's also hard to
> tell how much of a problem it really is.

I'm only a little concerned about grandfathering. I _am_ concerned that
the whitelisting mechanism supersedes the proposed algorithm and allows
for arbitrary charaters on labels above the level of the domain that the
registrar issues and can vet.

Can we do a hybrid system, where for whitelisted TLDs we accept
registered domains as found and then apply the algorithm to the rest of
the labels (effectively eTLD+2, though I don't know if we want to drag
the public suffix list into this).

-Dan Veditz

[Gervase Markham]

On 12/10/12 20:59, Justin Dolske wrote:
> Seems like it should be fairly simple for NetSol to generate a list of
> grandfathered domains. Have they done so and reviewed it?

They tell me that they have, and worked with some domain owners to
transition to new domains.

"As with the introduction of IDNA2008 and previous updates Verisign goes
through a process of communication with the registrar channel to
identify specific domain names that are no longer compliant, and
potential implications of allowing those domain names to continue in the
DNS. Verisign has provided a wholesale registration fee refund for the
remaining term left on such disallowed registrations when the registrar
works with the registrant to replace the disallowed registration with
one that is allowed."

Gerv

[Gervase Markham]

On 13/10/12 23:37, Dan Veditz wrote:
> I'm only a little concerned about grandfathering. I _am_ concerned that
> the whitelisting mechanism supersedes the proposed algorithm and allows
> for arbitrary charaters on labels above the level of the domain that the
> registrar issues and can vet.

Not arbitrary; we still have our character blacklist for protocol-alike
characters, and IDNA2008, when we implement it, will further restrict
the allowable set. As long as people can't make one domain component
look like two, or two like one, then people should only be able to mess
with people inside their own Public Suffix + 1.

> Can we do a hybrid system, where for whitelisted TLDs we accept
> registered domains as found and then apply the algorithm to the rest of
> the labels (effectively eTLD+2, though I don't know if we want to drag
> the public suffix list into this).

Hmm. Possible, but certainly more confusing. Ideally, the TLD whitelist
would go away eventually. I don't want to rip it out as the new
algorithm comes in because there's a higher risk of breaking people's
currently-working domains by mistake.

Gerv

[Dan Veditz]

On 10/15/12 11:36 AM, Gervase Markham wrote:
>> Can we do a hybrid system, where for whitelisted TLDs we accept
>> registered domains as found and then apply the algorithm to the rest of

>> the labels [...]

>
> Hmm. Possible, but certainly more confusing. Ideally, the TLD whitelist
> would go away eventually. I don't want to rip it out as the new
> algorithm comes in because there's a higher risk of breaking people's
> currently-working domains by mistake.

And I don't want to expand the whitelist more than double or triple by
numbers of registered domains to include .com when ideally we don't need
it. Especially if that then becomes an argument for a permanent spot on
the whitelist.

-Dan Veditz

[Gervase Markham]

On 16/10/12 21:14, Dan Veditz wrote:
> On 10/15/12 11:36 AM, Gervase Markham wrote:
>>> Can we do a hybrid system, where for whitelisted TLDs we accept
>>> registered domains as found and then apply the algorithm to the rest of
>>> the labels [...]
>>
>> Hmm. Possible, but certainly more confusing. Ideally, the TLD whitelist
>> would go away eventually. I don't want to rip it out as the new
>> algorithm comes in because there's a higher risk of breaking people's
>> currently-working domains by mistake.
>
> And I don't want to expand the whitelist more than double or triple by
> numbers of registered domains to include .com

I'm not sure what you mean by "expand by double or triple". It's a TLD
whitelist, it current contains 72 non-test TLDs, and this discussion is
about adding 3 more (.com, .net and .name).

> when ideally we don't need
> it. Especially if that then becomes an argument for a permanent spot on
> the whitelist.

Well, we could stop adding domains to the whitelist if we committed the
patch with the new algorithm. It's been waiting for your security review
for 6 months...

https://bugzilla.mozilla.org/show_bug.cgi?id=722299

Gerv

[John Nagle]

There's been much previous discussion about how tough Mozilla
should be on CAs. especially in the wildcard cert area. I'd
like to suggest raising the standards in private browsing mode.

If, in private browsing mode, Mozilla can detect a MITM attack,
the user should be warned with a high-visibility warning.
This includes any cert in the chain with a wildcard bigger than
one second-level domain. The base list of CAs from Mozilla
used in this mode may be shorter than the main list. Any
CAs added locally, if allowed at all, should produce a warning.

So, if a corporate or school firewall is listening in, the
user is informed. This is consistent with Mozilla's
"work for mankind, not the man" policy.

John Nagle

[Gervase Markham]

On 24/03/13 16:43, John Nagle wrote:
> If, in private browsing mode, Mozilla can detect a MITM attack,
> the user should be warned with a high-visibility warning.

Er, if we could detect an MITM attack in any mode, there should be a
high visibility warning. :-)

> This includes any cert in the chain with a wildcard bigger than
> one second-level domain.

I'm not entirely sure what you mean here, but NSS now only accepts certs
with the wildcard in the leftmost position.

> The base list of CAs from Mozilla
> used in this mode may be shorter than the main list.

In what way would such a shortening be accomplished?

> Any
> CAs added locally, if allowed at all, should produce a warning.

Now that _is_ an interesting idea. It is unlikely that someone would be
using private browsing mode to browse their own intranet. So it might
make sense to beef up the UI indicators (if not warnings) about using a
non-default-install cert in PBM.

Gerv

[Florian Weimer]

On 03/25/2013 12:44 PM, Gervase Markham wrote:
> On 24/03/13 16:43, John Nagle wrote:

>> If, in private browsing mode, Mozilla can detect a MITM attack,
>> the user should be warned with a high-visibility warning.
>

> Er, if we could detect an MITM attack in any mode, there should be a
> high visibility warning. :-)

In a corporate setting, intercepting proxies are fairly common, and
displaying a warning would be annoying to users. (Didn't some browser
vendor already try that?)

--
Florian Weimer / Red Hat Product Security Team

[Gervase Markham]

On 02/04/13 12:20, Florian Weimer wrote:
> In a corporate setting, intercepting proxies are fairly common, and
> displaying a warning would be annoying to users. (Didn't some browser
> vendor already try that?)

It depends what UI you use. For example, if we had a red padlock... ;-)

Seriously, that seems to be saying "users don't want to be bothered with
the fact that their connection to their bank is being MITMed". I'm
really not sure that's true. They may know in theory that network ops
has this ability, but that's easy to forget; much better, when they have
a lapse of memory and visit hsbc.com on their work computer, to have
some sort of alert.

Gerv

[ianG]

I think a big issue that is going on here is that the classic security
UI (aka secure browsing aka SSL) has been dumbed down so much that any
fiddling around with it causes all sorts of questions to loom large.

Initially I thought that the idea was really good. The private browsing
mode is a good signal of desired greater security. And the MITM
possibility is a classic security threat.

But the more I thought about it, the more I felt uncomfortable tying the
two together. Private browsing is more about anonymity in ones local
community, not the end point. If one is concerned about MITMs or
eavesdropping, then one is more likely to use Tor. When I think about
the uses and people doing this, cleaning the person's browser history
and traces comes to mind -- which directly clashes with SSL which sticks
a great big red thumb out there *and* reveals the target domain.

Whereas secure browsing is more about knowing that one is connected to
ones exact endpoint, and nobody can interfere with that. In today's
threat scenario, the only thing you care about is that you are connected
to your online bank, and nobody can interfere with that.

These are two different statements. In one, I don't care what anyone
can see or do, as long as nobody knows it's me. Privacy. In the other,
I don't care who knows what I'm doing, as long as they cannot interfere.

Obviously it is nice to have both, but that's much harder, that's dreamland.


iang

On 4/04/13 11:53 AM, Gervase Markham wrote:
> On 02/04/13 12:20, Florian Weimer wrote:

>> In a corporate setting, intercepting proxies are fairly common, and
>> displaying a warning would be annoying to users. (Didn't some browser
>> vendor already try that?)
>

> It depends what UI you use. For example, if we had a red padlock... ;-)
>
> Seriously, that seems to be saying "users don't want to be bothered with
> the fact that their connection to their bank is being MITMed". I'm
> really not sure that's true. They may know in theory that network ops
> has this ability, but that's easy to forget; much better, when they have
> a lapse of memory and visit hsbc.com on their work computer, to have
> some sort of alert.
>
> Gerv
>

> _______________________________________________
> dev-security mailing list
> dev-se...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security
>

[John Nagle]

On 4/4/2013 1:53 AM, Gervase Markham wrote:
> On 02/04/13 12:20, Florian Weimer wrote:
>> In a corporate setting, intercepting proxies are fairly common, and
>> displaying a warning would be annoying to users. (Didn't some browser
>> vendor already try that?)
>
> It depends what UI you use. For example, if we had a red padlock... ;-)
>
> Seriously, that seems to be saying "users don't want to be bothered with
> the fact that their connection to their bank is being MITMed". I'm
> really not sure that's true.

Agreed.

If it were up to me, I'd put a red band across the top of the
window, with

"Your communications are being monitored at
"firewall3.lehman.com" [146.127.226.4] as authorized by
a security certificate on your machine installed by "administrator".

Warning bars like that are seen on multi-level secure systems in some
DoD environments. (They say things like "SECRET NOFORN" in that
environment.) There's no way to turn them off.

John Nagle

"Work for mankind, not for the man" - Mozilla billboard.

[Monica Chew]

----- Original Message -----
> But the more I thought about it, the more I felt uncomfortable tying
> the
> two together. Private browsing is more about anonymity in ones local
> community, not the end point.

I agree with Ian. Please see figure 2 from http://www.collinjackson.com/research/private-browsing.pdf for use cases of private browsing mode, then consider how many of those sites support https in the first place.

Thanks,
Monica