Implications of new TLDs

[John Nagle]

Many new TLDs appear to be coming. This may affect how Mozilla
interprets input in the URL bar, and has security implications.
Single-word domain names are about to become a common form of
URL. Now the browser has to decide whether "example" is looked
up in DNS or a search engine.

Is DNS always preferred over search? Does this introduce any
security risks? Will other browsers from search engine vendors
(Microsoft and Google) make the same choice? Is adding
".com" to single words and retrying still appropriate behavior?
Should command completion/"suggest" use DNS information?

John Nagle

[Boris Zbarsky]

On 6/4/12 2:10 PM, John Nagle wrote:
> Is DNS always preferred over search?

At the moment, yes. Otherwise lots of intranet stuff that uses search
domains would fail too.

> Does this introduce any security risks?

Could be.

> Will other browsers from search engine vendors
> (Microsoft and Google) make the same choice?

I bet they already do; see intranets.

> Is adding ".com" to single words and retrying still appropriate behavior?

On DNS fail, you mean?

-Boris

[John Nagle]

On 6/4/2012 12:11 PM, Boris Zbarsky wrote:
> On 6/4/12 2:10 PM, John Nagle wrote:
>> Is DNS always preferred over search?
>
> At the moment, yes. Otherwise lots of intranet stuff that uses search
> domains would fail too.

OK, that's good enough for now.

The main change is that, for today's TLDs, few bare TLDs resolve
to an IP address. Corporate TLDs ("facebook", "pepsi", etc.)
probably will resolve to an IP address.

>
>> Does this introduce any security risks?
>
> Could be.

I can see problems appearing with firewalls, caching servers,
and other stuff in the middle with too much state. But that's
not the browser's problem.

John Nagle

[Zack Weinberg]

On 2012-06-04 12:29 PM, John Nagle wrote:
> The main change is that, for today's TLDs, few bare TLDs resolve
> to an IP address. Corporate TLDs ("facebook", "pepsi", etc.)
> probably will resolve to an IP address.

Are you aware of actual plans to allow A records for TLDs? I don't have
links to earlier discussion offhand, but I recall the consensus being
that that shouldn't be allowed at all.

zw

[John Nagle]

Interesting point. I'm not clear on how delegation of new TLDs
will work. Do the root servers host all their TLD-level DNS records,
or are they somehow delegated to a server under the control of the
TLD owner?

John Nagle

[Gervase Markham]

On 04/06/12 19:10, John Nagle wrote:
> Many new TLDs appear to be coming. This may affect how Mozilla
> interprets input in the URL bar, and has security implications.

But to a lesser degree than for Chrome. They use the PSL to determine
what is a URL and what is a search; that may not be scalable going forward.

> Single-word domain names are about to become a common form of
> URL.

IMO, Mozilla should not be in favour of this type of word hijacking.
"www.nike", fine. Bare "nike", no. But then, maybe it's up to the user's
DNS configuration rather than us...

> Is DNS always preferred over search? Does this introduce any
> security risks? Will other browsers from search engine vendors
> (Microsoft and Google) make the same choice?

You'd need to ask them.

> Is adding
> ".com" to single words and retrying still appropriate behavior?

Do we actually do that, without explicit user command?

Gerv

[Boris Zbarsky]

On 6/5/12 12:34 PM, Gervase Markham wrote:
>> Is adding
>> ".com" to single words and retrying still appropriate behavior?
>
> Do we actually do that, without explicit user command?

Yes, on DNS lookup fail. We'll try adding ".com" and "www." (or
equivalent's it's a preference which is localized per-locale iirc).

-Boris

[John Nagle]

On 6/5/2012 9:34 AM, Gervase Markham wrote:
> On 04/06/12 19:10, John Nagle wrote:
>> Single-word domain names are about to become a common form of
>> URL.
>
> IMO, Mozilla should not be in favour of this type of word hijacking.
> "www.nike", fine. Bare "nike", no. But then, maybe it's up to the user's
> DNS configuration rather than us...

It's not theoretical. Some of the country-code TLDs have A
records now. "AC" and "AI", for example, have A records, and web
sites behind then. "AI" resolves to "209.59.119.34",
and really is a valid domain with a web site.

In Firefox 12, putting a plain "ai" into the URL box
results in a Google search. At least the first time.
"ai." (the rarely used rooted domain format) brings up the web
site "ai", which is Offshore Information services. But now that
"ai." has been found as a domain, Firefox will, if asked for
"ai" again, will return the domain, not a Google search.
That's clearly a bug.

Until now, this was mostly a curiosity, but if there's a "FACEBOOK"
TLD, people are going to scream if it behaves like that.

John Nagle

[Johnathan Nightingale]

On Jun 6, 2012, at 2:32 AM, John Nagle wrote:

> On 6/5/2012 9:34 AM, Gervase Markham wrote:

>> On 04/06/12 19:10, John Nagle wrote:
>>> Single-word domain names are about to become a common form of
>>> URL.
>>
>> IMO, Mozilla should not be in favour of this type of word hijacking.
>> "www.nike", fine. Bare "nike", no. But then, maybe it's up to the user's
>> DNS configuration rather than us...
>

> It's not theoretical. Some of the country-code TLDs have A
> records now. "AC" and "AI", for example, have A records, and web
> sites behind then. "AI" resolves to "209.59.119.34",
> and really is a valid domain with a web site.
>
> In Firefox 12, putting a plain "ai" into the URL box
> results in a Google search. At least the first time.
> "ai." (the rarely used rooted domain format) brings up the web
> site "ai", which is Offshore Information services. But now that
> "ai." has been found as a domain, Firefox will, if asked for
> "ai" again, will return the domain, not a Google search.
> That's clearly a bug.
>
> Until now, this was mostly a curiosity, but if there's a "FACEBOOK"
> TLD, people are going to scream if it behaves like that.

Yep. That's a problem. It sounds to me like we want to do the DNS lookup on barewords, even knowing it will fail the vast majority of the time. That slows down round trips for those loads, but I don't think that rooted domain format is something we should expect users to attempt. The other option is to just tell those domain owners "too bad", but I think John's right that we are going to see more, not fewer, of these. The nikes and facebooks of the world will have the .com presence as well, and 302 their users to victory, but I don't know if that's a healthy assumption to keep baking into our software in the general case.

I'd like to know what UX thinks, though, since I'm on the fence myself about whether the solution is worth it to fix the smallness of the problem. John, can you file a bug with this information and I'll add some people to the cc?

J

---
Johnathan Nightingale
Sr. Director of Firefox Engineering
@johnath

[Gervase Markham]

On 06/06/12 16:13, Johnathan Nightingale wrote:
>> Until now, this was mostly a curiosity, but if there's a
>> "FACEBOOK" TLD, people are going to scream if it behaves like
>> that.
>
> Yep. That's a problem.

Unless it never works. If we announce we are not going to support this
hijacking of barewords, and perhaps get other browser vendors to join us
(I suspect Chrome would be keen; this is a problem for them) .

I also think there would be resistance within ICANN to A records at the
root, but I don't have the insight to know who would win that battle.
AIUI, the current docs don't ban them outright, but say that any
additional record types would require a stability review.

It sounds to me like we want to do the DNS
> lookup on barewords, even knowing it will fail the vast majority of
> the time.

I thought we did that anyway?

> That slows down round trips for those loads, but I don't
> think that rooted domain format is something we should expect users
> to attempt.

I agree with that.

Gerv

[Kevin Chadwick]

On Wed, 06 Jun 2012 17:06:45 +0100
Gervase Markham wrote:

> Unless it never works. If we announce we are not going to support this
> hijacking of barewords, and perhaps get other browser vendors to join us
> (I suspect Chrome would be keen; this is a problem for them) .

Where's this come from anyway, seems highly questionable that this has
got past quick contemplation to me??

It would be quite cool to just have a word as a site. OTOH how would
anyone know it was a url verbally or written.

This happens with our .co

what's your address

blah.co

... .uk

just .co

... .com

no, just .co

just .co

yeah...

it's the new company extension, it used to be columbia

oh ok, cool.

I imagine many if not all web forms validation will break for mail
addresses too, some don't allow multiple @s which the spec allows.
Some people had problems sending to our .co even, don't know what
server it was, probably some exchange crap as it was a big company. I
was surprised because it used to be columbias extension.

[John Nagle]

On 6/6/2012 8:13 AM, Johnathan Nightingale wrote:
> On Jun 6, 2012, at 2:32 AM, John Nagle wrote:
>
>> On 6/5/2012 9:34 AM, Gervase Markham wrote:
>>> On 04/06/12 19:10, John Nagle wrote:
>>>> Single-word domain names are about to become a common form of
>>>> URL.

....

>>
>> Until now, this was mostly a curiosity, but if there's a
>> "FACEBOOK" TLD, people are going to scream if it behaves like
>> that.
>
>
> Yep. That's a problem. It sounds to me like we want to do the DNS
> lookup on barewords, even knowing it will fail the vast majority of
> the time. That slows down round trips for those loads, but I don't
> think that rooted domain format is something we should expect users
> to attempt. The other option is to just tell those domain owners "too
> bad", but I think John's right that we are going to see more, not
> fewer, of these. The nikes and facebooks of the world will have the
> .com presence as well, and 302 their users to victory, but I don't
> know if that's a healthy assumption to keep baking into our software
> in the general case.

ICANN just released the first list of proposed new TLDs:

http://newgtlds.icann.org/en/program-status/application-results/strings-1200utc-13jun12-en

"FACEBOOK" isn't on the list, but "AOL", "AMAZON", "APPLE", "BAIDU",
"BLOOMBERG", "CADILLAC", "CALVINKLEIN", "CAPITALONE", etc. are.
Also "GOOGLE", "MICROSOFT", "YAHOO", and "YANDEX".

新闻 ("News", owned by Xinhua) and 谷歌 (Google) are also on the list.

Many of the TLDs are from domain speculators who want to resell
domains, but a sizable fraction are world-famous brands. Those
will be expected to work as bare words.

John Nagle

[Gervase Markham]

> Many of the TLDs are from domain speculators who want to resell
> domains, but a sizable fraction are world-famous brands. Those
> will be expected to work as bare words.

I'm really not convinced that's so. Do you have any evidence that people
are expecting that, or can you point to the place in the ICANN
guidelines about these TLDs where top-level A records are allowed?

Gerv

[Kevin Chadwick]

> Many of the TLDs are from domain speculators who want to resell
> domains, but a sizable fraction are world-famous brands.

I'm surprised at that at £118,000 and $25,000 per domain per year.

You can become a registrar of .co.uk for £160 per year.

Apparently they are non profit but there may be excesses over
handling costs, you got that right and this is about increasing
competition, yeah right.

In fact it will likely reduce competition because small firms won't
understand why their forms are having problems and won't be able to
afford to fix them or use these domains for years. I can fix any filters
I have but I'm sure it will make me think some obscenities about
ICANNT and I may refuse to.

has this actually been sensibly discussed by ICANN??


"there is insufficient space in the existing gTLD spaces"

What language are they speaking, there's a plethora including .ltd.uk
that requires a registered company name?? In fact you would struggle to
type in a word and have them all taken.


ICANN has received several petitions against the launch of the new gTLD
programme from the National Consumers League, the Association of
National Advertisers, the Coalition for Responsible Internet Domain
Oversight, the Federal Trade Commission, U.S. Congress as well as well
several brand owners such as Adobe Systems Incorporated, American
Express, The Coca-Cola Company, Dell Inc., Johnson & Johnson

Did they not listen, Greed?

What's the best way to show your disapproval?

I wouldn't be surprised if many simply block these sites.

--
Why not do something good every day and install BOINC.

[John Nagle]

Top-level A records are already allowed. Try

http://ai/

John Nagle

[Kevin Chadwick]

> Top-level A records are already allowed. Try
>
> http://ai/

Hmm GTLDs FAQ says one of the requirements is a minimum of three
letters so from what registrar is this?


________________________________________________________

Why not do something good every day and install BOINC.

________________________________________________________

[ianG]

On 15/06/12 09:18 AM, Kevin Chadwick wrote:
>> Top-level A records are already allowed. Try
>>
>> http://ai/
>
> Hmm GTLDs FAQ says one of the requirements is a minimum of three
> letters so from what registrar is this?

AI is the island of Anguilla, population of around 10,000. Vince Cate
started operating its TLD since around 1995. Vince has always done
interesting things with the DNS, so it is no surprise he's listed the
above redirect (if that's what it is).

The island would be much bigger than it is but it suffered under the
yolk of Cable and Wireless, a hangover from the British colonial days.
In those days, C&W just milked the place for what it could get, and
brought any development to a halt.

iang

ps; disclaimer, I lived next door to Vince and his TLD for 4 years or so
while building crypto stuff.

[Gervase Markham]

On 14/06/12 19:55, John Nagle wrote:
> Top-level A records are already allowed. Try
>
> http://ai/

The CCTLDs have a different arrangement with ICANN from the GTLDs. ICANN
has a lot less control over them. Can you find a GTLD where there is a
top-level A record?

Gerv

[John Nagle]

That's a discussion for another forum. We're not determining
gTLD policy here. Just figuring out what the browser has to do
with what's out there in DNS and what will have to be dealt with.

John Nagle

[Zack Weinberg]

I disagree; this is the place to work out whether it is Mozilla's
opinion that top-level A records are inappropriate. We have the
technical ability to refuse to honor such records, and the political
standing to make a case against them at ICANN, if we think that is the
right course of action.

I have remembered the security case for not honoring top-level A
records: it has to do with abbreviated DNS names used in intranets.
Suppose http://ai.example.com/ is an internal-use-only server for
example.com employees, whose computers have all been configured to retry
NXDOMAINs by tacking '.example.com' on the end, and a great deal of
internal URLs are therefore written http://ai/whatever. But the retry
only happens if an A query for ai. returns NXDOMAIN; a public A query
for ai. that returns an address will *supersede* the expected behavior
and redirect intended-to-be-private HTTP requests to the external
server. Depending on what the internal server does, this could cause a
disastrous data leak.

(Yes, example.com's sysadmins *can* prevent this by correctly
configuring their internal DNS and their firewall, but they probably
haven't.)

zw

[Gervase Markham]

On 16/06/12 15:46, Zack Weinberg wrote:
> I have remembered the security case for not honoring top-level A
> records: it has to do with abbreviated DNS names used in intranets.
> Suppose http://ai.example.com/ is an internal-use-only server for
> example.com employees, whose computers have all been configured to retry
> NXDOMAINs by tacking '.example.com' on the end,

Is that the common "search suffix" behaviour, then? A typed domain is
tried and if NXDOMAIN is returned, the suffixed versions are tried? It's
never the case that it tries suffixes first?

? and a great deal of

> internal URLs are therefore written http://ai/whatever. But the retry
> only happens if an A query for ai. returns NXDOMAIN; a public A query
> for ai. that returns an address will *supersede* the expected behavior
> and redirect intended-to-be-private HTTP requests to the external
> server. Depending on what the internal server does, this could cause a
> disastrous data leak.

Isn't this also a problem for http://foo.corp/ if "corp" gets registered
as a TLD?

Has ICANN 'reserved' some suffixes for internal use which it guarantees
will never be TLDs, to allow smart network admins to avoid this problem?

Gerv

[Zack Weinberg]

On 2012-06-18 3:48 AM, Gervase Markham wrote:
> On 16/06/12 15:46, Zack Weinberg wrote:
>> I have remembered the security case for not honoring top-level A
>> records: it has to do with abbreviated DNS names used in intranets.
>> Suppose http://ai.example.com/ is an internal-use-only server for
>> example.com employees, whose computers have all been configured to retry
>> NXDOMAINs by tacking '.example.com' on the end,
>
> Is that the common "search suffix" behaviour, then? A typed domain is
> tried and if NXDOMAIN is returned, the suffixed versions are tried? It's
> never the case that it tries suffixes first?

Yes, that is how suffix search has always worked AFAIK.

>> a public A query
>> for ai. that returns an address will *supersede* the expected behavior
>> and redirect intended-to-be-private HTTP requests to the external
>> server. Depending on what the internal server does, this could cause a
>> disastrous data leak.
>
> Isn't this also a problem for http://foo.corp/ if "corp" gets registered
> as a TLD?

Yes; however it is my impression (based on nothing terribly concrete)
that this is much rarer than http://foo/ and http://foo.corp.company.com/ .

You could make a case that the right fix here is to disable suffix
search, but I imagine that would not go over well with the large
intranet installations that are using it.

> Has ICANN 'reserved' some suffixes for internal use which it guarantees
> will never be TLDs, to allow smart network admins to avoid this problem?

I don't know.

zw

[Boris Zbarsky]

> Has ICANN 'reserved' some suffixes for internal use which it guarantees
> will never be TLDs, to allow smart network admins to avoid this problem?

As far as I know, no.

The only reserved TLDs, to my knowledge, are "example", "invalid",
"localhost", and "test".

There are various application-specific TLDs in use, however. See
http://en.wikipedia.org/wiki/Pseudo-top-level_domain and the "Although
they have no official status, they are generally regarded as having been
unofficially grandfathered, and are unlikely ever to be allocated as
top-level domains." comment. Of course trying to use one of those will
fail if the relevant application is running on your network, so they
don't really work for future-proofing either.

-Boris

[John Nagle]

On 6/18/2012 3:48 AM, Gervase Markham wrote:
> On 16/06/12 15:46, Zack Weinberg wrote:
>> I have remembered the security case for not honoring top-level A
>> records: it has to do with abbreviated DNS names used in intranets.
>> Suppose http://ai.example.com/ is an internal-use-only server for
>> example.com employees, whose computers have all been configured to retry
>> NXDOMAINs by tacking '.example.com' on the end,

Right. But that's handled at the DNS search level. "getaddrinfo",
given "ai", tries

ai - relative to local root, which is usually the
domain of the local machine minus the first
domain label

ai. - relative to global root.

Then Firefox feeds "ai.com" to getaddrinfo, so this gets tried

ai.com - relative to local root

ai.com. - relative to global root

Then Firefox feeds "ai" to the default search engine.

Now, all of this applies only to the first time Firefox
sees "ai". The SECOND time, caching in Firefox affects
the result. You can see this now. If you haven't
tried "ai" in this Firefox browser session, try it, and it will
spill to the search engine. Then try "ai." or "ai/", which
forces a domain search. Now you'll get the domain. Then try
a bare "ai" again, and you'll get the domain, because the
cache in Firefox doesn't do this right.

I think.

Google Chrome has a different (and probably better) system
for resolving this ambiguity - it asks you which one you want.

John Nagle

[Zack Weinberg]

On 2012-06-18 10:51 AM, John Nagle wrote:
> On 6/18/2012 3:48 AM, Gervase Markham wrote:
>> On 16/06/12 15:46, Zack Weinberg wrote:
>>> I have remembered the security case for not honoring top-level A
>>> records: it has to do with abbreviated DNS names used in intranets.
>>> Suppose http://ai.example.com/ is an internal-use-only server for
>>> example.com employees, whose computers have all been configured to retry
>>> NXDOMAINs by tacking '.example.com' on the end,
>
> Right. But that's handled at the DNS search level. "getaddrinfo",
> given "ai", tries
>
> ai - relative to local root, which is usually the
> domain of the local machine minus the first
> domain label
>
> ai. - relative to global root.

Are you *sure* about this? My understanding is that the lookups happen
in exactly the opposite order, i.e. 'ai.' *first*, and
'ai.<localdomain>.' only if that fails.

zw

[Gervase Markham]

On 16/06/12 15:46, Zack Weinberg wrote:

> I disagree; this is the place to work out whether it is Mozilla's
> opinion that top-level A records are inappropriate. We have the
> technical ability to refuse to honor such records,

Can you outline how we might do that? Would we need our own DNS
resolver? (Do we have that already? I know there was talk, for DNSSEC
reasons...)

Gerv

[Zack Weinberg]

I think we do need our own DNS resolver eventually (mostly because
DNSSEC) but it's not necessary for this. We'd just have to refuse to do
the DNS query at all for URLs whose hostname component did not contain a
dot, and/or which was equal to or a suffix of an entry in the public
suffix list.

This would also entail implementing our own *suffix search* logic to
replace the logic built into gethostbyname/getaddrinfo, so that we
didn't break the aforementioned intranet sites. I think there's a case
for doing that independent of whether we reject top-level A(AAA)
records: the security problem arises because an external entity changes
the meaning of an organization-internal URL, and we could fix that by
doing suffix search *first*.

(Alternatively we could disable suffix search altogether and see how
much screaming there is.)

zw

[Gervase Markham]

On 19/06/12 17:24, Zack Weinberg wrote:
> I think we do need our own DNS resolver eventually (mostly because
> DNSSEC) but it's not necessary for this. We'd just have to refuse to do
> the DNS query at all for URLs whose hostname component did not contain a
> dot, and/or which was equal to or a suffix of an entry in the public
> suffix list.

Er, I'm confused. If I type "http://email/" into my browser, you are
saying we should refuse to do a DNS query? How do I then reach my
intranet site? I'm fairly sure some intranet sites _only_ have a
single-word name.

> This would also entail implementing our own *suffix search* logic to
> replace the logic built into gethostbyname/getaddrinfo, so that we
> didn't break the aforementioned intranet sites.

Can we tell those calls not to do their own suffix search before they
return their answer?

> I think there's a case
> for doing that independent of whether we reject top-level A(AAA)
> records: the security problem arises because an external entity changes
> the meaning of an organization-internal URL, and we could fix that by
> doing suffix search *first*.

I suspect, with no evidence, that this might break things...

> (Alternatively we could disable suffix search altogether and see how
> much screaming there is.)

Surely ("don't call me Shirley!") it would be enormous amounts of screaming?

Gerv

[Zack Weinberg]

On 2012-06-20 3:44 AM, Gervase Markham wrote:
> On 19/06/12 17:24, Zack Weinberg wrote:
>
> Er, I'm confused. If I type "http://email/" into my browser, you are
> saying we should refuse to do a DNS query? How do I then reach my
> intranet site? I'm fairly sure some intranet sites _only_ have a
> single-word name.

Ugh, you're right; I forgot about /etc/hosts and WINS names.

There might be something clever we can do to detect these, but I'm not
sure what it would be offhand; the operating system APIs I know about
are deliberately designed to hide the details of where the names come
from :-(

>> This would also entail implementing our own *suffix search* logic
>> to replace the logic built into gethostbyname/getaddrinfo, so that
>> we didn't break the aforementioned intranet sites.
>
> Can we tell those calls not to do their own suffix search before
> they return their answer?

Yes, we just stick an extra dot on the end before calling getaddrinfo.

>> I think there's a case for doing that independent of whether we
>> reject top-level A(AAA) records: the security problem arises
>> because an external entity changes the meaning of an
>> organization-internal URL, and we could fix that by doing suffix
>> search *first*.
>
> I suspect, with no evidence, that this might break things...

It's certainly possible, e.g. http://example.cc/ where `cc` is both a
real TLD and an internal subdomain.

I confess I see this as another argument for disabling suffix search
altogether. It breaks *more*, but we get a substantial reduction in
context-dependence of URLs in exchange.

>> (Alternatively we could disable suffix search altogether and see
>> how much screaming there is.)
>
> Surely ("don't call me Shirley!") it would be enormous amounts of
> screaming?

My intuition says it would be large, but perhaps not too large, and I
wouldn't want to do anything without real data.

Which we could collect: instrument the DNS resolver to tell us when the
result we got was from suffix search, count the number of times it
happens, report via Telemetry (we don't record the names, so this should
be plenty anonymous). Algorithm for telling:

rA = getaddrinfo(name + ".");
if (rA) return rA;

rB = getaddrinfo(name);
if (rB)
suffix_search++;
return rB;

No additional overhead in the non-suffix-search case.

zw

[Kevin Chadwick]

> Are you aware of actual plans to allow A records for TLDs? I don't have
> links to earlier discussion offhand, but I recall the consensus being
> that that shouldn't be allowed at all.

I wouldn't be surprised if disallowing that was one of the stipulations
for pressing ahead but the money may have eroded that line. Anyone asked
for up to date clarification?

[John Nagle]

On 6/20/2012 9:54 AM, Kevin Chadwick wrote:
>> Are you aware of actual plans to allow A records for TLDs? I don't have
>> links to earlier discussion offhand, but I recall the consensus being
>> that that shouldn't be allowed at all.

Many of the ccTLDs already have them. I've mentioned "AI". That's
just an example. Here's the full list of TLDs that resolve to IP
addresses for HTTP requests:

AC -- [('193.223.78.210', 80)]
AI -- [('209.59.119.34', 80)]
CM -- [('195.24.205.60', 80)]
DK -- [('193.163.102.24', 80)]
GG -- [('87.117.196.80', 80)]
IO -- [('193.223.78.212', 80)]
JE -- [('87.117.196.80', 80)]
KH -- [('203.223.32.21', 80)]
PN -- [('80.68.93.100', 80)]
SH -- [('193.223.78.211', 80)]
TK -- [('217.119.57.22', 80)]
TM -- [('193.223.78.213', 80)]
TO -- [('216.74.32.107', 80)]
UZ -- [('91.212.89.8', 80)]
VI -- [('193.0.0.198', 80)]
WS -- [('64.70.19.33', 80)]
XN--O3CW4H -- [('203.146.249.130', 80)] (ไทย, the Thai TLD.)

About half of those IP addresses have a live web site behind them.
Others have parking pages or Apache error pages.

John Nagle
SiteTruth

[Dan B.]

Zack Weinberg wrote:
> ...

>
> I confess I see this as another argument for disabling suffix search
> altogether. It breaks *more*, but we get a substantial reduction in
> context-dependence of URLs in exchange.

and someone else mentioned doing name resolution without going through
normal OS service calls.


That sounds like a bad idea: If the browser resolves names differently
from how everything else on the system (host/nslookup, ping, ssh,
mailers, ftp, etc.) resolves names (well, excluding the common browser
case of treating a name that is unresolvable as a host name to be a
search term), that's going to be really confusing.


Daniel

[Boris Zbarsky]

On 6/21/12 12:11 PM, Dan B. wrote:
> and someone else mentioned doing name resolution without going through
> normal OS service calls.
>
>
> That sounds like a bad idea: If the browser resolves names differently
> from how everything else on the system (host/nslookup, ping, ssh,
> mailers, ftp, etc.) resolves names (well, excluding the common browser
> case of treating a name that is unresolvable as a host name to be a
> search term), that's going to be really confusing.

It'll be confusing, but the fact of the matter is that the "OS service
calls" are pretty broken for cases when you might have more than one
hostname to resolve and might care about doing other things at the same
time (like in a browser, say). There's pretty extensive discussion
about the details elsewhere, but I fully expect we will end up with our
own DNS resolver in the next 12-18 months, since none of the OSes in
question are planning to improve the APIs they offer...

-Boris

[Kevin Chadwick]

> It'll be confusing, but the fact of the matter is that the "OS service
> calls" are pretty broken for cases when you might have more than one
> hostname to resolve and might care about doing other things at the same
> time (like in a browser, say)

I can understand why an OS wouldn't listen to this and they would be
right. A domain is exact not fuzzed. As for using your own resolver
that would be an extremely bad move. I can accept somehow overlaying
dnssec as a switch offable hack but only because it isn't available to
many and should be. Many use say unbound or spybot or their own host
file blocking. There's also OS specifics like OpenBSDs resolver allowing
tcp only requests etc.. It would also add to and not reduce firefox's
memory footprint.

Personally I would as is the current situation I believe ignore
sites without a dot and leave it as their problem to provide
an alternative url or suffer reduced hits or disable search from the
url bar, of course chrome won't do that. I've never been a fan of search
from the url bar but then I'm not a fan of dotless urls either. The TLD
should be a group not an entity and I'm annoyed that .org is now allowed
for profit making companies watering this feature down. I'm sure ai
isn't a high traffic money making site needing to serve the non
technical too.

p.s. mozilla uses it's own malloc which is annoying because OpenBSDs is
so much better!!!

[Boris Zbarsky]

On 6/21/12 1:41 PM, Kevin Chadwick wrote:
>> It'll be confusing, but the fact of the matter is that the "OS service
>> calls" are pretty broken for cases when you might have more than one
>> hostname to resolve and might care about doing other things at the same
>> time (like in a browser, say)
>
> I can understand why an OS wouldn't listen to this and they would be
> right. A domain is exact not fuzzed.

I have no idea what you mean there. I'm talking about a situation where
you want to resolve foo.com, bar.cdn.com and resources.foo.com all in
parallel. No fuzzing of any sort.

> As for using your own resolver that would be an extremely bad move

Please go read the long existing discussions on this.

> Many use say unbound or spybot or their own host
> file blocking.

Our own resolver would obviously have to look at the host file, yes.

> It would also add to and not reduce firefox's
> memory footprint.

Maybe. Our own resolver may be less complex than working around all the
issues with the OS resolvers. For example, right now dealing with DNS
resolution involves allocating a threadpool and all that jazz, which
means not only lots of code to manage that but also a good bit of
runtime address space and memory use.

-Boris

[Zack Weinberg]

On 2012-06-21 10:41 AM, Kevin Chadwick wrote:
>
> p.s. mozilla uses it's own malloc which is annoying because OpenBSDs is
> so much better!!!

We've done extensive tests and believe jemalloc is the best available
malloc for the way we use memory; however, I don't know that we've
tested specifically against OpenBSD malloc. In what way is that better?
Can you point to empirical evidence of this?

zw

[Kevin Chadwick]

> >
> > p.s. mozilla uses it's own malloc which is annoying because OpenBSDs is
> > so much better!!!
>

> We've done extensive tests and believe jemalloc is the best available
> malloc for the way we use memory; however, I don't know that we've
> tested specifically against OpenBSD malloc. In what way is that better?
> Can you point to empirical evidence of this?

What were your metrics. Performance and/or security.

I haven't found the link I wanted to give you, but here's some, one may
be out of date now.

http://marc.info/?l=openbsd-misc&m=130687174807002&w=2

http://marc.info/?l=openbsd-misc&m=130688429221978&w=2


Do you mind If I forward this request to the OpenBSD list.
We may get some more accurate and interesting answers than I can give
that way.

[Kevin Chadwick]

> >> It'll be confusing, but the fact of the matter is that the "OS service
> >> calls" are pretty broken for cases when you might have more than one
> >> hostname to resolve and might care about doing other things at the same
> >> time (like in a browser, say)
> >
> > I can understand why an OS wouldn't listen to this and they would be
> > right. A domain is exact not fuzzed.
>

> I have no idea what you mean there. I'm talking about a situation where
> you want to resolve foo.com, bar.cdn.com and resources.foo.com all in
> parallel. No fuzzing of any sort.
>

I was under the impression the problem was dotless hostnames
conflicting with search. I don't see why multiple standard queries has
any bearing, dns queries are cheap even though browsers do far more than
they should pre-emptively by default (disabled in the OpenBSD
firefox port by default after some enthusiastic discussion, shall we
say).

> > As for using your own resolver that would be an extremely bad move
>
> Please go read the long existing discussions on this.

Please point me in the direction. At the moment I can only see bad
things coming from that. I guess which made it a long discussion.
Before dnssec and unbound I used tcp only, udp queries were blocked by
my firewalls. I had a way around it for Windows boxes but I guess an
inbuilt dns resolver would have really annoyed me?? Take a step back
when OpenBSD pioneered good dns randomisation, mozillas dns resolver
wouldn't have done, I guarantee it though the packet filter would have
probably fixed it up again. I just can't see how all situations could
possibly be foreseen. OSs can have multiple resolvers themselves.

[John Nagle]

On 6/21/2012 3:40 PM, Kevin Chadwick wrote:
> I don't see why multiple standard queries has

> any bearing, dns queries are cheap.

No-find TLD queries are surprisingly slow. Try a few.

John Nagle

[Boris Zbarsky]

On 6/21/12 6:40 PM, Kevin Chadwick wrote:
> I was under the impression the problem was dotless hostnames
> conflicting with search.

That's not the problem motivating custom resolvers.

> I don't see why multiple standard queries has
> any bearing, dns queries are cheap

Please go read about the problem space. Seriously. Start with the fact
that the OS DNS resolution services are blocking and don't allow timing
out requests, meaning that to do N DNS lookups without being screwed by
one of them being slow on some DNS server somewhere you need N separate
threads.

> Please point me in the direction.

http://linux.die.net/man/3/gethostbyname

-Boris

[Zack Weinberg]

On 2012-06-21 4:02 PM, Kevin Chadwick wrote:
>>>
>>> p.s. mozilla uses it's own malloc which is annoying because OpenBSDs is
>>> so much better!!!
>>
>> We've done extensive tests and believe jemalloc is the best available
>> malloc for the way we use memory; however, I don't know that we've
>> tested specifically against OpenBSD malloc. In what way is that better?
>> Can you point to empirical evidence of this?
>
> What were your metrics. Performance and/or security.

I wasn't directly involved, but I believe it was almost entirely
performance. In contexts where malloc-related security is an overriding
concern we have our own hacks -- see e.g. nsPresArena.cpp.

> http://marc.info/?l=openbsd-misc&m=130687174807002&w=2
> http://marc.info/?l=openbsd-misc&m=130688429221978&w=2

I could be wrong, but I am under the impression that we do not use
tagged pointers anymore.

> Do you mind If I forward this request to the OpenBSD list.
> We may get some more accurate and interesting answers than I can give
> that way.

Be my guest, but please respect follow-ups to dev.platform.

zw

[Kevin Chadwick]

> > I don't see why multiple standard queries has

> > any bearing, dns queries are cheap.
>
> No-find TLD queries are surprisingly slow. Try a few.

No problem here, maybe a second longer. I can't switch tab quick
enough. drill responds immediately. Do you suggest a method of testing
that?

[Justin Dolske]

On 6/21/12 3:25 PM, Boris Zbarsky wrote:

> Please go read about the problem space. Seriously. Start with the fact
> that the OS DNS resolution services are blocking and don't allow timing
> out requests, meaning that to do N DNS lookups without being screwed by
> one of them being slow on some DNS server somewhere you need N separate
> threads.

And then there's
http://boomswaggerboom.wordpress.com/2011/12/18/improving-dns-performance-in-firefox-for-android/
*sigh*

Justin

[Justin Dolske]

On 6/19/12 9:24 AM, Zack Weinberg wrote:

> I think we do need our own DNS resolver eventually (mostly because
> DNSSEC) but it's not necessary for this. We'd just have to refuse to do
> the DNS query at all for URLs whose hostname component did not contain a
> dot, and/or which was equal to or a suffix of an entry in the public
> suffix list.

There's also the fun case of intranets using things like
"benefits.corp". This was widely deployed at Sun Microsystems, which had
a number of such partially-qualified internal domains (eg .corp, .eng,
..sfbay). Including the confusion-inducing ".ebay" (for "east bay").
Where should "checkit.ebay" go, hmm? :)

I found an old page that references a number of these...
http://www.geocities.ws/lenin_salvador/sun_bookmarks.html

Justin

[ianG]

On 22/06/12 11:19 AM, Justin Dolske wrote:
> On 6/19/12 9:24 AM, Zack Weinberg wrote:
>

>> I think we do need our own DNS resolver eventually (mostly because
>> DNSSEC) but it's not necessary for this. We'd just have to refuse to do
>> the DNS query at all for URLs whose hostname component did not contain a
>> dot, and/or which was equal to or a suffix of an entry in the public
>> suffix list.
>

> There's also the fun case of ...

Oh, and me too - when I was in the business of writing highly secure
client applications, the protocol engines were written to be independent
of DNS (yes, we had our own conversions from domain to IP, and our
competitors did that as well).

I thought we were pretty immune until I discovered 30 second blockages
which was eventually traced to Sun's compulsory crypto engine (JCE). It
was doing default DNS resolves on some random name within its startup
code, hitting up against misconfigured local networking. In effect,
Java was saying that unless you are connected to the net with no funny
business, you're not supposed to use crypto.



iang

[Gervase Markham]

On 20/06/12 17:34, Zack Weinberg wrote:
> Ugh, you're right; I forgot about /etc/hosts and WINS names.
>
> There might be something clever we can do to detect these, but I'm not
> sure what it would be offhand; the operating system APIs I know about
> are deliberately designed to hide the details of where the names come
> from :-(

So our current thought is that we can't technically do anything about
this? But our options may change when we acquire our own DNS resolver?

>> Can we tell those calls not to do their own suffix search before
>> they return their answer?
>
> Yes, we just stick an extra dot on the end before calling getaddrinfo.

And we'd have to find the source of suffixes on each OS so we could
reimplement this functionality.

> It's certainly possible, e.g. http://example.cc/ where `cc` is both a
> real TLD and an internal subdomain.

I suspect few businesses use existing real TLDs as their external
subdomains. However, I also suspect quite a few use future TLDs! There
are lots of short TLAs, e.g. ".aco", ".ads" - I bet there are many
businesses with 3 or 4-letter initials who use them. And I bet a load
use ".corp" (6 applicants) and ".inc" (11 applicants).

> I confess I see this as another argument for disabling suffix search
> altogether. It breaks *more*, but we get a substantial reduction in
> context-dependence of URLs in exchange.

I really don't think breaking people's existing DNS resolution
configuration, and making Firefox inconsistent with all other apps on
the machine, is a goer. Unless someone from the networking team wants to
assert we should look into it...

> Which we could collect: instrument the DNS resolver to tell us when the
> result we got was from suffix search, count the number of times it
> happens, report via Telemetry (we don't record the names, so this should
> be plenty anonymous). Algorithm for telling:
>
> rA = getaddrinfo(name + ".");
> if (rA) return rA;
>
> rB = getaddrinfo(name);
> if (rB)
> suffix_search++;
> return rB;
>
> No additional overhead in the non-suffix-search case.

What figures would make the change acceptable? 1%? 0.1%? 0.0001%?

I suspect some users will never use this feature, and some will use it a
lot (probably without knowing that they are using it).

Gerv

[Daniel Veditz]

On 6/22/12 2:39 AM, Gervase Markham wrote:
> I suspect few businesses use existing real TLDs as their
> external subdomains. However, I also suspect quite a few use

> future TLDs! [...] And I bet a load use ".corp" (6

> applicants) and ".inc" (11 applicants).

We've already seen confusion with some organizations using .int
(internal) and the gTLD .int (intergovernmental) leading to
inappropriate SSL certificates being issued.

-Dan Veditz

[Zack Weinberg]

[I tried to reply to this this morning but it seems to have gotten lost
somewhere. I have had to reconstruct it from memory. Apologies for the
possible double post.]

On 2012-06-22 2:39 AM, Gervase Markham wrote:
> On 20/06/12 17:34, Zack Weinberg wrote:
>> Ugh, you're right; I forgot about /etc/hosts and WINS names.
>>
>> There might be something clever we can do to detect these, but I'm not
>> sure what it would be offhand; the operating system APIs I know about
>> are deliberately designed to hide the details of where the names come
>> from :-(
>
> So our current thought is that we can't technically do anything about
> this? But our options may change when we acquire our own DNS resolver?

Yeah, unless someone is cleverer than me. I wouldn't be at all
surprised if Windows at least did expose appropriate hooks, but they
might be very low-level.

> And we'd have to find the source of suffixes on each OS so we could
> reimplement this functionality.

Right. /etc/resolv.conf is pretty easy to parse, but I think both OSX
and Linux are moving away from it, and I don't know what you do for Windows.

> I really don't think breaking people's existing DNS resolution
> configuration, and making Firefox inconsistent with all other apps on
> the machine, is a goer. Unless someone from the networking team wants to
> assert we should look into it...

It certainly would break a lot -- but probably only for a minority of
users. And I think it might be worth it in the long run, for the sake
of removing this source of ambiguity in the meaning of an
apparently-absolute URL. Host identity is fundamental to the web's
security model, after all.

> What figures would make the change acceptable? 1%? 0.1%? 0.0001%?

I'm not really in a position to assess that.

> I suspect some users will never use this feature, and some will use it a
> lot (probably without knowing that they are using it).

Agree -- probably not unusual in corporate deployments, but nearly
unheard of elsewhere.

zw