Punycode & nslookup
Kai Szymanski
One of our customers wan't a Domain with "Umlaute" (german special
characters like "ￜ").
Is it correct when i have configured the zone like
zone "xn--umlauttest-z5a0tyc.de" {
type master;
file "master/umlauttestￜￜￜ.de.hosts";
allow-transfer { can_transfer; };
# allow-update { can_update; };
};
and the record like
xn--umlauttest-z5a0tyc.de. IN SOA ns.foobar.de.
hostmaster.foobar.de. (
2009120401 ; Serial
8H ; refresh
4H ; retry
5w6d16h ; expiry
1D ) ; minimum
IN NS ns.foobar.de.
IN NS ns2.foobar.de.
If so: When you enter the Domainname in a Browser: Did the Browser also
encode the url to punycode before asking a nameserver ?
Thanks for your hints!
Best regards,
Kai.
Joe Baptista
As for you question concerning the browser converting the domain to punycode before asking a nameserver - yes that is what some browsers do. I'm not sure why because it must confuse some users when that happens.
regards
joe baptista
Hi!
One of our customers wan't a Domain with "Umlaute" (german special characters like "ä").
Is it correct when i have configured the zone like
zone "xn--umlauttest-z5a0tyc.de" {
type master;
file "master/umlauttestäöü.de.hosts";
allow-transfer { can_transfer; };
# allow-update { can_update; };
};
and the record like
xn--umlauttest-z5a0tyc.de. IN SOA ns.foobar.de. hostmaster.foobar.de. (
2009120401 ; Serial
8H ; refresh
4H ; retry
5w6d16h ; expiry
1D ) ; minimum
IN NS ns.foobar.de.
IN NS ns2.foobar.de.
If so: When you enter the Domainname in a Browser: Did the Browser also encode the url to punycode before asking a nameserver ?
Thanks for your hints!
Best regards,
Kai.
_______________________________________________
bind-users mailing list
bind-...@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users
Kai Szymanski
my "problem" is: I can't test the zone with nslookup (only when i use
the puny-encoded domainname). Also other tools who uses dns to resolv
the entered domainname (like ping www.umlauttestￜￜￜ.de) did'nt work.
So i thought that
1. The User enters a url with Umlauts in browser
2. Browser examine url, "see" that there is umlaut in the domainname, an
encoded it (internal, so the user did'nt see it) to puny code and ask
the default nameserver for the domainname in punycode
Is this correct ?
Best regards,
Kai.
Joe Baptista
Hi Joe,
my "problem" is: I can't test the zone with nslookup (only when i use the puny-encoded domainname). Also other tools who uses dns to resolv the entered domainname (like ping www.umlauttestäöü.de) did'nt work.
I'm not sure what the state of nslookup is these days with respect to support for idn. One would think it should accept umlauts and translate to punycode. I can see how a lack of idn support can cause even programmers confusion.
So i thought that
1. The User enters a url with Umlauts in browser
2. Browser examine url, "see" that there is umlaut in the domainname, an encoded it (internal, so the user did'nt see it) to puny code and ask the default nameserver for the domainname in punycode
Is this correct ?
Thats how it should work. Its more of a cosmetic user thing and if browsers did that much user confusion would be reduced if they could see the idn domain instead of so an xn--* domain. Now there could very well be a browser that does this. I don't know - the last time I tested this was back when Ubuntu 7.?? was released. Don't remember the exact date and the only browser I tested it on was Firefox.
Have you tried other browsers? And what browser(s) have you tested this on.
You have hit on a very important point here.
regards
joe baptista
Chris Buxton
> Hi Joe,
>
> my "problem" is: I can't test the zone with nslookup (only when i use the puny-encoded domainname).
nslookup will only understand IDN if BIND is compiled with that option in the ./configure step.
> Also other tools who uses dns to resolv the entered domainname (like ping www.umlauttestäöü.de) did'nt work.
Other CLI tools will not work.
> So i thought that
>
> 1. The User enters a url with Umlauts in browser
> 2. Browser examine url, "see" that there is umlaut in the domainname, an encoded it (internal, so the user did'nt see it) to puny code and ask the default nameserver for the domainname in punycode
The browser has to understand IDN. Most current browsers do, including (I believe) IE 7 and later, Firefox 2 and later, and Safari 3 and later.
Chris Buxton
Professional Services
Men & Mice
Joe Baptista
nslookup will only understand IDN if BIND is compiled with that option in the ./configure step.
might be a good idea if it was the default option. as idn becomes popular the lack of idn support for the tools will result in confusion.
The browser has to understand IDN. Most current browsers do, including (I believe) IE 7 and later, Firefox 2 and later, and Safari 3 and later.
Does anyone have a list of idn domains? I'd like to try it out.
cheers
joe baptista
Joseph S D Yao
least three [?] major revisions in the BIND line, but also an
identically-named tool on MS Windows. I'm sure the older ones don't
support punycode. I wouldn't know about the MSW tool.
Maybe not a problem with the specific poster, who is presumably using
something from a compile of a current BIND distribution, but a problem
when generalising.
--
/*********************************************************************\
**
** Joe Yao js...@tux.org - Joseph S. D. Yao
**
\*********************************************************************/
Chris Buxton
> On Fri, Dec 4, 2009 at 12:26 PM, Chris Buxton <cbu...@menandmice.com> wrote:
>
>> nslookup will only understand IDN if BIND is compiled with that option in the ./configure step.
>
> might be a good idea if it was the default option. as idn becomes popular the lack of idn support for the tools will result in confusion.
The reason IDN support in the BIND query tools (dig, host, nslookup) is not the default is because it relies on a 3rd party library, which must be installed and configured by the package builder beforehand. This is just like SSL support, needed for DNSSEC and TSIG, except that most operating systems don't already ship with libidnkit.
Kai Szymanski
hi joe,
ok..i will compile bind (resp. the libs) by myself and try it out. Thanks!
What is the way for the future: Should the browser encode idn's into
punycode and send it to the nameserver (like example below) or should
the browser send the un-encoded idn to the nameserver and the nameserver
have to do the "encoding-stuff" ? Or both ?
When i use tcpdump tomonitor the traffic on port 53 and i enter for
example "www.wüstchen.de" i see:
13:16:32.856370 IP kshome-desktop.53700 > speedport.ip.domain: 60227+ A?
mail.xx.de. (33)
13:16:32.857902 IP speedport.ip.domain > kshome-desktop.53700: 60227
1/0/0 A[|domain]
13:16:57.404713 IP kshome-desktop.55215 > speedport.ip.domain: 13265+ A?
www.xn--wrstchen-65a.de. (41)
13:16:57.459098 IP speedport.ip.domain > kshome-desktop.55215: 13265
3/0/0[|domain]
13:16:57.601032 IP kshome-desktop.37413 > speedport.ip.domain: 790+ A?
www.xx.de. (34)
13:16:57.626011 IP speedport.ip.domain > kshome-desktop.37413: 790 1/0/0
A[|domain]
The browser (Firefox 3.0/Linux) seems to encode the entered Domain into
punycode.
When i enter "ping www.würstchen.de" i get:
13:19:35.835977 IP kshome-desktop.58121 > speedport.ip.domain: 10129+ A?
www.wM-CM-<rstchen.de. (35)
Look funny, not puny ;)
@joe: Here some idn's:
www.würstchen.de (Like Bratwurst ;)
www.tür.de (Door)
www.bügeleisen.de (flat iron)
Best regards,
Kai.
JFC Morfin
Hi! Kay,
I take back the entire thread since this is something which really match what is under warm discussion at the IETF WG/IDNSBIS.
Kai Szymanski <k...@codebiz.de> 4 d�cembre 2009 15:41
One of our customers wan't a Domain with "Umlaute" (german special characters like "�").
Is it correct when i have configured the zone like
zone " xn--umlauttest-z5a0tyc.de" {
type master;
file "master/umlauttest���.de.hosts";
allow-transfer { can_transfer; };
# allow-update { can_update; };
};
and the record like
xn--umlauttest-z5a0tyc.de . IN SOA ns.foobar.de. hostmaster.foobar.de. (
2009120401 ; Serial
8H ; refresh
4H ; retry
5w6d16h ; expiry
1D ) ; minimum
IN NS ns.foobar.de.
IN NS ns2.foobar.de.
If so: When you enter the Domainname in a Browser: Did the Browser also encode the url to punycode before asking a nameserver ?
<bapt...@publicroot.org> 4 d�cembre 2009 16:05
As for you question concerning the browser converting the domain to punycode before asking a nameserver - yes that is what some browsers do. I'm not sure why because it must confuse some users when that happens.
This is the IDNA concept. Conversion is to happen in Applications.
Kai Szymanski <k...@codebiz.de> 4 d�cembre 2009 16:23
my "problem" is: I can't test the zone with nslookup (only when i use the puny-encoded domainname). Also other tools who uses dns to resolv the entered domainname (like ping www.umlauttest���.de) did'nt work.
So i thought that
1. The User enters a url with Umlauts in browser
2. Browser examine url, "see" that there is umlaut in the domainname, an encoded it (internal, so the user did'nt see it) to puny code and ask the default nameserver for the domainname in punycode
Is this correct ?
Chris Buxton <cbu...@menandmice.com> 4 d�cembre 2009 18:26
�: Bind Mailing <bind-...@lists.isc.org>
On Dec 4, 2009, at 7:23 AM, Kai Szymanski wrote:
> Hi Joe,
>
> my "problem" is: I can't test the zone with nslookup (only when i use the puny-encoded domainname).
nslookup will only understand IDN if BIND is compiled with that option in the ./configure step.
> Also other tools who uses dns to resolv the entered domainname (like ping www.umlauttest���.de) did'nt work.
Other CLI tools will not work.
> So i thought that
>
> 1. The User enters a url with Umlauts in browser
> 2. Browser examine url, "see" that there is umlaut in the domainname, an encoded it (internal, so the user did'nt see it) to puny code and ask the default nameserver for the domainname in punycode
The browser has to understand IDN. Most current browsers do, including (I believe) IE 7 and later, Firefox 2 and later, and Safari 3 and later.
This is correct. However, beware: since you talk of test. The coming "Fast Track" ICANN project should use IDNA2008 (more versatile but restrict A-labels (xn--) to lower cases). The question is when is IDNA2008 to be released. We hoped this month or January. The present debate on Eszett that raised again at the WG may delay this.
To better understand I started looking in the code where the punycode routine is. Has someone a file name for it?
<bapt...@publicroot.org> 4 d�cembre 2009 19:12
might be a good idea if it was the default option. as idn becomes popular the lack of idn support for the tools will result in confusion.
Yes. But IDNA2008 is going to be much more complex to support for this kind of tool since zone managers may impose their own rules. So, in addition to know if an IDN works, it would be great to know if it is legitimate (TLD zone managers may decide rules, but higher level zone managers to disregard them).
Does anyone have a list of idn domains? I'd like to try it out.
Chris Buxton <cbu...@menandmice.com> 4 d�cembre 2009 20:29
The reason IDN support in the BIND query tools (dig, host, nslookup) is not the default is because it relies on a 3rd party library, which must be installed and configured by the package builder beforehand. This is just like SSL support, needed for DNSSEC and TSIG, except that most operating systems don't already ship with libidnkit.
Do you know the hook? I am just starting investigating the code, and I have C only as a minor :-)
Kai Szymanski <k...@codebiz.de> 5 d�cembre 2009 14:04
What is the way for the future: Should the browser encode idn's into
punycode and send it to the nameserver (like example below) or should
the browser send the un-encoded idn to the nameserver and the nameserver
have to do the "encoding-stuff" ? Or both ?
This a key architectural and network security issue. At this stage only the IDNA principles are documented. They imply (the IDNA name) that the applicatins (browser) should do the work. However, this implies a lot of possible different versions on the same machine. However, this also means the update of 16,5 millions of nameservers round the world. A big problem.
IMHO the current WG/IDNABIS burst is to obtain a clearer definition that the just announced Google DNS service can build on top (Google is _very_ involved in IDNA2008). This might lead punycode and value added punycode services to be placed in DNS front-end. This was discussed in WG/OPES the list is now unfortunately closed and the next document should have considered DNS and OPES.
This should be carefully considered in regards of pollution propagation. Also, we need to know better about DNS resolution in Chrome OS.
As usual in the Internet architecture, the best is to try and test. I look for people interested in just doing that and reporting on results.
jfc
Joseph S D Yao
> Hi Chris,
> hi joe,
>
> ok..i will compile bind (resp. the libs) by myself and try it out. Thanks!
>
> punycode and send it to the nameserver (like example below) or should
> the browser send the un-encoded idn to the nameserver and the nameserver
> have to do the "encoding-stuff" ? Or both ?
My preference would be to have what is entered on the address line and
seen by the human be also what is sent to the resolver. This would
require more changes, though. Second preference would be to have the
standard subroutines that the browser calls do it.
Under no circumstances should either the name server or the browser
proper have to worry about details such as how to encode or decode
different character formats.
Isn't one of Alan Perlis' quotes about, a high-level language is one
where you don't have to worry about unnecessary detail?
Chris Buxton
> What is the way for the future: Should the browser encode idn's into
> punycode and send it to the nameserver (like example below) or should
> the browser send the un-encoded idn to the nameserver and the nameserver
> have to do the "encoding-stuff" ? Or both ?
The theory, as JFC Morfin pointed out, is that applications are supposed to handle it. But most types of net-enabled applications don't - mainly just web browsers.
The most recent Windows stub resolver now handles the encoding on behalf of applications that don't do it themselves. This means that DNS servers (and firewalls) still don't need updates - the encoding is still handled by the client machine - but it also has the advantage that all applications get it automatically unless they deliberately try to validate hostnames using the pre-IDNA rules.
Unfortunately, I don't know of any other operating system stub resolver that does this.
Chris Buxton
> Chris Buxton <cbu...@menandmice.com> 4 décembre 2009 20:29
>
> Do you know the hook? I am just starting investigating the code, and I have C only as a minor :-)
All I know is what you find in the BIND source code directory. For example, with BIND 9.7.0b2:
$ ./configure --help | grep idn
--with-idn=MPREFIX enable IDN support using idnkit default PREFIX
--with-idnlib=ARG specify libidnkit
$ less README.idnkit
jefsey
>On Dec 5, 2009, at 6:34 AM, JFC Morfin wrote:
> >> The reason IDN support in the BIND query tools (dig, host,
> nslookup) is not the default is because it relies on a 3rd party
> library, which must be installed and configured by the package
> builder beforehand. This is just like SSL support, needed for
> DNSSEC and TSIG, except that most operating systems don't already
> ship with libidnkit.
> >
> > Do you know the hook? I am just starting investigating the code,
> and I have C only as a minor :-)
>
>All I know is what you find in the BIND source code directory. For
>example, with BIND 9.7.0b2:
>
>$ ./configure --help | grep idn
> --with-idn=MPREFIX enable IDN support using idnkit default PREFIX
> --with-idnlib=ARG specify libidnkit
>
>$ less README.idnkit
Due to my lack of time and because at the same time I use
BIND/Windows on my XP machine to test my own way to use the DNS in
the IDNA suggested context; I selected 9.5.2.
I understand from quick digging that the IDN libs are of 2003. Since
I mainly want to work on them, I feel this os OK. Except if they have
been updated?
Best.
jfc
jefsey
Thank you for this information.
I am totally new to BIND. Actually I thought that BIND was late under
Windows where the largest end-user need is. I know pretty well the
IDNA side as being architecturally conflicting on several key DNS
usage main issues. I have a "pre-theoretical" architectural solution
but it must now be confronted to experimentation. My approach is
based upon a strict respect of the existing DNS (not a single bit
change). However, I expect that experimentation may change my
"pre-theory", so this "pre-theory" is of no real interest here.
What is of interest is to be able to use BIND's capacities to carry
the experimentation and see what reality has to say. That capacity,
at this stage and as I see it is only to be able to trim the punyplus
module and to get it consistently at the propre place everywhere (I
want to consider IDNs as generallised Internet Domain Names, that
have to be processed in LC ASCII what ever the format the are in the
request or are to be in the response).
To do that I have practical problems :
- lack of time and of competence (I am a medium level C programmer, I
am not familiar with current tools and prefer to work under Windows
where I develop my current other tools)
- I do not know yet the internal functional and organisational
structure of the set of BIND of programs. Is there a document I
should start from?
- the real DNS operational usage change is due to IDNA2008, which
makes plain IDNA characteristics that one did not so much noticed
with IDNA2003. There are no IDNA2008 tools available yet. However the
core module programing may be the same (punycode is not changed,
punycode is only used with lowercases). Filtering and verification of
inputs may apply.
Nameprep is gone.
jfc
Danny Mayer
> On Dec 4, 2009, at 10:12 AM, Joe Baptista wrote:
>
>> On Fri, Dec 4, 2009 at 12:26 PM, Chris Buxton <cbu...@menandmice.com> wrote:
>>
option in the ./configure step.
popular the lack of idn support for the tools will result in confusion.
>
is not the default is because it relies on a 3rd party library, which
must be installed and configured by the package builder beforehand. This
is just like SSL support, needed for DNSSEC and TSIG, except that most
operating systems don't already ship with libidnkit.
>
And I haven't looked for one for Windows so it probably won't work with
Windows.
Danny
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
jefsey
> > The reason IDN support in the BIND query tools (dig, host, nslookup)
>is not the default is because it relies on a 3rd party library, which
>must be installed and configured by the package builder beforehand. This
>is just like SSL support, needed for DNSSEC and TSIG, except that most
>operating systems don't already ship with libidnkit.
> >
>
>And I haven't looked for one for Windows so it probably won't work with
>Windows.
IDNA2008 is an "elementary" solution: i.e. at individual application
level. This means that we now need an IDNA system that will
identically support all the elementary needs of each of the
applications on a machine. And we need to incorporate IDNA in the
network globalilty, meaning making sure the various (ICANN, Google,
Interplus, Microsoft(?), IDNA2003, Unicode approaches) do
interoperate. This is the fundamental issue currently discussed at
the WG/IDNABIS. This cannot be attained in curbing humanity to the
current uncomplete text limitations. This is to be based upon
experimentation, adaptation, and practice of the IDNA2008 proposition.
My current conclusion is that the general solution lies in a
replacement of the punycode implementation (i.e libidnkit) by an
extended punyplus module that will perform along the IDNA2010 wrap-up
exploratory effort that was initiated yesterday
(http://iucg.org/wiki/BUD-IDNA2010). This seems to be the most
natural and the most portable place.
jfc
Danny Mayer
> At 11:06 06/12/2009, Chris Buxton wrote:
>> On Dec 5, 2009, at 6:34 AM, JFC Morfin wrote:
>> > Chris Buxton <cbu...@menandmice.com> 4 d�cembre 2009 20:29
>> nslookup) is not the default is because it relies on a 3rd party
>> library, which must be installed and configured by the package builder
>> beforehand. This is just like SSL support, needed for DNSSEC and TSIG,
>> except that most operating systems don't already ship with libidnkit.
>> >
>> I have C only as a minor :-)
>>
>> All I know is what you find in the BIND source code directory. For
>> example, with BIND 9.7.0b2:
>>
>> $ ./configure --help | grep idn
>> --with-idn=MPREFIX enable IDN support using idnkit default PREFIX
>> --with-idnlib=ARG specify libidnkit
>>
>> $ less README.idnkit
>
> Due to my lack of time and because at the same time I use BIND/Windows
> on my XP machine to test my own way to use the DNS in the IDNA suggested
> context; I selected 9.5.2.
>
Which means spending even more time since BIND/Windows does not yet
support the IDN libs.
Danny