Domain names re-encoded incorrectly (here: german eszett)

[hesse...@googlemail.com]

Summary: mail.google.com is sending mails to not addressed domains

Dear Sir/Madam,

The moderator of Google's VRP pleased me to publish a vulnerability #129558305 here. So you can find the original report here, but without the description how it could be misused.

Unfortunately, the e-mail system sends e-mails despite of correct addressees to foreign domains:

1.) Login on mail.google.com (Version online 29.03.2019 21:09 CET)
2.) Create a new e-mail
3.) Enter an address with the eszett-letter (alias sharp-s), i.e. "google-test@a n h e ß dot d e"
4.) send the e-mail
5.) open the sent e-mail
6.) Have a look to the addressee - suddenly it was changed to "google-test@a n h e s s dot d e" instead remaining correctly "google-test@a n h e ß dot d e". The letter eszett has been translated.

Info: https://en.wikipedia.org/wiki/eszett

Problem:
============

- The e-mail is sent to a not addressed domain
- The user isn't aware about this
- It seems that the IDN standard that is available since August 2010 has not yet been implemented after nearly nine years.
Referring to page 28 of the rfc 5894, section 7.2.1 is talking about conversion of "ß" to capital letters, yes. This is only affecting capital written words.
But referring to section 7.2.3 the eszett "should be treated as distinct and protocol-valid character". Even section 4.4 (last paragraph, second sentence) is confirming that the eszett is valid starting IDNA2008.

The German registrar's DeNIC explanation: https://www.denic.de/fragen-antworten/faqs-zu-idns-ss/#faq-385

Expecation:
============

- Domains should be handled by the since 2010 existing standard
- Only this way is ensuring that all e-mails are delivered to the correct addressee. If a host is then invalid the delivery has to be rejected.
- {censored}
- {censored}
- Of course, this also applies to all other characters that may not yet be properly implemented.

Interesting: Directly after sending the e-mail you'll receive a little dark popup on the bottom left. After opening the sent e-mail via this you'll see the addressed domain "a n h e ß". But when you enter the sent folder you are seeing the "a n h e s s".

Mit freundlichen Grüßen
A.

[Alex Dupuy]

Hallo A.

Thanks for your interesting report. I checked with Gmail myself to confirm your report that anheß.de was being converted into anhess.de (it is being converted). However, I'm afraid that this isn't a vulnerability, as I discovered while preparing to open a security bug against Gmail, since the IDNA-2008 rules actually do require this behavior. If you were to try register the anheß.de domain with DENIC today they would tell you that it is already registered, as https://gwhois.org/anhe%C3%9F.de+dns shows the normalized domain name as anhess.de.

I found some more detail at Afilias (operator of .INFO), who write:

According to the IDN technical standards, the "ß" (Eszet) is effectively mapped to "ss" based on the Nameprep mechanism. Therefore, in short, YES, you may register domain names that contain the "ß", but that information, when passed to the registry by your registrar, will essentially be registered as double-s: "ss". 

The eurid.eu site also has a page specifically addressing this, saying: 

The IDNA2008 protocol supports both the German Eszett (ß) and the Greek ending sigma (ς) on input as fully allowed characters. With that said, due to the introduction of the homoglyph bundling mechanism, both characters are part of the homoglyph bundling algorithm, meaning that currently registered domain names containing the characters “ss”, or the Greek normal sigma (σ), prevent domain names with the German Eszett (ß) or Greek ending sigma (ς) from being registered.

More technically, any IDNA domain name is actually registered as an ASCII domain name using Punycode with an xn-- prefix. For example, the IDNA domain ähnlich.de would be converted into xn--hnlich-9ta.de. However, IDNA-2008 specification says that bundled "homoglyphs" (different ways of writing the same characters, such as ligatures for ff, fi, fl, ffi, and ffl in English) must be normalized before Punycode conversion is performed. For the German Eszett, that normalization converts it to two 's' characters (just as Gmail and gwhois.com, and Afilias, and DENIC all do).

The downside of the homoglyph normalization is that there is no way to de-normalize double 's' to ß in domain names since the rules for ß versus ss are complicated and changed in 1996, and in any case apply only to German words, not domain names, which are usually not words and even less often German words. The upside of the normalization is that the ASCII form of anheß.de is anhess.de and not xn--anhe-d6b.de or something like that.

If you really want to see something like an Eszett in your domain name, you could also (try to) register a version of the name with a Greek beta in place of the Eszett, like anheβ.de (xn--anhe-8ld.de), but it is likely to be rejected by a registry as an IDN homograph attack.

Wärmste Grüße.

@alex

[Andre Ess]

Hello, Alex,

Two years ago we had a conversation about this issue and, unfortunately, the "problem" is still active and your mailbox inactive. :/

May I kindly ask you for a short update this way and/or your help where to address the behaviour?

Thank you in advance.

Best regards,

A.