hostname resolving on 8.8.8.8 and 8.8.4.4 on east coast US, but not west coast

[Rory M]

We are noticing that:

• both `one.popinapp.co` and `two.popinapp.co` are resolving on east coast US, using both 8.8.8.8 and 8.8.4.4
• only `one.popinapp.co` is resolving on west coast US using those same DNS servers.  `two.popinapp.co` does not resolve.

pertinent debugging info:

• The date and time you encountered the problem:  noticed 2:27pm PDT on July 22, 2017
• Your location:  Los Angeles, CA.  USA.
• The platform on which you are noticing the problem (e.g. Mac, Windows, router, etc.):  Mac
• The hostname(s) for which you are having a problem:  two.popinapp.co
• Whether the problem is continuous or intermittent:  continuous
• The links to the tools' name server diagnosis report page:  
  
  http://dnsviz.net/d/two.popinapp.co/dnssec/
  http://dnsviz.net/d/one.popinapp.co/dnssec/
  https://intodns.com/popinapp.co

Here is the output from nameserver lookups on the west coast, for two.popinapp.co:

### using Charter dns:  SUCCESS

% nslookup two.popinapp.co

Server:        208.67.222.222

Address:    208.67.222.222#53

Non-authoritative answer:

two.popinapp.co    canonical name = pageserve.co.

Name:    pageserve.co

Address: 107.178.242.45

### using 8.8.8.8 Google dns:  FAILURE

% nslookup two.popinapp.co 8.8.8.8

Server: 8.8.8.8

Address: 8.8.8.8#53

** server can't find two.popinapp.co: NXDOMAIN

### using 8.8.4.4 Google dns:  FAILURE

% nslookup two.popinapp.co 8.8.4.4

Server: 8.8.4.4

Address: 8.8.4.4#53

** server can't find two.popinapp.co: NXDOMAIN

I'm not posting nslookup for one.popinapp.co for both 8.8.8.8 and 8.8.4.4, because both those work fine from my physical location.

Thanks, Rory

[Alex Dupuy]

Google Public DNS was probably serving cached NXDOMAIN results from our resolvers closest to your west coast location, from before you registered the popinapp.co domain (according to https://gwhois.org/popinapp.co, earlier the same day as you reported this problem).

Per RFC 2308, negative answers (NXDOMAIN or "NODATA", which is NOERROR with zero answers) can be cached for the minimum of the TTL of the SOA record for the zone and the "minimum" field in the SOA itself. For the .CO TLD, these are 900 (15 minutes) and 86400 (one day) respectively:

$ dig +multi +norec +nocmd +nostats does-not-exist.co @ns1.cctld.co

;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 65295

;; flags: qr aa; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:

;does-not-exist.co.     IN A

;; AUTHORITY SECTION:

co.                     900 IN SOA ns1.cctld.co. hostmaster.neustar.biz. (

                                2025940967 ; serial

                                900        ; refresh (15 minutes)

                                900        ; retry (15 minutes)

                                604800     ; expire (1 week)

                                86400      ; minimum (1 day)

                                )

However, Google Public DNS also implements an internet draft RFC (Aggressive use of DNSSEC-validated Cache) that allows DNSSEC-validating resolvers like Google Public DNS to synthesize negative responses from cached DNSSEC proofs of non-existence. These DNSSEC proofs are present in the .CO TLD, as well as many other TLDs, although some TLDs do not implement DNSSEC, and others (notably .COM, .INFO, .NET, .ORG, and most European ccTLDs) use NSEC3 for non-existence proofs, and Google Public DNS only implements synthesis of negative responses from cached NSEC records.

Section 5.4 of the draft recommends applying a limit of three hours (TTL 10800) to synthesized negative responses regardless of the negative caching (minimum) TTL, but Google Public DNS does not do that yet. It explicitly mentions only the minimum TTL, and not the TTL of the SOA record itself, so the 15 minute limit does not apply, and as a result, Google Public DNS will synthesize negative responses for a newly created .CO domain for up to a day, if it has relevant NSEC records in its cache.

This issue is not completely eliminated now that your domain has existed for more than a day, and the TTLs for NXDOMAIN returned by the .CO TLD name servers have definitively expired. The popinapp.co zone, as served by CloudFlare, will return NODATA (CloudFlare doesn't return NXDOMAIN for nonexistent domains for reasons related to DNSSEC) and has a negative cache time of one hour, since that is value of both the SOA TTL and the minimum TTL field in the SOA:

$ dig +nocmd +norec +noedns +nostats +multi nonexistent.popinapp.co @curt.ns.cloudflare.com

;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 45318

;; flags: qr aa; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:

;nonexistent.popinapp.co. IN A

;; AUTHORITY SECTION:

popinapp.co.            3600 IN SOA curt.ns.cloudflare.com. dns.cloudflare.com. (

                                2025242458 ; serial

                                10000      ; refresh (2 hours 46 minutes 40 seconds)

                                2400       ; retry (40 minutes)

                                604800     ; expire (1 week)

                                3600       ; minimum (1 hour)

                                )

As noted at https://superuser.com/questions/803767/how-to-set-nx-ttl-in-cloudflare, there is no way to change the negative caching TTL as the CloudFlare API does not allow users to modify the SOA record for a zone in any way.

Synthesis of NODATA responses does not happen because of the extremely limited scope of the CloudFlare NSEC records, so only the normal negative cache TTL applies here.

[ro...@smiletime.com]

Thank you so much for the detailed explanation, Alex.

After struggling with it yesterday (trying to figure out why one hostname was okay, but the other wasn't), today it looks like both hostnames resolve fine on the west coast.

Really appreciate your help.

Cheers, Rory