Major DNS outage in Google GCE and Public DNS 8.8.8.8 for 4x TLDs

2,678 views
Skip to first unread message

Lukas Tribus

unread,
May 29, 2021, 3:15:10 PM5/29/21
to gce-discussion
There is a massive outage in Google recursive DNS services which affect both public DNS (8.8.8.8 et all), and internal GCP resolution.

This outage is specific to 4 entire TLDs (researched by Antonio Prado)
.us
.biz
.bank
.insurance

This includes domains like google.us and zoom.us.

This is specific the certain regions in Europe, mainly 8.8.8.8 nodes in:
Italy, Switzerland, Austria, Slovenia, partially Germany.

Periodic (failing) DNS resolution of google.us against 8.8.4.4 with a 900s interval from 24 different vantage points in Europe:

It is also specifically noticeable on GCP VM's in Zurich, Switzerland.



The thread on the outages mailing list has additional information:



From Google DNS diagnostics on https://dns.google/we can see that Google nameservers cannot reach .us

{ "AD": false, "CD": true, "Comment": "Name servers did not respond [2001:500:3682::11, 156.154.126.70, 156.154.128.70, 2001:503:e239::3:1, 156.154.125.70, 156.154.124.70, 209.173.58.70, 2001:502:ad09::29, 156.154.127.70].", "Question": [ { "name": "us.", "type": 2 } ], "RA": true, "RD": true, "Status": 2, "TC": false }


This is causing huge outages in central and southern Europe.

Google NOC ticket [Google-383761353] has been filed more than 6 hours ago, while the issue is ongoing for likely more than 24 hours.

It's easily reproducible in the GCP location Zurich, Switzerland and from vantage points within Italy, Switzerland, Austria.


This has been first reported yesterday here:

and:


We are trying to get someone from Google to look into this very serious outage, but we have been completely unsuccessful to even get an acknowledgement so far.


Lukas Tribus

Shamma Nikhat (Cloud Platform Support)

unread,
May 31, 2021, 11:27:16 AM5/31/21
to gce-discussion
Hello,

The issue has been resolved now. For more information you can follow the PIT. 

Reply all
Reply to author
Forward
0 new messages