When will Google Public DNS implement a smaller EDNS buffer size

[Dmitriy Vladimirov]

Hello, all

we noticed that Linux servers, hosts in our company can not received big answers on DNSKEY queries from Google Public DNS. 

It is sent 3 queries toward Google Public DNS and no answers.

# dig @8.8.8.8 +dnssec +noanswer DNSKEY mylivewallpapers.com

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-16.P2.el7 <<>> @8.8.8.8 +dnssec +noanswer DNSKEY mylivewallpapers.com

; (1 server found)

;; global options: +cmd

;; connection timed out; no servers could be reached

=====

# tcpdump -nnvvv -tttt -i eth0 -s 0 host 8.8.8.8

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes

2021-04-04 14:46:48.197868 IP (tos 0x0, ttl 64, id 7976, offset 0, flags [none], proto UDP (17), length 77)

    10.2.57.10.34332 > 8.8.8.8.53: [bad udp cksum 0x5366 -> 0xdb99!] 61079+ [1au] DNSKEY? mylivewallpapers.com. ar: . OPT UDPsize=4096 DO (49)

2021-04-04 14:46:53.197824 IP (tos 0x0, ttl 64, id 9149, offset 0, flags [none], proto UDP (17), length 77)

    10.2.57.10.34332 > 8.8.8.8.53: [bad udp cksum 0x5366 -> 0xdb99!] 61079+ [1au] DNSKEY? mylivewallpapers.com. ar: . OPT UDPsize=4096 DO (49)

2021-04-04 14:46:58.197980 IP (tos 0x0, ttl 64, id 13907, offset 0, flags [none], proto UDP (17), length 77)

    10.2.57.10.34332 > 8.8.8.8.53: [bad udp cksum 0x5366 -> 0xdb99!] 61079+ [1au] DNSKEY? mylivewallpapers.com. ar: . OPT UDPsize=4096 DO (49)

I understand that udp answers from Google Public DNS are too big (4096 byte), fragmented and dropped by firewalls rules in ISPs network routers. To drop fragmented udp packets is good security practice against DDOS attacks. I found DNS flag day 2020 meeting. It is recommended to implement on DNS side for preventing udp fragmentation maximum edns-buffer-size: 1232

Also I found draft
https://tools.ietf.org/id/draft-ietf-dnsop-avoid-fragmentation-01.html

I can say that Cloudflare is not suffered from fragmentation.
I see switching from udp to tcp after first answer from 1.1.1.1

# dig @1.1.1.1 +dnssec +noanswer DNSKEY mylivewallpapers.com

;; Truncated, retrying in TCP mode.

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-16.P2.el7 <<>> @1.1.1.1 +dnssec +noanswer DNSKEY mylivewallpapers.com

; (1 server found)

;; global options: +cmd

;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43971

;; flags: qr rd ra ad; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:

; EDNS: version: 0, flags: do; udp: 1232

;; QUESTION SECTION:

;mylivewallpapers.com.          IN      DNSKEY

;; Query time: 26 msec

;; SERVER: 1.1.1.1#53(1.1.1.1)

;; WHEN: Sun Apr 04 14:50:43 MSK 2021

;; MSG SIZE  rcvd: 2313

=====

# tcpdump -nnvv -tttt -i eth0 -s 0 host 1.1.1.1

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes

2021-04-04 14:50:43.718177 IP (tos 0x0, ttl 64, id 13505, offset 0, flags [none], proto UDP (17), length 77)

    10.2.57.10.49602 > 1.1.1.1.53: [bad udp cksum 0x4558 -> 0x7479!] 10272+ [1au] DNSKEY? mylivewallpapers.com. ar: . OPT UDPsize=4096 DO (49)

2021-04-04 14:50:43.743828 IP (tos 0xb8, ttl 52, id 13411, offset 0, flags [DF], proto UDP (17), length 66)

    1.1.1.1.53 > 10.2.57.10.49602: [udp sum ok] 10272| q: DNSKEY? mylivewallpapers.com. 0/0/0 (38)

2021-04-04 14:50:43.744262 IP (tos 0x0, ttl 64, id 9576, offset 0, flags [DF], proto TCP (6), length 60)

    10.2.57.10.49971 > 1.1.1.1.53: Flags [S], cksum 0x453c (incorrect -> 0xf14d), seq 1115636626, win 64240, options [mss 1460,sackOK,TS val 3070826032 ecr 0,nop,wscale 7], length 0

2021-04-04 14:50:43.771540 IP (tos 0xb8, ttl 52, id 0, offset 0, flags [DF], proto TCP (6), length 52)

    1.1.1.1.53 > 10.2.57.10.49971: Flags [S.], cksum 0x0dff (correct), seq 3487892113, ack 1115636627, win 65535, options [mss 1460,nop,nop,sackOK,nop,wscale 10], length 0

2021-04-04 14:50:43.771583 IP (tos 0x0, ttl 64, id 9577, offset 0, flags [DF], proto TCP (6), length 40)

    10.2.57.10.49971 > 1.1.1.1.53: Flags [.], cksum 0x4528 (incorrect -> 0x4cde), seq 1, ack 1, win 502, length 0

2021-04-04 14:50:43.771686 IP (tos 0x0, ttl 64, id 9578, offset 0, flags [DF], proto TCP (6), length 91)

    10.2.57.10.49971 > 1.1.1.1.53: Flags [P.], cksum 0x455b (incorrect -> 0x44d1), seq 1:52, ack 1, win 502, length 5143971+ [1au] DNSKEY? mylivewallpapers.com. ar: . OPT UDPsize=4096 DO (49)

2021-04-04 14:50:43.796961 IP (tos 0xb8, ttl 52, id 56636, offset 0, flags [DF], proto TCP (6), length 40)

    1.1.1.1.53 > 10.2.57.10.49971: Flags [.], cksum 0x4e61 (correct), seq 1, ack 52, win 64, length 0

2021-04-04 14:50:43.797649 IP (tos 0xb8, ttl 52, id 56637, offset 0, flags [DF], proto TCP (6), length 1500)

    1.1.1.1.53 > 10.2.57.10.49971: Flags [.], cksum 0x2444 (correct), seq 1:1461, ack 52, win 64, length 146043971$ q: DNSKEY? mylivewallpapers.com. 6/0/1 mylivewallpapers.com. DNSKEY, mylivewallpapers.com. DNSKEY, mylivewallpapers.com. DNSKEY, mylivewallpapers.com. RRSIG, mylivewallpapers.com. RRSIG[|domain]

2021-04-04 14:50:43.797671 IP (tos 0x0, ttl 64, id 9579, offset 0, flags [DF], proto TCP (6), length 40)

    10.2.57.10.49971 > 1.1.1.1.53: Flags [.], cksum 0x4528 (incorrect -> 0x46f8), seq 52, ack 1461, win 501, length 0

2021-04-04 14:50:43.797706 IP (tos 0xb8, ttl 52, id 56638, offset 0, flags [DF], proto TCP (6), length 895)

    1.1.1.1.53 > 10.2.57.10.49971: Flags [P.], cksum 0x6e8b (correct), seq 1461:2316, ack 52, win 64, length 8554205 zoneRef+ [b2&3=0x796c] [25975a] [26998q] [24940n] [27760au][|domain]

2021-04-04 14:50:43.797717 IP (tos 0x0, ttl 64, id 9580, offset 0, flags [DF], proto TCP (6), length 40)

    10.2.57.10.49971 > 1.1.1.1.53: Flags [.], cksum 0x4528 (incorrect -> 0x43a6), seq 52, ack 2316, win 496, length 0

2021-04-04 14:50:43.798481 IP (tos 0x0, ttl 64, id 9581, offset 0, flags [DF], proto TCP (6), length 40)

    10.2.57.10.49971 > 1.1.1.1.53: Flags [F.], cksum 0x4528 (incorrect -> 0x43a0), seq 52, ack 2316, win 501, length 0

2021-04-04 14:50:43.825175 IP (tos 0xb8, ttl 52, id 56639, offset 0, flags [DF], proto TCP (6), length 40)

    1.1.1.1.53 > 10.2.57.10.49971: Flags [F.], cksum 0x4554 (correct), seq 2316, ack 53, win 64, length 0

2021-04-04 14:50:43.825215 IP (tos 0x0, ttl 64, id 9582, offset 0, flags [DF], proto TCP (6), length 40)

    10.2.57.10.49971 > 1.1.1.1.53: Flags [.], cksum 0x4528 (incorrect -> 0x439f), seq 53, ack 2317, win 501, length 0

I tried the same queries on Linux and MacOS host machines on several ISPs and results was identical - queries to Cloudflare DNS are successful,  to Google DNS are not.
Of course, I can use helpful options +tcp  +bufsize=2312 for utility dig,
but I did not found options to limit buffer size in Linux - etc/resolv.conf 
This options is configured on DNS servers side.

That is why I am interested offers for workaround and opinion about this situation. Can we await for implementing DNS flag day 2020 recommendation in Google Public DNS, because problem will not disappear -  DNS answers will be bigger and bigger and asking all ISPs to allow fragmented udp packets is not good idea in the world of DDOS attacks.

Thanks

/Dmitiy

[Alex Dupuy]

Dmitiy wrote:

we noticed that Linux servers, hosts in our company can not received big answers on DNSKEY queries from Google Public DNS. 

It is sent 3 queries toward Google Public DNS and no answers.

 

I understand that udp answers from Google Public DNS are too big (4096 byte), fragmented and dropped by firewalls rules in ISPs network routers. To drop fragmented udp packets is good security practice against DDOS attacks. I found DNS flag day 2020 meeting. It is recommended to implement on DNS side for preventing udp fragmentation maximum edns-buffer-size: 1232

Also I found draft
https://tools.ietf.org/id/draft-ietf-dnsop-avoid-fragmentation-01.html

I can say that Cloudflare is not suffered from fragmentation.
I see switching from udp to tcp after first answer from 1.1.1.1

# dig @1.1.1.1 +dnssec +noanswer DNSKEY mylivewallpapers.com

;; Truncated, retrying in TCP mode.

 

I tried the same queries on Linux and MacOS host machines on several ISPs and results was identical - queries to Cloudflare DNS are successful,  to Google DNS are not.

Of course, I can use helpful options +tcp  +bufsize=2312 for utility dig,
but I did not found options to limit buffer size in Linux - etc/resolv.conf 
This options is configured on DNS servers side.

The EDNS buffer size offer from the client can only be configured on the client side. Some DNS authoritative servers and resolvers set their own maximum DNS response size, and as long as they don't send a response larger than the client's requested maximum, such servers (like Cloudflare) are not wrong to do so. But neither are server-side maximums required or even recommended in DNS flag day documents. The recommendations are that clients and intermediate resolvers should set the EDNS UDP buffer size to 1232. It is true that there is no client configuration for this in resolv.conf, but recent versions of client code have been adapting the smaller buffersize, as seen in lines 43-52 of the libresolv code in https://code.woboq.org/userspace/glibc/resolv/resolv-internal.h.html

That is why I am interested offers for workaround and opinion about this situation. Can we await for implementing DNS flag day 2020 recommendation in Google Public DNS, because problem will not disappear -  DNS answers will be bigger and bigger and asking all ISPs to allow fragmented udp packets is not good idea in the world of DDOS attacks.

If you can upgrade your OSes (or install newer versions of libresolv), that may be the simplest solution for you. If that is not possible, the next best option is to install and run a local DNS proxy or caching name server that does support configurable EDNS UDP buffer size.