bind forwarding to consul for reverse DNS lookup

[Mike Tougeron]

Hi, I'm having some trouble configuring bind to do reverse DNS forwarding to consul and I was hoping someone could help out. 

The reverse DNS works as expected when I query consul directly

[root@vagrant-consul-server vagrant]# dig @127.0.0.1 -p 8600 145.252.168.192.in-addr.arpa

; <<>> DiG 9.10.2-RedHat-9.10.2-0.el6 <<>> @127.0.0.1 -p 8600 145.252.168.192.in-addr.arpa

; (1 server found)

;; global options: +cmd

;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 2912

;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:

;145.252.168.192.in-addr.arpa.      IN      A

;; ANSWER SECTION:

145.252.168.192.in-addr.arpa. 0       IN      PTR     vagrant-consul-server.node.vagrant.consul.

;; Query time: 0 msec

;; SERVER: 127.0.0.1#8600(127.0.0.1)

;; WHEN: Thu Jul 02 15:48:49 UTC 2015

;; MSG SIZE  rcvd: 129

But when I proxy it through bind it fails

[root@vagrant-consul-server vagrant]# dig @127.0.0.1 -p 53 145.252.168.192.in-addr.arpa

; <<>> DiG 9.10.2-RedHat-9.10.2-0.el6 <<>> @127.0.0.1 -p 53 145.252.168.192.in-addr.arpa

; (1 server found)

;; global options: +cmd

;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 2127

;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:

; EDNS: version: 0, flags:; udp: 4096

;; QUESTION SECTION:

;145.252.168.192.in-addr.arpa. IN A

;; Query time: 0 msec

;; SERVER: 127.0.0.1#53(127.0.0.1)

;; WHEN: Thu Jul 02 15:48:56 UTC 2015

;; MSG SIZE  rcvd: 57

Consul logs show that the query was sent to it (twice, once for dig to 8600 directly and the 2nd one via bind port 53)

==> WARNING: BootstrapExpect Mode is specified as 1; this is the same as Bootstrap mode.

==> WARNING: Bootstrap mode enabled! Do not enable unless necessary

==> WARNING: It is highly recommended to set GOMAXPROCS higher than 1

==> Starting raft data migration...

==> Starting Consul agent...

==> Starting Consul agent RPC...

==> Consul agent running!

         Node name: 'vagrant-consul-server'

        Datacenter: 'vagrant'

            Server: true (bootstrap: true)

       Client Addr: 127.0.0.1 (HTTP: 8500, HTTPS: -1, DNS: 8600, RPC: 8400)

      Cluster Addr: 192.168.252.145 (LAN: 8301, WAN: 8302)

    Gossip encrypt: false, RPC-TLS: false, TLS-Incoming: false

             Atlas: <disabled>

==> Log data will now stream in as it occurs:

    2015/07/02 15:44:00 [INFO] serf: EventMemberJoin: vagrant-consul-server 192.168.252.145

    2015/07/02 15:44:00 [INFO] serf: EventMemberJoin: vagrant-consul-server.vagrant 192.168.252.145

    2015/07/02 15:44:00 [INFO] raft: Node at 192.168.252.145:8300 [Follower] entering Follower state

    2015/07/02 15:44:00 [INFO] consul: adding server vagrant-consul-server (Addr: 192.168.252.145:8300) (DC: vagrant)

    2015/07/02 15:44:00 [INFO] consul: adding server vagrant-consul-server.vagrant (Addr: 192.168.252.145:8300) (DC: vagrant)

    2015/07/02 15:44:00 [ERR] agent: failed to sync remote state: No cluster leader

    2015/07/02 15:44:02 [WARN] raft: Heartbeat timeout reached, starting election

    2015/07/02 15:44:02 [INFO] raft: Node at 192.168.252.145:8300 [Candidate] entering Candidate state

    2015/07/02 15:44:02 [DEBUG] raft: Votes needed: 1

    2015/07/02 15:44:02 [DEBUG] raft: Vote granted. Tally: 1

    2015/07/02 15:44:02 [INFO] raft: Election won. Tally: 1

    2015/07/02 15:44:02 [INFO] raft: Node at 192.168.252.145:8300 [Leader] entering Leader state

    2015/07/02 15:44:02 [INFO] raft: Disabling EnableSingleNode (bootstrap)

    2015/07/02 15:44:02 [DEBUG] raft: Node 192.168.252.145:8300 updated peer set (2): [192.168.252.145:8300]

    2015/07/02 15:44:02 [INFO] consul: cluster leadership acquired

    2015/07/02 15:44:02 [INFO] consul: New leader elected: vagrant-consul-server

    2015/07/02 15:44:02 [DEBUG] consul: reset tombstone GC to index 2

    2015/07/02 15:44:02 [INFO] consul: member 'vagrant-consul-server' joined, marking health alive

    2015/07/02 15:44:02 [INFO] agent: Synced service 'consul'

    2015/07/02 15:44:02 [INFO] agent: Synced service 'dns'

    2015/07/02 15:45:06 [DEBUG] agent: Service 'dns' in sync

    2015/07/02 15:45:06 [DEBUG] agent: Service 'consul' in sync

    2015/07/02 15:46:55 [DEBUG] agent: Service 'consul' in sync

    2015/07/02 15:46:55 [DEBUG] agent: Service 'dns' in sync

    2015/07/02 15:47:59 [DEBUG] agent: Service 'dns' in sync

    2015/07/02 15:47:59 [DEBUG] agent: Service 'consul' in sync

    2015/07/02 15:48:49 [DEBUG] dns: request for {145.252.168.192.in-addr.arpa. 1 1} (122.909µs)

    2015/07/02 15:48:56 [DEBUG] dns: request for {145.252.168.192.in-addr.arpa. 1 1} (94.881µs)

and the bind logs show an error

FORMERR resolving '145.252.168.192.in-addr.arpa/A/IN': 127.0.0.1#8600

Here are the versions I'm using & the configs:

OS: CentOS 6.4

Bind: 9.8.2 (I also tried it with 9.10.2)

Consul: 0.5.2

/etc/named.conf

//

// named.conf

//

// Provided by Red Hat bind package to configure the ISC BIND named(8) DNS

// server as a caching only nameserver (as a localhost DNS resolver only).

//

// See /usr/share/doc/bind*/sample/ for example named configuration files.

//

options {

  listen-on port 53 { any; };

  listen-on-v6 port 53 { ::1; };

  directory "/var/named";

  dump-file "/var/named/data/cache_dump.db";

  statistics-file "/var/named/data/named_stats.txt";

  memstatistics-file "/var/named/data/named_mem_stats.txt";

  allow-query { any; };

  recursion yes;

  allow-recursion { any; };

  dnssec-enable no;

  dnssec-validation no;

  dnssec-lookaside auto;

  /* Path to ISC DLV key */

  bindkeys-file "/etc/named.iscdlv.key";

  managed-keys-directory "/var/named/dynamic";

};

logging {

  channel default_debug {

    file "data/named.run";

    severity dynamic;

  };

};

#zone "." IN {

#  type hint;

#  file "named.ca";

#};

include "/etc/named.rfc1912.zones";

include "/etc/named.root.key";

include "/etc/named/consul-zone.conf";

/etc/named/consul-zone.conf

zone "consul" IN {

  type forward;

  forward only;

  forwarders { 

    127.0.0.1 port 8600;

  };

};

zone "252.168.192.in-addr.arpa." IN {

  type forward;

  forward only;

  forwarders { 

    127.0.0.1 port 8600;

  };

};

/etc/consul.d/config.json

{

  "datacenter": "vagrant",

  "domain": "consul",

  "log_level": "trace",

  "disable_remote_exec": true,

  "disable_update_check": true,

  "advertise_addr": "192.168.252.145",

  "server": true,

  "recursors": [

    "10.REDACTED",

    "10.REDACTED"

  ],

  "bootstrap_expect": 1,

  "dns_config": {

    "node_ttl": "5s",

    "allow_stale": true,

    "max_stale": "5s"

  },

  "data_dir": "/var/lib/consul"

  "service": {

    "name": "dns",

    "tags": [

      "vagrantdns",

      "dns"

    ],

    "port": 53

  }

}

I've tried several different config settings in bind but no luck. At this point I can't tell if I'm doing something wrong or if there is a conflict between the response that consul is providing vs what bind is expecting.

Thanks,

Mike

[Armon Dadgar]

Hey Mike,

Can you crank up the debug logging on BIND to get a better idea of the error?

It’s possible there is something that is being missed, like maybe an SOA record

that BIND is expecting that Consul is not sending or is malformed.

Best Regards,

Armon Dadgar

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/consul/issues
IRC: #consul on Freenode
---
You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/consul-tool/23e96bd3-ad9f-4b1e-85a3-c9459df1021f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mike Tougeron]

Looks like bind isn't happy with the response it gets back from consul.

I updated the bind logging section  

logging {

  channel default_debug {

    file "/var/log/named/named.log";

    severity debug 10;

    print-time yes;

    print-category yes;

    print-severity yes;

  };

  category default      { default_debug; };

  category general      { default_debug; };

  category database     { default_debug; };

  category security     { default_debug; };

  category config       { default_debug; };

  category resolver     { default_debug; };

  category xfer-in      { default_debug; };

  category xfer-out     { default_debug; };

  category notify       { default_debug; };

  category client       { default_debug; };

  category unmatched    { default_debug; };

  category queries      { default_debug; };

  category network      { default_debug; };

  category update       { default_debug; };

  category dispatch     { default_debug; };

  category dnssec       { default_debug; };

  category lame-servers { default_debug; };  

};

When I do the dig I get the same response as before:

[root@vagrant-consul-server vagrant]# dig @127.0.0.1 -p 53 145.252.168.192.in-addr.arpa

; <<>> DiG 9.10.2-RedHat-9.10.2-0.el6 <<>> @127.0.0.1 -p 53 145.252.168.192.in-addr.arpa

; (1 server found)

;; global options: +cmd

;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 43090

;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:

; EDNS: version: 0, flags:; udp: 4096

;; QUESTION SECTION:

;145.252.168.192.in-addr.arpa. IN A

;; Query time: 2 msec

;; SERVER: 127.0.0.1#53(127.0.0.1)

;; WHEN: Mon Jul 06 18:30:54 UTC 2015

;; MSG SIZE  rcvd: 57

And the output of the bind log file is:

06-Jul-2015 18:30:54.025 client: debug 3: client 127.0.0.1#52104: UDP request

06-Jul-2015 18:30:54.025 client: debug 5: client 127.0.0.1#52104: using view '_default'

06-Jul-2015 18:30:54.025 security: debug 3: client 127.0.0.1#52104: request is not signed

06-Jul-2015 18:30:54.025 security: debug 3: client 127.0.0.1#52104: recursion available

06-Jul-2015 18:30:54.025 client: debug 3: client 127.0.0.1#52104: query

06-Jul-2015 18:30:54.025 queries: info: client 127.0.0.1#52104 (145.252.168.192.in-addr.arpa): query: 145.252.168.192.in-addr.arpa IN A +E (127.0.0.1)

06-Jul-2015 18:30:54.025 client: debug 10: client 127.0.0.1#52104 (145.252.168.192.in-addr.arpa): ns_client_attach: ref = 1

06-Jul-2015 18:30:54.025 security: debug 3: client 127.0.0.1#52104 (145.252.168.192.in-addr.arpa): query '145.252.168.192.in-addr.arpa/A/IN' approved

06-Jul-2015 18:30:54.025 client: debug 3: client 127.0.0.1#52104 (145.252.168.192.in-addr.arpa): replace

06-Jul-2015 18:30:54.025 general: debug 3: clientmgr @0x7fe476f7d010: get client

06-Jul-2015 18:30:54.025 general: debug 3: clientmgr @0x7fe476f7d010: create new

06-Jul-2015 18:30:54.025 general: debug 3: clientmgr @0x7fe476f7d010: clientmctx

06-Jul-2015 18:30:54.025 client: debug 3: client @0x7fe46c590ed0: create

06-Jul-2015 18:30:54.025 resolver: debug 1: fetch: 145.252.168.192.in-addr.arpa/A

06-Jul-2015 18:30:54.025 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): create

06-Jul-2015 18:30:54.025 resolver: debug 10: log_ns_ttl: fctx 0x7fe470633010: fctx_create: 145.252.168.192.in-addr.arpa (in '252.168.192.IN-ADDR.ARPA'?): 1 86400

06-Jul-2015 18:30:54.025 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): join

06-Jul-2015 18:30:54.025 resolver: debug 3: fetch 0x7fe470dbd250 (fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A)): created

06-Jul-2015 18:30:54.025 client: debug 3: client @0x7fe46c590ed0: udprecv

06-Jul-2015 18:30:54.025 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): start

06-Jul-2015 18:30:54.025 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): try

06-Jul-2015 18:30:54.025 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): cancelqueries

06-Jul-2015 18:30:54.025 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): getaddresses

06-Jul-2015 18:30:54.025 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): query

06-Jul-2015 18:30:54.025 resolver: debug 3: resquery 0x7fe470639010 (fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A)): send

06-Jul-2015 18:30:54.025 resolver: debug 3: resquery 0x7fe470639010 (fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A)): sent

06-Jul-2015 18:30:54.025 resolver: debug 3: resquery 0x7fe470639010 (fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A)): udpconnected

06-Jul-2015 18:30:54.025 resolver: debug 3: resquery 0x7fe470639010 (fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A)): senddone

06-Jul-2015 18:30:54.026 resolver: debug 3: resquery 0x7fe470639010 (fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A)): response

06-Jul-2015 18:30:54.027 resolver: debug 10: received packet:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id:  14492

;; flags: qr aa rd ra; QUESTION: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:

;145.252.168.192.in-addr.arpa. IN A

;; ANSWER SECTION:

;145.252.168.192.in-addr.arpa. 0 IN PTR vagrant-consul-server.node.vagrant.consul.

06-Jul-2015 18:30:54.027 resolver: debug 3: received packet from 127.0.0.1#8600 (no opt):

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id:  14492

;; flags: qr aa rd ra; QUESTION: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:

;145.252.168.192.in-addr.arpa. IN A

;; ANSWER SECTION:

145.252.168.192.in-addr.arpa. 0 IN PTR vagrant-consul-server.node.vagrant.consul.

06-Jul-2015 18:30:54.027 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): answer_response

06-Jul-2015 18:30:54.027 resolver: debug 1: DNS format error from 127.0.0.1#8600 resolving 145.252.168.192.in-addr.arpa/A for client 127.0.0.1#52104: reply has no answer

06-Jul-2015 18:30:54.027 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): cancelquery

06-Jul-2015 18:30:54.027 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): add_bad

06-Jul-2015 18:30:54.027 lame-servers: info: FORMERR resolving '145.252.168.192.in-addr.arpa/A/IN': 127.0.0.1#8600

06-Jul-2015 18:30:54.027 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): try

06-Jul-2015 18:30:54.027 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): cancelqueries

06-Jul-2015 18:30:54.027 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): getaddresses

06-Jul-2015 18:30:54.027 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): no addresses

06-Jul-2015 18:30:54.027 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): done

06-Jul-2015 18:30:54.027 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): stopeverything

06-Jul-2015 18:30:54.027 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): cancelqueries

06-Jul-2015 18:30:54.027 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): sendevents

06-Jul-2015 18:30:54.027 query-errors: debug 1: client 127.0.0.1#52104 (145.252.168.192.in-addr.arpa): query failed (SERVFAIL) for 145.252.168.192.in-addr.arpa/IN/A at query.c:7591

06-Jul-2015 18:30:54.027 client: debug 3: client 127.0.0.1#52104 (145.252.168.192.in-addr.arpa): error

06-Jul-2015 18:30:54.027 client: debug 3: client 127.0.0.1#52104 (145.252.168.192.in-addr.arpa): send

06-Jul-2015 18:30:54.027 client: debug 3: client 127.0.0.1#52104 (145.252.168.192.in-addr.arpa): sendto

06-Jul-2015 18:30:54.027 client: debug 3: client 127.0.0.1#52104 (145.252.168.192.in-addr.arpa): senddone

06-Jul-2015 18:30:54.027 client: debug 3: client 127.0.0.1#52104 (145.252.168.192.in-addr.arpa): next

06-Jul-2015 18:30:54.027 client: debug 10: client 127.0.0.1#52104 (145.252.168.192.in-addr.arpa): ns_client_detach: ref = 0

06-Jul-2015 18:30:54.027 client: debug 3: client 127.0.0.1#52104 (145.252.168.192.in-addr.arpa): endrequest

06-Jul-2015 18:30:54.027 query-errors: debug 2: fetch completed at resolver.c:3366 for 145.252.168.192.in-addr.arpa/A in 0.001498: failure/success [domain:252.168.192.IN-ADDR.ARPA,referral:0,restart:2,qrysent:1,timeout:0,lame:0,neterr:0,badresp:1,adberr:0,findfail:0,valfail:0]

06-Jul-2015 18:30:54.027 resolver: debug 3: fetch 0x7fe470dbd250 (fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A)): destroyfetch

06-Jul-2015 18:30:54.027 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): shutdown

06-Jul-2015 18:30:54.027 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): doshutdown

06-Jul-2015 18:30:54.027 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): stopeverything

06-Jul-2015 18:30:54.027 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): cancelqueries

06-Jul-2015 18:30:54.027 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): unlink

06-Jul-2015 18:30:54.027 resolver: debug 3: fctx 0x7fe470633010(145.252.168.192.in-addr.arpa/A): destroy

You can see in the log file that bind got a response from consul but it thinks it is an invalid format. :(

-Mike

[Armon Dadgar]

Hey Mike,

It looks like the “Question” section is asking for an A record, which is probably confusing BIND.

I think the only valid types are ANY or PTR for a reverse lookup, but I could be wrong!

To view this discussion on the web visit https://groups.google.com/d/msgid/consul-tool/680da318-4d02-4e46-bc34-961db334a9cb%40googlegroups.com.

[Mike Tougeron]

Ugh... so I *was* doing it wrong... :(

Doing "dig @127.0.0.1 -p 53 -x 192.168.252.145" worked. Thanks for your help!

-Mike

...

[Xavier Krantz]

Hi Mike,

For PTR records, the dig command need the "-x" flag...

In fact, Consul should not be answering to your query, that is why it is confusing.