DNS lookup "dig amq.service.consul" not returning the correct ANSWER with host and port.

[David Johnsson]

Hi,

I'm not able to resolve my new service amq.service.consul after installing consul and dnsmasq. When I enter "dig amq.service.consul" it seems my dns config is not correct because I dont get the same ANSWER SECTION that I get when entering "dig @127.0.0.1 -p 8600 amq.service.consul SRV"

Here I explain the steps I've taken and then show the different outputs from the dig commands.

Essentially I want to be able to call curl http://amq.service.consul/hawtio/jolokia/read/java.lang:type=Memory/HeapMemoryUsage to be able to retrieve java memory metrics. I'm assuming this is possible.

What I've done

1. installed consul (basic install. No conf changes)

2. followed the consul tutorial to create a simple service. Added /etc/consul.d/amq.json 

{"service": {"name": "amq", "tags": ["java"], "port": 61000,

  "check": {"script": "curl localhost:61000 >/dev/null 2>&1", "interval": "10s"}}}

3. installed dnsmasq

4. executed:  echo "server= echo "server=/consul/127.0.0.1#8600" > /etc/dnsmasq.d/10-consul

5. added the following to /etc/resolv.conf nameserver 127.0.0.1 

6. started dnsmasq

7. started consul using: consul agent -server -bootstrap-expect 1 -data-dir /tmp/consul -config-dir /etc/consul.d -ui-dir /var/opt/consul-ui/

The output below compares the dig commands. You'll notice that I dont get the ANSWER SECTION when just specifying dig amq.service.consul SRV. nslookup on amq.service.consul also returns host not found. 

Dig results when pointing at Consul DNS 

[root@ip-10-10-100-10 consul.d]# dig @127.0.0.1 -p 8600 amq.service.consul SRV

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.23.rc1.32.amzn1 <<>> @127.0.0.1 -p 8600 amq.service.consul SRV

; (1 server found)

;; global options: +cmd

;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39308

;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; WARNING: recursion requested but not available

;; QUESTION SECTION:

;amq.service.consul.            IN      SRV

;; ANSWER SECTION:

amq.service.consul.     0       IN      SRV     1 1 61000 ip-10-10-100-10.node.dc1.consul.

;; ADDITIONAL SECTION:

ip-10-10-100-10.node.dc1.consul. 0 IN   A       10.10.100.10

;; Query time: 1 msec

;; SERVER: 127.0.0.1#8600(127.0.0.1)

;; WHEN: Fri Dec  5 00:43:28 2014

;; MSG SIZE  rcvd: 152

Dig results when not specifying the Consul DNS (Notice there is no ANSWER SECTION)

[root@ip-10-10-100-10 consul.d]# dig amq.service.consul

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.23.rc1.32.amzn1 <<>> amq.service.consul

;; global options: +cmd

;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 53548

;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:

;amq.service.consul.            IN      A

;; AUTHORITY SECTION:

.                       3553    IN      SOA     a.root-servers.net. nstld.verisign-grs.com. 2014120401 1800 900 604800 86400

;; Query time: 0 msec

;; SERVER: 127.0.0.1#53(127.0.0.1)

;; WHEN: Fri Dec  5 00:47:57 2014

;; MSG SIZE  rcvd: 111

Thanks heaps!

Dave

[allan bailey]

the dnsmask setting needs to have "server=/consul./127.0.0.1#8600"

Notice the . at the end of the top-level 'consul' domain.

-allan

> --
> You received this message because you are subscribed to the Google Groups
> "Consul" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to consul-tool...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Allan Bailey
zirpu...@gmail.com

There are 2 hard problems in computer science:
caching, naming, off-by-1 errors.

[David Johnsson]

Really? Thats different from what I have read in a number of places. I'll give it a try though. Never say never :)

Thanks

[David Johnsson]

I've essentially followed this guide: http://www.morethanseven.net/2014/04/25/consul/

[David Johnsson]

Hi,

Adding the . in "server=/consul./127.0.0.1#8600" didnt seem to make a difference. 

I must be doing something silly and missing one of the configuration setting somewhere.

Thanks

Dave

On Friday, December 5, 2014 12:10:11 PM UTC+11, allan bailey wrote:

[Alvaro Miranda Aguilera]

Hello, are not you missing the dc in the search?

ping consul.service.dc1.consul

all the steps you need should be

1. make sure the client is asking the proper dns, so /etc/resolv.conf need to be modified to include the ip of the servers where dnsmasq is running

I do that like this:

cat > /etc/resolv.conf <<EOF2

search consul

nameserver 192.168.10.11

nameserver 192.168.10.12

nameserver 10.0.2.3

EOF2

2. you need to tell the dnsmasq server to forward.. I use this:

grep consul /etc/dnsmasq.conf || (echo 'server=/consul/127.0.0.1#8600' | tee -a /etc/dnsmasq.conf && service dnsmasq force-reload)

3. then the services are available like this:

consul.service.dc1.consul.

[David Johnsson]

Thanks Alvaro.

Your suggestions helped me. This is what I did.

updated /etc/resolv.conf to include the search consul and include our domain + name server.

search consul our.domain
nameserver 127.0.0.1
nameserver 10.10.0.2

updated /etc/dnsmasq.conf according to your suggestion below.

restarted dnsmasq

restarted consul agent.

nslookup and dig work now for "amq.service.consul"

[ec2-user@ip-10-10-100-10 ~]$ nslookup amq.service.consul
Server:         127.0.0.1
Address:        127.0.0.1#53

Name:   amq.service.consul
Address: 10.10.100.10

Thank you so much for your help.

[Alvaro Miranda]

hello

on top of that i add nodes.<dc>.consul to search in resolv, so nodes can ping/access each other by name

glad you got it working

[David Lin]

Thanks for the help! I was able to resolve the same issue. How do you do this though:

on top of that i add nodes.<dc>.consul to search in resolv, so nodes can ping/access each other by name

Can you provide an example? Thanks!

[Alvaro Miranda Aguilera]

Hello

Sure,

Basically you can do this:

ping server1.nodes.dc1.consul

rite?

Then, you can add that to the resolv file like this:


/etc/resolv.conf
search consul nodes.dc1.consul our.domain

Do that in the servers/clients on dc1, and they will be able to do:

ping server1

Hope this helps.

Any further question just let us know.

Thanks
Alvaro.

[Calvin Leung Huang]

How can you curl http://amq.service.consul/hawtio/jolokia/read/java.lang:type=Memory/HeapMemoryUsage when the service is registered on port 6100, don't you need a reverse proxy to route it from 80 to 6100?

[sivaram reddy]

Hi Alvaro,

I was unable to ping dns.node.mydc1.internal (Consul server) node.Could you please help to resolve the issue.

My Consul server IP is something like this 10.21.4.24 and Client1 is 10.21.4.26 and Client2 is 10.21.4.27.

What are the configuration changes(both in Server and clinet) that i have to do in resolv.conf and dnsmasq.conf ..so that i can do ping dns.node.mydc1.internal. and client node is role.node.mydc1.internal .

Thanks in Advance.

Regards,

Sivaram.M

[James Phillips]

Hi Sivaram,

Have you checked out this DNS guide - https://www.consul.io/docs/guides/forwarding.html. This might be helpful in configuring dnsmasq. Hope that helps!

-- James

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/consul/issues
IRC: #consul on Freenode
---

You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/consul-tool/ef258b7c-fe4c-4936-842e-9865bc85abe4%40googlegroups.com.