question on GRPC DNS resolution

285 views
Skip to first unread message

Sureshbabu Seshadri

unread,
Sep 13, 2021, 2:37:05 AM9/13/21
to grpc.io
In our intranet, GRPC servers are running in different computers. While many of the servers are connecting without any issue, there are some servers failed with error "DNS resolution failed for service : <hostname>:<port>"

The issue can be resolved by adding the environment variable GRPC_DNS_RESOLVER with value native.

Questions: 
  1. Since all the servers are running in the same network, what could be the reason behind DNS resolution failure for some servers only?
  2. Is adding the  GRPC_DNS_RESOLVER degrades name resolution performance or any other side effect.
We need to take a decision whether this environment variable is to be added as part of our product installation or not to avoid any similar issues in customer environment. 

Please let know if any other input like log files required for this issue analysis

sanjay...@google.com

unread,
Sep 15, 2021, 1:11:46 PM9/15/21
to grpc.io
Which gRPC language are you using? Is it C++? If yes, this could be an issue with the "ares" DNS used by default. I suggest reproducing the issue with some logs and logging it at github.

Sureshbabu Seshadri

unread,
Sep 16, 2021, 10:04:46 AM9/16/21
to grpc.io
OK, without GRPC_DNS_RESOLVER it is always reproducible, can you elaborate which log type to be enabled as GRPC usually writes tons of logs if we enable ALL.

apo...@google.com

unread,
Sep 16, 2021, 12:13:26 PM9/16/21
to grpc.io
If you are able to reproduce and capture logs, then it would be great if you could enable the following env vars to get more debug info:

GRPC_VERBOSITY=DEBUG
GRPC_TRACE=cares_resolver,cares_address_sorting

(you can add more tracers to GRPC_TRACE, but we need at least those two)

Optionally, a packet trace showing DNS activity (e.g. captured by tcpdump) may also be helpful.

Sureshbabu Seshadri

unread,
Sep 17, 2021, 2:40:22 AM9/17/21
to grpc.io
Thanks, please find two log files one where DNS resolution failed and another one resolved with the help of native value set to GRPC_DNS_RESOLVER environment variable in the following share


Please let me know if you need any other input for analyzing the issue.

Sureshbabu Seshadri

unread,
Sep 25, 2021, 4:23:06 AM9/25/21
to grpc.io
Any update on this?

Sanjay Pujare

unread,
Sep 25, 2021, 1:24:18 PM9/25/21
to Sureshbabu Seshadri, grpc.io
You should be able to create an issue at https://github.com/grpc/grpc/issues with the log files you have collected and a description of the issue.

--
You received this message because you are subscribed to a topic in the Google Groups "grpc.io" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/grpc-io/ae0feBLvcRI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to grpc-io+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/dcec4eb7-9d80-41ff-8e11-ce7ca163017fn%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages