Vert.x 3.3.2 is not able to find redis in a dockerized environment

1,445 views
Skip to first unread message

poiuytrez

unread,
Jul 19, 2016, 11:34:20 AM7/19/16
to vert.x
Hi, 

I have issues to run a webserver connected to a redis server with Vert.x 3.3.2 in a docker environment. I get the following error: 
webserver               | java.net.UnknownHostException: failed to resolve 'my-redis.home'. Exceeded max queries per resolve 3
webserver               | at io.vertx.core.dns.impl.fix.DnsNameResolverContext.finishResolve(DnsNameResolverContext.java:435)
webserver               | at io.vertx.core.dns.impl.fix.DnsNameResolverContext.tryToFinishResolve(DnsNameResolverContext.java:362)
webserver               | at io.vertx.core.dns.impl.fix.DnsNameResolverContext$2.operationComplete(DnsNameResolverContext.java:160)
webserver               | at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:518)
webserver               | at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:492)
webserver               | at io.netty.util.concurrent.DefaultPromise.notifyListenersWithStackOverFlowProtection(DefaultPromise.java:431)
webserver               | at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
webserver               | at io.netty.util.concurrent.DefaultPromise.setSuccess(DefaultPromise.java:99)
webserver               | at io.vertx.core.dns.impl.fix.DnsQueryContext.setSuccess(DnsQueryContext.java:197)
webserver               | at io.vertx.core.dns.impl.fix.DnsQueryContext.finish(DnsQueryContext.java:180)
webserver               | at io.vertx.core.dns.impl.fix.DnsNameResolver$DnsResponseHandler.channelRead(DnsNameResolver.java:680)
webserver               | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
webserver               | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:334)
webserver               | at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:326)
webserver               | at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
webserver               | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
webserver               | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:334)
webserver               | at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:326)
webserver               | at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1320)
webserver               | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
webserver               | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:334)
webserver               | at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:905)
webserver               | at io.netty.channel.nio.AbstractNioMessageChannel$NioMessageUnsafe.read(AbstractNioMessageChannel.java:93)
webserver               | at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:563)
webserver               | at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:504)
webserver               | at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:418)
webserver               | at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:390)
webserver               | at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:742)
webserver               | at java.lang.Thread.run(Thread.java:745)

I works perfectly with Vert.x 3.2.1.

I made a repoducer: 

Let me know if I am missing something or if I need to file an issue. 

Thank you,
poiuytrez

Julien Viet

unread,
Jul 19, 2016, 12:18:10 PM7/19/16
to ve...@googlegroups.com
can you open an issue in vertx-core for it ?

thanks


--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/d0b3a4d7-7262-431a-b4c4-126b66db69c1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Guillaume Charhon

unread,
Jul 19, 2016, 12:38:25 PM7/19/16
to ve...@googlegroups.com

--
You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/XQjarD00WHU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+un...@googlegroups.com.

Julien Viet

unread,
Jul 19, 2016, 4:12:36 PM7/19/16
to ve...@googlegroups.com
in this situation can you tell what is the content of the /etc/resolv.conf file running in the container ?


Julien Viet

unread,
Jul 20, 2016, 3:14:44 AM7/20/16
to ve...@googlegroups.com
Hi,

the problem is that this particular setup uses ndots = 0, here is the resolv.conf file:

search local
nameserver 127.0.0.11
options ndots:0

currently the new DNS resolver does not parse this value when instantiating the Vert.x AddressResolverOptions and 0 is actually unsupported (it’s an exotic case).

In your situation you need to fallback on the JVM resolver using :

command: java -Dvertx.disableDnsResolver=true -jar /src/build/libs/dns-reproducer.jar
I renamed the issues you created to 
"Load the ndots value on linux from /etc/resolv.conf and allow a value of 0"
Julien

poiuytrez

unread,
Jul 20, 2016, 4:13:33 AM7/20/16
to vert.x
Hi Julien,

Thank you for the diagnostic and the workaround. 

Julien Viet

unread,
Jul 20, 2016, 3:24:47 PM7/20/16
to ve...@googlegroups.com
hopefully it will be fixed in 3.3.3

Tiago Reis

unread,
Jul 25, 2016, 1:19:02 PM7/25/16
to vert.x
Hi Julien,

I think we might have hit a related problem.

Stack trace:

2016-07-25 12:25:57.969 [vert.x-eventloop-thread-1] ERROR c.n.vex.clients.util.ExtranetClient - Request Exception: { failed to resolve

'REMOVED'. Exceeded max queries per resolve 3  }

java.net.UnknownHostException: failed to resolve 'REMOVED'. Exceeded max queries

 per resolve 3

        at io.vertx.core.dns.impl.fix.DnsNameResolverContext.finishResolve(DnsNameResolverContext.java:435)

        at io.vertx.core.dns.impl.fix.DnsNameResolverContext.tryToFinishResolve(DnsNameResolverContext.java:362)

        at io.vertx.core.dns.impl.fix.DnsNameResolverContext$2.operationComplete(DnsNameResolverContext.java:160)

        at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:518)

        at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:492)

        at io.netty.util.concurrent.DefaultPromise.notifyListenersWithStackOverFlowProtection(DefaultPromise.java:431)

        at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)

        at io.netty.util.concurrent.DefaultPromise.setSuccess(DefaultPromise.java:99)

        at io.vertx.core.dns.impl.fix.DnsQueryContext.setSuccess(DnsQueryContext.java:197)

        at io.vertx.core.dns.impl.fix.DnsQueryContext.finish(DnsQueryContext.java:180)

        at io.vertx.core.dns.impl.fix.DnsNameResolver$DnsResponseHandler.channelRead(DnsNameResolver.java:680)

        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)

        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:334)

        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:326)

        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)

        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)

        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:334)

        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:326)

        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1320)

        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)

        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:334)

        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:905)

        at io.netty.channel.nio.AbstractNioMessageChannel$NioMessageUnsafe.read(AbstractNioMessageChannel.java:93)

        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:563)

        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:504)

        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:418)

        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:390)

        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:742)

        at java.lang.Thread.run(Thread.java:745)



resolv.conf content:


search eu-west-1.compute.internal

nameserver 10.55.0.2


Any ideas?


Thanks

Julien Viet

unread,
Jul 25, 2016, 1:34:20 PM7/25/16
to ve...@googlegroups.com
it looks like you are rather trying to resolve the “REMOVED” hostname.

check your code.

Tiago Reis

unread,
Jul 25, 2016, 1:50:59 PM7/25/16
to vert.x
Ah no, the hostname is valid, I just removed it because it's private from our client, it's a valid hostname

sorry I should have been explicit that I had "hidden" the real hostname.

Julien Viet

unread,
Jul 25, 2016, 4:50:36 PM7/25/16
to ve...@googlegroups.com
without more details about the hostname, it is difficult to help you.

and your case does not seem like the current case as the current case uses ndots=0

Tiago Reis

unread,
Jul 25, 2016, 4:58:58 PM7/25/16
to vert.x
Hi,

what do you need to know about the hostname?

I know, I just thought that it might be related since the stack trace is the same, the error message is the same, and was something that only started happening when we moved to version 3.3.2

We went from 3.2.1 to 3.3.2

Julien Viet

unread,
Jul 25, 2016, 5:03:18 PM7/25/16
to ve...@googlegroups.com
the hostname is useful to understand how the name resolution was done, specifically with search domain that you seem to use "eu-west-1.compute.internal"



Saurabh Kukreti

unread,
Apr 12, 2017, 5:32:56 AM4/12/17
to vert.x
I faced the same behavior in Vertx 3.3.3. Few of the requests were 200 OK and few failed with the exception below.

2017-04-11 18:54:50,731 [vert.x-eventloop-thread-0] ERROR com.sample.service.UserManager - Error occurred while validating email java.net.UnknownHostException: failed to resolve 'sample.io'. Exceeded max queries per resolve 3 
at io.netty.resolver.dns.DnsNameResolverContext.finishResolve(DnsNameResolverContext.java:476)
at io.netty.resolver.dns.DnsNameResolverContext.tryToFinishResolve(DnsNameResolverContext.java:398)
at io.netty.resolver.dns.DnsNameResolverContext.query(DnsNameResolverContext.java:164)
at io.netty.resolver.dns.DnsNameResolverContext.tryToFinishResolve(DnsNameResolverContext.java:392)
at io.netty.resolver.dns.DnsNameResolverContext.query(DnsNameResolverContext.java:164)
at io.netty.resolver.dns.DnsNameResolverContext.access$800(DnsNameResolverContext.java:54)
at io.netty.resolver.dns.DnsNameResolverContext$3.operationComplete(DnsNameResolverContext.java:190)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:514)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:488)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:427)
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:129)
at io.netty.resolver.dns.DnsQueryContext.setFailure(DnsQueryContext.java:217)
at io.netty.resolver.dns.DnsQueryContext.access$300(DnsQueryContext.java:44)
at io.netty.resolver.dns.DnsQueryContext$3.run(DnsQueryContext.java:160)
at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:120)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:440)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
at java.lang.Thread.run(Thread.java:745)

Simon Templer

unread,
May 15, 2017, 12:54:15 PM5/15/17
to vert.x
Hi,

I also have that problem in a Docker environment after upgrading to VertX > 3.2.1 (tried each version up to 3.4.1). The workaround setting the system property as described in the ticket (https://github.com/eclipse/vert.x/issues/1525#issuecomment-236938128) works for me.

Best,
Simon
Reply all
Reply to author
Forward
0 new messages