Problems with HttpClient on OpenShift

[Khen Ofek]

Hi,

I am trying the new 3.3.0 version.
I am using HttpClient in my code to do a post request to a server using TLS/SSL.
On my local machine everything works fine.
When I try to run it on OpenShift gear I get the following error: "Exceeded max queries per resolve 3"
With 3.2.1 I don't have any issues with HttpClient.

This is the beginning of the stack trace:
Received exception: failed to resolve 'b1-becopia.rhcloud.com.prod.rhcloud.com'. Exceeded max queries per resolve 3 
java.net.UnknownHostException: failed to resolve 'b1-becopia.rhcloud.com.prod.rhcloud.com'. Exceeded max queries per resolve 3
    at io.vertx.core.dns.impl.fix.DnsNameResolverContext.finishResolve(DnsNameResolverContext.java:435)
    at io.vertx.core.dns.impl.fix.DnsNameResolverContext.tryToFinishResolve(DnsNameResolverContext.java:362)
    at io.vertx.core.dns.impl.fix.DnsNameResolverContext.query(DnsNameResolverContext.java:131)
    at io.vertx.core.dns.impl.fix.DnsNameResolverContext.tryToFinishResolve(DnsNameResolverContext.java:356)
    at io.vertx.core.dns.impl.fix.DnsNameResolverContext.query(DnsNameResolverContext.java:131)
    at io.vertx.core.dns.impl.fix.DnsNameResolverContext.access$500(DnsNameResolverContext.java:57)
    at io.vertx.core.dns.impl.fix.DnsNameResolverContext$2.operationComplete(DnsNameResolverContext.java:157)

The code looks like that:
       --------------------------------
       HttpClientOptions options = new HttpClientOptions().setDefaultHost(delegationHost);
       if (delegationPort != null) options.setDefaultPort(delegationPort);
        options.setSsl(ssl);
        options.setTrustAll(trustAll);
        HttpClient client = vertx.createHttpClient(options);
        HttpClientRequest request = client.post(endpoint, response -> {
            logger.info("Received response with status code " + response.statusCode());
            response.bodyHandler(buffer -> {
                logger.trace("Received response body: " + buffer);
                res.handle(Future.succeededFuture(buffer.toJsonObject()));
            });             
        });
        request.exceptionHandler(e -> {
              logger.error("Received exception: " + e.getMessage());
              e.printStackTrace();
              res.handle(Future.failedFuture(e));
        });
        request.end(params.toString());
        ------------------------------

Khen Ofek

[Julien Viet]

Hi,

I suspect this is due to the search domains that is configured with an ndots value > 1 .

At the moment we cannot get this value automatically from the OS.

can you give us the content of the /etc/resolv.conf file ?

alternatively you can set the system property vertx.disableDnsResolver to true to fallback on the JVM address resolver : -Dvertx.disableDnsResolver=true

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/9a2b947f-cf22-4c33-a567-567a091c71d3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Julien Viet]

what is the value of delegationHost ?

On Jun 27, 2016, at 6:14 PM, Khen Ofek <khen...@gmail.com> wrote:

[Julien Viet]

I was able to reproduce it with my own openshift online account.

it looks like a bug in the dns resolver - I will investigate tomorrow more in details.

[Julien Viet]

Hi,

I created this issue for this problem : https://github.com/eclipse/vert.x/issues/1479

I believe it should be fixed soon.

[Khen Ofek]

Hi,

Thanks for the quick reply and investigation.

I am looking forward for this fix.

Khen Ofek

--
You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/K1y_bsYSMPM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+un...@googlegroups.com.

Visit this group at https://groups.google.com/group/vertx.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/D09C35C6-64F2-4AD8-8A0A-37B5C292EF4F%40julienviet.com.

[Julien Viet]

Hi, there is a fix in this branch that you can try

https://github.com/eclipse/vert.x/tree/address-resolver-should-not-bind-locally

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/CA%2BWr-%3DStax2pETmk2iitgnSkN2mwG%3DyHfz4%3Do%3DfFXbWN%3DfaHdA%40mail.gmail.com.

[Khen Ofek]

Hi,

I tested the branch on my openshift installation and it solved the issue.

Thanks,
Khen

[Julien Viet]

thanks for confirming

we are looking forward doing a 3.3.1 release within a couple of weeks to fix this issue and a couple of others that should be fixed soon in 3.3.x

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/205f03ec-87ed-452d-b96d-c9621fd0b069%40googlegroups.com.

[hyginous neto]

Hi Julien,

I am seeing same dns resolution issue with "vertx-web 3.4.1" in the Openshift 3.1. Any suggestion?

<dependency>

<groupId>io.vertx</groupId>

<artifactId>vertx-web</artifactId>

<version>3.4.1</version>

</dependency>

[Julien Viet]

Hi,

can you provide details of the issue ?

you can deactivate the async DNS using the system property vertx.disableDnsResolver set to true

I think that in 3.5, we shall get back to the JVM resolver by default and make the async dns resolver a name resolver plugin.

Julien

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/d04148de-d0ef-4425-b79f-b46ad0e997e5%40googlegroups.com.

[hyginous neto]

We have a application written in Vertx which listens to a tcp port. 

Below issue is coming when a JBoss applications is trying to call this tcp service. Both vertx and jboss app is deployed in Openshift.

One question, if i disable async DNS resolving, i would see some latency in the response times...right? 

Also our throughput will reduce.

Error logs

[0m[0m12:22:42,597 INFO  [stdout] (pool-6-thread-1) Trying to create new socket

[0m[0m12:23:02,623 INFO  [stdout] (vert.x-eventloop-thread-4) Failed to connect to laas service. Error Message is [failed to resolve ‘laasweb-dev.aexp.com’. Exceeded max queries per resolve 4 ]:

[0m[31m12:23:02,624 ERROR [stderr] (vert.x-eventloop-thread-4) java.net.UnknownHostException: failed to resolve ‘laasweb-dev.aexp.com’. Exceeded max queries per resolve 4

[0m[31m12:23:02,624 ERROR [stderr] (vert.x-eventloop-thread-4)         at io.netty.resolver.dns.DnsNameResolverContext.finishResolve(DnsNameResolverContext.java:473)

[0m[31m12:23:02,624 ERROR [stderr] (vert.x-eventloop-thread-4)         at io.netty.resolver.dns.DnsNameResolverContext.tryToFinishResolve(DnsNameResolverContext.java:393)

[Clement Escoffier]

Hi,

On 23 May 2017, at 00:15, hyginous neto <hygi...@gmail.com> wrote:

We have a application written in Vertx which listens to a tcp port. 

Below issue is coming when a JBoss applications is trying to call this tcp service. Both vertx and jboss app is deployed in Openshift.

One question, if i disable async DNS resolving, i would see some latency in the response times...right? 

Also our throughput will reduce.

Only on the first call, because once the name has been resolved, it’s cached on the client side (for the TTL time).

Is your called services exposed behind a route ? DNS resolution inside the same namespace should be working. 

Clement

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/dab8c866-613a-44a7-821c-994bb9a334a9%40googlegroups.com.

[hyginous neto]

Yes, it is behind route.

it is in different Openshift namespace.

I want to give little background on the setup.

We have TCP Vertx Service and we also have a vertx client jar(which uses NetClient to connect to TCP Service). For this particular app, we logged in to POD and did curl to the service hostname and port, it is resolving. But when it is used within a Jboss container, it fails. But it works some JBoss apps which uses our client jar.

Any idea how can i set vertx.disableDnsResolve property to the NetClient? I know Netty needs it, but we are using this jar within jboss. It works for some apps.

[Clement Escoffier]

Hello,

On 23 May 2017, at 23:31, hyginous neto <hygi...@gmail.com> wrote:

Yes, it is behind route.

it is in different Openshift namespace.

I want to give little background on the setup.

We have TCP Vertx Service and we also have a vertx client jar(which uses NetClient to connect to TCP Service). For this particular app, we logged in to POD and did curl to the service hostname and port, it is resolving. But when it is used within a Jboss container, it fails. But it works some JBoss apps which uses our client jar.

Any idea how can i set vertx.disableDnsResolve property to the NetClient? I know Netty needs it, but we are using this jar within jboss. It works for some apps.

You can configure it using a system property. Depending the s2i or binary build you are using, the way to pass the system property differs.

Clement

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/7f0a2352-7764-411c-8960-b558ac36cbf7%40googlegroups.com.

[hyginous neto]

Thanks Clement/Julien, My issue resolved with system property setting. thanks again. :)