Webclient and Keep-Alive to vertx server.

856 views
Skip to first unread message

javadevmtl

unread,
Mar 15, 2018, 11:05:07 PM3/15/18
to vert.x
Hi, I have very simple vertx server that returns 200 ok and is running over 4 verticles instances. I have tested with Jmeter and it works fine...

I'm trying to send requests to from my kafka consumer using the vertx-web-client, but I get Netty Exceptions Address Already in use on the client.

I don't think I'm doing anything extra ordinary with the client: https://pastebin.com/fJvdabeQ



javadevmtl

unread,
Mar 15, 2018, 11:09:31 PM3/15/18
to vert.x
Also I profile the application with Yourkit and I see connections opening and closing on the client, when I would expect them to open and stay open?

javadevmtl

unread,
Mar 15, 2018, 11:43:39 PM3/15/18
to vert.x
Ok this seems to only happen if I run inside docker on mesos. Even if I provide a static I.P.

Anyone have experienced this before?

javadevmtl

unread,
Mar 16, 2018, 12:05:00 AM3/16/18
to vert.x
And the exception: io.netty.channel.AbstractChannel$AnnotatedSocketException: Address not available: /xxx.xxx.xxx.42:14916\n\tat sun.nio.ch.Net.$$YJP$$connect0(Native Method)

Julien Viet

unread,
Mar 16, 2018, 4:11:56 AM3/16/18
to ve...@googlegroups.com
so that's the local address that is being used ?

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/b2aceb4e-47a1-449a-a188-ee851e7d9780%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

javadevmtl

unread,
Mar 16, 2018, 7:06:38 AM3/16/18
to vert.x
It's the host address and containers are running on bridge network.

javadevmtl

unread,
Mar 16, 2018, 8:47:43 AM3/16/18
to vert.x
See attached. It's not normal that it opens and closes connections like that...
Screen Shot 2018-03-16 at 8.46.08 AM.png

John Smith

unread,
Mar 16, 2018, 3:39:21 PM3/16/18
to ve...@googlegroups.com
If there is to many requests for the amount of client instances could that cause the issue?

--
You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/styvMHFJw_U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/d8c107b5-1f9b-44a5-bc7d-e399e7cb1e3f%40googlegroups.com.

John Smith

unread,
Mar 16, 2018, 4:36:05 PM3/16/18
to ve...@googlegroups.com
Ok so I added headers to the server the vertx web-client is not sending Connection: keep-alive?

Julien Viet

unread,
Mar 16, 2018, 5:09:55 PM3/16/18
to ve...@googlegroups.com
it depends on the version (of vertx) and the protocol (http/1.1 or http/2)

until 3.5.0 for HTTP/1.1, connections stayed in the pool

since 3.5.1 for HTTP/1.1, connections are closed when put back in the pool

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.

javadevmtl

unread,
Mar 16, 2018, 5:34:23 PM3/16/18
to vert.x
The client doesn't seem to send the keep-alive... On the server side I print the headers. I don't see it.

Here is a simple reproducer: https://pastebin.com/rYUcY5Sm

Here are the debug logs...


         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 47 45 54 20 2f 63 67 69 2d 62 69 6e 2f 73 65 6e |GET /cgi-bin/xxx|
|00000010| 64 73 6d 73 20 48 54 54 50 2f 31 2e 31 0d 0a 48 |xxxx HTTP/1.1..H|
|00000020| 6f 73 74 3a 20 31 37 32 2e 31 37 2e 31 37 2e 34 |ost: xxx.xx.xx.x|
|00000030| 32 3a 32 39 32 30 30 0d 0a 0d 0a                |2:29200....     |
+--------+-------------------------------------------------+----------------+


         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d |HTTP/1.1 200 OK.|
|00000010| 0a 43 6f 6e 74 65 6e 74 2d 4c 65 6e 67 74 68 3a |.Content-Length:|
|00000020| 20 31 31 0d 0a 0d 0a 30 3a 20 61 63 63 65 70 74 | 11....0: accept|
|00000030| 65 64                                           |ed              |
+--------+-------------------------------------------------+----------------+

javadevmtl

unread,
Mar 16, 2018, 6:05:52 PM3/16/18
to vert.x
Lol sorry if I have been noisy on the group.

So yeah HTTP 1.1 doesn't have to send keep-alive.

So I dunno. When the application runs on dcos/mesos/docker vertx-web-clienty keep-alive doesn't seem to work. The connections are one for one for each request I make.

Julien Viet

unread,
Mar 17, 2018, 3:40:50 AM3/17/18
to ve...@googlegroups.com
so you are saying that the http client keepAlive=true does not send the keepAlive header ?



--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.

John Smith

unread,
Mar 17, 2018, 4:49:04 AM3/17/18
to ve...@googlegroups.com
Right! But then I read that http 1.1 doesn't require it to send it anyways?

You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/styvMHFJw_U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+un...@googlegroups.com.

Visit this group at https://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/901F0695-A957-4633-B370-5BE7599EA157%40julienviet.com.

Julien Viet

unread,
Mar 17, 2018, 5:25:19 AM3/17/18
to ve...@googlegroups.com
no it does not.

But configured as is Vert.x HttpClient will always send it.

So I believe that there might be a (forward) http proxy that removes the headers.

Julien

John Smith

unread,
Mar 17, 2018, 6:47:40 AM3/17/18
to ve...@googlegroups.com
ok so vertx http client does send the header? it doesnt seem to, check the debug output of the logs i posted further up. I enabled the log tracing.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/B8C2951C-ADE7-47C1-9317-D312ED9E6BC1%40julienviet.com.

Julien Viet

unread,
Mar 17, 2018, 1:04:51 PM3/17/18
to ve...@googlegroups.com
that's the debug log on the client ?

John Smith

unread,
Mar 17, 2018, 1:37:46 PM3/17/18
to ve...@googlegroups.com
yeah

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/BC725FD8-CAC5-403E-8EC6-A47EC296D097%40julienviet.com.

Julien Viet

unread,
Mar 17, 2018, 2:03:08 PM3/17/18
to ve...@googlegroups.com
are you sure that the WebClient has keepAlive set to true ?

John Smith

unread,
Mar 17, 2018, 2:17:06 PM3/17/18
to ve...@googlegroups.com
look ay my reproducer. I even tried the basic http client. And keepalive is on by dwfault also no?

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/14AEDAA7-AA35-47F9-BED1-A1347DD4D574%40julienviet.com.
Message has been deleted

javadevmtl

unread,
Mar 18, 2018, 2:23:05 PM3/18/18
to vert.x
Ok it seems it's doing keep-alive. But it doesn't send the header. I manually put the header and wen't through.

So I did the following test...

My client is deployed over 4 verticles.
I sent 200 requests.
On the server side I recorded 20 connect events and received all 200 requests.
On yourkit I saw 20 opens and 20 closes.

So I'm guessing that  the 200 requests where above the maxpool of the client so they where opened and closed?

Julien Viet

unread,
Mar 18, 2018, 3:12:18 PM3/18/18
to ve...@googlegroups.com
if you have 4 verticles it means you have 4 * 5 (default) = 20 total max connections

John Smith

unread,
Mar 18, 2018, 3:18:19 PM3/18/18
to ve...@googlegroups.com
ok so it is working? But on yourkit i see 20 connections constantly opening and closing?

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/01692F3B-5B88-4B72-9630-12C03233A0AF%40julienviet.com.

Julien Viet

unread,
Mar 18, 2018, 3:24:41 PM3/18/18
to ve...@googlegroups.com
when the the client has no pending request it closes the connection, that might explain that

John Smith

unread,
Mar 18, 2018, 3:28:59 PM3/18/18
to ve...@googlegroups.com
oh ok. Is there a way to keep open a bit longer with a setting? Well I guess can assure myself simce server recorded 20 connects with 200 request so it should be good. cool

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/06B9E9EE-B506-401B-AAAE-9534049553F7%40julienviet.com.

Julien Viet

unread,
Mar 18, 2018, 3:36:47 PM3/18/18
to ve...@googlegroups.com
no, but we should have such feature to specify how long a connection can remain in the pool without being closed immediately

John Smith

unread,
Mar 18, 2018, 3:39:09 PM3/18/18
to ve...@googlegroups.com
Ok want me to make a feature request on github?

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/styvMHFJw_U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/styvMHFJw_U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/styvMHFJw_U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/64235A29-FA44-45EF-A3E7-1D150D0967CC%40julienviet.com.

Julien Viet

unread,
Mar 18, 2018, 3:53:57 PM3/18/18
to ve...@googlegroups.com
yes sure

javadevmtl

unread,
Mar 19, 2018, 12:17:14 PM3/19/18
to vert.x
Btw testing with 3.5.0 and is the behaviour is what I expected more. I guess if you guys go ahead and implement the new feature it will be on par.

Is there a reason, the behaviour changed between 3.5.0 and 3.5.1?

Julien Viet

unread,
Mar 19, 2018, 12:41:40 PM3/19/18
to ve...@googlegroups.com
the http client pool has been improved and the actual behaviour correspond to the previous behaviour we use to have a few versions ago (that was closing connections).

here is the issue so you can read the motivations : https://github.com/eclipse/vert.x/issues/2208



On 19 Mar 2018, at 17:17, javadevmtl <java.d...@gmail.com> wrote:

Btw testing with 3.5.0 and is the behaviour is what I expected more. I guess if you guys go ahead and implement the new feature it will be on par.

Is there a reason, the behaviour changed between 3.5.0 and 3.5.1?

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.

javadevmtl

unread,
Mar 19, 2018, 1:17:59 PM3/19/18
to vert.x
Cool. I tested 3.5.0 and albeit the connections stay open longer, I saw inconsistency in the requests/sec sent.
Message has been deleted
Message has been deleted
Message has been deleted

Julien Viet

unread,
Mar 20, 2018, 4:26:50 PM3/20/18
to ve...@googlegroups.com
when it's closed its not put back in the pool I think.

On 19 Mar 2018, at 19:40, Manish Kumar <kumar.m...@gmail.com> wrote:

since 3.5.1 for HTTP/1.1, connections are closed when put back in the pool


@Julien - If connection is closed while putting back into the pool then why to put back in pool at all? Anyway it's closed, and new connection will have to be open if to be reused.

I read through https://github.com/eclipse/vert.x/issues/2208, and seems like we are trying to make pool implementation HTTP agnostic, but it doesn't explain the logic behind closing connection before putting back into pool.

Maybe more documentation/reasoning/explanation will help.


On Friday, March 16, 2018 at 5:09:55 PM UTC-4, Julien Viet wrote:
it depends on the version (of vertx) and the protocol (http/1.1 or http/2)

until 3.5.0 for HTTP/1.1, connections stayed in the pool

since 3.5.1 for HTTP/1.1, connections are closed when put back in the pool

On 16 Mar 2018, at 04:09, javadevmtl <java.d...@gmail.com> wrote:

Also I profile the application with Yourkit and I see connections opening and closing on the client, when I would expect them to open and stay open?

On Thursday, 15 March 2018 23:05:07 UTC-4, javadevmtl wrote:
Hi, I have very simple vertx server that returns 200 ok and is running over 4 verticles instances. I have tested with Jmeter and it works fine...

I'm trying to send requests to from my kafka consumer using the vertx-web-client, but I get Netty Exceptions Address Already in use on the client.

I don't think I'm doing anything extra ordinary with the client: https://pastebin.com/fJvdabeQ




--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.
Reply all
Reply to author
Forward
0 new messages