Number of opened files are constantly increasing

[Eugene Strokin]

Hello, I have a strange situation. While I'm running my application (Vert.x 2) and checking number of opened files by the process:

cd /proc/{my pid}/fd
ls -l | wc -l

I see the number is constantly increasing.

Most of the files are something like this:

ls -l | less

lrwx------ 1 root root 64 Mar  5 19:51 999 -> socket:[178198

After the number reaches 4096 I'm getting 

java.io.IOException: Too many open files
 at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
 at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)
 at io.netty.channel.socket.nio.NioServerSocketChannel.doReadMessages(NioServerSocketChannel.java:135)
 at io.netty.channel.nio.AbstractNioMessageChannel$NioMessageUnsafe.read(AbstractNioMessageChannel.java:68)
 at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
 at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
 at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
 at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
 at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
 at java.lang.Thread.run(Thread.java:745)

Could you please tell me what it could be? Looks like connections are not getting closed for some reason.

I'm using HttpClient to send requests to other servers, nothing fancy, but what possibly could lead to such situation?

Thank you,

Eugene

[Witold Szczerba]

Hi,
I think you may have a bug in your code. Double check lines 23 to 28 (or other, it's hard to guess).
Let us know how is your progress with the issue.

Regards,
Witold Szczerba

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Eugene Strokin]

I wander how did you get lines 23 or 28? I’m not sure what are you referring to.

You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/jyy5oBAOLoM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+un...@googlegroups.com.

[Eugene Strokin]

I had to add

request.response().close();

after each

request.response().end();

and now I see the number of opened file is stable.

I found this information here:

http://vertx.io/core_manual_java.html

But I nether saw it in the examples, here for instance:

https://github.com/vert-x/vertx-examples/blob/master/src/raw/java/proxy/ProxyServer.java

Looks like after line 51 we should be closing the response:

req.response().close()

Am I correct?

On Thursday, March 5, 2015 at 8:26:57 PM UTC-5, Eugene Strokin wrote:

I wander how did you get lines 23 or 28? I’m not sure what are you referring to.

On Mar 5, 2015, at 8:23 PM, wrote:

Hi,
I think you may have a bug in your code. Double check lines 23 to 28 (or other, it's hard to guess).
Let us know how is your progress with the issue.

Regards,
Witold Szczerba

[Alexander Lehmann]

The vert.x http client uses a connection pool to accomodate http Keep-Alive, so it would be correct to keep the connection open when the server supports Keep-Alive, however the connections should be closed at a later time when the server has closed the connection or when the connection is reused and is checked to be closed.

Not sure if that works correctly, if you are closing the connections manually, you are disabling Keep-Alive, but it should be ok since it is not necessary.

[Eugene Strokin]

Right, that what I was expecting. But looks like the connections are not getting back to the pool on  request.response().end(); but a new connection is creating instead for the next request.

Unless we have to do something special to return a connection?

[Alexander Lehmann]

I think that should work without any additional method call. Vert.x uses netty as network layer, that implements the pool, which should work. (have to do a test if it really does though)

[Tim Fox]

Hi Eugene,

I think Witold is making the point (a little sarcastically ;) ) that it's pretty much impossible to diagnose the issue without seeing your client code.

I suspect you're not closing the connections on the client side somehow.

[Eugene Strokin]

Sorry, I didn’t get the sarcasm, I thought the problem is quite clear without code example. But as Alexander mentioned in later emails, I should not explicitly close the connection, so it would return to the pool and be reused if I want (and I do want) to use keep alive feature.

To make my code work, I had to add response().close() everywhere where I have response.end(). I think Alexander is right, this way I’m disabling keep alive feature. But looks like there is a bug somewhere which preventing reuse of a connection if not close.

[Tim Fox]

I'm not going to conjecture about what is and or isn't going on, because without seeing a reproducer, it's really just that - conjecture ;)