Gremlin and JanusGraph with NodeJS: too many open files.

[Augusto Will]

I run a .js file that read from mysql, and insert in Janus, record by record. The script read 1 record from mysql, insert a vertex in Janus, wait for the response then read another record from mysql. After some time, I have

4668109 [gremlin-server-session-1] WARN  org.janusgraph.graphdb.transaction.StandardJanusGraphTx  - Query requires iterating over all vertices [()]. For better performance, use indexes

24738035 [gremlin-server-boss-1] WARN  io.netty.channel.DefaultChannelPipeline  - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.

java.io.IOException: Too many open files

at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)

at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)

at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)

at io.netty.channel.socket.nio.NioServerSocketChannel.doReadMessages(NioServerSocketChannel.java:140)

at io.netty.channel.nio.AbstractNioMessageChannel$NioMessageUnsafe.read(AbstractNioMessageChannel.java:69)

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:652)

at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:575)

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:489)

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:451)

at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)

at java.lang.Thread.run(Thread.java:748)

[Jason Plurad]

Can you share the complete code transaction where you insert the vertex? Does it iterate through all of the results from the request?

[Augusto Will]

No, Jason, it's just another error from other query that i run on gremlin console with g.V()

But the error is not related to that.

I think the problem is the client is not closing the connection, seems to me that something is keeping the connection open until I get too much connections.

The insert vertex is this:

let v = `g.addV(T.label, 'usuario', 'nome', nome, 'id', ida )`;
let p = `{   nome: ${registro["name"]}, ida: ${registro["id"]}  }`;
gremlin.run(v, p).then((a) => {
   res();
});

[Augusto Will]

The problem is the Nodejs Gremlin client. I just write my own and entire script start to works. 

Thank you 

[Stephen Mallette]

Are you talking about this?

https://github.com/jbmusso/gremlin-javascript

On Thu, Oct 12, 2017 at 7:50 AM, Augusto Will <pw...@pwill.com.br> wrote:

The problem is the Nodejs Gremlin client. I just write my own and entire script start to works. 

Thank you 

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/1b1fec12-bc71-40da-ae8c-f5b17ddddbb9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Augusto Will]

Yes, the connection is not closed, maybe I'm wrong but seems to me that problem is related to that, this is why I have too many files open error, every query is generating a new connection and connections are not closed as related here:

 

https://github.com/jbmusso/gremlin-javascript/pull/81

and here: https://github.com/jbmusso/gremlin-javascript/pull/82

Em quinta-feira, 12 de outubro de 2017 09:13:30 UTC-3, Stephen Mallette escreveu:

Are you talking about this?

https://github.com/jbmusso/gremlin-javascript

On Thu, Oct 12, 2017 at 7:50 AM, Augusto Will <pw...@pwill.com.br> wrote:

The problem is the Nodejs Gremlin client. I just write my own and entire script start to works. 

Thank you 

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

[Jean-Baptiste Musso]

Hello Augusto,

Lib author here. Could you please share a more detailed snippet of your JavaScript code so I can troubleshoot the issue?

Typically, people only open one connection and re-use that one during the lifetime of their application. Unless you call createClient() every time you insert a new vertex, gremlin-javascript doesn't create a new WebSocket connection under the hood (the client does not manage a pool of connections, you have to handle this yourself at the moment). Could it be that you're calling .createClient() multiple times, maybe in a for loop?

Jean-Baptiste