g.E().count() timeout

[Sture Lygren]

Hi,

While trying to get Arcade Analytics to work (gremlin, and local JanusGraph) here I notice that AA falls over and dies while trying to complete the indexing process. Main issue seems to be a timeout occurring on issuing a g.E().count() command. My current graph is considered tiny with only 500K vertices and 1.5M edges. I've increased the timeout setting to 10min, but calling g.E().count() from the gremlin console I still receive a timeout. Is there any way to fix this?

Thanks,

Sture

   

[Stephen Mallette]

It sounds like you're connecting Gremlin Console to a JanusGraph hosted in Gremlin Server. Have you increased the -Xmx for Gremlin Server from it's default? Perhaps that setting is insufficient? Of course, even at a size of 1.5M edges you probably should not expect a millisecond response unless perhaps the entire graph is loaded to memory when you do your global count. 

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/d5ec02a6-a099-4320-8cf6-908fe72fd4f5%40googlegroups.com.

[Sture Lygren]

Hi,

Setup here is gremlin server running JanusGraph (0.4.0) with a cql (3 Cassandra docker instances) cluster backend - all running on a dual CPU 24 core 64Gb server. Tried increasing both -Xms and -Xmx in gremlin_server.sh from standard 32m and 512m to 4096m and now the edges count finishes, but only after ~5 min. Not sure throwing more memory at it will help significantly. Are there any other options to improve this?

Thanks,

Sture 

On Monday, December 2, 2019 at 1:39:32 PM UTC+1, Stephen Mallette wrote:

It sounds like you're connecting Gremlin Console to a JanusGraph hosted in Gremlin Server. Have you increased the -Xmx for Gremlin Server from it's default? Perhaps that setting is insufficient? Of course, even at a size of 1.5M edges you probably should not expect a millisecond response unless perhaps the entire graph is loaded to memory when you do your global count. 

On Mon, Dec 2, 2019 at 6:19 AM Sture Lygren <sly...@gmail.com> wrote:

Hi,

While trying to get Arcade Analytics to work (gremlin, and local JanusGraph) here I notice that AA falls over and dies while trying to complete the indexing process. Main issue seems to be a timeout occurring on issuing a g.E().count() command. My current graph is considered tiny with only 500K vertices and 1.5M edges. I've increased the timeout setting to 10min, but calling g.E().count() from the gremlin console I still receive a timeout. Is there any way to fix this?

Thanks,

Sture

   

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gremli...@googlegroups.com.

[Stephen Mallette]

Well, at least the memory bump allowed you to get a count. At 1.5M edges and without holding the graph in memory (with gremlin server that would mean a session to make multiple requests over the same transaction - not a great way to do that as we don't typically like to recommend sessions) your graph is probably just large enough to start causing you to ask questions about how best to do global operations and typically that question is like, "If I execute this traversal as OLTP and end up with X minutes of wait time, is that better than Y minutes of wait time if I use Spark and the complexity that it brings?" That said, I really haven't used JanusGraph before (only Titan and that was years ago), so I'm not sure if there is more you can do to speed things up further or if 1.5M edges is a typical point to start testing Spark based traversals. I seem to recall my threshold for such things being around 5M edges, but all environments are different and my memory may be off on that one. Perhaps someone else has some advice or, if not, consider taking your question to the JanusGraph users list.

On Mon, Dec 2, 2019 at 8:58 AM Sture Lygren <sly...@gmail.com> wrote:

Hi,

Setup here is gremlin server running JanusGraph (0.4.0) with a cql (3 Cassandra docker instances) cluster backend - all running on a dual CPU 24 core 64Gb server. Tried increasing both -Xms and -Xmx in gremlin_server.sh from standard 32m and 512m to 4096m and now the edges count finishes, but only after ~5 min. Not sure throwing more memory at it will help significantly. Are there any other options to improve this?

Thanks,

Sture 

On Monday, December 2, 2019 at 1:39:32 PM UTC+1, Stephen Mallette wrote:

It sounds like you're connecting Gremlin Console to a JanusGraph hosted in Gremlin Server. Have you increased the -Xmx for Gremlin Server from it's default? Perhaps that setting is insufficient? Of course, even at a size of 1.5M edges you probably should not expect a millisecond response unless perhaps the entire graph is loaded to memory when you do your global count. 

On Mon, Dec 2, 2019 at 6:19 AM Sture Lygren <sly...@gmail.com> wrote:

Hi,

While trying to get Arcade Analytics to work (gremlin, and local JanusGraph) here I notice that AA falls over and dies while trying to complete the indexing process. Main issue seems to be a timeout occurring on issuing a g.E().count() command. My current graph is considered tiny with only 500K vertices and 1.5M edges. I've increased the timeout setting to 10min, but calling g.E().count() from the gremlin console I still receive a timeout. Is there any way to fix this?

Thanks,

Sture

   

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremli...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/d5ec02a6-a099-4320-8cf6-908fe72fd4f5%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/bb9a32d7-5c52-418e-a607-28c415b715f6%40googlegroups.com.