Large memory consumption from Gremlin Server

[Zeph]

Gremlin server seems to be creating multiple threads that increasingly consume memory. Restarting the server temporarily resolves this. Does anyone know why this happens and how to fix it?

[Robert Dale]

Could you provide some details about your setup?

Java version?

TinkerPop version?

Gremlin server configuration?

How are you accessing the server? Client?

Queries?

GC logs?

Thread dump?

How can we reproduce?

On Fri, Mar 16, 2018 at 12:18 Zeph <ashley...@gmail.com> wrote:

Gremlin server seems to be creating multiple threads that increasingly consume memory. Restarting the server temporarily resolves this. Does anyone know why this happens and how to fix it?

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/9b1600e1-c8b3-42cb-882b-954d8af4aa30%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Robert Dale

[Jason Plurad]

Are you using parameterized scripts? https://tinkerpop.apache.org/docs/current/reference/#parameterized-scripts

[Zeph]

I'm using a java application to access the server, which is hosting multiple graphs. I'm also using the JanusGraph 0.2.0 distribution.

GraphTraversalSources are instantiated once and then reused.

On Saturday, March 17, 2018 at 1:13:12 PM UTC-4, Robert Dale wrote:

[Ted Wilmes]

Along with the other requested info, can you post a heap histogram?  jmap -histo $JANUS_PID

--Ted

[Zeph]

Heap

 PSYoungGen      total 742400K, used 703940K [0x000000076ab00000, 0x00000007a4a80000, 0x00000007c0000000)

  eden space 604160K, 93% used [0x000000076ab00000,0x000000078d372830,0x000000078f900000)

  from space 138240K, 99% used [0x000000078f900000,0x0000000797ffeb60,0x0000000798000000)

  to   space 152064K, 0% used [0x000000079b600000,0x000000079b600000,0x00000007a4a80000)

 ParOldGen       total 1439744K, used 423320K [0x00000006c0000000, 0x0000000717e00000, 0x000000076ab00000)

  object space 1439744K, 29% used [0x00000006c0000000,0x00000006d9d66248,0x0000000717e00000)

 Metaspace       used 53824K, capacity 55358K, committed 55680K, reserved 109772

[satishch...@gmail.com]

Hi , 

We are also seeing the same type of issue , our memory usage has increased drastically.

we have over 7 million  edges and over 70k vertices.

Thanks,

Satish.

[satishch...@gmail.com]

Hi , 

We are also seeing the same type of issue , our memory usage has increased drastically.

- we have over 7 million  edges and over 70k vertices.

- our data base is neo4j 

pls let us know what could cause this issue 

Thanks,

Satish.

On Tuesday, March 20, 2018 at 1:39:11 PM UTC-7 Zeph wrote:

[Stephen Mallette]

Lots of things can cause increased memory usage with Gremlin Server, but I will however take some wild swings:

1. https://tinkerpop.apache.org/docs/current/reference/#parameterized-scripts

2. Maybe you simply need to give Gremlin Server more memory and bump the -Xmx - you have a graph of large size and it's neo4j which is basically an in-memory graph with massive caches

3. Piggybacking on the previous item - are neo4js configurations correct? perhaps you need to adjust those? whatever neo4j best practices are for performance tuning apply to its use in Gremlin Server, so it's worth making sure you've gotten neo4j configurations tuned properly: https://neo4j.com/docs/operations-manual/current/performance/memory-configuration/

As you can see the original thread didn't go very far in the past. That's mostly because there wasn't very much information provided to us. It's quite hard to deduce the problem area without information of the sorts that Goodman Dale and Goodman Wilmeeez originally requested:

Could you provide some details about your setup?
Java version?
TinkerPop version?
Gremlin server configuration?
How are you accessing the server? Client?
Queries?
GC logs?
Thread dump?
How can we reproduce?

The point of course is to not answer all those questions, but to provide context that can enable a discussion which will hopefully lead you to an answer. Good luck!

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/0f726e02-520a-41a3-bb80-4ac8441b5047n%40googlegroups.com.