Gremlin is taking long time to come up after restart

[Satish Kumar]

Hi ,

When we restart gremlin some times its taking long time to come up  it is getting struck as below.

ubuntu@ip-192-168-180-2:/data/neo4j/graph.db/logs$ tail -f gremlin.log

INFO main org.apache.tinkerpop.gremlin.server.AbstractChannelizer - Configured application/vnd.gremlin-v2.0+json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0

INFO gremlin-server-boss-1 org.apache.tinkerpop.gremlin.server.GremlinServer - Gremlin Server configured with worker thread pool of 30, gremlin pool of 16 and boss thread pool of 1.

INFO gremlin-server-boss-1 org.apache.tinkerpop.gremlin.server.GremlinServer - Channel started at port 8182.

INFO gremlin-server-worker-28 org.apache.tinkerpop.gremlin.server.op.session.Session - New session established for fa24bc56-1ddd-46ad-a469-64c9fd7f2ecd

INFO main org.apache.tinkerpop.gremlin.server.GremlinServer - 3.3.10

         \,,,/

         (o o)

-----oOOo-(3)-oOOo-----

INFO main org.apache.tinkerpop.gremlin.server.GremlinServer - Configuring Gremlin Server from conf/gremlin-server-custom.yml

After 25 mins - 30 mins its coming back up. 

I went through below stack over flow https://stackoverflow.com/questions/58085575/issues-getting-correct-data-from-gremlin-server, is this related our issue since this is happening in production  I don't to take risk so need clarification on this.

tinkergraph-empty.properties file : 

bash-4.4$ cat tinkergraph-empty.properties

# Licensed to the Apache Software Foundation (ASF) under one

# or more contributor license agreements.  See the NOTICE file

# distributed with this work for additional information

# regarding copyright ownership.  The ASF licenses this file

# to you under the Apache License, Version 2.0 (the

# "License"); you may not use this file except in compliance

# with the License.  You may obtain a copy of the License at

#

#  http://www.apache.org/licenses/LICENSE-2.0

#

# Unless required by applicable law or agreed to in writing,

# software distributed under the License is distributed on an

# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY

# KIND, either express or implied.  See the License for the

# specific language governing permissions and limitations

# under the License.

gremlin.graph=org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph

gremlin.tinkergraph.vertexIdManager=LONGbash-4.4$ 

our gremlin-server-custom.yml file :

host: 0.0.0.0

port: 8182

threadPoolWorker: 30

gremlinPool: 0

scriptEvaluationTimeout: 300000

channelizer: org.apache.tinkerpop.gremlin.server.channel.WebSocketChannelizer

graphs: {

  graph: conf/neo4j.properties}

scriptEngines: {

  gremlin-groovy: {

  plugins: { org.apache.tinkerpop.gremlin.server.jsr223.GremlinServerGremlinPlugin: {classMapCacheSpecification: "initialCapacity=10000,maximumSize=10000"},

               org.apache.tinkerpop.gremlin.neo4j.jsr223.Neo4jGremlinPlugin: {},

               org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {classImports: [java.lang.Math], methodImports: [java.lang.Math#*]},

               org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: [scripts/neo4j.groovy]}}}}

serializers:

  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { ioRegistries: [org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV3d0] }}            # application/vnd.gremlin-v3.0+gryo

  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { serializeResultToString: true }}                                                                      # application/vnd.gremlin-v3.0+gryo-stringd

  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0, config: { ioRegistries: [org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV3d0] }}        # application/jsonprocessors:

  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0, config: { ioRegistries: [org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV2d0] }} # application/vnd.gremlin-v2.0+json

processors:

  - { className: org.apache.tinkerpop.gremlin.server.op.session.SessionOpProcessor, config: { sessionTimeout: 900000, maxParameters: 24 }}

  - { className: org.apache.tinkerpop.gremlin.server.op.standard.StandardOpProcessor, config: { maxParameters: 24 }}

metrics: {

  consoleReporter: {enabled: false, interval: 180000},

  csvReporter: {enabled: false, interval: 180000, fileName: /data/gremlin-server-metrics.csv},

  jmxReporter: {enabled: false},

  slf4jReporter: {enabled: false, interval: 180000},

  gangliaReporter: {enabled: false, interval: 180000, addressingMode: MULTICAST},

  graphiteReporter: {enabled: false, interval: 180000}}

strictTransactionManagement: false

threadPoolBoss: 1

maxInitialLineLength: 4096

maxHeaderSize: 8192

maxChunkSize: 8192

maxContentLength: 65536000

maxAccumulationBufferComponents: 1024

resultIterationBatchSize: 64

writeBufferLowWaterMark: 32768

writeBufferHighWaterMark: 65536000

ssl: {

  enabled: false}

Thanks,

Satish.

[Satish Kumar]

Any help would be really appreciated.

[Stephen Mallette]

I'm not aware of anything in Gremlin Server that would take 30+ minutes to start. I see you are using neo4j - I think i would try to investigate slow startup times for it. If you have an especially large graph you may need to adjust some memory settings for neo4j. I seem to remember that if you don't have cache sizes and such set appropriate to your graph size you can experience significant slowness on startup. If you can isolate the problem to Gremlin Server a bit and can provide a reproducible way to recreate the problem we could look into it more closely.

On Thu, Jun 25, 2020 at 11:40 AM Satish Kumar <satishch...@gmail.com> wrote:

Any help would be really appreciated.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/92353d65-4a92-46d6-9b1b-7a674b0b070co%40googlegroups.com.

[Alex Vinnik]

Looks like Neo4J 3.2.3 kernel is the cause of that slow graph loading. I tried Gremlin 3.5.0-snapshot with Neo4J 3.4.11 kernel and it loads the same graph in 2 seconds.
Gremlin 3.4.7 with Neo4J 3.2.3 kernel takes forever (~15 minutes) to load.

 

Stephen, when do you plan to release 3.5? Any outstanding issues there?

On Thursday, June 25, 2020 at 1:09:05 PM UTC-5, Stephen Mallette wrote:

I'm not aware of anything in Gremlin Server that would take 30+ minutes to start. I see you are using neo4j - I think i would try to investigate slow startup times for it. If you have an especially large graph you may need to adjust some memory settings for neo4j. I seem to remember that if you don't have cache sizes and such set appropriate to your graph size you can experience significant slowness on startup. If you can isolate the problem to Gremlin Server a bit and can provide a reproducible way to recreate the problem we could look into it more closely.

On Thu, Jun 25, 2020 at 11:40 AM Satish Kumar <satishch...@gmail.com> wrote:

Any help would be really appreciated.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gremli...@googlegroups.com.

[Stephen Mallette]

There is no planned date to release 3.5 at this time. There are many outstanding issues in my mind, but none related to Neo4j specifically. 

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/6a3a5770-df5a-4d91-926c-9f47fee2f121o%40googlegroups.com.