[gremlin-jython] "Could not create GremlinScriptEngine for gremlin-jython"

[Dave vU]

Hi,

I am running a neo4j DB that I query through the gremlin-python interface…

Things used to work under a previous 3.3.0-SNAPSHOT, but stopped working under the latest 3.3.0 tag… I am working on fixing the conf file to reflect the latest changes, but bumping into a strange error when I include gremlin-jython in the list of scriptEngines:

[INFO] ServerGremlinExecutor - Initialized Gremlin thread pool.  Threads in pool named with pattern gremlin-*

[INFO] ServerGremlinExecutor - Initialized GremlinExecutor and preparing GremlinScriptEngines instances.

[INFO] ServerGremlinExecutor - Initialized gremlin-groovy GremlinScriptEngine and registered metrics

[ERROR] DefaultGremlinScriptEngineManager - Could not create GremlinScriptEngine for gremlin-jython

java.lang.IllegalStateException: javax.script.ScriptException: NameError: name 'LabelP' is not defined in <script> at line number 1

at org.apache.tinkerpop.gremlin.python.jsr223.GremlinJythonScriptEngine.<init>(GremlinJythonScriptEngine.java:87)

at org.apache.tinkerpop.gremlin.python.jsr223.GremlinJythonScriptEngineFactory.getScriptEngine(GremlinJythonScriptEngineFactory.java:110)

at org.apache.tinkerpop.gremlin.jsr223.DefaultGremlinScriptEngineManager.createGremlinScriptEngine(DefaultGremlinScriptEngineManager.java:415)

at org.apache.tinkerpop.gremlin.jsr223.DefaultGremlinScriptEngineManager.getEngineByName(DefaultGremlinScriptEngineManager.java:219)

at org.apache.tinkerpop.gremlin.jsr223.CachedGremlinScriptEngineManager.lambda$getEngineByName$366(CachedGremlinScriptEngineManager.java:57)

at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)

at org.apache.tinkerpop.gremlin.jsr223.CachedGremlinScriptEngineManager.getEngineByName(CachedGremlinScriptEngineManager.java:57)

at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:262)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Caused by: javax.script.ScriptException: NameError: name 'LabelP' is not defined in <script> at line number 1

at org.python.jsr223.PyScriptEngine.scriptException(PyScriptEngine.java:202)

at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:42)

at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:31)

at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)

at org.apache.tinkerpop.gremlin.python.jsr223.GremlinJythonScriptEngine.<init>(GremlinJythonScriptEngine.java:75)

... 13 more

Caused by: Traceback (most recent call last):

  File "<script>", line 1, in <module>

NameError: name 'LabelP' is not defined

at org.python.core.Py.NameError(Py.java:288)

at org.python.core.PyFrame.getname(PyFrame.java:257)

at org.python.pycode._pyx10.f$0(<script>:1)

at org.python.pycode._pyx10.call_function(<script>)

at org.python.core.PyTableCode.call(PyTableCode.java:167)

at org.python.core.PyCode.call(PyCode.java:18)

at org.python.core.Py.runCode(Py.java:1403)

at org.python.core.__builtin__.eval(__builtin__.java:497)

at org.python.core.__builtin__.eval(__builtin__.java:501)

at org.python.util.PythonInterpreter.eval(PythonInterpreter.java:259)

at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:40)

... 16 more

Am I missing something in the conf file for gremlin-jython to work?

Full Conf file:

host: localhost

port: 8182

threadPoolWorker: 1

gremlinPool: 8

scriptEvaluationTimeout: 30000

graphs: {

 graph: conf/matterx-neo4j.properties}

scriptEngines: {

    gremlin-groovy: {

       plugins: {

         org.apache.tinkerpop.gremlin.server.jsr223.GremlinServerGremlinPlugin: {},

         org.apache.tinkerpop.gremlin.tinkergraph.jsr223.TinkerGraphGremlinPlugin: {},

         org.apache.tinkerpop.gremlin.neo4j.jsr223.Neo4jGremlinPlugin: {},

         org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {classImports: [java.lang.Math], methodImports: [java.lang.Math#*]},

         org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: [scripts/matterx.groovy]}}},

     gremlin-jython: {},

     gremlin-python: {}

}

serializers:

     - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { useMapperFromGraph: graph }}            # application/vnd.gremlin-v1.0+gryo

     - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0, config: { useMapperFromGraph: graph }}        # application/vnd.gremlin-v1.0+gryo-lite

     - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}        # application/vnd.gremlin-v1.0+gryo-stringd

     - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0, config: { useMapperFromGraph: graph }} # application/vnd.gremlin-v1.0+json

     - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0, config: { useMapperFromGraph: graph }} # application/vnd.gremlin-v2.0+json

     - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { useMapperFromGraph: graph }}        # application/json

processors:

  - { className: org.apache.tinkerpop.gremlin.server.op.session.SessionOpProcessor, config: { sessionTimeout: 28800000 }}

  - { className: org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor, config: { cacheExpirationTime: 600000, cacheMaxSize: 1000 }}

metrics: {

 consoleReporter: {enabled: true, interval: 180000},

slf4jReporter: {enabled: true, interval: 180000}}

strictTransactionManagement: false

threadPoolBoss: 1

maxInitialLineLength: 4096

maxHeaderSize: 8192

maxChunkSize: 8192

maxContentLength: 65536

maxAccumulationBufferComponents: 1024

resultIterationBatchSize: 64

Thanks in advance!

-- 

Dave

[Dave vU]

Quick follow-up to add that I have identified the exact point of incompatibility:

Adding the neo4j plugin to the list of plugins

org.apache.tinkerpop.gremlin.neo4j.jsr223.Neo4jGremlinPlugin: {}

triggers the gremlin-jython error…

Does that mean it is currently impossible to use a neo4j database at the same time as gremlin-jython?

-- 

Dave

[Stephen Mallette]

There are some bugs here. To start, the Neo4jGremlinPlugin isn't compatible with the gremlin-python ScriptEngine. It fails to import LabelP as a class and when python tries to make "LabelP.of()" a global function it can't find it. This works fine in Gremlin Groovy given the more advanced nature of the ScriptEngine. In trying to find you a workaround, I noticed a second bug that prevents the workaround where plugins aren't being applied in a deterministic order which of course destroys my attempts at a consistent workaround.  I've pushed fixes for all issues.

The core of the fix for your purposes is this:

https://github.com/apache/tinkerpop/commit/2a2ecb0d3d42d052e5a688678304543078b1eae4#diff-04b03b8413b61a2ad3cf0821e0478726R54

I think it would be easy enough for you to just make that little change and recompile the neo4j-gremlin module and just replace the 3.3.0 jar with that.

Thanks for reporting this and sorry for the trouble.

Stephen

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/ee7b4e67-2423-4815-8aeb-4ab542e7fae4%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Dave vU]

Hi Stephen,

Thanks for the quick reply.

So, if I'm understanding correctly, Tinkerpop 3.3.0 does not support gremlin-python with a neo4j DB? But applying this specific patch (or just updating to that commit version) should fix the issue?

We were really hoping to stick to 3.3.0 and not move for a long while: the project we are trying to use Tinkerpop with, needs some stability, and the main branch seems to break compatibility quite often… 

One strange thing is: if I remove the neo4j plugin altogether, things seem to work (gremlin-python/gremlin-jython load without issue) and I am still able to read/write from the neo4j DB. Am I missing something about what the neo4j plugin does? Is it only necessary to use neo4j-specific features, but not to use it as a generic backend?

Best regards,

-- 

Dave

On Tuesday, 29 August 2017 21:38:46 UTC+9, Stephen Mallette wrote:

There are some bugs here. To start, the Neo4jGremlinPlugin isn't compatible with the gremlin-python ScriptEngine. It fails to import LabelP as a class and when python tries to make "LabelP.of()" a global function it can't find it. This works fine in Gremlin Groovy given the more advanced nature of the ScriptEngine. In trying to find you a workaround, I noticed a second bug that prevents the workaround where plugins aren't being applied in a deterministic order which of course destroys my attempts at a consistent workaround.  I've pushed fixes for all issues.

The core of the fix for your purposes is this:

https://github.com/apache/tinkerpop/commit/2a2ecb0d3d42d052e5a688678304543078b1eae4#diff-04b03b8413b61a2ad3cf0821e0478726R54

I think it would be easy enough for you to just make that little change and recompile the neo4j-gremlin module and just replace the 3.3.0 jar with that.

Thanks for reporting this and sorry for the trouble.

Stephen

On Tue, Aug 29, 2017 at 1:18 AM, Dave vU <zedr...@gmail.com> wrote:

Quick follow-up to add that I have identified the exact point of incompatibility:

Adding the neo4j plugin to the list of plugins

org.apache.tinkerpop.gremlin.neo4j.jsr223.Neo4jGremlinPlugin: {}

triggers the gremlin-jython error…

Does that mean it is currently impossible to use a neo4j database at the same time as gremlin-jython?

-- 

Dave

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

[Stephen Mallette]

> So, if I'm understanding correctly, Tinkerpop 3.3.0 does not support gremlin-python with a neo4j DB? 

Yes - the neo4j plugin provides an incompatible static import to gremlin-python

> But applying this specific patch (or just updating to that commit version) should fix the issue?

yes - the one line fix i referenced rectifies the problem but the commit version on the whole should work fine. just check that commit out, build, and replace the tinkerpop neo4j jar

> One strange thing is: if I remove the neo4j plugin altogether, things seem to work (gremlin-python/gremlin-jython load without issue) and I am still able to read/write from the neo4j DB. Am I missing something about what the neo4j plugin does? Is it only necessary to use neo4j-specific features, but not to use it as a generic backend?

ahh - i didn't think of this. well - that could be your workaround. the neo4j plugin only imports neo4j specific classes. if you don't use any of those then it will work. if you do use them you'd have to specify the full package name to the class in your script. you could also import them manually (which i tried yesterday in a different way and failed but the following works):

scriptEngines: {

  gremlin-groovy: {

    plugins: { org.apache.tinkerpop.gremlin.server.jsr223.GremlinServerGremlinPlugin: {},

               org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: { 

                 classImports: [

                   "org.apache.tinkerpop.gremlin.neo4j.structure.Neo4jElement",

                   "org.apache.tinkerpop.gremlin.neo4j.structure.Neo4jGraph",

                   "org.apache.tinkerpop.gremlin.neo4j.structure.Neo4jGraphVariables",

                   "org.apache.tinkerpop.gremlin.neo4j.structure.Neo4jHelper",

                   "org.apache.tinkerpop.gremlin.neo4j.structure.Neo4jProperty",

                   "org.apache.tinkerpop.gremlin.neo4j.structure.Neo4jVertex",

                   "org.apache.tinkerpop.gremlin.neo4j.structure.Neo4jVertexProperty",

                   "org.apache.tinkerpop.gremlin.neo4j.process.traversal.LabelP"],

                 methodImports: [org.apache.tinkerpop.gremlin.neo4j.process.traversal.LabelP#*]},

               org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: [scripts/empty-sample.groovy]}}},

  gremlin-python: {}}

Now the ImportGremlinPlugin mimics what the Neo4jGremlinPlugin does automatically. That's your workaround to stay on 3.3.0 without change.

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/16cb984a-8c4c-4bae-96ac-8e7b453bdce7%40googlegroups.com.

[Stephen Mallette]

Actually - it's interesting that you use the Gremlin Python ScriptEngine - could you share how you are using it? are you actually submitting python Gremlin scripts to Gremlin Server for execution? It would be interesting to know how that is working for you.