Gremlin/Neo4j kernel property index usage

Alex Vinnik

unread,

Jan 13, 2021, 5:30:22 PM1/13/21

to Gremlin-users

Gremlin server version: 3.3.10

Neo4J kernel version: 3.2.3

Hi there,

I am trying to enable neo4j indexes for node properties by setting the following configuration.

gremlin.neo4j.conf.dbms.auto_index.nodes.enabled=true
gremlin.neo4j.conf.dbms.auto_index.nodes.keys=PropName

I do see that index files are created (see below). But query "g.V().has('PropName', 'value').profile()" is not any faster than without indexing enabled. What is missing? How can I validate that index is actually being used? Please advise.

Many thanks in advance.

/opt/gremlin-server/data/index/lucene/node/node_auto_index

bash-4.4$ ls

_an.fdt _o3.fdt _sv.fdt _t5.cfe _tp.cfe _uj.cfe _um.cfe _up.cfe _us.cfe

_an.fdx _o3.fdx _sv.fdx _t5.cfs _tp.cfs _uj.cfs _um.cfs _up.cfs _us.cfs

_an.fnm _o3.fnm _sv.fnm _t5.si _tp.si _uj.si _um.si _up.si _us.si

_an.si _o3.si _sv.si _te.cfe _tz.cfe _uk.cfe _un.cfe _uq.cfe _ut.cfe

_an_Lucene50_0.doc _o3_Lucene50_0.doc _sv_Lucene50_0.doc _te.cfs _tz.cfs _uk.cfs _un.cfs _uq.cfs _ut.cfs

_an_Lucene50_0.tim _o3_Lucene50_0.tim _sv_Lucene50_0.tim _te.si _tz.si _uk.si _un.si _uq.si _ut.si

_an_Lucene50_0.tip _o3_Lucene50_0.tip _sv_Lucene50_0.tip _tf.cfe _u9.cfe _ul.cfe _uo.cfe _ur.cfe segments_3

_an_Lucene54_0.dvd _o3_Lucene54_0.dvd _sv_Lucene54_0.dvd _tf.cfs _u9.cfs _ul.cfs _uo.cfs _ur.cfs write.lock

_an_Lucene54_0.dvm _o3_Lucene54_0.dvm _sv_Lucene54_0.dvm _tf.si _u9.si _ul.si _uo.si _ur.si

Stephen Mallette

unread,

Jan 14, 2021, 8:52:14 AM1/14/21

to gremli...@googlegroups.com

Does it get faster if you include the vertex label in your query? like:

g.V().has("person", "PropName", "value")

Note that neo4j index support needs some attention to get smarter:

https://issues.apache.org/jira/browse/TINKERPOP-1575

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/639a7751-fb3e-4db7-87b5-dc82a0e1333an%40googlegroups.com.

Alex Vinnik

unread,

Jan 14, 2021, 11:20:02 AM1/14/21

to gremli...@googlegroups.com

Yes. Use of a label makes it 30 times faster according to the profile in my case. Thanks for your response.

gremlin> g.V().has("TenantID", "tenant1").profile()
==>Traversal Metrics
Step                                                               Count  Traversers       Time (ms)    % Dur
=============================================================================================================
Neo4jGraphStep(vertex,[TenantID.eq(tenant1)])                       1001        1001         181.748   100.00
                                            >TOTAL                     -           -         181.748        -
gremlin> g.V().has("tenant", "TenantID", "tenant1").profile()
==>Traversal Metrics
Step                                                               Count  Traversers       Time (ms)    % Dur
=============================================================================================================
Neo4jGraphStep(vertex,[~label.eq(tenant), Tenan...                     1           1           6.231   100.00
                                            >TOTAL                     -           -           6.231        -

You received this message because you are subscribed to a topic in the Google Groups "Gremlin-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gremlin-users/r042o3VjcdU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H43_4UHVMANOc%3D6D6YVrA3wYHNaDXd2h_ZNFfSm42itKO%2BA%40mail.gmail.com.

Alex Vinnik

unread,

Jan 14, 2021, 11:58:19 AM1/14/21

to gremli...@googlegroups.com

Stephen,

One question still puzzles me. Why doesn't property look up use an index? Something misconfigured?

On Thu, Jan 14, 2021 at 7:52 AM Stephen Mallette <spmal...@gmail.com> wrote:

You received this message because you are subscribed to a topic in the Google Groups "Gremlin-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gremlin-users/r042o3VjcdU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H43_4UHVMANOc%3D6D6YVrA3wYHNaDXd2h_ZNFfSm42itKO%2BA%40mail.gmail.com.

Stephen Mallette

unread,

Jan 14, 2021, 12:21:11 PM1/14/21

to gremli...@googlegroups.com

At the time when this was implemented I seem to recall that the Neo4j APIs required the label for the index lookup to work. I dont know if that has changed. You can see the code for how it works here:

https://github.com/apache/tinkerpop/blob/3.4.9/neo4j-gremlin/src/main/java/org/apache/tinkerpop/gremlin/neo4j/structure/trait/NoMultiNoMetaNeo4jTrait.java#L151-L190

The path to getting to this code is a bit convoluted on 3.4.x is a bit more straightforward on the master branch for 3.5.0 where we drop multi/meta property support for neo4j:

https://github.com/apache/tinkerpop/blob/a11796c887b38585720cbae067b377d05809538c/neo4j-gremlin/src/main/java/org/apache/tinkerpop/gremlin/neo4j/process/traversal/step/sideEffect/Neo4jGraphStep.java#L70-L127

We'd be happy to review a pull request if you feel you could improve things. Thanks!

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CALd8TVF9ZLYZD9P%2BYftNom%3Dk0NS%2B9pPiikpQwOp0y37SO5Msig%40mail.gmail.com.

amit.c...@gmail.com

unread,

Jan 14, 2021, 12:37:32 PM1/14/21

to Gremlin-users

I think thats still the case as per documentation here

Reply all

Reply to author

Forward