Poor read performance on get vertex after creating a new index

44 views
Skip to first unread message

Tanroop Dhillon

unread,
Jul 13, 2020, 11:11:21 AM7/13/20
to JanusGraph users
Hi,

I have a property ID on both my vertex and edge.
Initially I has created a composite index on Vertex property = ID and read performance on get vertex by ID was very good. Then I created a composite index on edge property = ID. After this, get edge by ID is working great and get vertex by ID response time has drastically increased (from 50ms to 1 sec). Any help on this?

My code

try {
JanusGraphIndex index = mgmt.buildIndex("byId", Vertex.class).addKey(mgmt.getPropertyKey("ID")).unique().buildCompositeIndex();
mgmt.setConsistency(index, ConsistencyModifier.LOCK);
} catch (Exception e) {
log.info("Composite index: byId on Vertex is already defined");
}

try {
JanusGraphIndex edgeIndex = mgmt.buildIndex("byEdgeId", Edge.class).addKey(mgmt.getPropertyKey(
"ID")).buildCompositeIndex();
mgmt.setConsistency(edgeIndex, ConsistencyModifier.LOCK);
} catch (Exception e) {
log.info("Composite index: byEdgeId on Edge is already defined");
}

sparshneel chanchlani

unread,
Jul 13, 2020, 11:21:02 AM7/13/20
to janusgra...@googlegroups.com
Tanroop,
50ms is still on the higher side, Also would like to know what index backend you are using, If EalsticSearch I would suggest create mixed index, you ID data type should be string and do query with textContains. Also I see one difference in your code the vertex index is unique and the Edge one is non unique.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/827ad4db-ed89-4f56-8328-13968a0ff215o%40googlegroups.com.

Tanroop Dhillon

unread,
Jul 13, 2020, 11:53:51 AM7/13/20
to janusgra...@googlegroups.com
Sparshneel, 
Unique constraint works on vertex type only. 
Currently I am not using any indexing backend. 
The problem is that 50 ms should have been there even after creating edge instance afterwards. 
I am suspecting that reason could be same property key for both vexted and edge. 

sparshneel chanchlani

unread,
Jul 13, 2020, 11:56:38 AM7/13/20
to janusgra...@googlegroups.com
Tanroop,
These queries on graph ,it is better to use index backend. What is your query can u paste.

Tanroop Dhillon

unread,
Jul 13, 2020, 12:04:22 PM7/13/20
to janusgra...@googlegroups.com
Sparshneel, 
Just to check if there is any problem with just getting vertex by ID, I am running the basic query as follows
Suppose my ID is "user" and label "USER" (vertex index is defined on ID)
GraphTraversal<Vertex, Vertex> traversal = graph.traversal().V().has("USER", "ID", "user");
if (traversal.hasNext()) {
  Vertex v = traversal.next();
}


sparshneel chanchlani

unread,
Jul 13, 2020, 12:17:26 PM7/13/20
to janusgra...@googlegroups.com
Actually, IF u are using cassandra as backend, Repair your keyspace to 80%. Also These queries on cassandra on unique key which  is part of your partition key, now if you have row by user id it basically scan the partitions until it finds the key. Where as with index backend like elastic, its inverted index your searching becomes fast. We have seen this same issue for our use case and we had to query the index backend to improve performance coz as your data grows it will be slower.

Tanroop Dhillon

unread,
Jul 13, 2020, 12:26:13 PM7/13/20
to janusgra...@googlegroups.com
I totally agree that elastic search would improve performance. But I am trying to figure out why the performance has degraded after creating edge index
A query on edge ID after creating edge index is working absolutely fine. The 99 percentile is at 29ms , that too, for a higher load.

The query in which I am querying edge by ID is as follows

Suppose edge has property value "edge1" for ID
traversal().E().has("e", "ID", "edge1")


Tanroop Dhillon

unread,
Jul 13, 2020, 12:31:15 PM7/13/20
to janusgra...@googlegroups.com
Just to reiterate, first I created a composite index on ID with vertex type. Then I created composite index on ID with edge type.
The index on vertex type seems to have stopped working. The problem could exist in elastic search as well. 

Maybe index for vertex has been overwritten and not getting populated anymore because of same property key (ID)

On Mon, Jul 13, 2020 at 9:47 PM sparshneel chanchlani <sparshneel...@gmail.com> wrote:

sparshneel chanchlani

unread,
Jul 13, 2020, 12:41:22 PM7/13/20
to janusgra...@googlegroups.com
Delete index on edge graph query to perform better should traverse from vertex using edge
For eg see below query: 
g.V().has(‘User’,’id’,’user1’).out(“edgeId”).values() This should be the query. Also Can you share more details on what is you id data type etc.

sparshneel chanchlani

unread,
Jul 13, 2020, 12:44:50 PM7/13/20
to janusgra...@googlegroups.com

Tanroop Dhillon

unread,
Jul 13, 2020, 12:49:08 PM7/13/20
to janusgra...@googlegroups.com
Yeah. I tried the query which you shared. p99 was 100 ms. So we thought of creating edge index too. That screwed up everything. Let me go through the technical limitations link which you shared with me. 

Also it took me 24 hrs to create new edge index and reindex it . The edges existing in the system were around 270 million. 
How long the deletion process would take? Just in case you've done this. 

Tanroop Dhillon

unread,
Jul 13, 2020, 12:56:05 PM7/13/20
to janusgra...@googlegroups.com
Sparshneel , 
We are planning to add another vertex property with name VID. We will create index on VID and do one time property update for historical data. 
This is too check if same property on both edge and vertex is causing discrepancy in querying vertex by index

sparshneel chanchlani

unread,
Jul 13, 2020, 1:50:15 PM7/13/20
to janusgra...@googlegroups.com
Hmm, not sure if that is the issue coz both are considered as seperate objects.

Reply all
Reply to author
Forward
0 new messages