How to make an index to improve performance of range() queries?
683 views
Skip to first unread message
Alexandr Porunov
Mar 30, 2018, 4:42:48 PM3/30/18
to JanusGraph users
Hello,
I've noticed that range() performance is depend on two things:
1. Indexed elements count
2. Offset of the query
I have vertices with label "location" and property "location_type" which has values 0, 1, 2 or 3.
My vertices count:
with "location_type=0" is 7 vertices
with "location_type=1" is 252 vertices
with "location_type=2" is ~380,000 vertices
with "location_type=3" is ~4,700,000 vertices
I tried to test range queries against those sets of data with two types of indexes: composite and mixed. For mixed indexes I have Elasticsearch 6.2. As a backend storage I have ScyllaDB 2.1.1.
When I tested composite indexes I used:
mgmt.buildIndex("location_type_index", Vertex.class).indexOnly(locationVertexLabel)
.addKey(locationTypeProperty)
.buildCompositeIndex();
When I tested mixed indexes I used:
mgmt.buildIndex("location_type_index", Vertex.class).indexOnly(locationVertexLabel)
.addKey(locationTypeProperty)
.buildMixedIndex("search");
A query which I try to perform ("from" and "to" are long variables):
graph.traversal().V().hasLabel("location").has("location_type", 3).range(from, to).toList()
Below are my tests (each test retrieves 11000 vertices).
For composite indexes:
Test 1:
from = 0
to = 11000
Time to get vertices: 27 seconds
Test 2:
from = 1000000
to = 1011000
Time to get vertices: 77 seconds
Test 3:
from = 2000000
to = 2011000
Time to get vertices: 98 seconds
Test 4:
from = 4000000
to = 4011000
Time to get vertices: 108 seconds
For mixed indexes:
Test 1:
from = 0
to = 11000
Time to get vertices: 5 seconds
Test 2:
from = 100000
to = 111000
Time to get vertices: 77 seconds
Test 3:
from = 4000000
to = 4011000
Time to get vertices: fed up of waiting (waited for 1 hour and 15 minutes and then killed the process)
Also, I tried to perform similar tests with "location_type = 2" and the speed was much better. With "location_type=1" or "location_type=0" speed is less than 1 second.
Is there a possibility to improve performance by adding / changing some indexes or improve a query?
Best regards,
Alexandr
I've noticed that range() performance is depend on two things:
1. Indexed elements count
2. Offset of the query
I have vertices with label "location" and property "location_type" which has values 0, 1, 2 or 3.
My vertices count:
with "location_type=0" is 7 vertices
with "location_type=1" is 252 vertices
with "location_type=2" is ~380,000 vertices
with "location_type=3" is ~4,700,000 vertices
I tried to test range queries against those sets of data with two types of indexes: composite and mixed. For mixed indexes I have Elasticsearch 6.2. As a backend storage I have ScyllaDB 2.1.1.
When I tested composite indexes I used:
mgmt.buildIndex("location_type_index", Vertex.class).indexOnly(locationVertexLabel)
.addKey(locationTypeProperty)
.buildCompositeIndex();
When I tested mixed indexes I used:
mgmt.buildIndex("location_type_index", Vertex.class).indexOnly(locationVertexLabel)
.addKey(locationTypeProperty)
.buildMixedIndex("search");
A query which I try to perform ("from" and "to" are long variables):
graph.traversal().V().hasLabel("location").has("location_type", 3).range(from, to).toList()
Below are my tests (each test retrieves 11000 vertices).
For composite indexes:
Test 1:
from = 0
to = 11000
Time to get vertices: 27 seconds
Test 2:
from = 1000000
to = 1011000
Time to get vertices: 77 seconds
Test 3:
from = 2000000
to = 2011000
Time to get vertices: 98 seconds
Test 4:
from = 4000000
to = 4011000
Time to get vertices: 108 seconds
For mixed indexes:
Test 1:
from = 0
to = 11000
Time to get vertices: 5 seconds
Test 2:
from = 100000
to = 111000
Time to get vertices: 77 seconds
Test 3:
from = 4000000
to = 4011000
Time to get vertices: fed up of waiting (waited for 1 hour and 15 minutes and then killed the process)
Also, I tried to perform similar tests with "location_type = 2" and the speed was much better. With "location_type=1" or "location_type=0" speed is less than 1 second.
Is there a possibility to improve performance by adding / changing some indexes or improve a query?
Best regards,
Alexandr
Alexandr Porunov
Mar 30, 2018, 4:51:09 PM3/30/18
to JanusGraph users
A little correction for the Test 2 (mixed indexes): 92 seconds
Test 3 (mixed index): Tried to perform it again: 1 hour 28 minutes.
Test 3 (mixed index): Tried to perform it again: 1 hour 28 minutes.
Daniel Kuppitz
Mar 30, 2018, 5:49:19 PM3/30/18
to JanusGraph users
Is location_type the only indexed property? If so, try to add another one (preferably a unique property) that can also serve as a sort criterion. Let's say you also have a name property in your (mixed) index, then your queries would look like this:
g.V().has("location","location_type",type).order().by("name").limit(1000) // page 1g.V().has("location","location_type",type).has("name", gt(last_name)).order().by("name").limit(1000) // page 2; last_name is the last name on page 1g.V().has("location","location_type",type).has("name", gt(last_name)).order().by("name").limit(1000) // page 3; last_name is the last name on page 2
Cheers,
Daniel
--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/86148d4d-c0c5-4d85-b047-3e6675ae19bb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Alexandr Porunov
Mar 31, 2018, 1:24:50 PM3/31/18
to JanusGraph users
Hello Daniel,
I tried to add "location_id" property to the index also and used your suggestions:
Here is the index with the additional property:
mgmt.buildIndex("location_type_index", Vertex.class).indexOnly(locationVertexLabel)
.addKey(locationType)
.addKey(locationId)
.buildMixedIndex("search");
Here are queries which I tried to perform with that index:
// Query 1:
graph.traversal().V().hasLabel("location")
.has("location_type", 3)
.has("name", P.gt(6255148))
.order().by("location_id")
.limit(11000).toList()
// Query 2:
graph.traversal().V().hasLabel("location")
.has("location_type", 3)
.order().by("location_id")
.limit(11000).toList();
// Query 3:
graph.traversal().V().hasLabel("location")
.has("location_type" 3)
.order().by("location_id")
.range(100000, 111000).toList();
Unfortunately, all queries are performing very bad. I couldn't wait for results. I waited approximately 40 minutes for each query to be done but neither of them was able to return results in 40 minutes. This is too long for retrieving 11000 vertices.
I see that JanusGraph has created an index in Elasticsearch but it seems that the index is too slow:
yellow open janusgraph_location_type_index RFeq3AztR4q9m496dQxzXA 5 1 5044332 0 273.4mb 273.4mb
Do you know if there is another option to speed up range() or limit retrieval?
Best regards,
Alexandr
I tried to add "location_id" property to the index also and used your suggestions:
Here is the index with the additional property:
mgmt.buildIndex("location_type_index", Vertex.class).indexOnly(locationVertexLabel)
.addKey(locationType)
.addKey(locationId)
.buildMixedIndex("search");
Here are queries which I tried to perform with that index:
// Query 1:
graph.traversal().V().hasLabel("location")
.has("location_type", 3)
.has("name", P.gt(6255148))
.order().by("location_id")
.limit(11000).toList()
// Query 2:
graph.traversal().V().hasLabel("location")
.has("location_type", 3)
.order().by("location_id")
.limit(11000).toList();
// Query 3:
graph.traversal().V().hasLabel("location")
.has("location_type" 3)
.order().by("location_id")
.range(100000, 111000).toList();
Unfortunately, all queries are performing very bad. I couldn't wait for results. I waited approximately 40 minutes for each query to be done but neither of them was able to return results in 40 minutes. This is too long for retrieving 11000 vertices.
I see that JanusGraph has created an index in Elasticsearch but it seems that the index is too slow:
yellow open janusgraph_location_type_index RFeq3AztR4q9m496dQxzXA 5 1 5044332 0 273.4mb 273.4mb
Do you know if there is another option to speed up range() or limit retrieval?
Best regards,
Alexandr
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
Daniel Kuppitz
Mar 31, 2018, 2:24:46 PM3/31/18
to JanusGraph users
Hi Alexandr,
query 1 should be fast, however, it looks like there's a typo in it (you don't have a name property in your index):
g.V().hasLabel("location").has("location_type", 3).has("location_id", P.gt(6255148)).order().by("location_id").limit(11000).toList()
Cheers,
Daniel
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/bcb64138-c64f-4f6c-adbb-2481d03b0926%40googlegroups.com.
Alexandr Porunov
Mar 31, 2018, 5:57:50 PM3/31/18
to JanusGraph users
Hi Daniel,
Thank you for pointing the mistake! I changed it to:
Thank you for pointing the mistake! I changed it to:
g.V().hasLabel("location").has("location_type", 3).has("location_id", P.gt(6255148)).order().by("location_id").limit(11000).toList()
But still don't have any result after 70 minutes.
I think the index doesn't work properly. What do you think? Is there a possibility to check it?
Best regards,
Alexandr
Alexandr Porunov
Apr 1, 2018, 2:22:06 PM4/1/18
to JanusGraph users
I tried to perform next query with Elasticsearch:
curl -XGET '10.10.10.50:9200/janusgraph_location_type_index/_search?pretty' -H 'Content-Type: application/json' -d'
{
"sort" : [
{ "location_id" : "asc" },
"_score"
],
"from" : 2000000, "size" : 11000,
"query" : {
"bool" : {
"must" : {
"match" : {
"location_type" : 3
}
},
"filter" : {
"range" : {
"location_id" : { "gt" : 6255148 }
}
}
}
}
}
'
It takes about 4 seconds to finish. The result is the next:
{
"took" : 3736,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 2372187,
"max_score" : null,
"hits" : [
{
"_index" : "janusgraph_location_type_index",
"_type" : "location_type_index",
"_id" : "v9s5mw",
"_score" : 1.0,
"_source" : {
"location_type" : 3,
"location_id" : 11090923
},
"sort" : [
11090923,
1.0
]
},
{
"_index" : "janusgraph_location_type_index",
"_type" : "location_type_index",
"_id" : "v9s8so",
"_score" : 1.0,
"_source" : {
"location_type" : 3,
"location_id" : 11090924
},
"sort" : [
11090924,
1.0
]
},
{
"_index" : "janusgraph_location_type_index",
"_type" : "location_type_index",
"_id" : "v8lb80",
"_score" : 1.0,
"_source" : {
"location_id" : 11090925,
"location_type" : 3
},
"sort" : [
11090925,
1.0
]
},
// AND 10997 OTHER SIMILAR RESULTS
I don't see here a vertex id property which is strange. How Janusgraph retrieves vertices with this result? Elasticsearch definitely works fast. The question only, how JanusGraph works with indexes. Because the query (which is analog to above query performs very long. Definitely more than 1 hour but it never was finished.):
curl -XGET '10.10.10.50:9200/janusgraph_location_type_index/_search?pretty' -H 'Content-Type: application/json' -d'
{
"sort" : [
{ "location_id" : "asc" },
"_score"
],
"from" : 2000000, "size" : 11000,
"query" : {
"bool" : {
"must" : {
"match" : {
"location_type" : 3
}
},
"filter" : {
"range" : {
"location_id" : { "gt" : 6255148 }
}
}
}
}
}
'
It takes about 4 seconds to finish. The result is the next:
{
"took" : 3736,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 2372187,
"max_score" : null,
"hits" : [
{
"_index" : "janusgraph_location_type_index",
"_type" : "location_type_index",
"_id" : "v9s5mw",
"_score" : 1.0,
"_source" : {
"location_type" : 3,
"location_id" : 11090923
},
"sort" : [
11090923,
1.0
]
},
{
"_index" : "janusgraph_location_type_index",
"_type" : "location_type_index",
"_id" : "v9s8so",
"_score" : 1.0,
"_source" : {
"location_type" : 3,
"location_id" : 11090924
},
"sort" : [
11090924,
1.0
]
},
{
"_index" : "janusgraph_location_type_index",
"_type" : "location_type_index",
"_id" : "v8lb80",
"_score" : 1.0,
"_source" : {
"location_id" : 11090925,
"location_type" : 3
},
"sort" : [
11090925,
1.0
]
},
// AND 10997 OTHER SIMILAR RESULTS
I don't see here a vertex id property which is strange. How Janusgraph retrieves vertices with this result? Elasticsearch definitely works fast. The question only, how JanusGraph works with indexes. Because the query (which is analog to above query performs very long. Definitely more than 1 hour but it never was finished.):
g.V().hasLabel("location").
has("location_type", 3).
has("location_id", P.gt(6255148)).
order().by("location_id").
limit(11000).toList()
Am I doing something wrong? Maybe I should configure JanusGraph to work properly with indexes?
Best regards,
Alexandr
Best regards,
Alexandr
Alexandr Porunov
Apr 2, 2018, 7:59:49 AM4/2/18
to JanusGraph users
I've checked how the query performs with profile(). It takes about 170 minutes to be executed.
Here is the query:
Here is the query:
g.V().hasLabel("location").
has("location_type", 3).
has("location_id", P.gt(6255148)).
order().by("location_id").
limit(11000).profile().next()
Here is the result:
"Traversal Metrics
Step Count Traversers Time (ms) % Dur
=============================================================================================================
JanusGraphStep([],[~label.eq(location), locatio... 2372187 2372187 6831676.311 66.42
\_condition=(~label = location AND location_type = 3 AND location_id > 6255148)
\_isFitted=true
\_query=[(location_type = 3 AND location_id > 6255148)]:location_type_index
\_index=location_type_index
\_orders=[]
\_isOrdered=true
\_index_impl=search
optimization 116.069
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
backend-query 2372187 2890923.446
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
OrderGlobalStep([[value(location_id), incr]]) 11001 11001 3453278.237 33.58
RangeGlobalStep(0,11000) 11000 11000 6.969 0.00
>TOTAL - - 10284961.518 -
Does somebody know why it takes so long time?
Best regards,
Alexandr
Here is the result:
"Traversal Metrics
Step Count Traversers Time (ms) % Dur
=============================================================================================================
JanusGraphStep([],[~label.eq(location), locatio... 2372187 2372187 6831676.311 66.42
\_condition=(~label = location AND location_type = 3 AND location_id > 6255148)
\_isFitted=true
\_query=[(location_type = 3 AND location_id > 6255148)]:location_type_index
\_index=location_type_index
\_orders=[]
\_isOrdered=true
\_index_impl=search
optimization 116.069
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
backend-query 2372187 2890923.446
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)]:location_type_index
OrderGlobalStep([[value(location_id), incr]]) 11001 11001 3453278.237 33.58
RangeGlobalStep(0,11000) 11000 11000 6.969 0.00
>TOTAL - - 10284961.518 -
Does somebody know why it takes so long time?
Best regards,
Alexandr
Kevin Schmidt
Apr 2, 2018, 9:38:03 AM4/2/18
to Alexandr Porunov, JanusGraph users
Did you create/add your indexes to a populated graph? If so, did you follow the process outlined in the documentation to reindex?
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/f997a8f5-9cea-4988-a5a8-e960e789b1f5%40googlegroups.com.To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
Alexandr Porunov
Apr 2, 2018, 9:56:36 AM4/2/18
to JanusGraph users
Hello Kevin,
Yes, I was following the documentation when I was creating indexes. The method which I used is the next:
private void createIndexes(){
mgmt = graph.openManagement();
PropertyKey locationId = mgmt.getPropertyKey("location_id");
PropertyKey locationType = mgmt.getPropertyKey("location_type");
VertexLabel locationVertexLabel = mgmt.getVertexLabel("location");
mgmt.buildIndex("location_type_index", Vertex.class)
.addKey(locationType)
.addKey(locationId)
.indexOnly(locationVertexLabel)
.buildMixedIndex("search");
mgmt.buildIndex("location_id_index", Vertex.class)
.addKey(locationId)
.indexOnly(locationVertexLabel)
.buildMixedIndex("search");
mgmt.commit();
graph.tx().commit();
try {
ManagementSystem.awaitGraphIndexStatus(graph, "location_type_index").call();
} catch (InterruptedException e) {
e.printStackTrace();
}
try {
ManagementSystem.awaitGraphIndexStatus(graph, "location_id_index").call();
} catch (InterruptedException e) {
e.printStackTrace();
}
graph.tx().commit();
mgmt = graph.openManagement();
try {
mgmt.updateIndex(mgmt.getGraphIndex("location_type_index"), SchemaAction.REINDEX).get();
mgmt.updateIndex(mgmt.getGraphIndex("location_id_index"), SchemaAction.REINDEX).get();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
mgmt.commit();
graph.tx().commit();
}
Also, as I mentioned above, I checked indexes in Elasticsearch. They are created and indexed.
Also, as seen above from the profile(), when I was performing the query which takes ~170 minutes to be executed, JanusGraph was using the index "location_type_index".
So, I assume the index was created correctly.
Best regards,
Alexandr
Yes, I was following the documentation when I was creating indexes. The method which I used is the next:
private void createIndexes(){
mgmt = graph.openManagement();
PropertyKey locationId = mgmt.getPropertyKey("location_id");
PropertyKey locationType = mgmt.getPropertyKey("location_type");
VertexLabel locationVertexLabel = mgmt.getVertexLabel("location");
mgmt.buildIndex("location_type_index", Vertex.class)
.addKey(locationType)
.addKey(locationId)
.indexOnly(locationVertexLabel)
.buildMixedIndex("search");
mgmt.buildIndex("location_id_index", Vertex.class)
.addKey(locationId)
.indexOnly(locationVertexLabel)
.buildMixedIndex("search");
mgmt.commit();
graph.tx().commit();
try {
ManagementSystem.awaitGraphIndexStatus(graph, "location_type_index").call();
} catch (InterruptedException e) {
e.printStackTrace();
}
try {
ManagementSystem.awaitGraphIndexStatus(graph, "location_id_index").call();
} catch (InterruptedException e) {
e.printStackTrace();
}
graph.tx().commit();
mgmt = graph.openManagement();
try {
mgmt.updateIndex(mgmt.getGraphIndex("location_type_index"), SchemaAction.REINDEX).get();
mgmt.updateIndex(mgmt.getGraphIndex("location_id_index"), SchemaAction.REINDEX).get();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
mgmt.commit();
graph.tx().commit();
}
Also, as I mentioned above, I checked indexes in Elasticsearch. They are created and indexed.
Also, as seen above from the profile(), when I was performing the query which takes ~170 minutes to be executed, JanusGraph was using the index "location_type_index".
So, I assume the index was created correctly.
Best regards,
Alexandr
Daniel Kuppitz
Apr 2, 2018, 10:39:58 AM4/2/18
to JanusGraph users
Can you try that without the location_id_index? This index definition actually makes no sense, as it only covers a field that is already covered by location_type_index. Maybe this is messing thing up.
Cheers,
Daniel
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/88ab138b-7100-4afc-8612-771813802585%40googlegroups.com.
Alexandr Porunov
Apr 2, 2018, 11:36:22 AM4/2/18
to JanusGraph users
Hello Daniel,
I just tried to recreate the database with only one index (location_type_index) and run the query again. It didn't help. The query still takes too much time. I think that JanusGraph works incorrectly with indexes. It works OK when there are no many indexed vertices (less than half a million) but when there are many vertices (in my case more than 5 million vertices) it works very very slow. When I try to execute search operations on JanusGraph index directly in ElasticSearch it works very fast. When I try to execute the same (as I think) query, but through JanusGraph it works super slow.
Also, I didn't find vertex id property in responses from ElasticSearch. If there is no vertex id property how JanusGraph can return vertices from ElasticSearch? Is it possible that JanusGraph has a bug with ElasticSearch 6.2?
Best regards,
Alexandr
I just tried to recreate the database with only one index (location_type_index) and run the query again. It didn't help. The query still takes too much time. I think that JanusGraph works incorrectly with indexes. It works OK when there are no many indexed vertices (less than half a million) but when there are many vertices (in my case more than 5 million vertices) it works very very slow. When I try to execute search operations on JanusGraph index directly in ElasticSearch it works very fast. When I try to execute the same (as I think) query, but through JanusGraph it works super slow.
Also, I didn't find vertex id property in responses from ElasticSearch. If there is no vertex id property how JanusGraph can return vertices from ElasticSearch? Is it possible that JanusGraph has a bug with ElasticSearch 6.2?
Best regards,
Alexandr
Daniel Kuppitz
Apr 2, 2018, 12:49:42 PM4/2/18
to JanusGraph users
The vertex id is encoded in ES and it's actually included in a snippet you've posted earlier. Anyway, I'm not sure what's going on there performance-wise, whether you need to tune ES or if JanusGraph is doing something wrong. All I can say is that the query times you've reported look excessively long to me. Would be nice if Janus devs could chime in here.
Cheers,
Daniel
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/8ad4ce72-7916-4e00-b8bd-90c11fdd3f6b%40googlegroups.com.
Jason Plurad
Apr 2, 2018, 12:55:35 PM4/2/18
to JanusGraph users
If you're running with 0.2.0 release, you might be hitting this issue "order().by(property) doesn't work with mixed index"
https://github.com/JanusGraph/janusgraph/issues/252
If you're able to build the master branch, you could verify if it is resolved.
https://github.com/JanusGraph/janusgraph/issues/252
If you're able to build the master branch, you could verify if it is resolved.
On Monday, April 2, 2018 at 12:49:42 PM UTC-4, Daniel Kuppitz wrote:
The vertex id is encoded in ES and it's actually included in a snippet you've posted earlier. Anyway, I'm not sure what's going on there performance-wise, whether you need to tune ES or if JanusGraph is doing something wrong. All I can say is that the query times you've reported look excessively long to me. Would be nice if Janus devs could chime in here.Cheers,Daniel
Alexandr Porunov
Apr 2, 2018, 2:10:01 PM4/2/18
to JanusGraph users
Hello Jason,
Thank you very much for the help!
I've compiled the latest JanusGraph version from the master branch and now the latest query works! Speeded up performance from 171 minutes to 16 seconds!
Still, I have an issue with range queries.
Here are my new tests:
// TEST 1
Query:
Thank you very much for the help!
I've compiled the latest JanusGraph version from the master branch and now the latest query works! Speeded up performance from 171 minutes to 16 seconds!
Still, I have an issue with range queries.
Here are my new tests:
// TEST 1
Query:
g.V().hasLabel("location").
has("location_type", 3).
has("location_id", P.gt(6255148)).
order().by("location_id").
limit(11000).profile().next()
Result: 16 seconds
Detailed results:
Detailed results:
"Traversal Metrics
Step Count Traversers Time (ms) % Dur
=============================================================================================================
JanusGraphStep([],[~label.eq(location), locatio... 11000 11000 15813.934 100.00
\_condition=(~label = location AND location_type = 3 AND location_id > 6255148)
\_isFitted=true
\_query=[(location_type = 3 AND location_id > 6255148)][ASC(location_id)]:location_type_index
\_limit=11000
\_index=location_type_index
\_orders=[ASC(location_id)]
\_isOrdered=true
\_index_impl=search
optimization 33.050
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)][ASC(location_id)]:location_ty
pe_index
>TOTAL - - 15813.934 -
//TEST 2
Query:
\_limit=11000
\_index=location_type_index
\_orders=[ASC(location_id)]
\_isOrdered=true
\_index_impl=search
optimization 33.050
backend-query 0.000
\_query=location_type_index:[(location_type = 3 AND location_id > 6255148)][ASC(location_id)]:location_ty
pe_index
>TOTAL - - 15813.934 -
//TEST 2
Query:
g.V().hasLabel("location").
has("location_type", 3).
order().by("location_id").
limit(11000).toList()
Result: 26 seconds
//TEST 3
Query:
limit(11000).toList()
Result: 26 seconds
//TEST 3
Query:
g.V().hasLabel("location").
has("location_type", 3).
order().by("location_id").
range(100000, 111000).toList()
Result: 8 minutes
//TEST 4
Query:
range(100000, 111000).toList()
Result: 8 minutes
//TEST 4
Query:
g.V().hasLabel("location").
has("location_type", 3).
order().by("location_id").
range(4000000, 4011000).toList()
Result: Very long. Don't have results yet
Do you know if JanusGraph works with range() queries? Maybe it has the same bug which it had with order().by(property)?
Best regards,
Alexandr
range(4000000, 4011000).toList()
Result: Very long. Don't have results yet
Do you know if JanusGraph works with range() queries? Maybe it has the same bug which it had with order().by(property)?
Best regards,
Alexandr
Jason Plurad
Apr 2, 2018, 3:54:45 PM4/2/18
to JanusGraph users
Reply all
Reply to author
Forward
0 new messages