[BLOG] JanusGraph write performance

[Ted Wilmes]

Hello,

There has been lots of talk of JanusGraph performance best practices. I think we could use some more consolidated tips in the 

docs but in the meantime, I wrote this post to illustrate some of the ins and outs of this sort of tuning.

--Ted

[Vincent Brouillet]

I found your article to be very useful. I'd like to get hold of the gremlin script you wrote for Gatling to inject the users. Would you be able to post that somewhere?

I'd like to compare with the same benchmark against my kubernetes setup on AWS.

I'm only getting 200/vertex insert per seconds on single thread (parts of our write process has to be run in single thread but still too slow in my opinion). While I can write easily 70,000 rows/sec directly into Cassandra. 

But to be sure we are comparing apples, running the exact same test would be useful (to tune and verify the underlying infrastructure is sound)

[Ted Wilmes]

Hi Vincent,

Thanks for checking the article out. That chunk of Gremlin at the beginning of the post that inserted the same person over and over is the entirety of the Gremlin that was executed. I'd like to open source the Gatling TinkerPop protocol but I have some cleanup work to do. I'll let you know when it's out. Can you share a snippet of how you're doing inserts?

--Ted

[Vincent Brouillet]

Something like that

// batched with unique composite index 

graph = JanusGraphFactory.open('conf/janusgraph-cassandra.properties')
mgmt = graph.openManagement()
node = mgmt.makeVertexLabel('node').make()
name = mgmt.makePropertyKey('name').dataType(String.class).make()
nameIndex = mgmt.buildIndex('nameIndex', Vertex.class).addKey(name).unique().buildCompositeIndex()
mgmt.setConsistency(name, ConsistencyModifier.LOCK)
mgmt.setConsistency(nameIndex, ConsistencyModifier.LOCK)
mgmt.commit()
g = graph.traversal()
txn = 10000
t0 = t2 = System.currentTimeMillis(); (1..200000).each{ g.addV('node').property('name', 'n_'+it).iterate(); if (it % txn == 0) graph.tx().commit(); if (it % 10000 == 0) {t3 = System.currentTimeMillis(); println ''+it+' '+(t3-t2); t2 = t3; } }; t1 = System.currentTimeMillis(); t1-t0

Some colleagues get 1,000/sec on their laptop.  I get ~200 on AWS in Docker