Transactions and uploading a lot of vertices

[Alexander Scherbatiy]

I am trying to upload several thousands vertices into JanusGraph for my testing.

Is there any performance benefits to commit the transaction after adding all vertices, after adding each vertex or may be there is an optimal vertices batch size that can be used to make the commit after uploading a certain amount of vertices?

Thanks,

Alexander.

[Pavel Ershov]

It depends on graph structure how many properties/edges

Good starting point do commit every 10k vertices. Addition info about bulk loading https://docs.janusgraph.org/advanced-topics/bulk-loading/

среда, 25 сентября 2019 г., 23:14:14 UTC+3 пользователь Alexander Scherbatiy написал:

[Alexander Scherbatiy]

I have a graph that I need to upload which has about 1000 vertices and
3000 edges.

I tried to load it using commit after
a) each iteration
b) every 100th iteration
c) after all data uploading

It took about a) 2.3s b) 2.3s c) 9.8 sec on my system.

I used 'graph.tx().commit()' code to commit uploaded data in transaction.

My graph parameters are:
JanusGraph graph = JanusGraphFactory.build()
.set("storage.backend", "inmemory")
.set("graph.set-vertex-id", "true")
.set("ids.block-size", "100000")
.set("ids.authority.wait-time", "5")
.set("ids.renew-timeout", "50")
.open();

The profiler shows that the most time in c) which uses only one commit
after whole data uploading takes the method
JanusGraphStep.executeGraphCentryQuery where it creates
Sets.newHashSet() and then takes a lot time over iterating elements.

By the way, what is the right way to call transactions in JanusGraph?
The Transactions chapter in JanusGraph doc describes only examples of
'graph.tx().commit()' and 'graph.tx().createThreadedTx()'.

There is also method 'graph.newTransaction()'.
Is there drawbacks to explicitly create transactions by
'graph.newTransaction()' over using the default one and committing by
'graph.tx().commit()'?

Thanks,
Alexander.

> --
> You received this message because you are subscribed to the Google
> Groups "JanusGraph users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to janusgraph-use...@googlegroups.com
> <mailto:janusgraph-use...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/janusgraph-users/3ecea828-b9a7-4e98-9376-01a66bb07eb5%40googlegroups.com
> <https://groups.google.com/d/msgid/janusgraph-users/3ecea828-b9a7-4e98-9376-01a66bb07eb5%40googlegroups.com?utm_medium=email&utm_source=footer>.