strange behavior pertaining to one transaction vs many

[Javad Karabi]

im importing data from a csv file, and when i wrap the whole thing in one transaction, the performance is actually slower than multiple transactions.

when i use one transaction, most of the time is spent in the closing of the transaction, but when i wrap the whole thing in one transaction, most of the time is spent in IteratorUtil.firstOrNull, (although, again, using multiple transactions is faster)

does the performance of firstOrNull degrade as the transaction it is in gets larger?

[Javad Karabi]

i should clarify that the arguement to firstOrNull is findNodesByLabelAndProperty, so i am assuming that most of the time is actually in that function, that findNodesByLabelAndProperty may be whats taking longer as the transaction grows

[Michael Hunger]

Yep, definitely. You'd batch your transaction to sizes like 1k elements at a time.

Usually it's more (20k-50k) but there is an issue in 2.0 that does makes findNodesByLabelAndProperty slower when working in larger transactions :(

--
You received this message because you are subscribed to the Google Groups "Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Javad Karabi]

ah, would you mind elaborating a bit? i am just curious.

is this because any index updates would not have been committed to the actual index until the transaction is committed, therefore there are now _2_ indexes that must be consulted when looking up an index? The index that is already committed, and the one that is soon to be committed?

--
You received this message because you are subscribed to a topic in the Google Groups "Neo4j" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/neo4j/1-VbLSimfTw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to neo4j+un...@googlegroups.com.

[Michael Hunger]

Yep, exactly.

It was not an issue before but the implementation changed and there were some places that have not yet been fully converted. 

That's why you see a slow-down if both the in-transaction-state as well as the on-disk/persisted index state have to be consulted.

Will def. be fixed in 2.1 most certainly earlier.