queries size limit / data size limit

[drk...@gmail.com]

Hello, 

I'm using the rest API and Cypher, and i'm wondering what are the limits of Batch mechanism and Cypher :

- The maximal length for a Cypher query

(For example i want to use SET on lots of properties n.name="glou", n.type="glop", mixing existing one to be update and new ones to be created)

- The maximal length for the http body ? Let's assume i batch lots of Cypher queries, i guess that at a point, it would be better to split and send several batch ?

I'm wondering that to find the best practice for massive operations : for example create 100000 nodes with theirs properties...

Is it better to send a cypher query for each nodes, and all in a batch request ?

Or create (for example) 10 nodes per cypher query, and send in batch ? 

Or create all nodes in a single cypher query ? 

Thank you

[Michael Hunger]

See neo4j.org/develop/import

And more detail at jexp.de/blog

Michael

Sent from mobile device

--
You received this message because you are subscribed to the Google Groups "Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[drk...@gmail.com]

Thanx Michael, your blog is excellent :)

I'm currently using 1.9, waiting the 2.0 but for the moment i focus on stable version.

I had understood the best practice was to use Cypher queries with parameters, using batch rest API. 

But just to be sure I have well understood, because I loose my english sometimes :

try to bundle as many operations into as few Cypher statements or HTTP request as possible 

I'm still wondering what is best ?

• a huge cypher query in 1 http request (using collection as you wrote, could be nice)
• many small cypher queries in 1 Rest-Batch-Operation 
• quite huges cypher request in few operations in 1 Rest-Batch-Operation  (between the 2 first solutions)

And moreover, if the first solution is the best, is there any limit to cypher query length ?

To update data with Cypher it is also necessary to take transaction size into account. For the embedded case, batching transactions is discussed in the next installment of this series. For the remote execution via the Neo4j REST API there are a few important things to remember. Especially with large index lookups and match results, it might happen that the query updates hundreds of thousands of elements

I think the Rest-Batch-Operation creates a single transaction, so transaction size account "problem" or "point" is the same for my 2 previous cases I guess, so does not help me to choose the good approach

Sorry if I misunderstood some points, 

Thanks, 

Jean

[Peter Neubauer]

Hi there,

I think testing variant 3 with a parameterized cypher query and maybe a couple of thousand IDs would be best. Please report back!

/peter

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

Kids in Malmö this summer?       - http://www.kidscraft.se
Neo4j questions? Please use SO - http://stackoverflow.com/search?q=neo4j

[Michael Hunger]

Thanks ;)

Sent from mobile device

Am 27.06.2013 um 12:07 schrieb drk...@gmail.com:

Thanx Michael, your blog is excellent :)

I'm currently using 1.9, waiting the 2.0 but for the moment i focus on stable version.

I had understood the best practice was to use Cypher queries with parameters, using batch rest API. 

But just to be sure I have well understood, because I loose my english sometimes :

try to bundle as many operations into as few Cypher statements or HTTP request as possible 

It was mostly focussing on creating whole subgraphs in a go.

Also using the param collection

I'm still wondering what is best ?

• a huge cypher query in 1 http request (using collection as you wrote, could be nice)
• many small cypher queries in 1 Rest-Batch-Operation 
• quite huges cypher request in few operations in 1 Rest-Batch-Operation  (between the 2 first solutions)

And moreover, if the first solution is the best, is there any limit to cypher query length ?

To update data with Cypher it is also necessary to take transaction size into account. For the embedded case, batching transactions is discussed in the next installment of this series. For the remote execution via the Neo4j REST API there are a few important things to remember. Especially with large index lookups and match results, it might happen that the query updates hundreds of thousands of elements

I think the Rest-Batch-Operation creates a single transaction, so transaction size account "problem" or "point" is the same for my 2 previous cases I guess, so does not help me to choose the good approach

To large tx mostly happens if your updates are driven by large matches 

Eg when updating or deleting parts of a large graph

Hth

Michael