Import delta

[koen]

Hi,  a generic question.  Lets assume you import data from a CSV file into neo4j using cypher "LOAD CSV".
The first/initial run will give you a full load of all nodes / properties Lets assume this results in several node types each of them with several properties and several relationships..

Now i have a new CSV file of the same format as the initial load one.   In this file some new records might have been added and for existing records (nodes or rels) the properties might have been changed.

My question .. what would be the best approach to process such "delta" import ..

1) resolve the differences during pre-processing and generate a delte import file ..
OR
2) is there a clever way to write cypher code to import initial loads (which is the straightforward part) and also verify at the same time if existing nodes need to be updated when doing a delta import

thanks

Koen

[koen]

In addition..     so if i would run a LOAD CSV twice in a row with exactly the same file i would like nothing to happen to the nodes / rels  (meaning no creation of duplicate rels etc).. If i would run the import with almost the same csv file (with only one property changed somewhere in the file )  only that specific property would be updated in that specific node and nothing would change to the other rels / nodes ..  so what are the cypher statements that make this happen ..

Thanks Koen



Op maandag 21 augustus 2017 06:22:12 UTC+2 schreef koen:

[LAW Andy]

Koen,

I think that you are looking for the MERGE command.

http://neo4j.com/docs/developer-manual/current/cypher/clauses/merge/

Later,

Andy

> --
> You received this message because you are subscribed to the Google Groups "Neo4j" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+un...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

[Koen Kleingeld]

Hi andy, i am already using the merge command which works fine to create unique records but i also want to be able to for example update / change an attribute on an existing node .. i maybe have to do some more testing with all merge options ..

koen

> To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+unsubscribe@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

--
You received this message because you are subscribed to a topic in the Google Groups "Neo4j" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/neo4j/hTjPHgqcy1Y/unsubscribe.
To unsubscribe from this group and all its topics, send an email to neo4j+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Groeten / Best Regards,

Koen

[Michael Hunger]

This statement is idempotent, but also updates properties on nodes / rels

HTH Michael

LOAD CSV FROM "url" as row

// get-or-create-node

MERGE (n1:Node1 {row.id1}) 

// update prop

SET n1.prop1 = row.prop1

// get-or-create-node

MERGE (n2:Node2 {row.id2}) 

// update prop

SET n2.prop2 = row.prop2

// get-or-create-relationship

MERGE (n1)-[rel:REL]->(n2) 

SET rel.prop = row.relProp

[Koen Kleingeld]

Hi thx , i will do some more testing .. i also have some CREATE statements since the input data set results sometimes in multiple rels between 2 nodes..

[Liping Huang]

Hi Koen,

I have the same requirements and I do it in follow way:

MERGE (n:NodeLabel{id: {id}})
ON CREATE SET
    n.XXX = YYY,
    ......
ON MATCH SET
    n.XXX = ZZZ

MERGE (n1:Node1{id: {id}})
MERGE (n2:Node2{id: {id}})
MERGE (n1) -[r:REL]-(n2)
ON CREATE SET
    r.XXX = YYY,
ON MATCH SET
    r.XXX = ZZZ


I still in testing but get the big problem when sync the data to neo4j but this cypher query are what I used for now.

What's your solution right now for your question? thanks.





在 2017年8月22日星期二 UTC+8下午9:53:08，koen写道：

> To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+un...@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

--

You received this message because you are subscribed to a topic in the Google Groups "Neo4j" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/neo4j/hTjPHgqcy1Y/unsubscribe.

To unsubscribe from this group and all its topics, send an email to neo4j+un...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Groeten / Best Regards,

Koen

--
You received this message because you are subscribed to the Google Groups "Neo4j" group.

To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+un...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Neo4j" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/neo4j/hTjPHgqcy1Y/unsubscribe.

To unsubscribe from this group and all its topics, send an email to neo4j+un...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Koen Kleingeld]

Hi, actualy i do more or less the same

regards  koen

To unsubscribe from this group and all its topics, send an email to neo4j+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.