Programatic export to csv file?

[Glenn Hoetker]

Hi all.

I'm struggling with something that seems like it should be so simple that I'm convinced I'm just missing something.  I wish to use Neo4j as part of analyzing a large database of patents.  The FINAL step will be to use data derived via Neo4j to do statistical analysis.  The easiest way to get the derived data into my statistical software would be as a CSV.  Consider a super simple example, calculating how many patents each inventor (identified by "hanId") has invented.

In the browser, I can run the command

match (i:Inventor)-[:Invented]->(p:Patent) return i.hanId as Inventor, count(p) as Patents limit 100

To get the results of that query to a csv file, I   

a) press the export file button in the results pane

b) choose CSV

c) go to the new tab

d) save it, including selecting a directory and filename using the file dialog.

It gives me just what I want

Inventor,Patents
3342952,1
1484384,1
3554015,1
9816,11

but with many more steps than seems like should be required. In my ideal world, there would be an "Export" command, allowing me to do something like this to replace steps (a) through (d)

match (i:Inventor)-[:Invented]->(p:Patent) return i.hanId as Inventor, count(p) as Patents limit 100 export "/Users/Me/patentCount.csv" replace

That doesn't exist (yet??!! Maybe?? Please??). But, I have to think there is some way to do this programmatically that I just haven't found.  The "manual" approach isn't really viable given the number of derived statistics required, how long some of the queries will take, and the need for reproducability (step (d) in particular is not cool for reproducible science). 

Like I said, I feel like I'm just overlooking something simple here.  Can anyone point me in the right direction?

Thank you.

Glenn

[Michael Hunger]

There is currently no "simple" export command, but I want to add one to the apoc procedure library, just haven't gotten around to do it.

https://github.com/neo4j-contrib/neo4j-apoc-procedures/issues/117

Meanwhile if you can run neo4j-shell you could use: https://github.com/jexp/neo4j-shell-tools#export

--
You received this message because you are subscribed to the Google Groups "Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Glenn Hoetker]

Dear Michael,

Thank you very much for the quick and helpful answer. Much appreciated.

Glenn

To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+un...@googlegroups.com.

[Michael Hunger]

Your use-case is interesting, grab me on neo4j.com/slack and we can chat about the procedure.

Should be easy to add.

Michael

To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+unsubscribe@googlegroups.com.

[Michael Hunger]

I added CSV export to apoc via apoc.export.csv.query(query, file, config).

Please have a look at let me know if it makes sense.

see: https://neo4j-contrib.github.io/neo4j-apoc-procedures/#_export_import

you'll have to set `apoc.export.file.enabled=true` in your neo4j.conf see https://neo4j-contrib.github.io/neo4j-apoc-procedures/#_configuration_options

Michael

[David Rader]

Downloaded the "latest" apoc jar file. 275 procedures but no export.csv.*  ... here is the e to g range of procedure names...only cypher and graphml ...

apoc.es.stats, apoc.example.movies, apoc.export.cypher.all, apoc.export.cypher.data, apoc.export.cypher.graph, apoc.export.cypher.query, apoc.export.cypherAll, apoc.export.cypherData, apoc.export.cypherGraph, apoc.export.cypherQuery, apoc.export.graphml.all, apoc.export.graphml.data, apoc.export.graphml.graph, apoc.export.graphml.query, apoc.generate.ba, 

[David Rader]

apoc version 3.0.4.2 (latest general release) installs and runs fine (no export.csv, though).

apoc version 3.1.0.2 (Nov 6) generates an error on starting Neo4j 3.0.6

Error message is "Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase@5839e98f' was successfully initialized, but failed to start. Please see attached cause exception." 

Swapping back to 3.0.4.2 resolved the error but with the smaller number of GA functions.

Note: for Win10, a reboot is needed every time the plugin jar is changed out. neo4j-ce is not completely stopping. So, access to the jar file is still held by the process. A reboot clears everything, then the swap can occur.

[Michael Hunger]

Hi Dave 

sorry for the confusion.

the docs represent what's in the master branch (3.0.3-SNAPSHOT)

apoc 3.1.x is for Neo4j 3.1.x

apoc 3.0.x is for Neo4j 3.0.x

The reboot required for the CE with plugin sounds like a bug, could you raise an issue for that on github.com/neo4j/neo4j/issues ? Thanks

I gonna release a new apoc release (that also contains the CSV export) this week for 3.0 and 3.1

Cheers, Michael

To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+unsubscribe@googlegroups.com.

[David Rader]

Thanks for your help, Michael.  Just posted the new issue.  The task manager image is in twice since I wasn't sure the paste worked and did an upload of an image file, too.

Dave

--
You received this message because you are subscribed to a topic in the Google Groups "Neo4j" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/neo4j/FsP23qaI9kE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to neo4j+unsubscribe@googlegroups.com.

[Michael Hunger]

Thanks a lot. 

Michael 

Von meinem iPhone gesendet