Clone a Neo4j database programmatically

[Johnny Weng Luu]

I have a staging app and a production app at Heroku.

I want the staging app to mimic the production app as much as possible, which means I want to clone the production app's database each time I test something out in the staging app.

Is there a way to programmatically import all data from one Neo4j database to another?

Or is the best way to do this by doing it manually with backup/restoration buttons in Heroku admin page?

Johnny

[Peter Neubauer]

Johnny,
I think right now the backup/restores are the best options.
Alternatively, if the DB is not big, maybe you could run a Cypher
export and import?

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

If you can write, you can code - @coderdojomalmo
If you can sketch, you can use a graph database - @neo4j

[Radhakrishna Kalyan]

Hi,

Do you think if clone/backup an existing database will be a good feature to neoclipse.
Please comment,

Kalyan

--
Thanks and Regards
N Radhakrishna Kalyan

[Alan Robertson]

A general, non-Neo4J note about cloning production data for testing:
1) It's a really good idea from a testing perspective
2) It's sometimes a really bad idea from a security perspective.
3) Everything I say here may not apply to your environment, but
will apply to someone's environment eventually
if Neo4J continues to be more and more successful.

With respect to (2), the problems that can come up are things related to
the security of the test environment, and the sensitivity of the data in
the production environment.

For example, if you have personally identifiable or sensitive
information - names, addresses, social security numbers, bank account
information, medical information, and so on in your production
environment, one needs to be careful about who gets access to this data.

Normally when you have sensitive information it is a good idea to
restrict who has access to the data using access controls through the
application and through the database (if available). However, such
restrictions don't make sense in a test environment, because of the
nature of chasing bugs. As a result, the sensitive data may not be
properly protected in the test environment.

For relational databases, there are tools to make copies of your
production data while disguising the sensitive parts. For example,
replace the names with random names from the phonebook. Similarly for
social security numbers, etc.

This looks like this will become an opportunity for someone in the
future - to provide such tools for Neo4J and/or other graph databases.

--
Alan Robertson<al...@unix.sh> - @OSSAlanR

"Openness is the foundation and preservative of friendship... Let me claim from you at all times your undisguised opinions." - William Wilberforce

[Peter Neubauer]

Regarding that,
I think there are some projects working on piping Neo4j databases for
export through http://www.netezza.com/ which can do this type of
obfuscating stuff.

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

If you can write, you can code - @coderdojomalmo
If you can sketch, you can use a graph database - @neo4j

[Nikhil]

Peter,

This is a problem I have been trying to address in our environment(s). We have been trying to achieve backups via GraphML/GraphSON but that requires a heavy sanitization of data for serialization and deserialization. Eventually, we ended up taking EBS snapshots of our volumes on EC2.

Regarding Netezza, the last time when I checked their website (which was a couple of weeks ago), I got a very SQLish feeling from their products. I had even got in touch with a friend who works closely with Netezza associates. Didn't know they had some projects working on piping Neo4j database for export. Can you share more information on this?

--

Nikhil

--
Nikhil Lanjewar
Engineering Lead at YourNextLeap
http://yournextleap.com

[Peter Neubauer]

Well,
mainly, the integration works using the JDBC driver
(https://github.com/rickardoberg/neo4j-jdbc)and piping it into
Netezza, need to dig up details on the status.

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

If you can write, you can code - @coderdojomalmo
If you can sketch, you can use a graph database - @neo4j