[Rexster] Graph database backup and import

[Nikhil Lanjewar]

Hi,

I have been playing around with Rexster lately for graph requirements
in my web product. I'm trying to use it in a ruby on Rails project by
tweaking ActiveResource to interact over REST. So far, I'm finding it
perfect for all my product requirements. :)

However, database backup is one of the major concerns while choosing
an API abstraction layer for any database. I could not find any
documentation touching topics related to database backups and
importing backed-up data with Rexster.

It would be great if someone could point me to a documentation link
for reference.

P.S.: I remember stumbling upon a forum/thread mentioning similar
functionality (export/import graph to and from a csv file) available
via rexster-console, but I can't find it anymore.

--
Nikhil

[Peter Neubauer]

Nikhil,

at least for the pure graph data, you can use GraphML import and export, see http://docs.neo4j.org/chunked/snapshot/gremlin-plugin.html#rest-api-load-a-sample-graph for an example.

This does not save index info. For neo4j, we have been looking into a richer format (still not superfully fledged), see http://py2neo.org/geoff/ and https://github.com/neo4j/neo4j-geoff for details.

Also, you can of course use the underlying graph to do database-specific backup and restore, see http://docs.neo4j.org/chunked/snapshot/gremlin-plugin.html#rest-api-send-an-arbitrary-groovy-script---lucene-sorting on how to work with different APIs through e.g. Groovy.

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

Neo4j 1.6 released                 - dzone.com/6S4K
The Neo4j Heroku Challenge   - http://neo4j-challenge.herokuapp.com/

[Pierre De Wilde]

Hey,

In TinkerPop stack, there are no specific backup/restore API.

However, you may export/import data in both GraphML and JSON format using the following methods:

g.loadGraphML()

g.saveGraphML()

g.loadGraphSON()

g.saveGraphSON()

See doc at https://github.com/tinkerpop/gremlin/wiki/Gremlin-Methods

or code at https://github.com/tinkerpop/gremlin/blob/master/gremlin-groovy/src/main/groovy/com/tinkerpop/gremlin/groovy/loaders/GraphLoader.groovy

HTH,

Pierre

On Wed, Feb 8, 2012 at 9:28 AM, Nikhil Lanjewar <nikhil...@gmail.com> wrote:

[Nikhil Lanjewar]

Thanks Peter and Pierre!

g.saveGraphMl(filename) worked like a charm.

However, I'm getting an error while trying to load this GraphML using

g.loadGraphML(filename)

rexster[groovy]> g.loadGraphML('graph.xml')
==>An error occurred while processing the script for language [groovy]: javax.script.ScriptException: java.lang.RuntimeException: id is a reserved property key

Similar behavior for load/saveGraphSON . I'm using rexster-0.8-SNAPSHOT.

--
Nikhil

[Pierre De Wilde]

Hey,

It seems that you are using Rexster console. In that case, your code is executed at the server side via RexPro. The path is therefore relative to the Rexster server directory.

You may also use an URL instead of a filename:

g.loadGraphML(URL)

HTH,

Pierre

[Nikhil Lanjewar]

Hey,

I'm running both, Rexster server and console from the same directory. So, I don't think this problem should arise. Also, the error reported by rexster console doesn't talk about a FileNotFoundException or something similar.

I found an explanation for 'id is a reserved property key' at http://groups.google.com/group/gremlin-users/browse_thread/thread/77838ba9591f09bd

My saved GraphML contains an 'id' from the graph it was saved. IMO, g.saveGraphML shouldn't copy the 'id's if 'id' is a reserved property. Could you please let me know of your opinion on this?

--
Nikhil

[Marko Rodriguez]

Hi,

gremlin> g.saveGraphML('test.xml')
==>null
gremlin> g = new TinkerGraph()
==>tinkergraph[vertices:0 edges:0]
gremlin> g.loadGraphML('test.xml')
==>null

I don't get such a problem. Can you demonstrate how you are saving/loading your graphs?

Thanks,
Marko.

http://markorodriguez.com

[Nikhil Lanjewar]

Hi Marko,

Apologies for a delayed response. My Rexster configuration in
rexster.xml includes two Neo4j databases, neo4j-reloaded and neo4j-
revolutions.

neo4j-reloaded corresponds to an existing Neo4j Community 1.6M03
instance's graph.db/ with ~2900 nodes.
neo4j-revolutions corresponds to a blank location.

Both, rexster server and console, are running from the same directory.
Following is the series of command-execution from rexster-console.

rexster[groovy]> rexster.graphNames
==>neo4j-revolutions
==>emptygraph
==>neo4j-reloaded
rexster[groovy]> g_reloaded = rexster.getGraph('neo4j-reloaded')
==>neo4jgraph[EmbeddedGraphDatabase [/home/incognito/Download/neo4j-
community-1.6.M03/data.orig/graph.db]]
rexster[groovy]> g_revolutions = rexster.getGraph('neo4j-revolutions')
==>neo4jgraph[EmbeddedGraphDatabase [/home/incognito/Download/neo4j-
community-1.6.M03/blank]]
rexster[groovy]> g_reloaded.saveGraphML('neo_reloaded.xml')
==>
rexster[groovy]> g_revolutions.loadGraphML('neo_reloaded.xml')

==>An error occurred while processing the script for language

[groovy]: javax.script.ScriptException: java.lang.RuntimeException: id

is a reserved property key

rexster[groovy]>


I've tried this with Neo4j Community 1.4 instance too and got same
results. Further I have also tried executing Gremlin scripts on my
Neo4j REST server instance with similar results. Here's how:

$:~> curl --data "script=g.saveGraphML('neo.xml')"
http://localhost:7474/db/data/ext/GremlinPlugin/graphdb/execute_script
"null"

Restarted my Neo4j with 'data' pointing to an empty directory.

$:~> curl --data "script=g.loadGraphML('neo.xml')"
http://localhost:7474/db/data/ext/GremlinPlugin/graphdb/execute_script
"javax.script.ScriptException: java.lang.RuntimeException: id is a
reserved property key"
$:~>

My intention is to backup an existing database and import it to
another fresh database instance.

--
Nikhil

On Feb 8, 8:41 pm, Marko Rodriguez <okramma...@gmail.com> wrote:
> Hi,
>
> gremlin> g.saveGraphML('test.xml')
> ==>null
> gremlin> g = new TinkerGraph()
> ==>tinkergraph[vertices:0 edges:0]
> gremlin> g.loadGraphML('test.xml')
> ==>null
>
> I don't get such a problem. Can you demonstrate how you are saving/loading your graphs?
>
> Thanks,
> Marko.
>
> http://markorodriguez.com
>
> On Feb 8, 2012, at 8:07 AM, Nikhil Lanjewar wrote:
>
>
>
>
>
>
>
> > Hey,
>
> > I'm running both, Rexster server and console from the same directory. So, I don't think this problem should arise. Also, the error reported by rexster console doesn't talk about a FileNotFoundException or something similar.
>

> > I found an explanation for 'id is a reserved property key' athttp://groups.google.com/group/gremlin-users/browse_thread/thread/778...

[Peter Neubauer]

mmh,
this looks like a bug in that a graphml with an "id" property on a
node will not be able to be imported in Blueprints. I think you should
file a bug for this.

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

Neo4j 1.6 released                 - dzone.com/6S4K
The Neo4j Heroku Challenge   - http://neo4j-challenge.herokuapp.com/

[Marko Rodriguez]

Hi,

I can't seem to reproduce this error. Can you provide me the smallest possible GraphML example and step-by-step test to demonstrate your problem (preferably as an issue). I can then see where the problem lays.

Thank you,
Marko.

http://markorodriguez.com

[Nikhil Lanjewar]

Peter and Marko,

Thanks for the response. Will add an issue shortly.

Could you please let me know what's the best location to add an issue about this? I don't think this is related exclusively to Rexster now. Seems more like a Blueprints issue. Can someone confirm?

--
Nikhil

[Marko Rodriguez]

Hey,

If you can't reproduce the error using raw Blueprints and it only happens in Rexster, then put it in Rexster. If you can reproduce the error in raw Blueprints, then put it in Blueprints.

Raw Blueprints would look something like this:

GraphMLReader.inputGraph(graph, new FileInputStream("my.graphml"));

HTH,
Marko.

http://markorodriguez.com

[Andras Gefferth]

Hi,

I would like to completely restore a DB from backup, including node and edge IDs.

I can do it with Tinkergraph using the GraphSON format.

Is it also possible with neo4j? I didn't succeed so far.

thanks

[Stephen Mallette]

Neo4j does not allow ID assignment so it will not be possible via any TinkerPop IO format. 

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.