Loading GraphSON files into Titan
598 views
Skip to first unread message
defermat
Mar 15, 2013, 12:31:19 PM3/15/13
to aureliu...@googlegroups.com
I've been dealing with GraphML for the most part so far, and that has
been loading without issue, I can query the graph and everything.
However I wanted to try a different format out, specifically GraphSON,
but unfortunately thus far I haven't been able to successfully make
queries on files I try to load that are GraphSON.
Running Titan 0.2.0 with HBase 0.94.2 CDH4.2.0
My GraphSON file is as follows:
{"name":"saturn","type":"titan","_id":0,"_inE":[{"_label":"father","_id":12,"_outV":1}]}
{"name":"jupiter","type":"god","_id":1,"_outE":[{"_label":"lives","_id":13,"_inV":4},{"_label":"brother","_id":16,"_inV":3},{"_label":"brother","_id":14,"_inV":2},{"_label":"father","_id":12,"_inV":0}],"_inE":[{"_label":"brother","_id":17,"_outV":3},{"_label":"brother","_id":15,"_outV":2},{"_label":"father","_id":24,"_outV":7}]}
{"name":"neptune","type":"god","_id":2,"_outE":[{"_label":"lives","_id":20,"_inV":5},{"_label":"brother","_id":19,"_inV":3},{"_label":"brother","_id":15,"_inV":1}],"_inE":[{"_label":"brother","_id":18,"_outV":3},{"_label":"brother","_id":14,"_outV":1}]}
{"name":"pluto","type":"god","_id":3,"_outE":[{"_label":"pet","_id":23,"_inV":11},{"_label":"lives","_id":21,"_inV":6},{"_label":"brother","_id":17,"_inV":1},{"_label":"brother","_id":18,"_inV":2}],"_inE":[{"_label":"brother","_id":19,"_outV":2},{"_label":"brother","_id":16,"_outV":1}]}
{"name":"sky","type":"location","_id":4,"_inE":[{"_label":"lives","_id":13,"_outV":1}]}
{"name":"sea","type":"location","_id":5,"_inE":[{"_label":"lives","_id":20,"_outV":2}]}
{"name":"tartarus","type":"location","_id":6,"_inE":[{"_label":"lives","_id":21,"_outV":3},{"_label":"lives","_id":22,"_outV":11}]}
{"name":"hercules","type":"demigod","_id":7,"_outE":[{"_label":"mother","_id":25,"_inV":8},{"time":1,"_label":"battled","_id":26,"_inV":9},{"time":2,"_label":"battled","_id":27,"_inV":10},{"time":12,"_label":"battled","_id":28,"_inV":11},{"_label":"father","_id":24,"_inV":1}]}
{"name":"alcmene","type":"human","_id":8,"_inE":[{"_label":"mother","_id":25,"_outV":7}]}
{"name":"nemean","type":"monster","_id":9,"_inE":[{"time":1,"_label":"battled","_id":26,"_outV":7}]}
{"name":"hydra","type":"monster","_id":10,"_inE":[{"time":2,"_label":"battled","_id":27,"_outV":7}]}
{"name":"cerberus","type":"monster","_id":11,"_outE":[{"_label":"lives","_id":22,"_inV":6}],"_inE":[{"_label":"pet","_id":23,"_outV":3},{"time":12,"_label":"battled","_id":28,"_outV":7}]}
The following is what I'm getting:
gremlin> conf = new BaseConfiguration();
==>null
gremlin> conf.setProperty("storage.backend","hbase");
==>null
gremlin> conf.setProperty("storage.hostname","x.x.x.x,x.x.x.x,x.x.x.x");
==>null
gremlin> conf.setProperty("storage.lock-retries","10")
==>null
gremlin> conf.setProperty("storage.batch-loading","true")
==>null
gremlin> conf.setProperty("storage.tablename", "titan3")
==>null
gremlin> z = TitanFactory.open(conf);
...
==>titangraph[hbase:x.x.x.x]
gremlin> z.createKeyIndex('name', Vertex.class)
==>null
gremlin> z.loadGraphSON('gods-graphson.json')
==>null
gremlin> saturn = z.V('name','saturn').next()
java.util.NoSuchElementException
Display stack trace? [yN] y
java.util.NoSuchElementException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:796)
at java.util.HashMap$KeyIterator.next(HashMap.java:828)
at com.tinkerpop.pipes.util.iterators.HistoryIterator.next(HistoryIterator.java:25)
at com.tinkerpop.pipes.transform.IdentityPipe.processNextStart(IdentityPipe.java:20)
at com.tinkerpop.pipes.AbstractPipe.next(AbstractPipe.java:84)
at com.tinkerpop.pipes.util.Pipeline.next(Pipeline.java:115)
at java_util_Iterator$next.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
at groovysh_evaluate.run(groovysh_evaluate:46)
at groovysh_evaluate$run.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
at groovysh_evaluate$run.call(Unknown Source)
at org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:67)
at org.codehaus.groovy.tools.shell.Interpreter$evaluate.call(Unknown
Source)
at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:152)
at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:114)
at org.codehaus.groovy.tools.shell.Shell$leftShift$0.call(Unknown
Source)
at org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:88)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1071)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:128)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:148)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:100)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:272)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:52)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:137)
at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:57)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1071)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:128)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:148)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:66)
at com.thinkaurelius.titan.tinkerpop.gremlin.Console.<init>(Console.java:60)
at com.thinkaurelius.titan.tinkerpop.gremlin.Console.<init>(Console.java:67)
at com.thinkaurelius.titan.tinkerpop.gremlin.Console.main(Console.java:72)
It appears to me that the data is not actually being loaded, is there
something I should be doing differently or missing?
Thanks,
Charlie
been loading without issue, I can query the graph and everything.
However I wanted to try a different format out, specifically GraphSON,
but unfortunately thus far I haven't been able to successfully make
queries on files I try to load that are GraphSON.
Running Titan 0.2.0 with HBase 0.94.2 CDH4.2.0
My GraphSON file is as follows:
{"name":"saturn","type":"titan","_id":0,"_inE":[{"_label":"father","_id":12,"_outV":1}]}
{"name":"jupiter","type":"god","_id":1,"_outE":[{"_label":"lives","_id":13,"_inV":4},{"_label":"brother","_id":16,"_inV":3},{"_label":"brother","_id":14,"_inV":2},{"_label":"father","_id":12,"_inV":0}],"_inE":[{"_label":"brother","_id":17,"_outV":3},{"_label":"brother","_id":15,"_outV":2},{"_label":"father","_id":24,"_outV":7}]}
{"name":"neptune","type":"god","_id":2,"_outE":[{"_label":"lives","_id":20,"_inV":5},{"_label":"brother","_id":19,"_inV":3},{"_label":"brother","_id":15,"_inV":1}],"_inE":[{"_label":"brother","_id":18,"_outV":3},{"_label":"brother","_id":14,"_outV":1}]}
{"name":"pluto","type":"god","_id":3,"_outE":[{"_label":"pet","_id":23,"_inV":11},{"_label":"lives","_id":21,"_inV":6},{"_label":"brother","_id":17,"_inV":1},{"_label":"brother","_id":18,"_inV":2}],"_inE":[{"_label":"brother","_id":19,"_outV":2},{"_label":"brother","_id":16,"_outV":1}]}
{"name":"sky","type":"location","_id":4,"_inE":[{"_label":"lives","_id":13,"_outV":1}]}
{"name":"sea","type":"location","_id":5,"_inE":[{"_label":"lives","_id":20,"_outV":2}]}
{"name":"tartarus","type":"location","_id":6,"_inE":[{"_label":"lives","_id":21,"_outV":3},{"_label":"lives","_id":22,"_outV":11}]}
{"name":"hercules","type":"demigod","_id":7,"_outE":[{"_label":"mother","_id":25,"_inV":8},{"time":1,"_label":"battled","_id":26,"_inV":9},{"time":2,"_label":"battled","_id":27,"_inV":10},{"time":12,"_label":"battled","_id":28,"_inV":11},{"_label":"father","_id":24,"_inV":1}]}
{"name":"alcmene","type":"human","_id":8,"_inE":[{"_label":"mother","_id":25,"_outV":7}]}
{"name":"nemean","type":"monster","_id":9,"_inE":[{"time":1,"_label":"battled","_id":26,"_outV":7}]}
{"name":"hydra","type":"monster","_id":10,"_inE":[{"time":2,"_label":"battled","_id":27,"_outV":7}]}
{"name":"cerberus","type":"monster","_id":11,"_outE":[{"_label":"lives","_id":22,"_inV":6}],"_inE":[{"_label":"pet","_id":23,"_outV":3},{"time":12,"_label":"battled","_id":28,"_outV":7}]}
The following is what I'm getting:
gremlin> conf = new BaseConfiguration();
==>null
gremlin> conf.setProperty("storage.backend","hbase");
==>null
gremlin> conf.setProperty("storage.hostname","x.x.x.x,x.x.x.x,x.x.x.x");
==>null
gremlin> conf.setProperty("storage.lock-retries","10")
==>null
gremlin> conf.setProperty("storage.batch-loading","true")
==>null
gremlin> conf.setProperty("storage.tablename", "titan3")
==>null
gremlin> z = TitanFactory.open(conf);
...
==>titangraph[hbase:x.x.x.x]
gremlin> z.createKeyIndex('name', Vertex.class)
==>null
gremlin> z.loadGraphSON('gods-graphson.json')
==>null
gremlin> saturn = z.V('name','saturn').next()
java.util.NoSuchElementException
Display stack trace? [yN] y
java.util.NoSuchElementException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:796)
at java.util.HashMap$KeyIterator.next(HashMap.java:828)
at com.tinkerpop.pipes.util.iterators.HistoryIterator.next(HistoryIterator.java:25)
at com.tinkerpop.pipes.transform.IdentityPipe.processNextStart(IdentityPipe.java:20)
at com.tinkerpop.pipes.AbstractPipe.next(AbstractPipe.java:84)
at com.tinkerpop.pipes.util.Pipeline.next(Pipeline.java:115)
at java_util_Iterator$next.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
at groovysh_evaluate.run(groovysh_evaluate:46)
at groovysh_evaluate$run.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
at groovysh_evaluate$run.call(Unknown Source)
at org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:67)
at org.codehaus.groovy.tools.shell.Interpreter$evaluate.call(Unknown
Source)
at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:152)
at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:114)
at org.codehaus.groovy.tools.shell.Shell$leftShift$0.call(Unknown
Source)
at org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:88)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1071)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:128)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:148)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:100)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:272)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:52)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:137)
at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:57)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1071)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:128)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:148)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:66)
at com.thinkaurelius.titan.tinkerpop.gremlin.Console.<init>(Console.java:60)
at com.thinkaurelius.titan.tinkerpop.gremlin.Console.<init>(Console.java:67)
at com.thinkaurelius.titan.tinkerpop.gremlin.Console.main(Console.java:72)
It appears to me that the data is not actually being loaded, is there
something I should be doing differently or missing?
Thanks,
Charlie
Marko Rodriguez
Mar 15, 2013, 12:38:19 PM3/15/13
to aureliu...@googlegroups.com
Hi,
There are two "version" of GraphSON (we should probably make this more clear as many people keep stumbling).
There is the original GraphSON format that is "graph-centric" and has readers/writers deployed with Blueprints.
- its like GraphML -- each vertex and edge is its own object under the full graph document.
There is the Faunus GraphSON format that is "vertex-centric" and has readers/writers deployed with Faunus.
- its row-based (like what you have below) and has the edges of a vertex associated with the vertex.
So, to do what you want:
g = new TinkerGraph()
g.loadGraphML('graph-of-the-gods.xml')
g.saveGraphSON('graph-of-the-gods.json')
g = TitanFactory.open(…)
g.loadGraphSON('graph-of-the-gods.json')
HTH,
Marko.
http://markorodriguez.com
> --
> You received this message because you are subscribed to the Google Groups "Aurelius" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
There are two "version" of GraphSON (we should probably make this more clear as many people keep stumbling).
There is the original GraphSON format that is "graph-centric" and has readers/writers deployed with Blueprints.
- its like GraphML -- each vertex and edge is its own object under the full graph document.
There is the Faunus GraphSON format that is "vertex-centric" and has readers/writers deployed with Faunus.
- its row-based (like what you have below) and has the edges of a vertex associated with the vertex.
So, to do what you want:
g = new TinkerGraph()
g.loadGraphML('graph-of-the-gods.xml')
g.saveGraphSON('graph-of-the-gods.json')
g = TitanFactory.open(…)
g.loadGraphSON('graph-of-the-gods.json')
HTH,
Marko.
http://markorodriguez.com
> You received this message because you are subscribed to the Google Groups "Aurelius" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
defermat
Mar 15, 2013, 12:47:31 PM3/15/13
to aureliu...@googlegroups.com
Ah, makes sense. Thanks!
Reply all
Reply to author
Forward
0 new messages