What to use to manage rdf triples with tinkerpop ecosystem?

1,117 views
Skip to first unread message

Alexandre Blanquart

unread,
Jun 17, 2011, 5:05:17 AM6/17/11
to Gremlin-users
Hi everyone,

First, I have to confess that I am pretty amazed of all the good job
on tinkerpop ecosystem.
I read the github wiki to understand the different components.
Actually, I am trying to figure out how to manage rdf triples in this
ecosystem. I would like to load these triples and be able to query,
traverse and put some closures during those kind of processes. I am
indeed very interested around the possibility to load linked data as
simply you described (like for example with DBPedia).
However I am bit lost to know what is the best to do between loading
through Sail interface and persist in a Neo4j database. Yet, I have
not been able for now to completely understand the concepts behind
Sail, GraphSail, SailGraph ... and I am also very interested with the
Neo4j database.

Could you give me some hints?
Thanks so much
Alexandre

Peter Neubauer

unread,
Jun 17, 2011, 5:22:59 AM6/17/11
to gremli...@googlegroups.com
Alexandre,
for RDF related work, I suggest to use the SAIl implementation by
Tinkerpop (not sure how stable it is yet, but it is very good work
done by Josh Shinavier) at
https://github.com/tinkerpop/blueprints/wiki/Sail-Ouplementation,
changing

Sail sail = new GraphSail(new TinkerGraph()); against

Sail sail = new GraphSail(new Neo4jGraph('target/db'));

or any other implementation. For Neo4j, please refer to http://neo4j.org.

Does that help?

Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://startupbootcamp.org/    - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.

Alexandre Blanquart

unread,
Jun 17, 2011, 5:44:34 AM6/17/11
to Gremlin-users
Thanks for the link and the suggestion.
Does it seem right for you to try to load my rdf triples using Sail
implementation. Then, transform it into a graph to bind it with a
neo4j database. Thus, I will be able to use BluePrints API and have
fun also with Neo4j.

Alexandre

On 17 juin, 11:22, Peter Neubauer <peter.neuba...@neotechnology.com>
wrote:
> Alexandre,
> for RDF related work, I suggest to use the SAIl implementation by
> Tinkerpop (not sure how stable it is yet, but it is very good work
> done by Josh Shinavier) athttps://github.com/tinkerpop/blueprints/wiki/Sail-Ouplementation,
> changing
>
> Sail sail = new GraphSail(new TinkerGraph()); against
>
> Sail sail = new GraphSail(new Neo4jGraph('target/db'));
>
> or any other implementation. For Neo4j, please refer tohttp://neo4j.org.
>
> Does that help?
>
> Cheers,
>
> /peter neubauer
>
> GTalk:      neubauer.peter
> Skype       peter.neubauer
> Phone       +46 704 106975
> LinkedIn  http://www.linkedin.com/in/neubauer
> Twitter     http://twitter.com/peterneubauer
>
> http://www.neo4j.org              - Your high performance graph database.http://startupbootcamp.org/   - Öresund - Innovation happens HERE.http://www.thoughtmade.com- Scandinavia's coolest Bring-a-Thing party.

Peter Neubauer

unread,
Jun 17, 2011, 6:14:10 AM6/17/11
to gremli...@googlegroups.com
Alexandre,
yes, you can access such a graph via both APIs. This is what you
actually do using the Gremlin Plugin,
http://docs.neo4j.org/chunked/snapshot/gremlin-plugin.html for
example.

However, I think there is a lot of logic and indexing going on the
Joshs SAIL implementation that you might want to examine, since it is
not reflected in the raw graph structure.

Just put something up, inspect and report back is my initial reaction :)

Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://startupbootcamp.org/    - Öresund - Innovation happens HERE.

http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.

Pierre De Wilde

unread,
Jun 17, 2011, 6:27:59 AM6/17/11
to gremli...@googlegroups.com
Hi Alexandre,

SailGraph is a Blueprints implementation: Blueprints TransactionalGraph on top of RDF-based Sail interface.
GraphSail is a Blueprints outplementation: RDF-based Sail interface on top of Blueprints IndexableGraph.

So, you can do 'crazy' thinks like:

gremlin> g = new SailGraph(new GraphSail(new Neo4jGraph('/data/neo4j/test')))
==>sailgraph[graphsail]
gremlin> g.loadRDF(new FileInputStream('data/graph-example-1.ntriple'), 'http://tinkerpop.com#', 'n-triples', null)
==>null
gremlin> g.E
==>e[http://tinkerpop.com#1 - http://tinkerpop.com#name -> "marko"^^<xsd:string>]
==>e[http://tinkerpop.com#2 - http://tinkerpop.com#name -> "vadas"^^<xsd:string>]
==>e[http://tinkerpop.com#4 - http://tinkerpop.com#name -> "josh"^^<xsd:string>]
==>e[http://tinkerpop.com#3 - http://tinkerpop.com#name -> "lop"^^<xsd:string>]
==>e[http://tinkerpop.com#3 - http://tinkerpop.com#lang -> "java"^^<xsd:string>]
==>e[http://tinkerpop.com#5 - http://tinkerpop.com#name -> "ripple"^^<xsd:string>]
==>e[http://tinkerpop.com#5 - http://tinkerpop.com#lang -> "java"^^<xsd:string>]
==>e[http://tinkerpop.com#6 - http://tinkerpop.com#name -> "peter"^^<xsd:string>]
gremlin> g.V                                                                                                       
java.lang.UnsupportedOperationException
Display stack trace? [yN] 
gremlin> g.shutdown()
==>null
gremlin> g = new Neo4jGraph("/data/neo4j/test")
==>neo4jgraph[EmbeddedGraphDatabase [/data/neo4j/test]]
gremlin> g.V.value   
==>urn:com.tinkerpop.blueprints.pgm.oupls.sail:namespaces

and so on...

HTH,
Pierre

Joshua Shinavier

unread,
Jun 17, 2011, 6:30:12 AM6/17/11
to gremli...@googlegroups.com
On Fri, Jun 17, 2011 at 6:14 PM, Peter Neubauer
<peter.n...@neotechnology.com> wrote:
[...]

> However, I think there is a lot of logic and indexing going on the
> Joshs SAIL implementation that you might want to examine, since it is
> not reflected in the raw graph structure.


That's correct. It uses a hybrid graph-based and index-based solution
for fast query answering. However, all of RDF data is represented in
the graph structure; the indexes merely add a performance boost. Just
treat GraphSail as you would any other Sail implementation, loading
data in through addStatement and reading it back out through
getStatements or evaluate (or using any of the various helper classes
which work with Sail and Repository). In parallel, though, you can
examine and traverse the resulting graph at the Neo4j level.

Josh

HasanB

unread,
Jun 17, 2011, 2:53:18 PM6/17/11
to Gremlin-users
Alexandre,

I have a demo of how to load an OWL file into a Blueprints managed
database by means of a POST or PUT method on a Rexster extension I've
been messing with, as part of the learning curve.

It's in a very ragged state at the moment. You can have it
immediately, if you want to have a quick look. Or you can wait a
week, or so until I've made it cleaner and more understandable.

If you happen to be an RDF/OWL expert, I'd LOVE to have your help with
determining if what I've done is correct.

Rgds,
Hasan


On Jun 17, 4:05 am, Alexandre Blanquart <alex.blanqu...@gmail.com>
wrote:

Alexandre Blanquart

unread,
Jun 20, 2011, 8:03:24 AM6/20/11
to Gremlin-users
Thanks for the hints Pierre,

I can now load my triples. I am still a litte confused because all
information is stored in the data with the key set to "value". Please
see the graphml serialization:
<node id="5">
<data key="value">T http://www.w3.org/2001/XMLSchema#string Miller
Sandwich Bar Ltd.</data>
</node>

How can I modelize the RDF triples in the property graph and
dissociate for example the type, lang, value information? like there
https://github.com/tinkerpop/blueprints/wiki/Sail-Implementation#working_rdf

Regards,
Alex

On 17 juin, 12:27, Pierre De Wilde <pierredewi...@gmail.com> wrote:
> Hi Alexandre,
>
> SailGraph is a Blueprints implementation: Blueprints TransactionalGraph on
> top of RDF-based Sail interface.
> GraphSail is a Blueprints outplementation: RDF-based Sail interface on top
> of Blueprints IndexableGraph.
>
> So, you can do 'crazy' thinks like:
>
> gremlin> g = new SailGraph(new GraphSail(new
> Neo4jGraph('/data/neo4j/test')))
> ==>sailgraph[graphsail]
> gremlin> g.loadRDF(new FileInputStream('data/graph-example-1.ntriple'), 'http://tinkerpop.com#', 'n-triples', null)
> ==>null
> gremlin> g.E
> ==>e[http://tinkerpop.com#1-http://tinkerpop.com#knows->http://tinkerpop.com#2]
> ==>e[http://tinkerpop.com#1-http://tinkerpop.com#knows->http://tinkerpop.com#4]
> ==>e[http://tinkerpop.com#1-http://tinkerpop.com#created->http://tinkerpop.com#3]
> ==>e[http://tinkerpop.com#1-http://tinkerpop.com#name->
> "marko"^^<xsd:string>]
> ==>e[http://tinkerpop.com#1-http://tinkerpop.com#age-> "29"^^<xsd:int>]
> ==>e[http://tinkerpop.com#2-http://tinkerpop.com#name->
> "vadas"^^<xsd:string>]
> ==>e[http://tinkerpop.com#2-http://tinkerpop.com#age-> "27"^^<xsd:int>]
> ==>e[http://tinkerpop.com#4-http://tinkerpop.com#created->http://tinkerpop.com#3]
> ==>e[http://tinkerpop.com#4-http://tinkerpop.com#created->http://tinkerpop.com#5]
> ==>e[http://tinkerpop.com#4-http://tinkerpop.com#name->
> "josh"^^<xsd:string>]
> ==>e[http://tinkerpop.com#4-http://tinkerpop.com#age-> "32"^^<xsd:int>]
> ==>e[http://tinkerpop.com#3-http://tinkerpop.com#name->
> "lop"^^<xsd:string>]
> ==>e[http://tinkerpop.com#3-http://tinkerpop.com#lang->
> "java"^^<xsd:string>]
> ==>e[http://tinkerpop.com#5-http://tinkerpop.com#name->
> "ripple"^^<xsd:string>]
> ==>e[http://tinkerpop.com#5-http://tinkerpop.com#lang->
> "java"^^<xsd:string>]
> ==>e[http://tinkerpop.com#6-http://tinkerpop.com#created->http://tinkerpop.com#3]
> ==>e[http://tinkerpop.com#6-http://tinkerpop.com#name->
> "peter"^^<xsd:string>]
> ==>e[http://tinkerpop.com#6-http://tinkerpop.com#age-> "35"^^<xsd:int>]

Pierre De Wilde

unread,
Jun 20, 2011, 9:37:44 AM6/20/11
to gremli...@googlegroups.com
Hi Alexandre,

> How can I modelize the RDF triples in the property graph and
> dissociate for example the type, lang, value information? like there
> https://github.com/tinkerpop/blueprints/wiki/Sail-Implementation#working_rdf

The Sail implementation (SailGraph) does it for you. 
If you may see at graph-example-1-rdf.jpg, literals are not stored as vertex properties (as you may expect) but as other vertices.

Accessing those literals is therefore a little bit tricky:

gremlin> g = new SailGraph(new GraphSail(new Neo4jGraph('/data/neo4j/test')))
==>sailgraph[graphsail]

gremlin> g.loadRDF(new FileInputStream('data/graph-example-1.ntriple'), 'http://tinkerpop.com#', 'n-triples', null) // skip this if already loaded
==>null

gremlin> g.addNamespace('tg', 'http://tinkerpop.com#')
==>null

gremlin> v = g.v('tg:1')

gremlin> v['tg:name']
==>null

gremlin> v['tg:name'] = 'marko'
RDF graph URI and blank node vertices can not have properties

gremlin> v.map()
==>kind=uri

gremlin> v.out              

gremlin> v.out.kind.unique()
==>uri
==>literal

gremlin> v.out{it.kind=="uri"}

gremlin> v.out{it.kind=="literal"}

gremlin> v.out{it.kind=="literal"}.value    
==>29
==>marko

gremlin> v.out{it.kind=="literal"}.type 

gremlin> v.out{it.kind=="literal"}.inE.label


I still don't grasp why it's implemented that way. 
Storing literals as vertex properties would be much more simple and intuitive...

Pierre



2011/6/20 Alexandre Blanquart <alex.bl...@gmail.com>

Alexandre Blanquart

unread,
Jun 20, 2011, 10:01:53 AM6/20/11
to Gremlin-users
Thanks Pierre,

Since my last post, I was able to determine how to access the
properties as you described.
I will enjoy the closures to put more logic when searching.
Indeed, it would have been possible to put the literals as vertex
properties. For me, it has been implemented this way to stay the
closer possible to the RDF representation as everything is centered
around edges and URIs.

Did you append to store the graph using GraphMLWriter (https://
github.com/tinkerpop/gremlin/wiki/Graph-Framework-Connectors)? I tried
to do so directly on neo4j graph as it is unsupported at the SailGraph
level. However, I am losing all the good properties and only having a
"value" node key. I am then wondering, how to have the good
visualization like https://github.com/tinkerpop/gremlin/raw/master/doc/images/graph-example-1-rdf.jpg.

Alex

On 20 juin, 15:37, Pierre De Wilde <pierredewi...@gmail.com> wrote:
> Hi Alexandre,
>
> > How can I modelize the RDF triples in the property graph and
> > dissociate for example the type, lang, value information? like there
>
> https://github.com/tinkerpop/blueprints/wiki/Sail-Implementation#work...
>
> The Sail implementation (SailGraph) does it for you.
> If you may see at
> graph-example-1-rdf.jpg<https://github.com/tinkerpop/gremlin/raw/master/doc/images/graph-exam...>,
> 2011/6/20 Alexandre Blanquart <alex.blanqu...@gmail.com>
>
>
>
>
>
>
>
> > Thanks for the hints Pierre,
>
> > I can now load my triples. I am still a litte confused because all
> > information is stored in the data with the key set to "value". Please
> > see the graphml serialization:
> > <node id="5">
> >  <data key="value">Thttp://www.w3.org/2001/XMLSchema#stringMiller
> > Sandwich Bar Ltd.</data>
> > </node>
>
> > How can I modelize the RDF triples in the property graph and
> > dissociate for example the type, lang, value information? like there
>
> >https://github.com/tinkerpop/blueprints/wiki/Sail-Implementation#work...

Pierre De Wilde

unread,
Jun 20, 2011, 10:46:58 AM6/20/11
to gremli...@googlegroups.com
Hi Alex,

GraphMLWriter doesn't work because it uses .getVertices() which is not supported at SailGraph level:

gremlin> g.V
java.lang.UnsupportedOperationException

Only a custom writer may help.

Pierre

2011/6/20 Alexandre Blanquart <alex.bl...@gmail.com>
Thanks Pierre,

Alexandre Blanquart

unread,
Jun 20, 2011, 11:15:27 AM6/20/11
to Gremlin-users
Hi Pierre,

That's what I was thinking.
Does such or similar writer already exist?

Alex

On 20 juin, 16:46, Pierre De Wilde <pierredewi...@gmail.com> wrote:
> Hi Alex,
>
> GraphMLWriter doesn't work because it uses .getVertices() which is not
> supported at SailGraph level:
>
> gremlin> g.V
> java.lang.UnsupportedOperationException
>
> Only a custom writer may help.
>
> Pierre
>
> 2011/6/20 Alexandre Blanquart <alex.blanqu...@gmail.com>
>
>
>
>
>
>
>
> > Thanks Pierre,
>
> > Since my last post, I was able to determine how to access the
> > properties as you described.
> > I will enjoy the closures to put more logic when searching.
> > Indeed, it would have been possible to put the literals as vertex
> > properties. For me, it has been implemented this way to stay the
> > closer possible to the RDF representation as everything is centered
> > around edges and URIs.
>
> > Did you append to store the graph using GraphMLWriter (https://
> > github.com/tinkerpop/gremlin/wiki/Graph-Framework-Connectors)? I tried
> > to do so directly on neo4j graph as it is unsupported at the SailGraph
> > level. However, I am losing all the good properties and only having a
> > "value" node key. I am then wondering, how to have the good
> > visualization like
> >https://github.com/tinkerpop/gremlin/raw/master/doc/images/graph-exam...
> > .
>
> > Alex

Pierre De Wilde

unread,
Jun 20, 2011, 11:24:04 AM6/20/11
to gremli...@googlegroups.com
Don't know. The original writer is here.

Pierre


2011/6/20 Alexandre Blanquart <alex.bl...@gmail.com>
Hi Pierre,

Alexandre Blanquart

unread,
Jun 20, 2011, 11:27:49 AM6/20/11
to Gremlin-users
Thanks anyway Pierre for the valuable information.
Helped a lot!

On 20 juin, 17:24, Pierre De Wilde <pierredewi...@gmail.com> wrote:
> Don't know. The original writer is
> here<https://github.com/tinkerpop/blueprints/blob/master/blueprints-core/s...>
> .
>
> Pierre
>
> 2011/6/20 Alexandre Blanquart <alex.blanqu...@gmail.com>
>
>
>
>
>
>
>

Marko Rodriguez

unread,
Jun 20, 2011, 11:27:52 AM6/20/11
to gremli...@googlegroups.com
Hi,

I'd be happy to accept a pull request for a RDFGraphWriter and RDFGraphReader in blueprints-graph-sail (or blueprints-sail-graph).

Note that SailGraph does has loadRDF(). I'd like to remove that in order to be consistent with the GraphMLReader/Writer model and as such, have loadRDF() as a metaMethod of Gremlin.

Thanks,
Marko.

Joshua Shinavier

unread,
Jun 20, 2011, 12:17:25 PM6/20/11
to gremli...@googlegroups.com
On Mon, Jun 20, 2011 at 9:37 PM, Pierre De Wilde
<pierre...@gmail.com> wrote:
[...]

> I still don't grasp why it's implemented that way.
> Storing literals as vertex properties would be much more simple and
> intuitive...


It might be, if it weren't for the fact that in RDF, you can have
multiple values for the same predicate (so if you made these into
vertex properties, you would have to deal with value collections).
It's also useful, and necessary in Named Graph contexts, to be able to
attach metadata to the relationship between the subject
vertex/resource and the predicate value.

Best,

Josh

Pierre De Wilde

unread,
Jun 20, 2011, 1:08:12 PM6/20/11
to gremli...@googlegroups.com
Hi Josh,

Thank you for this clarification. 
Makes sense that, since RDF is a multigraph, multiple values need to be stored in lists/maps or, as you do, in separated vertices.

Are named graphs fully supported in SailGraph ( e.g. in executeSparql: select distinct ?g where {graph ?g {?s ?p ?o}} ) ?
Does loadRDF support N-Quads ?

Thanks,
Pierre

2011/6/20 Joshua Shinavier <jo...@fortytwo.net>

Joshua Shinavier

unread,
Jun 20, 2011, 1:21:03 PM6/20/11
to gremli...@googlegroups.com
On Tue, Jun 21, 2011 at 1:08 AM, Pierre De Wilde
<pierre...@gmail.com> wrote:
> Hi Josh,
> Thank you for this clarification.
> Makes sense that, since RDF is a multigraph, multiple values need to be
> stored in lists/maps or, as you do, in separated vertices.
> Are named graphs fully supported in SailGraph ( e.g. in executeSparql:
> select distinct ?g where {graph ?g {?s ?p ?o}} ) ?


They should be.

> Does loadRDF support N-Quads ?


No, but N-Quads support could be added (via SesameTools). Feel free
to make a ticket if you think this would be useful.


Josh

Marko Rodriguez

unread,
Jun 20, 2011, 1:34:27 PM6/20/11
to gremli...@googlegroups.com
Hi,

>> Are named graphs fully supported in SailGraph
>

> They should be.

They are. Every edge has an 'ng' property.

Marko.

http://markorodriguez.com

Pierre De Wilde

unread,
Jun 20, 2011, 1:51:22 PM6/20/11
to gremli...@googlegroups.com
Cool. 

N-Quads support will allow to add the missing weight property in:

data/graph-example-1.nq

Pierre

2011/6/20 Marko Rodriguez <okram...@gmail.com>

Joshua Shinavier

unread,
Jun 20, 2011, 2:30:14 PM6/20/11
to gremli...@googlegroups.com
On Tue, Jun 21, 2011 at 1:51 AM, Pierre De Wilde
<pierre...@gmail.com> wrote:
> Cool.
> N-Quads support will allow to add the missing weight property in:
> data/graph-example-1.nq


Well, you can express the same things (modulo some corner cases) with
TriX or TriG. But as long as we're talking about it, I have just
added N-Quads support to SailGraph. Marko is deploying a snapshot
now.

Best,

Josh

Reply all
Reply to author
Forward
0 new messages