Librdf plugin

[Christoph Burgmer]

I've created librdf plugin prototype for making use of the Python bindings of
librdf (http://librdf.org/bindings/):
https://github.com/cburgmer/surf/tree/librdf/plugins/surf.librdf

With rdflib SPARQL support being way to experimental this could offer a nice
replacement for in-memory storage.

Not all functionality is implemented and some test fail, however resources can
be stored and queried.

Initializing the store:
>>> store = surf.Store(**{"reader": "librdf", "writer" : "librdf"})

Librdf also claims to support certain other storage engines, I haven't tried
them yet though.

-Christoph

[Chris Wj]

Cool, thanks Christoph!

--
You received this message because you are subscribed to the Google Groups "surfrdf" group.
To post to this group, send email to sur...@googlegroups.com.
To unsubscribe from this group, send email to surfrdf+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/surfrdf?hl=en.

[Dominique Guardiola Falco]

thanks for making this plugin Christopher !

I was excited to test it, but could not do it before now,I really like Surf API
In rdflib, any query always turns to parsing errors like "Invalid control character" and "EOL while scanning string literal", I cannot fix the world's rdf data !

so librdf install works well, i tried to call it from its python bindings, everything responds as expected
I installed surf with 3 plugins : rdflib, librdf and sparql

Surf can create a store, load some data, use things like get_class over it, but when I query it I always get a NoneType (either in raw and programmatic queries)
wondering if it's the same for everyone else (or just me)

>>> import surf,rdflib
>>> from surf.query import *
>>> uri="http://www.w3.org/People/Berners-Lee/card.rdf"
>>> mystore = surf.Store(**{"reader": "librdf", "writer" : "librdf"})
>>> session = surf.Session(mystore)
>>> mystore.load_triples(source=uri)
True
>>> mystore.size()
78
>>> mystore.execute_sparql("SELECT ?s ?p ?o WHERE { ?s ?p ?o }")
>>> list(mystore.execute_sparql("SELECT ?s ?p ?o WHERE { ?s ?p ?o }"))
Traceback (most recent call last):
  File "<console>", line 1, in <module>
TypeError: 'NoneType' object is not iterable
>>>

It's under Python 2.7.1, rdflib 3.1.0 with rdfextras , redland 1.0.13, raptor 2.0.2, rasqual 0.9.25

[Christoph Burgmer]

Hi Dominique

Am Dienstag, 23. August 2011 schrieb Dominique Guardiola Falco:
> Surf can create a store, load some data, use things like get_class over it,
> but when I query it I always get a NoneType (either in raw and programmatic
> queries)
> wondering if it's the same for everyone else (or just me)

I've never gotten around to properly implement execute_sparql (which is not a
core concept to SuRF). If you look at the plugin implementation, you'll see
the code is commented out.

SuRF's default behaviour is to return "None" if the method is not implemented.
I believe in some places it would be better to raise a NotImplementedError.

Here's a similar query that does work:

>>> import surf

>>> uri="http://www.w3.org/People/Berners-Lee/card.rdf"
>>> mystore = surf.Store(**{"reader": "librdf", "writer" : "librdf"})

>>> mystore.load_triples(source=uri)
True
>>> mystore.size()
78
>>> from surf.query import *
>>> q = mystore.reader._execute(select('?s', '?p', '?o').where(('?s', '?p',
'?o')))
>>> q = mystore.reader._execute(select('?s', '?p', '?o').where(('?s', '?p',
'?o')))
>>> list(q)
[{'p': <RDF.Node object at 0x1f06290>, 's': <RDF.Node object at 0x1f06390>,
'o': <RDF.Node object at 0x1f062d0>}, {'p': <RDF.Node object at 0x1f06450>,
's': <RDF.Node object at 0x1f06190>, 'o': <RDF.Node object at 0x1f063d0>},
...

You can try to use the mystore.reader._to_table() method to convert the output
q into rdflib types.

It shouldn't be too difficult to complete execute_sparql, it is basically all
about returning a proper JSON result, similar to the other plugins.

Hope this helps
-Christoph

[Dominique Guardiola Falco]

Great, this works and programmatic queries too
It's nice to be able to generate SPARQL queries like that
I saw another project on Bitbucket, called Telescope, just for this task
But surf has a broader API

I do not need JSON output right now, getting query results in python/rdflib objects is great

I had no EOL or character errors at all, librdf seems definitely more solid than rdflib for parsing !

[Christoph Burgmer]

Just for reference.

There are 7 failing tests for the experimental librdf plugin (of total 56) of
which
- 3 can be attributed to missing support for execute_sparql() (see mail from 2
days ago)
- 3 seem to stem from a different interpretation on context on librdf's side
(or so I believe)
- 1, I am not completely sure, might be the same as the preceding 3.

I tried to read on librdf's documentation on context, but didn't make it
through the whole page: http://librdf.org/notes/contexts.html

-Christoph

[Christoph Burgmer]

I believe your are reporting two issues here.

> I'm testing the sparql plugin to access a D2R server (Joseki + ARQ)
> surf fails on a simple query, but this query works when testing it directly
> on the server (through a js interface)

I think this is due to a spelling error in your code "endproint".

However your might find that the SuRF sparql protocol plugin doesn't properly
work with Joseki. I haven't been able to setup Joseki to be able to write to
it, so I couldn't properly test. However somebody on GitHub implemented a
joseki_sparql plugin, maybe this is what you need:
https://github.com/bollwyvl/surf/commits/surfrdf

> AFAIK, librdf is not in the loop, I just created a sparql store with surf.
> But the strange thing is that librdf is around because everytime I load it,
> it throws 3 errors - but works (perhaps a problem on my side when
> installing it seems, I opened an issue here :
> http://bugs.librdf.org/mantis/view.php?id=460 )
>
> could there be a conflict between surf and librdf "RDF" namespace ?
>
> >>> import surf
> >>> from surf.query import *
> >>> store = surf.Store( reader = 'sparql_protocol',
>
> ... endproint = 'http://localhost:2020/sparql',
> ... default_graph = 'http://localhost:2020')
> librdf error - storage postgresql already registered
> librdf error - storage virtuoso already registered
> librdf error - query language vsparql already registered

I can't reproduce this error. I did setup a clean environment, install surf
from scratch and add the librdf plugin:

$ virtualenv env
$ source env/bin/activate
$ easy_install -U surf surf.sparql_protocol
$ cd /tmp/surf/plugins/surf.librdf/
$ python setup.py sdist
$ pip install dist/surf.librdf-1.0.0.tar.gz

No error messages here. Have you by any chance to installations of the SuRF
librdf plugin, so that SuRF finds it twice?

> By the way, has the developement migrated on github or is it still on svn ?

No, the fork on github is just a private one. It is currently updated
automatically, so changes from svn should be pulled over. Not to obvious, but
the surfrdf branch has the upstream source, while master has things I played
with. I just probably move the librdf plugin to svn, as it seems usable.

-Christoph

[Christoph Burgmer]

> > AFAIK, librdf is not in the loop, I just created a sparql store with
> > surf. But the strange thing is that librdf is around because everytime I
> > load it, it throws 3 errors - but works (perhaps a problem on my side
> > when installing it seems, I opened an issue here :
> > http://bugs.librdf.org/mantis/view.php?id=460 )
> >
> > could there be a conflict between surf and librdf "RDF" namespace ?
> >
> > >>> import surf
> > >>> from surf.query import *
> > >>> store = surf.Store( reader = 'sparql_protocol',
> >
> > ... endproint = 'http://localhost:2020/sparql',
> > ... default_graph = 'http://localhost:2020')
> > librdf error - storage postgresql already registered
> > librdf error - storage virtuoso already registered
> > librdf error - query language vsparql already registered
>
> I can't reproduce this error. I did setup a clean environment, install surf
> from scratch and add the librdf plugin:
>
> $ virtualenv env
> $ source env/bin/activate
> $ easy_install -U surf surf.sparql_protocol
> $ cd /tmp/surf/plugins/surf.librdf/
> $ python setup.py sdist
> $ pip install dist/surf.librdf-1.0.0.tar.gz
>
> No error messages here. Have you by any chance to installations of the SuRF
> librdf plugin, so that SuRF finds it twice?

Sorry, should've said "Have you by any chance *two* installations"

[Dominique Guardiola Falco]

The syntax error was the problem - I tried to delete this message in GGroups , too late...
however the sparql_plugin had no success

I'm using now the joseki_sparql plugin over the D2R server and the reader works , so perhaps you could bring it to the main branch too
The update queries I tried with it returns me "Bad Query" errors, but I must triple-check my syntax...

[Christoph Burgmer]

> I'm using now the joseki_sparql plugin over the D2R server and the reader
> works , so perhaps you could bring it to the main branch too
> The update queries I tried with it returns me "Bad Query" errors, but I
> must triple-check my syntax...

I tried to setup Joseki and get the same endpoint for read & write access,
however this failed. If you succeed in making this work, please do document
how you did it.

-Christoph

[Dominique Guardiola Falco]

In fact, I use a D2R server, a package to map a SQL database to RDF, where Joseki is pre-installed, so I can work with "traditional" django apps and expose a SPARQL endpoint for the data.

the installation is not difficult :
This is the tutoral for the SPARQL/Update flavour of the D2R server:
http://d2rqupdate.cs.technion.ac.il/tutorial
(a trick : for the jsqlparser, rename the downloaded .jar to .zip, unzip, and grab the jsqlparser.jar in the lib folder)
you also need to have a jdbc jar for your db, the package already includes MySQL and PgSQL 8.4

You don't need to follow all the tutorial, you can also generate your own SQL>RDF mapping using the "generate mapping" script, like it's explained here : http://www4.wiwiss.fu-berlin.de/bizer/d2r-server/#quickstart

once you have :
- all the jars in the lib folder
- a mapping.n3 file pointing to a SQL db, with correct credentials
you just launch it : "./d2r-server mapping.n3" and then access the web/snorql interface on http://localhost:2020/