Some thoughts about O/R functionality

[ak1394]

Hi Niklas,

to follow up that discussion we had, I'll give you some background
info on
why I was looking for RDF O/R mapper and what requirements I had.

Currently I'm working on a small web project project for a local
school,
wich involves keeping number of records of a different types in the
datastore
(parents, pupils, teachers, etc) and running various queries on these
data,
like like "find pupils of particualr class" or "find parents of pupils
of particular class".

I started off using SQLAlchemy, but soon realized that RDF and
SPARQL
probably would be a better choice, so I decided to replace SQLAlchemy
and SQL
database with equivalent RDF based tools.

The SQL Database was to be replaced by OpenLink Virtuoso, but I could
not find sutable O/R
mapper.

My requirements were:

1) It has to talk to SPARQL backend.

2) Support RDFS or OWL for schema modelling, so I don't have to define
all my classes in the code.

3) It has to be able to write changes back to the datastore.

Unfortunately I did not find anything like that, the "Seth" was
looking interesting but I did not wanted to
deal with Java/Python integration and something quite heavyweight as
Pellet, and the project itself did
not look too active.

So, I ended up writing my own, quite basic O/R mapper which is not
quite complete at this stage.

I'm not actually looking forward to have to do my own implementation
of O/R mapper, so if you plan
to add support to OWL/SPARQL/write to Oort in the near future, or do
something in that direction collaborating
with RDFAlchemy I'd be happy to switch, and possibly (if you're
interested) contribute to the project.

Best regards,
Anton

[Niklas Lindström]

Hi Anton!

Thank you for your thoughts, and pardon my late reply.

It does indeed sound reasonable to use RDF for your purpose.

While I have thought about generating SPARQL queries from declared
RdfQueries in Oort, I have discerned some potential problems with the
approach. Though it would probably be straightforward to generate a
query with lots of "OPTIONAL"s to fetch at least one level of data
(i.e limited use of subqueries in the first call), I'm afraid it may
not really align with what the Oort-based approach fits best for. That
is, a more lazy, incremented reading of data (with optional
programmatic redirection); something remote querying does poorly.

As it is, I think Oort works best when run with RDFLib and e.g. a
BerkleyDB backend (i.e. an embedded database with no connection
overhead). Also, my time is unfortunately quite limited right now, so
an eventual SPARQL-using backend for Oort would be quite some time
away anyway.

I must mention though that since the RdfQuery objects of Oort can be
serialized (to any depth) as both RDF/XML and JSON, it could be
possible to create two Oort-based apps communicating remotely, sending
"facets" of graphs to each other. If the declared approach I have
wasn't a raw Python "DSL", it could even be considered as an
alternative to SPARQL I guess. ;) I would love to examine that further
(perhaps in combination with Versa as an optional selector
language..).

Of course, this is of no help for you if you need SPARQL for getting
the data in the first place..

It is more likely that I'll work on OWL support in the not to distant
future though. I have used Oort to get at OWL model info for creating
form-based editing lately, and it seems promising to use similar code
to generate RdfQuery base classes from such data. This has also
expanded my ideas about write functionality.

Regarding writing back RDF to a persistent graph in general: I think
that much can be facilitated if one uses contexts/named graphs for
such things - that may overcome some of the problems in regard to when
a statement should be replaced or not, and also some of the issues
BNodes raise when making graph changes. With contexts, you just edit
the entire context and replace the original with the new one. It would
be a more "record"-based approach. Those records would be aligned with
the structure of the RdfQueries - ideally created from serializations
of such in the first place.

(Contexts are also something I think makes possible a limited use of
reasoners - where you'd infer new data from added material without
running through the entire persistent graph. This is quite new
territory for me though.)

I should really expand and concretize my thoughts on editing, but I
have to leave that for another time.

Finally, I wrote some thoughts related to what we've discussed here in
January; see <http://dustfeed.blogspot.com/2007/01/knowledge-bits-and-pieces.html>.

Thanks again for your response! It would be interesting to hear more
about your work and thoughts. Perhaps parts of Oort can be used to
help you anyway, either in the current form or as a base for new code.

Best regards,
Niklas

[ak1394]

Hi Niklas,

thanks for your response. Regarding the SPARQL support, I'm mostly
interested using it as a query
language to give users more flexible tool than something I could do
with say SQL or pre-programmed
queries. It rules out however (in my opinion) using any embeddable RDF
library, because of poor
or notexistent SPARQL support in most of them.

As of using named graphs for implementing write support, wouldn't it
make it more difficult to
use SPARQL to query the data organized in such fashion?

Regards,
Anton

[Niklas Lindström]

Hi!

I see. So you wouldn't necessarily use it internally in your
application if a way based on programmatic access would work? In, that
case, Oort could work. That is, if you find the current status of the
SPARQL-support in RDFLib acceptable. The next release will definitely
fix some current issues (I'm using RDFLib from trunk in my current
work, with the SPARQL fixes in place).

I think I see your concern - but gladly, no. The ConjunctiveGraph in
RDFLib works with named graphs but queries against it use the entire
conjunction (see also e.g. the 'triples' and 'quads' methods) . See
<http://oort.to/oort_api/oort.util.graphs-pysrc.html#load_if_modified>
for a simple example of how to use graph identifiers to manage content
from files.

Best regards,
Niklas

[ak1394]

Well I'm not quite sure that RDFLib SPARQL support is adequate,
at least for Virtuoso there is a promise for inference support,
which I don't think RDFLib will have.

[Niklas Lindström]

> Well I'm not quite sure that RDFLib SPARQL support is adequate,
> at least for Virtuoso there is a promise for inference support,
> which I don't think RDFLib will have.

I see. From what I've tested so far I can recommend taking a look at
FuXi: <http://code.google.com/p/python-dlp/wiki/FuXi>. As described "a
forward-chaining production system for Notation 3 Description Logic
Programming". I have sucessfully tested it with the rules in
<http://python-dlp.googlecode.com/files/pD-rules.n3> (e.g. the
presence of subPropertyOf and subClassOf statements resulted in
expected additional statements when I ran FuXi on some test data).
This in effect provides (programmatically controlled) inference
support for RDFLib.

Best regards,
Niklas

[ak1394]

I would still prefer to have inference done by a datastore, but anyway
the current
release of Virtuoso doesn't do it yet.

I will keep an eye on Oort to see when OWL support comes.

Best regards,
Anton