Re: Issue 4 in fuxi: Switch over to current rdflib

[fu...@googlecode.com]

Comment #1 on issue 4 by costezki...@gmail.com: Switch over to current
rdflib
http://code.google.com/p/fuxi/issues/detail?id=4

I think it is about the time to switch to rdflib v3+ and python 2.7+

how big is the effort to upgrade it? when will it be taken to that level?

[fu...@googlecode.com]

Updates:
Owner: gjhigg...@gmail.com

Comment #2 on issue 4 by gjhigg...@gmail.com: Switch over to current rdflib
http://code.google.com/p/fuxi/issues/detail?id=4

In case it's of interest, I propagated my rdflib3 refactoring of FuXi to
the new 1.3 release. It's a relatively trivial refactoring that aims simply
to maintain compatibility with layer cake rdflib 2.4.X. It doesn't address
(and in fact is pretty much ignorant of) the module-level changes that
Chimezie envisaged. Four/five tests are failing, indicative of further work
required to get this up to production level-code.

http://code.google.com/r/gjhiggins-fuxi-rdflib3/source/list?name=rdflib3

FWIW.

[Chime Ogbuji]

Excellent! I will take a look and try to get a sense of the effort it would take to take this to its conclusion: switching back to rdflib while maintaining the divergent module-changes and components (such as the pure python parser, the Generic SPARQL Store, the MySQL/SPARQL implementation, etc.). Do you have any sense of this?

--
Chime Ogbuji
Sent with Sparrow

> --
> You received this message because you are subscribed to the Google Groups "fuxi-discussion" group.
> To post to this group, send email to fuxi-di...@googlegroups.com.
> To unsubscribe from this group, send email to fuxi-discussi...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/fuxi-discussion?hl=en.
>

[fu...@googlecode.com]

Comment #3 on issue 4 by gjhigg...@gmail.com: Switch over to current rdflib
http://code.google.com/p/fuxi/issues/detail?id=4

In response to Chimezie's post to the forum:

++ Excellent! I will take a look and try to get a sense of the effort
++ it would take to take this to its conclusion: switching back to
++ rdflib while maintaining the divergent module-changes and components
++ (such as the pure python parser, the Generic SPARQL Store,
++ the MySQL/SPARQL implementation, etc.). Do you have any sense of this?

I am able to report that all the above-referenced work has been restored,
refactored and recently merged back into the default branch of my clone
of rdfextras, ready for pushing to the "official" repos:

http://code.google.com/r/gjhiggins-rdfextras/source/browse/#hg%2Frdfextras

There is a Hudson CI build which tracks commits:

http://bel-epa.com/hudson/job/rdfextras-test/

and which maintains reports of test runs, currently standing at 369 tests
with 4 failures and 13 skips (of known issues, mostly with SQL stores):

http://bel-epa.com/hudson/job/rdfextras-test/lastCompletedBuild/testReport/

and (fwiw) coverage reports:

http://bel-epa.com/hudson/job/rdfextras-test/Test_coverage_Report/

With respect to detail - the MySQL/SPARQL implementation is available as
rdfextras.sparql2sql and shows little or no difference in test results to
the extant default rdfextras SPARQL implementation.

Most of the stores have been recovered and refactored but I'm unsure of
what you mean by "the Generic SPARQL Store" - the recovered stores are in:

http://code.google.com/r/gjhiggins-rdfextras/source/browse/#hg%2Frdfextras%2Fstore

Whilst the key-value stores required little change other than a mild
refactoring, the SQL stores are evincing problems when running tests that
involved contexts and Statements.

Many of the tests make assertions about the length of the graph but this
seems to be broken for contexts, as this Pdb interaction apparently
demonstrates (if the comment formatting screws up the layout, I'll
attach a text file):

python run_tests.py --pdb-failure
test/test_store/test_sqlite.py:SQLiteContextTestCase.testLenInMultipleContexts
Running nose with: --pdb-failure
test/test_store/test_sqlite.py:SQLiteContextTestCase.testLenInMultipleContexts
--attr=!performancetest --where=./ --with-doctest
--doctest-extension=.doctest --doctest-tests
> /usr/lib/python2.7/unittest/case.py(496)_baseAssertEqual()
-> raise self.failureException(msg)
(Pdb) u
> /usr/lib/python2.7/unittest/case.py(503)assertEqual()
-> assertion_func(first, second, msg=msg)
(Pdb) u
> ~rdfextras/test/test_store/test_context.py(146)testLenInMultipleContexts()
-> self.assertEquals(len(self.graph), oldLen + 1)
(Pdb) oldLen
0
(Pdb) self.graph.serialize()
*** Exception: Can't split 'hates'
(Pdb) self.graph.serialize(format="n3")
'\n<pizza> <hates> <tarek> .\n\n'
(Pdb) len(self.graph)
3
(Pdb) self.assertEquals(len([y for y in self.graph.triples((None, None,
None))]), oldLen + 1)

The failure to serialize the test statements as XML is rather inconvenient
and perhaps even a bug.

Still, even with the limitation of several significant test failures,
it is possible to run FuXi's test suite with rdflib 3.2 dev and the
"restoration" rdfextras clone.

Again, there is a Hudson CI build:

http://bel-epa.com/hudson/job/fuxi-rdflib3/

similarly tracking commits and maintaining reports of test runs, currently
standing at 87 tests and 31 failures

http://bel-epa.com/hudson/job/fuxi-rdflib3/lastCompletedBuild/testReport/

and (again, fwiw) test coverage

http://bel-epa.com/hudson/job/fuxi-rdflib3/Test_coverage_Report/

The complete console output is captured here:

http://bel-epa.com/hudson/job/fuxi-rdflib3/17/console

For my own convenience, I adjusted matters so that I could run nose, its
--pdb and --pdb-failure options are extremely useful conveniences. The
existing test/suite.py seems to find 469 doctests, I can't explain the
difference as yet.

I can't detect any significant difference between the result of suite.py run
with FuXi+layercake and the same test run with refactoredFuXi +
rdflib3/restorationrdfextras; the numbers of tests, passes and fails were
pretty much the same (to a casual inspection).

The overwhelming majority of the test failures would seem to be due to
simple
case mismatches and other format mismatches, e.g.

Expected:
( ex:Fire and ex:Water )
Got:
( ex:Fire AND ex:Water )

If this were any domain other than RDF, I would readily opine that a fix
would appear to be trivial - but I've learned to be circumspect, even
with what seems obvious.

I have recently removed the rdflib2/rdflib3 import switching because FuXi
does
not run with either rdflib-2.4.1 or rdflib-2.4.2, only with the "layercake"
fork. This is immediately apparent with the failure of imports of a
non-existent
"parse" function in rdflib.sparql.parser and then an rdflib.OWL module
which is
missing completely from the 2.4.1/2.4.2 package (that's as far as I got
before
the realisation settled in).

HTH,

Cheers,

Graham Higgins

[fu...@googlecode.com]

Updates:
Status: Started
Labels: -Type-Enhancement -Priority-Medium Type-Defect Priority-Low

Comment #4 on issue 4 by chime...@gmail.com: Switch over to current rdflib
http://code.google.com/p/fuxi/issues/detail?id=4

This is something I'm committed to doing. The next milestone will be about
OWL 2 EL reasoning and proof generation and the milestone after that will
be completely focused on switching FuXi over to rdflib3