FuXi portability?
owen
threads but I couldn't figure out how to do se (perhaps those threads
are closed?). One of those is the rdflib compatibility thread in
which evolution to a pure python version of rdflib is considered.
This thread is to advocate a pure python version of FuXi (possibly a
subset), in the long term, if practical. One major benefit would be
the capability to execute FuXi on the Java or .NET platforms and to
integrate the FuXi runtime with the various libraries and languages
those platforms support--not just Java and C# but also Scala, Ruby,
Clojure, etc. I that were the case I'd be willing to help with cross-
platform testing on the JVM.
William Waites
fuxi work with rdflib 3, this would make the whole thing pure python
and go some ways towards portability.
given a pure python system you might try running it with jython and
from there you have java and maybe jruby,,,
Hth,
-w
> --
> You received this message because you are subscribed to the Google Groups
> "fuxi-discussion" group.
> To post to this group, send email to fuxi-di...@googlegroups.com.
> To unsubscribe from this group, send email to
> fuxi-discussi...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/fuxi-discussion?hl=en.
>
>
--
Sent from my mobile device
William Waites
Email: wwa...@gmail.com
UK tel: +44 131 516 3563
UK mob: +44 789 798 9965
Chimezie Ogbuji
version of the rdflib maintenance mercurial branch that I mostly use
FuXi with into the python-dlp Google Code project. See the wiki below
http://code.google.com/p/python-dlp/wiki/LayerCakePythonDivergence
Beyond the improvements to the SPARQL-to-SQL engine, it also
incorporates a new SPARQL rdflib store adapter based on Ivan Herman's
'SPARQL Endpoint interface to Python'. The store constructor takes a
single argument which is the URL of a SPARQL endpoint.
It is one of few steps towards essentially adopting the SPARQL algebra
and protocol as APIs for RDF in Python that can be coupled with FuXi's
emerging query re-writing capabilities in order to do SPARQL RIF / OWL
2 RL entailment over existing SPARQL endpoints as a query mediation
framework for the Semantic Web.
Responses inline below
On Wed, Jul 21, 2010 at 3:45 PM, owen <one...@gmail.com> wrote:
> This thread could have been posted as a response to a couple existing
> threads but I couldn't figure out how to do se (perhaps those threads
> are closed?). One of those is the rdflib compatibility thread in
> which evolution to a pure python version of rdflib is considered.
> This thread is to advocate a pure python version of FuXi (possibly a
> subset), in the long term, if practical.
So, this 'branch' I just committed (if you want to call it that) is a
major step towards this. I completely removed all references to the
old C-based SPARQL parser and added a dependency on pyparsing so it is
(once again) 100% pure-Python. I believe FuXi is 100% pure Python as
well, so using it with this branch should be sufficient for this.
> One major benefit would be
> the capability to execute FuXi on the Java or .NET platforms and to
> integrate the FuXi runtime with the various libraries and languages
> those platforms support--not just Java and C# but also Scala, Ruby,
> Clojure, etc. I that were the case I'd be willing to help with cross-
> platform testing on the JVM.
I'd be very interested in seeing how that would work and would
appreciate any input from people who are interested and/or want to
contribute to this branch (perhaps so it can slowly converge with
rdflib3 someday even)
-- Chime
owen
Fetching that using svn and running setup.py results in the
following. What am I doing wrong?
/Users/Shared/workspaces/workspace/layercake-python/rdflib/store/
FOPLRelationalModel/QuadSlot.py:11: DeprecationWarning: the md5 module
is deprecated; use hashlib instead
import md5
Traceback (most recent call last):
File "setup.py", line 9, in <module>
from rdflib import __version__, __date__
File "/Users/Shared/workspaces/workspace/layercake-python/rdflib/
__init__.py", line 41, in <module>
from rdflib.sparql.parser import parse as PyParseSPARQL
File "/Users/Shared/workspaces/workspace/layercake-python/rdflib/
sparql/parser.py", line 125, in <module>
from rdflib.sparql.sql.RdfSqlBuilder import BNodeRef
File "/Users/Shared/workspaces/workspace/layercake-python/rdflib/
sparql/sql/RdfSqlBuilder.py", line 10, in <module>
from rdflib.sparql.bison.QName import *
File "/Users/Shared/workspaces/workspace/layercake-python/rdflib/
sparql/bison/__init__.py", line 1, in <module>
from Processor import CreateSPARQLParser,Parse
ImportError: cannot import name CreateSPARQLParser
owen
attached error messages (Attachment 1).
Then, went back and took a closer look at the import error in
layercake. Since it appears the offending import is not actually
used, commented it out. Layercake then builds with setup.py. Rebuilt
FuXi tip with setup.py to pick this up. However, some of the unit
tests seem to be failing since they point to the old parser
(Attachment 2). Is there a new version of the tests that runs clean
with layercake? thx
Attachment 1
===========================================================
Last login: Mon Jul 26 08:09:46 on ttys000
/Users/Shared/workspaces/workspace/fuxi\ scripts/ez.command ; exit;
Macintosh-6:~ onewnan$ /Users/Shared/workspaces/workspace/fuxi\
scripts/ez.command ; exit;
Searching for fuxi
Reading http://pypi.python.org/simple/fuxi/
Reading http://code.google.com/p/python-dlp/wiki/FuXi
Reading http://code.google.com/p/fuxi
Reading http://python-dlp.googlecode.com/svn/trunk/fuxi/
Best match: FuXi 1.0-rc-II.dev
Downloading http://pypi.python.org/packages/source/F/FuXi/FuXi-1.0-rc-II.dev.tar.gz#md5=74b218ca85c35899da41e397b317c18a
Processing FuXi-1.0-rc-II.dev.tar.gz
Running FuXi-1.0-rc-II.dev/setup.py -q bdist_egg --dist-dir /var/
folders/yN/yNVCI5CMGGm7dmdKQkYp-++++TI/-Tmp-/easy_install-QeHKv6/
FuXi-1.0-rc-II.dev/egg-dist-tmp-TazwD6
easy_install", line 8, in <module>
load_entry_point('setuptools==0.6c11', 'console_scripts',
'easy_install')()
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/
easy_install.py", line 1712, in main
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/
easy_install.py", line 1700, in with_ei_usage
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/
easy_install.py", line 1716, in <lambda>
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/distutils/core.py", line 152, in setup
dist.run_commands()
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/distutils/dist.py", line 975, in run_commands
self.run_command(cmd)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/distutils/dist.py", line 995, in run_command
cmd_obj.run()
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/
easy_install.py", line 211, in run
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/
easy_install.py", line 446, in easy_install
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/
easy_install.py", line 476, in install_item
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/
easy_install.py", line 655, in install_eggs
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/
easy_install.py", line 930, in build_and_install
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/
easy_install.py", line 919, in run_setup
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/
sandbox.py", line 62, in run_setup
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/
sandbox.py", line 105, in run
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/
sandbox.py", line 64, in <lambda>
File "setup.py", line 1, in <module>
import ez_setup
ImportError: No module named ez_setup
logout
Attachment 2
==================================================================================================
Last login: Mon Jul 26 08:48:08 on ttys000
/Users/Shared/workspaces/workspace/fuxi\ scripts/tests.command ; exit;
Macintosh-6:~ onewnan$ /Users/Shared/workspaces/workspace/fuxi\
scripts/tests.command ; exit;
testOWL.py ========================================
testOWL.py:2: DeprecationWarning: the sets module is deprecated
from sets import Set
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
packages/rdflib-2.4.1.dev_r324-py2.6.egg/rdflib/store/
is deprecated; use hashlib instead
from FuXi.SPARQL.BackwardChainingStore import
TopDownSPARQLEntailingStore
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/FuXi-1.0.dev-py2.6.egg/FuXi/SPARQL/
BackwardChainingStore.py", line 12, in <module>
from FuXi.Rete.Magic import *
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/FuXi-1.0.dev-py2.6.egg/FuXi/Rete/Magic.py",
line 28, in <module>
from rdflib.sparql.bison import Parse
ImportError: cannot import name Parse
test_builtin_ordering.py ========================================
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
packages/rdflib-2.4.1.dev_r324-py2.6.egg/rdflib/store/
is deprecated; use hashlib instead
packages/rdflib-2.4.1.dev_r324-py2.6.egg/rdflib/sparql/bison/
Resource.py:3: DeprecationWarning: the sets module is deprecated
Time to build production rule (RDFLib): 7.5101852417e-05 seconds
test_builtin_ordering.py:53: DeprecationWarning: Rules in a network
should be built *after* construction via
self.buildNetworkClause(HornFromN3(n3graph)) for instance
network = ReteNetwork(rule_store)
Time to build production rule (RDFLib): 0.001708984375 seconds
.Time to build production rule (RDFLib): 7.60555267334e-05 seconds
Time to build production rule (RDFLib): 0.00133013725281 seconds
.Time to build production rule (RDFLib): 6.69956207275e-05 seconds
Time to build production rule (RDFLib): 0.00194811820984 seconds
.Time to build production rule (RDFLib): 7.91549682617e-05 seconds
Time to build production rule (RDFLib): 0.00214099884033 seconds
.
----------------------------------------------------------------------
Ran 4 tests in 0.086s
OK
testSkolemization.py ========================================
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
packages/rdflib-2.4.1.dev_r324-py2.6.egg/rdflib/store/
is deprecated; use hashlib instead
packages/rdflib-2.4.1.dev_r324-py2.6.egg/rdflib/sparql/bison/
Resource.py:3: DeprecationWarning: the sets module is deprecated
Time to build production rule (RDFLib): 7.20024108887e-05 seconds
F
======================================================================
FAIL: testUnionSkolemization (__main__.UnionSkolemizedTest)
----------------------------------------------------------------------
p=network.setupDescriptionLogicProgramming(self.tBoxGraph)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/FuXi-1.0.dev-py2.6.egg/FuXi/Rete/Network.py",
line 352, in setupDescriptionLogicProgramming
for rule in AdditionalRules(owlN3Graph):
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/FuXi-1.0.dev-py2.6.egg/FuXi/DLP/
ConditionalAxioms.py", line 67, in AdditionalRules
initNs={"owl":OWL_NS}).askAnswer[0]:
File "build/bdist.macosx-10.3-fat/egg/rdflib/Graph.py", line 752, in
query
parsedQuery)
File "build/bdist.macosx-10.3-fat/egg/rdflib/sparql/bison/
Processor.py", line 23, in query
assert USE_PYPARSING,'C-based BisonGen SPARQL parser has been
removed'
AssertionError: C-based BisonGen SPARQL parser has been removed
----------------------------------------------------------------------
Ran 1 test in 0.075s
FAILED (failures=1)
testExistentialInHead.py ========================================
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
packages/rdflib-2.4.1.dev_r324-py2.6.egg/rdflib/store/
is deprecated; use hashlib instead
packages/rdflib-2.4.1.dev_r324-py2.6.egg/rdflib/sparql/bison/
Resource.py:3: DeprecationWarning: the sets module is deprecated
testExistentialInHead.py:49: DeprecationWarning: Rules in a network
should be built *after* construction via
self.buildNetworkClause(HornFromN3(n3graph)) for instance
inferredTarget = deltaGraph)
Time to build production rule (RDFLib): 0.00206899642944 seconds
Time to calculate closure on working memory: 2.43186950684 m seconds
@prefix : <file:///Users/Shared/workspaces/workspace/fuxi/test/#>.
@prefix m: <http://example.com/#>.
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
:det1 m:inference [ a m:Inference;
m:inference_name "Inference1"].
:det2 m:inference [ a m:Inference;
m:inference_name "Inference2"].
.
----------------------------------------------------------------------
Ran 1 test in 0.039s
OK
additionalDLPTests.py ========================================
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
packages/rdflib-2.4.1.dev_r324-py2.6.egg/rdflib/store/
is deprecated; use hashlib instead
packages/rdflib-2.4.1.dev_r324-py2.6.egg/rdflib/sparql/bison/
Resource.py:3: DeprecationWarning: the sets module is deprecated
FFTime to build production rule (RDFLib): 0.000259876251221 seconds
.Time to build production rule (RDFLib): 6.89029693604e-05 seconds
.FFF
======================================================================
FAIL: testBasePredicateEquivalence
(__main__.AdditionalDescriptionLogicTests)
----------------------------------------------------------------------
testBasePredicateEquivalence
self.assertEqual(repr(Class(EX_NS.Foo)),"Class: ex:Foo
EquivalentTo: ex:Bar")
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/FuXi-1.0.dev-py2.6.egg/FuXi/Syntax/
InfixOWL.py", line 914, in __repr__
manchesterSyntax(classOrIdentifier(s),self.graph) for s in ec]
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/FuXi-1.0.dev-py2.6.egg/FuXi/Syntax/
InfixOWL.py", line 246, in manchesterSyntax
initNs=nsBinds):
File "build/bdist.macosx-10.3-fat/egg/rdflib/Graph.py", line 752, in
query
parsedQuery)
File "build/bdist.macosx-10.3-fat/egg/rdflib/sparql/bison/
Processor.py", line 23, in query
assert USE_PYPARSING,'C-based BisonGen SPARQL parser has been
removed'
AssertionError: C-based BisonGen SPARQL parser has been removed
======================================================================
FAIL: testExistentialInRightOfGCI
(__main__.AdditionalDescriptionLogicTests)
----------------------------------------------------------------------
testExistentialInRightOfGCI
self.assertEqual(repr(Class(EX_NS.Foo)),"Class: ex:Foo SubClassOf:
( ex:someProp some ex:Omega )")
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/FuXi-1.0.dev-py2.6.egg/FuXi/Syntax/
InfixOWL.py", line 905, in __repr__
manchesterSyntax(classOrIdentifier(s),self.graph) for s in sc]
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/FuXi-1.0.dev-py2.6.egg/FuXi/Syntax/
InfixOWL.py", line 236, in manchesterSyntax
return '( %s some %s )'%
(propString,manchesterSyntax(someClass,store))
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/FuXi-1.0.dev-py2.6.egg/FuXi/Syntax/
InfixOWL.py", line 246, in manchesterSyntax
initNs=nsBinds):
File "build/bdist.macosx-10.3-fat/egg/rdflib/Graph.py", line 752, in
query
parsedQuery)
File "build/bdist.macosx-10.3-fat/egg/rdflib/sparql/bison/
Processor.py", line 23, in query
assert USE_PYPARSING,'C-based BisonGen SPARQL parser has been
removed'
AssertionError: C-based BisonGen SPARQL parser has been removed
======================================================================
FAIL: testOtherForm (__main__.AdditionalDescriptionLogicTests)
----------------------------------------------------------------------
NormalFormReduction(self.ontGraph)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/FuXi-1.0.dev-py2.6.egg/FuXi/DLP/
DLNormalization.py", line 188, in NormalFormReduction
UniversalNominalRangeTransformer().transform(ontGraph)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/FuXi-1.0.dev-py2.6.egg/FuXi/DLP/
DLNormalization.py", line 69, in transform
u'rdfs':RDFS.RDFSNS}):
File "build/bdist.macosx-10.3-fat/egg/rdflib/Graph.py", line 752, in
query
parsedQuery)
File "build/bdist.macosx-10.3-fat/egg/rdflib/sparql/bison/
Processor.py", line 23, in query
assert USE_PYPARSING,'C-based BisonGen SPARQL parser has been
removed'
AssertionError: C-based BisonGen SPARQL parser has been removed
======================================================================
FAIL: testOtherForm2 (__main__.AdditionalDescriptionLogicTests)
----------------------------------------------------------------------
NormalFormReduction(self.ontGraph)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/FuXi-1.0.dev-py2.6.egg/FuXi/DLP/
DLNormalization.py", line 188, in NormalFormReduction
UniversalNominalRangeTransformer().transform(ontGraph)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/FuXi-1.0.dev-py2.6.egg/FuXi/DLP/
DLNormalization.py", line 69, in transform
u'rdfs':RDFS.RDFSNS}):
File "build/bdist.macosx-10.3-fat/egg/rdflib/Graph.py", line 752, in
query
parsedQuery)
File "build/bdist.macosx-10.3-fat/egg/rdflib/sparql/bison/
Processor.py", line 23, in query
assert USE_PYPARSING,'C-based BisonGen SPARQL parser has been
removed'
AssertionError: C-based BisonGen SPARQL parser has been removed
======================================================================
FAIL: testValueRestrictionInLeftOfGCI
(__main__.AdditionalDescriptionLogicTests)
----------------------------------------------------------------------
testValueRestrictionInLeftOfGCI
self.assertEqual(repr(leftGCI),
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/FuXi-1.0.dev-py2.6.egg/FuXi/Syntax/
InfixOWL.py", line 1176, in __repr__
return
manchesterSyntax(self._rdfList.uri,self.graph,boolean=self._operator)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/FuXi-1.0.dev-py2.6.egg/FuXi/Syntax/
InfixOWL.py", line 196, in manchesterSyntax
children = [manchesterSyntax(child,store) for child in
Collection(store,thing)]
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/FuXi-1.0.dev-py2.6.egg/FuXi/Syntax/
InfixOWL.py", line 234, in manchesterSyntax
return '( %s value %s )'%(propString,manchesterSyntax(val,store))
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/FuXi-1.0.dev-py2.6.egg/FuXi/Syntax/
InfixOWL.py", line 246, in manchesterSyntax
initNs=nsBinds):
File "build/bdist.macosx-10.3-fat/egg/rdflib/Graph.py", line 752, in
query
parsedQuery)
File "build/bdist.macosx-10.3-fat/egg/rdflib/sparql/bison/
Processor.py", line 23, in query
assert USE_PYPARSING,'C-based BisonGen SPARQL parser has been
removed'
AssertionError: C-based BisonGen SPARQL parser has been removed
----------------------------------------------------------------------
Ran 7 tests in 0.035s
FAILED (failures=5)
logout
[Process completed]
[Process completed]
William Waites
> So this email sufficiently motivated me to push the most current
> version of the rdflib maintenance mercurial branch that I mostly use
> FuXi with into the python-dlp Google Code project. See the wiki below
>
Just a quick note, I managed to get FuXi running on the Android emulator
under the native Python from ASE with this branch. The main problems
were problems with rdlib/__init__.py redefining things, e.g. complaining
about "from rdflib import RDF". This is possibly due to a lack of
easy_install or the like. A module called fiximports.py, attached, is
a very ugly kludge that makes it work.
Apart from that, smooth sailing. Tested with the little program I sent to
Vasiliy and the list the other day.
> very interested in seeing how that would work and would
> appreciate any input from people who are interested and/or want to
> contribute to this branch (perhaps so it can slowly converge with
> rdflib3 someday even)
>
Some preliminary attempts to get Jython running on Android failed
miserably I'm sorry to say...
Cheers,
-w
--
William Waites <wwa...@gmail.com>
Mob: +44 789 798 9965
Fax: +44 131 464 4948
William Waites
and friends running on Android, mostly due to missing site.py
and distutils. I've hacked together some shell scripts that
will get FuXi running on one though, just as a (repeatable)
proof of concept.
Some changes to setuptools were necessary but apart from
that the only things of note are the shell scripts (in bin/) and
the minor patches to layercake that owenan mentions in the
FuXi portability thread.
This fixes the weird problem with rdflib imports that I
mentioned earlier.
Mercurial repository here: http://bitbucket.org/ww/ncpy/src
---- README ----
Not Crippled Python for Android
===============================
This package adds some bits to the ASE (Android Scripting
Environment) that makes it behave a bit more like a normal
Unix python. A modified setuptools is included which
bundles the absent distutils. It also installs site.py and
unittest.py that are missing and usually come with python
2.6.
It can also install rdflib and fuxi. The versions of rdflib
and fuxi are the pure python ones from the experimental
branch documented at
http://code.google.com/p/python-dlp/wiki/LayerCakePythonDivergence
It requires ASE and their Python interpreter to be installed.
To use, you need the Android SDK, and an emulator or real
phone running. To install from source, try::
adb push ncpy /mnt/sdcard/ncpy
And then do::
adb shell
to get a shell.
To set up the environment (a little like virtualenv) do::
# . /mnt/sdcard/ncpy/bin/shenv
To compile and install the modules::
# sh /mnt/sdcard/ncpy/bin/install
# sh /mnt/sdcard/ncpy/bin/install_fuxi
To run python with the correct environment (after the
. shenv step above) you just run::
# $PYTHON
You can run a simple test of FuXi by doing::
# $PYTHON /mnt/sdcard/ncpy/test/test_fuxi.py
Chimezie Ogbuji
BisonGen / C SPARQL parser and committed the changes into
layercake-python. See:
http://code.google.com/p/python-dlp/source/detail?r=333
On Mon, Jul 26, 2010 at 11:02 AM, owen <one...@gmail.com> wrote:
> However, some of the unit tests seem to be failing since they point to the old parser
> (Attachment 2). Is there a new version of the tests that runs clean
> with layercake? thx
I also committed 2 sets of changes to FuXi/trunk:
1. References to the BisonGen / C SPARQL parser
2. A few minor updates to the RDF data description language (the
relevant section in the user manual for how to use this to identify
derived predicates is here:
http://code.google.com/p/fuxi/wiki/FuXiUserManual#IdentifyDerivedPredicates)
See:
http://code.google.com/p/fuxi/source/detail?r=4e5abf45c4127bb77d05dd3a6efbe06f484e2352
I was able to run the OWL tests (with --topDown=ground) using this
version of layercake-python. Hope this helps
Vasiliy Faronov
So what is the current plan for RDFLib use with FuXi?
Do you plan to stick with RDFLib 2.4 / layercake-python or to converge
eventually to stock RDFLib 3.0, perhaps incorporating Graham Higgins'
fixed imports branch?
--
Vasiliy Faronov
Chimezie Ogbuji
On Tue, Jul 27, 2010 at 12:37 PM, Vasiliy Faronov <vfar...@gmail.com> wrote:
> Hi Chimezie,
> So what is the current plan for RDFLib use with FuXi?
The million dollar question :)
> Do you plan to stick with RDFLib 2.4 / layercake-python or to converge
> eventually to stock RDFLib 3.0, perhaps incorporating Graham Higgins'
> fixed imports branch?
I've gone back and forth on this. The last time around [1], I was
inclined to go ahead with an rdflib 'branch' that was based on the 3.0
api, incorporated existing and future SPARQL capabilities, and was
maintained in an rdflib svn branch (in the official Google Code
project).
As was mentioned earlier [2], most of the API changes require changing
imports, however I have a significant amount of deployed, production
code that relies on these old APIs, so despite the fact that I can do
this for FuXi straight forwardly (via Graham Higgins' FuXi patch), I
can't do it without essentially introducing a major revision into
deployed production code that relies on FuXi and rdflib. It really
needs to be said that this is the risk you have when you make a
significant API change to widely-deployed software that is not
backwards compatible.
There is an existing hack [3] that is meant to allow references to the
old API to 'resolve' to the new API. I haven't tried this and one
options would be to:
- Take a fresh / current rdflib 3.0
- Incorporate the current SPARQL capabilities from layercake-python
- Add the hack
I would then need to ensure this combination works with deployed code
(do integration testing). Even if it does, it doesn't guarantee that
future versions of rdflib 3.+ will incorporate the hack (much less the
SPARQL capabilities - which goes to my next point below). The fact
that 3.0 didn't include a mechanism for keeping backward compatibility
seems to suggest that there is a high degree of difficulty in doing
so.
Also, the manner in which SPARQL support was removed from the
'current' rdflib indicates that (perhaps) SPARQL is not a high
priority for the rdlfib 3+ road map, despite the fact it is very
central to the FuXi roadmap.
So as a result, I'm now more inclined to:
* Use layercake-python as a fork of rdflib that uses the old API and
incorporates the extant SPARQL capabilities as well as subsequent
SPARQL capabilities .
* Rely on others to investigate how to 'backport' the API disconnect
(which I have been told is a significant majority of the difference
between rdflib 2.4.* and rdflib 3.*)
I'm hoping that by removing the BisonGen SPARQL parser and all its
dependencies (which was a major motivation for removing SPARQL from
rdflib 3.* in the first place), people who are kind enough to help to
'backport' will find it easier to do with a pure python library.
Sorry for the convoluted answer, but it is a very convoluted situation :(
[1] http://groups.google.com/group/fuxi-discussion/msg/0520501010b42ff8
[2] http://groups.google.com/group/fuxi-discussion/msg/e9572ec63d8984a3
[3] http://groups.google.com/group/rdflib-dev/msg/40c48c76eafb0605
Vasiliy Faronov
> Sorry for the convoluted answer, but it is a very convoluted situation :(
It clarifies a lot, thanks.
I wonder if the SPARQL implementation in rdfextras could be aligned with
the one in layercake-python (I'm guessing the latter is more complete?).
Then, we (maybe I :-)) could try maintaining an "experimental"
rdflib-3.0 branch of FuXi, based on Graham's patches, by backporting
changes from the main (l-p) branch while keeping the 3.0 imports.
--
Vasiliy Faronov
Graham Higgins
> * Rely on others to investigate how to 'backport' the API disconnect
> (which I have been told is a significant majority of the difference
> between rdflib 2.4.* and rdflib 3.*)
my understanding. As you have observed, rdflib 3.0.0 ships without any
out-of-the-box SPARQL facility. The current rdflib "Intro to Sparql"
wiki page is incorrect, the code included therein won't execute with a
fresh rdflib 3.0.0. Merely registering the sparql plugin is
insufficient, the /processor name/ has to be specified as a kwarg to
graph.query.
> I'm hoping that by removing the BisonGen SPARQL parser and all its
> dependencies (which was a major motivation for removing SPARQL from
> rdflib 3.* in the first place), people who are kind enough to help to
> 'backport' will find it easier to do with a pure python library.
implementation has been transcribed into my working rdfextras repos
[1] (into a subdir called "bisonsparql"), refactored and the old
test_sparql test suite adapted for the bisonsparql implementation -
largely via specifying:
graph.query(qObject, processor="bisonsparql")
67 tests run, 7 failures.
[1] http://bitbucket.org/gjhiggins/rdfextras/overview
Cheers,
Graham.
William Waites
> Well on the way to completion. The layercake non-C SPARQL
> implementation has been transcribed into my working rdfextras repos
> [1] (into a subdir called "bisonsparql"), refactored and the old
> test_sparql test suite adapted for the bisonsparql implementation -
> largely via specifying:
>
Could I suggest a different name? bisonsparql might be
confusing as it doesn't use bison anymore. Perhaps
"layercake" or maybe that would be confusing too...
I'm bad at thinking of names.
Cheers,
-w
Graham Higgins
Hash: SHA1
On 29 Jul 2010, at 09:39, William Waites wrote:
> Could I suggest a different name? bisonsparql might be
> confusing as it doesn't use bison anymore. Perhaps
> "layercake" or maybe that would be confusing too...
> I'm bad at thinking of names.
Some of the FuXi code calls routines imported from the bison
subdirectory, specifically the main SPARQL API in
rdfextras.bisonsparql.bison.Processor, so that's why "bisonsparql".
But I agree, it is clumsy at best and potentially misleading at worst,
choosing a different module name is indicated. I'm still cogitating.
I'm also still trying to comprehend the full motivation for the C-
based implementation. I'm sure that it wasn't just to give Win/XP
users a hard time and, given that the rdflib 3.0.0 API affords plugin
Processors, I re-instituted the csparql module in order to give it a
spin with rdflib 3.0.0.
I used Niklas Lindström's sys.modules approach [1] to persuade
SPARQLParserc to play nicely with the other modules, bulk-edited the
tests and got:
54 tests run in 2.4 seconds.
22 errors (32 tests passed)
where nearly all of the errors are due to a single (API-related) issue
which I haven't yet looked at closely:
rdfextras/csparql/bison/SPARQLEvaluate.py line 200 in unRollTripleItems
for item in items:
TypeError: 'Resource' object is not iterable
Does anyone see /any/ merit in keeping the csparql module as an option
(possibly in its own separate package) or should it be discarded as
"no longer supported"?
[1] http://groups.google.com/group/rdflib-dev/msg/40c48c76eafb0605
- --
Cheers,
Graham
http://www.linkedin.com/in/ghiggins
-----BEGIN PGP SIGNATURE-----
iEYEARECAAYFAkxWLlwACgkQOsmLt1NhivwE5wCg3LTVmHJ/+Yf6E4GWFkjwdasP
BoMAoP8CPR7Ik1GKE/TeKCgRb3/m6cB1iQCVAgUBTFYuXFnrWVZ7aXD1AQJAAgP/
cT4WSfNpaEzDLlc969vEXJaR8rCQxo+coewkLrbdiIa4xHcf065YqlMRNWur/t0M
Mf1vFGejTsW7+XfW8oAbpdfjAmamc41Q6D6jQiXYpV8jcJVOFwm0wWYkf44zQ5La
9+iVNHQDWK1RaW0pZ8y5RhflIVP1T6KS82ZXv+RiuUs=
=sgxK
-----END PGP SIGNATURE-----
owen
Here are some initial thoughts and findings about running FuXi /
layercake over jython / JRE.
Note to begin with jython is still at version 2.5 whereas FuXi runs
over python 2.6.
Surfing the Web a bit reveals that while jython 2.6 has been a while
coming, Oracle/Sun continues to actively support and enhance the
product. As of last spring 2.6 was targeted for late 2010. Major
performance enhancements are planned with 2.6, which is basically a
good thing, since numbers I’ve seen on the Web and in testing FuXi
show an order of magnitude longer path length with jython 2.5 vis-à-
vis python 2.6. JDK 1.7 enhancements for dynamic languages are
central to achieving the jython 2.6 performance goal. So, reading
between the lines, politics over the release of JDK 1.7 may be slowing
down jython 2.6 product release. In any case, Oracle is focusing
jython efforts on 2.6 with the expectation that 2.6 code can be
mechanically transitioned to 3.0 when the time comes.
As you might expect, there are issues getting FuXi to run on jython; a
debugging/development environment is needed. In my case this is
Eclipse J EE for the Web 20100218-1602 with Pydev 1.60; and also
MercurialEclipse 1.6.0 to fetch/pull FuXi. Pydev provides analogous
features to Eclipse’s Java debugger for both C python and jython: code
completion, stack and expression (watch) inspection, refactor (such as
extract method), go to definition, etc. An issue did arise with
Eclipse jython debugging but there seems to be a workaround, see
http://stackoverflow.com/questions/3377840/pydev-hangs-up-when-i-try-to-debug-jython.
To enable debug/breakpoints with all libraries of course requires
their various sources on the python path. This can be accomplished
under Pydev by including FuXi and layercake in the test project using
Eclipse project references. In each case, the source tab of the
included project’s python path must point to the root(s) of the
appropriate libraries. On OS X, it’s also necessary to install jython
and configure Preferences to point to it, manually entering the path
to cachedir.
Comparing test results, under jython testOwl.py runs differently than
under python. It fails in jython rdflib/plugin line 20:
__import__(module_path, globals(), locals(), True)
with AttributeError: __len__. It turns out that _import__ expects
a list as its fourth parameter, so the fix is to simply change that
line to read:
module = __import__(module_path, globals(), locals(), [True])
After this fix another issue crops up:
...
python/rdflib/syntax/parsers/RDFXMLParser.py", line 14, in
create_parser
parser.start_namespace_decl("xml", "http://www.w3.org/XML/1998/
namespace")
AttributeError: 'JavaSAXParser' object has no attribute
'start_namespace_decl'
It turns out there is already a fix for this in a more current version
of rdflib. We can ignore the attribute start_namespace_decl under
jython:
try:
# Workaround for bug in expatreader.py. Needed when
# expatreader is trying to guess a prefix.
parser.start_namespace_decl("xml", "http://www.w3.org/XML/1998/
namespace")
except AttributeError:
pass # Not present in Jython (at least)
Applying these fixes FuXi unit tests yield the following results:
testOWL.py—fails under both python and jython with list index out of
range as previously reported
test_builtin_ordering.py: executes without errors or failure under
both interpreters
testSkolemization.py: executes without errors or failure under both
interpreters
testExistentialInHead.py: executes without errors or failure under
both interpreters
additionalDLPTests.py: executes clean on python but reports two errors
with jython, see the attached.
This last and only remaining apparent difference is troublesome, since
it is a semantic difference rather than an overt failure. Perhaps
this is due to variations between python 2.5 and 2.6 specifications,
or perhaps a bug in the jython 2.5 implementation. This may in any
case be a mute point since FuXi on a poorly performing jython may not
be attractive compared to established native Java reasoning engines.
The current jython is in effect an interpreter layered atop another
interpreter, since the traditional JDK was never designed for dynamic
types. However, extending JDK for dynamic (and functional)
programming languages is a major goal of Java 1.7--to include various
dynamic languages such as ruby and groovy as well as python. This may
lead to a breakthrough in jython performance; or maybe not. In the
best case, FuXi could compete as a reasoning tool across the gamut of
JVM compatible languages. Time will tell. With a little luck that
time is not long. I suggest we should try this again when jython 2.6
appears.
=======================================
Error Messages for additionalDLPTests.py failure:
ex:NumDisV2D(?X)
:- ex:contains(?X ?NRLvfAFW136)
ex:MajorStenosis(?NRLvfAFW136),
ex:locatedIn(?NRLvfAFW136 ex:RCA),
ex:Cath(?X),
ex:contains(?X ?NRLvfAFW159),
ex:MajorStenosis(?NRLvfAFW159),
ex:locatedIn(?NRLvfAFW159 ex:LAD),
..Time to build production rule (RDFLib): 0.00200009346008 seconds
.
FAIL: testBasePredicateEquivalence
(__main__.AdditionalDescriptionLogicTests)
----------------------------------------------------------------------
Traceback (most recent call last):
testBasePredicateEquivalence
self.assertEqual(repr(rules),
AssertionError: '[Forall ?X ( ex:Foo(?X) :- ex:Bar(?X) ), Forall ?X
( ex:Bar(?X) :- ex:Foo(?X) )]' != '[Forall ?X ( ex:Bar(?X) :- ex:Foo(?
X) ), Forall ?X ( ex:Foo(?X) :- ex:Bar(?X) )]'
======================================================================
FAIL: testGCIConDisjunction (__main__.AdditionalDescriptionLogicTests)
Traceback (most recent call last):
self.assertEqual(repr(rules),
AssertionError: '[Forall ?X ( ex:Bar(?X) :- And( ex:Foo(?X) ex:Omega(?
X) ) ), Forall ?X ( ex:Bar(?X) :- And( ex:Foo(?X) ex:Alpha(?X) ) )]' !
= '[Forall ?X ( ex:Bar(?X) :- And( ex:Foo(?X) ex:Alpha(?X) ) ),
Forall ?X ( ex:Bar(?X) :- And( ex:Foo(?X) ex:Omega(?X) ) )]'
----------------------------------------------------------------------
Ran 7 tests in 10.048s
FAILED (failures=2)
owen
http://zyasoft.com/pythoneering/2010/09/jython-2.5.2-beta-2-is-released/