Adding denoted/calculated functions
Mark Birbeck
First, thanks for a great library. I've come to it via the also
excellent ORDF library, and the two together are proving to be a
powerful combination.
My problem is that I'd like to use the forward-chaining features of
ORDF+FuXi to not only generate simple values, but also to calculate
values based on data coming in. I've read other threads where you warn
about performance issues, but I'm not so concerned about that at this
stage, since I think the convenience of causing new values to be
generated simply by creating rules is so great that it beats doing it
some other -- possibly faster -- way.
(And if you add in ORDF's queueing abilities the problem becomes even
less significant in my environment.)
So, in order to start finding my way around this I began first with
the builtin predicates. It took a bit of experimentation to get my
functions to be called (and I'm still unable to add my own functions
outside of the BuiltinPredicates.py module), but at least I got to a
point where I could test for some condition and return a boolean.
I realise it was a little optimistic to imagine that I could use the
same mechanism to create a calculated function, but I thought I might
as well try. :)
The first thing I needed to establish was what the syntax should be in
N3; does the following seem reasonable?:
{?C v:organization-name ?name . ?name matching:funcEnc ?enc .}
=> {?C matching:encoded ?enc .}.
I.e., for every organization name predicate, generate an encoded version of it?
Next I cheated and created a function that satisfied the library in
its registration stage (by supporting both subject and object) but
then simply returned a value:
def MatchingEncode(subject,object_):
if not isinstance(subject,Variable):
assert isinstance(term,Literal),"matching:encode can only
be used with Literals! (%s)"%term
def matchingEncodeF(s):
assert isinstance(s,Literal),"matching:encode can only be
used with Literals!: %s"%(s)
return "7336"
return matchingEncodeF
Since the loop in the propagate() method in BetaNode relies on these
functions returning a boolean, I added a check to see if the variable
on the RHS was actually defined, and if it wasn't I then called the
function in a different way so as to get the returned value:
if (isinstance(rhs,Variable) and not rhs in binding):
val = builtin.func(lhs)
.
.
.
The problem is that I don't know what comes next! Or indeed if
anything can come next, because I could be too late in the process to
influence the conclusions.
For example, if I replace the second line above with this:
partialInst.addConsistentBinding({Variable(rhs):Literal(builtin.func(lhs))})
matches.add(partialInst)
(and modify addConsistentBinding() so that it allows assigned values
rather than setting everything to None), I do get a predicate set in
the output, but its value is a bnode rather than "7336".
I realise I'm layering one hack on top of another here, but at the
moment I'm really just trying to find the right place in the sequence
to focus on; once I have triples coming out then I'll tidy up.
However, since I'm not familiar with the Rete algorithm I don't know
whether I'm way too late in the process, whether I should be focusing
on creating a token rather than modifying the bindings...or what.
So, any pointers as to where I should be directing my efforts would be
greatly appreciated.
Thanks!
Best regards,
Mark
Chimezie Ogbuji
On Mon, Dec 13, 2010 at 5:23 AM, Mark Birbeck <mark.b...@gmail.com> wrote:
> Hi Chime,
>
> First, thanks for a great library. I've come to it via the also
> excellent ORDF library, and the two together are proving to be a
> powerful combination.
Excellent. I'll be interested in learning how you are using both.
> My problem is that I'd like to use the forward-chaining features of
> ORDF+FuXi to not only generate simple values, but also to calculate
> values based on data coming in.
Ok.
> I've read other threads where you warn
> about performance issues, but I'm not so concerned about that at this
> stage, since I think the convenience of causing new values to be
> generated simply by creating rules is so great that it beats doing it
> some other -- possibly faster -- way.
Ok. Can you point out which thread you are talking about? I'm not
certain I exactly understand what you are trying to do. I can think
of 3 possible situations where you want to 'generate' values:
1. Unsafe rules (rules where not every variable which occurs in the
head of the rule also occurs in the body of the same rule). An
example is:
{ ?P a Person } => { ?P hasFather [] } .
{ ?P hasFather ?FATHER } => { ?FATHER a Person } .
The rule safety criteria ensures that they have a finite grounding, so
for example, these rule will result in an infinite chain of hasFather
statements. FuXi supports this via N3 rules with unamed BNodes in the
head:
The /test/testExistentialInHead.py file has an example of how to do this.
2. Builtins that create values. The math:product CWM builtin [1] is
an example of this:
{ ?Person hasWeight ?KG. ( ?KG 100 ) math:product ?G } => { ?Person
hasWeightInGrams ?G }
However, FuXi currently only supports binary builtins that evaluate to
true / false .
> ..snip...
> It took a bit of experimentation to get my
> functions to be called (and I'm still unable to add my own functions
> outside of the BuiltinPredicates.py module),
In order to register your functions, you can use the
additionalBuiltins keyword on either of these functions:
- FuXi.Rete.RuleStore.SetupRuleStore
- FuXi.Horn.NetworkFromN3
- FuXi.Horn.HornFromN3
It takes a dictionary which maps the builtin URIs to a function that
returns a function which is used to implement the builtin. So, for
example, <http://www.w3.org/2000/10/swap/string#startsWith> (from [1])
can be implemented with a function like this:
def StringStartsWith(subject,object_):
def startsWith(s,o):
return s.startswith(o)
return startsWith
And this can be registered and used in an N3 document in this way:
STRING_NS = Namespace("http://www.w3.org/2000/10/swap/string#")
rules = HornFromN3(open(n3FileName), additionalBuiltins={
STRING_NS.startsWith : StringStartsWith }))
> ..snip..
> The first thing I needed to establish was what the syntax should be in
> N3; does the following seem reasonable?:
>
> {?C v:organization-name ?name . ?name matching:funcEnc ?enc .}
> => {?C matching:encoded ?enc .}.
>
> I.e., for every organization name predicate, generate an encoded version of it?
Unfortunately, since only binary, boolean builtins are supported you
won't be able to support this via builtins.
>..snip..
> Since the loop in the propagate() method in BetaNode relies on these
> functions returning a boolean,
Yes, this is because only binary, boolean builtins are supported.
>..snip..
> I realise I'm layering one hack on top of another here, but at the
> moment I'm really just trying to find the right place in the sequence
> to focus on; once I have triples coming out then I'll tidy up.
> However, since I'm not familiar with the Rete algorithm I don't know
> whether I'm way too late in the process, whether I should be focusing
> on creating a token rather than modifying the bindings...or what.
Much of what you are describing is eerily similar to the struggle I
had in trying to implement builtins that generate values and can work
with the Rete algorithm some time ago. In the end, I chose to not
support them. However, I recently added support for externally
defined actions (meant as infrastructure for capabilities of the same
name in the new RIF-PRD) that are triggered when a rule is fired. I
have since added a wiki describing how it works, but it is not as easy
to setup as it should be and I will be adding an issue to that effect.
Essentially, you need to get a handle on the terminal node for the
rule you want to generate values, and register an execute action that
can do what you want to do (add triples with generated terms, for
instance). So, with the following rule:
{?person foaf:mbox ?email . } => {?person foaf:mbox_sha1sum [] }.
you can define an action like this:
def encodeAction(tNode, inferredTriple, token, binding):
from hashlib import sha1
person = binding[Variable('person')
email = binding[Variable('email')]
newTriple = (person,FOAF['mbox_sha1sum'],sha1(email).hexdigest())
tNode.network.addWME(ReteToken(newTriple))
And register it to override the handling of the rule firing
The wiki is here:
http://code.google.com/p/fuxi/wiki/ReteActions
Let me know if you have any questions, but this should support the
behavior you want.
William Waites
] an example of this:
]
] { ?Person hasWeight ?KG. ( ?KG 100 ) math:product ?G } => { ?Person
] hasWeightInGrams ?G }
]
] However, FuXi currently only supports binary builtins that evaluate to
] true / false .
I've been struggling with a similar thing, fwiw. The first cut at
the curate [1] tool has a built in predicate called httpGET which
evaluates to True if a GET on its object succeeds (it ignores the
subject).
Talking this over with some collegues what we would really like
to be able to do is something like,
{ ?resource httpReq [
verb "GET";
status "200";
content_type "text/n3" ] } =>
{ ?resource is ok }.
Still kind of a binary predicate returning a boolean, but I can't see
an obvious way of getting at the nodes one hop out from the object.
I guess in this example httpReq is really trying to be a quaternary
predicate!
Haven't looked in detail at how hard it might be to give the builtins
a little bit of context so they could explore relevant bits of their
neighbourhood in the Rete network.
] However, I recently added support for externally
] defined actions (meant as infrastructure for capabilities of the same
] name in the new RIF-PRD) that are triggered when a rule is fired. �I
] have since added a wiki describing how it works, but it is not as easy
] to setup as it should be and I will be adding an issue to that effect.
Oh this is very good news. I hadn't seen this. What I had done to
accomplish this was to stuff actions-in-disguise as built in
predicates in the body of a rule chaining off the first. Not
sure it will even work predictably depending on the order in
which each clause in the rule body gets evaluated and certainly
very ugly.
So I guess the question now is how we should cause the actions
to be configured based on some input. How can we say that such-
and-such is an action in an N3 file so that HornFromN3 goes and
puts it somewhere on the rule, and then the code for building
the network fills in executeAction on the beta node.
Cheers,
-w
[1] http://packages.python.org/curate/overview.html
--
William Waites
http://eris.okfn.org/ww/foaf#i
9C7E F636 52F6 1004 E40A E565 98E3 BBF3 8320 7664
Chimezie Ogbuji
On Fri, Dec 17, 2010 at 9:24 AM, William Waites <w...@styx.org> wrote:
>...snip
> So I guess the question now is how we should cause the actions
> to be configured based on some input. How can we say that such-
> and-such is an action in an N3 file so that HornFromN3 goes and
> puts it somewhere on the rule, and then the code for building
> the network fills in executeAction on the beta node.
In the end, I added a registerReteAction(..) method to network
instances. See the new test case for registering RETE actions:
http://code.google.com/p/fuxi/source/browse/test/testReteAction.py
Does this work for you?
-- Chime
William Waites
] In the end, I added a registerReteAction(..) method to network
] instances. See the new test case for registering RETE actions:
]
] http://code.google.com/p/fuxi/source/browse/test/testReteAction.py
]
] Does this work for you?
It's very close. I'm just testing a variant of it that goes
through the network, looks at rule heads to see if predicates
match a known action, and then goes and adds them in in the
same way, something like this:
for tNode in network.terminalNodes:
for rule in tNode.rules:
if not isinstance(rule.formula.head, Uniterm):
continue
headTriple = rule.formula.head.toRDFTuple()
for pred in actions:
if headTriple[1] == pred:
action = actions[pred](headTriple[0], headTriple[2])
tNode.executeActions[headTriple] = (True, action)
(note the check for Uniterm which I think might be missing
from registerReteAction()).
and an action looks like
class httpGET(object):
def __init__(self, sbind, obind):
self.sbind = sbind
self.obind = obind
def __call__(self, tNode, inferredTriple, token, binding, debug = False):
... do something
Brilliant work! Thanks!
-w
William Waites
Actually it seems to work very well. The curate tool is now
updated to take advantage of this.
Actions look like this:
https://github.com/wwaites/curate/blob/master/curate/rules.py
The network gets built like this:
https://github.com/wwaites/curate/blob/master/curate/rules.py
A simple rule set:
https://github.com/wwaites/curate/blob/master/examples/groups.n3
A slightly more elaborate ruleset (actually the use case I'm
concerned with at the moment):
https://github.com/wwaites/curate/blob/master/examples/tagging.n3
Cheers,
-w