Storing Rules and Statements in SQL / DB

[akoumjian]

I know it's been asked before, but I wanted to see if anyone had
recently done work on storing and retrieving both rulesets and
statements into databases such as MySQL. I am one of what appears to
be a growing number of people using Django (or other framework) that
would like to make use of PyKE. The current method of using source and
compiled files is not just potentially slow and memory hogging for
large data or rulesets (where I imagine the ones in my project getting
quite unwieldy), but also doesn't lend itself well to web projects
where the idea is to keep persistent storage in a DB while keep your
application "stateless".

I am less concerned about the multi-threading / asynchronous behavior
requested in the past, as my project would have knowledge bases unique
to each user. If anyone has made some progress in this direction, I
would love to see your work. If not, perhaps someone might have a
guess as to how they would begin to approach the problem.

[Ken Bridgeman]

Refreshing this question.

I have a need for over 100,000 facts that when passed through the rule system can generate 20 billion combinations.  Obviously the rules handle the 100K to 20billion, but I need fast access to the 100,000 facts.

Basically, I'd like to be able to do a query on a SQL database, then for each row in the result bind the variables and continue with the next predicate in the rule something like this:

Authorization_Rule17

[Ken Bridgeman]

Lost previous response ... starting over ...

I have a similar desire to have access to a large set of facts in a database; i.e. 100,000 facts which with BC rules will generate 20 billion combinations.

I want to be able to make a query to a SQL database and have the variables bind to columns in the first row.  Backtracking would bind variables to columns in the next row, until the number of rows in the result set was depleted.

A rule might look like this...simplified for this discussion.

# Allow person to have access to client

AuthRule17

   use  Permit(?person, CLIENT, ?client_id)

   when  DB_STOREDPROC('client_db', 'client_table', 'client_lookup_proc', ?person, ?client_id)

It would be a lot more complicated than this since the SQL would need to change if ?person, or ?client_id were bound vs unbound, etc.

But DB_STOREDPROC would return a set of results.  The first row would be bound to ?person and ?client_id and remaining predicates processed.

Upon backtracking, subsequent rows would be bound to ?person and ?client_id just as they are bound today to items from the fact_base.

On Wednesday, July 20, 2011 12:47:05 PM UTC-5, akoumjian wrote:

[Bruce Frederiksen]

Hi Ken,

Pyke supports user-defined knowledge bases.  You can subclass knowledge_base and knowledge_entity_list (both defined in knowledge_base.py) to create your own types of knowledge bases.  Pyke provides 4 sets of subclasses for fact bases (fact_base.py), question bases (question_base.py), rule bases (rule_base.py) and the "special" knowledge base (special.py).  You can look at these 4 examples to see how to do the subclassing.  These are each only a few hundred lines of code.

The knowledge_base subclass represents one knowledge base.  This is the first name of the name1.name2(...) premises in your rules.  For example, you could have just one knowledge base object for all of the facts in the database, or one per database table.

The knowledge_entity_list subclass represents all of the facts with the same name.  These names are the name2 of name1.name2(...) in your rules.  Again, you could make these correspond to table names, or column values, or whatever makes sense for you.

The lookup method returns a context manager wrapping a generator.  You can use the closing context manager in the standard python contextlib library for the context manager.  The generator yields (nothing) for each set of pattern variable bindings that match the goal presented to the lookup method.  It binds the pattern variables, does the yield back to the caller, then unbinds the pattern variables for each matching fact.

You could create your own kind of knowledge base that does SQL queries for its fact lookups.  If you only want facts in your database, look at fact_base.py as your main example.  Facts are added in the fact_base class (add_universal_fact, add_case_specific_fact, assert_), and the lookup is done in the fact_list class (lookup).

To create the SELECT query, use the is_data method on each pattern passed to lookup to see if it is a data value or a pattern variable.  Collect the data values (converting the pattern to the data value with the as_data method) to build your WHERE clause.  Then collect the columns related to the pattern variables (is_data is False) in the SELECT clause.  Loop on the returned rows binding the data returned in each row to the pattern variables using the match_data method.  Be sure to mark the bindings and call undo_to_mark within a finally clause (so that it is done on exceptions, this is the reason for the context managers).

The pattern objects are defined in pattern.py, and the binding objects in contexts.py if you want to take a look.  Basically, the patterns are static objects (immutable) that store all of the pattern variable bindings in a context (binding) object.

Hope this helps!

-Bruce

[Ken Bridgeman]

Thanks.  Just getting back to this personal project.  We'll see if I make progress.  Cheers.