[ANN] Clojure RETE implementation - CLIPS-like expert system shell. New version 4.3

[ru]

New feature: added Java interface. Eclipse project example.

Home: https://github.com/rururu/rete4frames

Have fun!

Sincerely,

  Ru

[u1204]

Forgy's RETE is a self-modifying data structure.
How is this handled in Clojure?

Tim Daly

[ru]

This implementation does not strictly support principles of functional programming, immutability and so on. Clojure used mostly for simplicity of programming, specifically in right hand sides of the rules and helper functions. Moreover, it is extensively uses Java HashMaps directly for better performance.

понедельник, 12 мая 2014 г., 17:47:44 UTC+4 пользователь ru написал:

[Alan Moore]

Nice... good to see another implementation.

Have you seen clara-rules by Ryan Brush? It is actually a modified/optimized RETE but faithful to the basic design. 

See:

https://github.com/rbrush/clara-rules

Alan

[Alan Moore]

On Monday, May 12, 2014 10:29:03 AM UTC-7, da...@axiom-developer.org wrote:

Forgy's RETE is a self-modifying data structure.
How is this handled in Clojure?

The clara-rules approach treats (no pun intended) the working memory as a value, from the readme.md:

"Embrace immutability. The rule engine's working memory is a persistent Clojure data structure that can participate in transactions. All changes produce a new working memory that shares state with the previous."

Alan

[ru]

Before creating own rete implementation I have used in my applications Drools for several years. Drools is becoming more and more complex and in the end ceased to support my applications. I decided to create my own implementation as simple as possible to have a full control and quick search in case of an error in application. As I understand from quick look Clara rete implementation is a similar to Drools.  It is intresting to compare Clara, Drools and rete4frames on traditional benchmarks such as Mrs.Manners and Waltz algorithm. I tried to compare my implementation with Drools, but Manners example of Drools in the last version I looked at was brocken. I wrote to Mark Proctor about this and he promised to fix this but I did not look at that ever since.. :( Hope I will find time to accomplish this comparison.

понедельник, 12 мая 2014 г., 17:47:44 UTC+4 пользователь ru написал:

New feature: added Java interface. Eclipse project example.

[Alan Moore]

It would be interesting to compare rete4frames with clara-rules and maybe some other clojure-based RETE implementation.

Having spent some time with Clara and only looked over rete4frames it is clear that Clara has been heavily optimized for working with collections/accumulators.

Especially interesting is that Clara is built using protocols for key RETE components, e.g. working memory, transport, etc. This has been demonstrated to implement Storm clusters/topologies doing distributed hash-based joins that leverage Clara's immutability... See:

https://github.com/rbrush/clara-storm

Your rete4frames provides a good example of the power of the RETE algorithm. Also, in both rete4frames and Clara, I was surprised at the small size of the code as compared to the Drools/Jess/CLIPS code bases. This compactness is due in large measure to the host language being Clojure. :-)

Take care.

Alan

[ru]

Alan. You absolutely right! Clojure must return thinking of programmers back to solving problems from vanity and narcissism of unnecessary complexity :) I certainly will look on Clara more attentively and may be switch to it in my applications.

cheers,

   Ru

понедельник, 12 мая 2014 г., 17:47:44 UTC+4 пользователь ru написал:

New feature: added Java interface. Eclipse project example.

[Dave Sann]

this code (and the related create) is unusual:

(defn reset []
  "Reset: clear and initialize all memories"
  (def =AMEM= (object-array @ACNT))
  (def =BMEM= (object-array =BCNT=))
  (def =FMEM= (create-fmem =TEMPL=))
  (def =FMMB= (create-fmem =TEMPL=))
  (def =FIDS= (HashMap.))
  (def CFSET (atom nil))
  (def FCNT (atom 0)))

It implies that you can only have one set of rules/engine in any usage of the library. Even if you plan to use mutable data - this seems to create additional restrictions. Why not put all of these into a map and pass that around as the ruleset/engine state?

[Ryan Brush]

It's cool to see different approaches to Rete in Clojure. I think a solid implementation could really disrupt Drools. Why would I want to write in a limited rule language if I can get rule-like semantics and the expressiveness of Clojure at the same time? Of course I'm going to be biased towards Clara, but more important is the fact that Clojure provides a stronger foundation for expert systems than anything else out there. This is an opportunity for Clojure to make inroads. 

Regarding the benchmarking questions, I'm sure Drools will run Ms. Manners and other benchmarks faster than Clara at this point. This is partially because I've emphasized correctness and simplicity in Clara first, although I've recently started doing some significant profiling and optimizations. But Ms. Manners is really a poor example of typical rule engine use...it was created specifically to exercise some worst case scenarios. It's also constraint-oriented, so I'd actually use core.logic to solve that problem before I'd use any forward-chaining approach.

More important is to ask whether a system is fast enough for your needs. I gladly pay some overhead for persistent data structures because they are fast enough and save me a lot of time. (And I can spend that time optimizing the real bottlenecks of the program!)  These advantages tend to pay off in surprising ways, such as the recent excitement around how fast React and Om are. Clara will get optimized, but not at the cost of Clojure ideals like immutability.

In any case, it's great to see activity around Rete in Clojure. Clara was designed for a particular problem space I've been working in, but it's certainly not the only approach we should consider.

[ru]

Dave. It is a very good advice. Initially I tried to do as simple as possible. Now I will try to follow your advice and check what performance impact it may has.

Ryan. RETE algorithm is a very basic and can be applied to a broad spectrum of tasks not only to expert systems. For example, I use it in one application to execute a set of special parallel processes. So, requirements to implementations may be different. Here would be useful to have universal set of benchmarks to compare different implementations. I selected Manners and Waltz because they are in CLIPS distribution. It will be very appreciated if someone point out other good examples as candidates on benchmark tests.

Sincerely,
  Ru

понедельник, 12 мая 2014 г., 17:47:44 UTC+4 пользователь ru написал:

New feature: added Java interface. Eclipse project example.

[Ryan Brush]

Hey Ru,

I avoided Miss Manners simply because it doesn't represent the usage patterns I have or expect, so I just have been profiling against the data I have for my use case.

Of course, Ms Manners may be a a better reflection of your needs, in which case it could be a good benchmark. Micro-benchmarking is full of pitfalls, so your mileage may vary.

-Ryan

[ru]

Hello Ryan,

Have you look at Waltz test? You may be interested of this. Its CLIPS source is in examples folder of rete4frames.

-Ru

понедельник, 12 мая 2014 г., 17:47:44 UTC+4 пользователь ru написал:

New feature: added Java interface. Eclipse project example.

[Ryan Brush]

I'm less familiar with Waltz. It looks like a good way to exercise some worst-case scenarios in the rule engines as well. Like Ms Manners I don't think this is a very representative usage pattern but it could expose problems in extreme cases.

-Ryan