Some context about my question.  I have been reading about, and
experimenting with, expert systems for the past few months.  I bought
from Amazon half a dozen well know books such as "Expert Systems -
Principles and Programming" by Giarratano, "Programming Expert Systems
in OPS5" by Brownston et al., "Building Expert Systems" by Hayes-Roth
et al., and a few more.

Although I have a basic knowledge of computing and AI history, I did
not closely follow recent AI research and applications.  Many early
expert systems books contain a lot of what might now be considered
hype.  But one of the books I bought, Feigenbaum's "The Rise of the
Expert Company", published in 1989, seem to tell several stories of
how this technology improved the bottom line of many companies.

To someone not deeply familiar with AI like me, all this seems to stop
around the AI winter.  So, I always wondered what happened to expert
systems since then.  I got Feigenbaum's book from Amazon at 0.01$ plus
shipping: is this any indication of the fate of expert systems? :)
Are expert systems alive and well?

Paolo
--
Lisp Propulsion Laboratory log - http://www.paoloamoroso.it/log

In financial fraud detection, particularly the problem of finding
credit card transaction fraud, expert systems were an early candidate.
Based on experts' experience, rules were posed which characterized
known fraud and the output on unseen data was used to figure out how
the rules would go wrong.  These systems did as well as the state of
the art at the time which was considerably worse than the state of the
art now.  That state of the art was defined by supervised training
algorithms running against specially defined and somewhat intricate
features of transactions and history.

The interesting and important thing that happened is that the fraud
characteristics evolved over time as, indeed, things in the real world
tend to do.  The rule developers continued to add rules but as they
reached systems with hundreds to thousands of rules, the interactions
between the rules became the dominant determinant of the behavior of
the system.  As a result, these systems became harder and harder to
maintain.  The systems based on supervised learning had no such
difficulties; as new features were proposed or as the world shifted,
they simply retrained their model and performance was maintained and
the level of complexity did not increase.

Ultimately, the rules systems suffered so severely from bit-rot that it
became cheaper to buy the services of a trained system.   One company
in particular, HNC Software, did a good enough job in this area so that
they eventually dominated the market for credit card fraud detection
software and services.  Eventually, CITI was just about the only place
that a system based primarily on large numbers of human designed rules
persisted, and that was largely due to a large staff of modelers who
developed special techniques to allow them to retire old rules and
manage the interactions.

The really important lesson from all of this is not that rules are bad
and neural nets are good.  The HNC system in the end had a rule
post-processor which was critical in assuring that the system could
comply with various regulatory and business requirements.  For example,
there is a limitation period in the US after a first contact about a
possible fraud during which a customer cannot be contacted about the
same issue.  Getting a neural net to understand and comply with
arbitrary and stiff requirements like this is a losing battle.

The features that were inputs into the system were also examples of
very simple rules if you define any hand-coded processing as a rule.
For example, you could define apparent velocity as the ratio of
distance and time between transactions and set a hard or soft threshold
for what constitutes "high" velocity.  You could do the same to define
"large" amounts of cash advances.  These might make very good input
variables for a learning machine, but you could also make the case that
they are just rules with soft outputs.

The difference, then, is really one of emphasis rather than of essence.
 The systems that ultimately failed depended on large numbers of fairly
complex rules with binary outputs that were combined using human
specified rules in the style of expert systems from the 1980's.  They
failed because the combination step quickly became unmaintainable when
they reached the software-engineering break-even where each bug fix
introduces as many new bugs as it removes old bugs.

The systems that succeeded used very simple rules with soft outputs
combined using a learning system that weighted these inputs and their
interactions so as to optimize performance.  The commonly repeated
nostrum about neural nets and similar learning algorithms being
"uninspectable" as compared to traditional rule-based systems was shown
to be completely misguided because supervised learning systems often
provide an ability to quantify the average benefit of any particular
input.  This ability to diagnose which inputs were useful and which
counter-productive combined with the ability to learn new patterns is
what really prevented bit-rot from making these systems progressively
less effective.

At this point, the lessons of history are pretty clear, at least with
respect to fraud detection.  You can't do without rules in such a
problem, at least in the form of feature detectors, but these rules
have to be kept exceedingly simple and must be rigorously evaluated for
an benefit.  You also absolutely must have a very sophisticated way of
combining the output of these rules that allows you to determine net
benefit and blame.  Finally, you really will need some sort of ability
to impose business rules on the outputs.

I should point out that the market has spoken in other ways as well.
According to Amazon, at least, Feigenbaum's book is now literally not
worth the paper it is printed on.  That is a pretty harsh judgement and
I think that he had some interesting things to say at the time, but we
have learned a lot since then.  Peter Norvig's book, on the other hand,
is still selling.  Conclude what you will, but I think that the fact
that he put in chapters on reasoning in the face of uncertainty and on
learning systems is a pretty major component of his success.

   The Handbook of Applied Expert Systems
   CRC Press
   ISBN 0-8493-3106-4

HTH.

Given that, I'll claim that it's partly "once it works, it's not AI,"
partly a name change ("rule-based" rather than "expert"), partly that
(like almost every other technology) rules alone usually aren't enough,
so the tools have gotten more complex and may be described differently.

There are numerous tools available, both commercial and non-commercial.
  Some of them can be traced directly back to the giants of AI summer -
Gensym Corporation makes G2;  Haley Systems has an OPS-descendant; as I
recall, there's still a descendant of ART*Enterprise going strong as
well, mostly in banking and insurance (I think).

A separate family holds CLIPS (originally NASA), JESS, a Java descendant
of CLIPS from Sandia, and LISA for Common Lispers.

In another renaming, rule-based technology has also been recast as
"business rules" - see BPEL, workflow, process control, etc.

You might look at comp.ai.shells (as in expert system shells or
rule-based shells) - while not super active, there is occasional useful
information.

There are also those that would argue that (if you squint a bit) logic
programming is just rules, opening up Prolog, constraints, and all those
wonderful technologies.

Rules are definitely alive. In the past year my company delivered a
custom application that used CLIPS to reason about information in
military messages - the use of rules made it possible for the customer
to tweak things to accommodate new message types without having to do
recompiles or have a full software development environment (Visual
Studio - they're a Microsoft shop) around.

bob bechtel

P.S. "The Rise of the Expert Company," being more a business book than a
technology book, has suffered the fate of most business books - after a
year or so, there's a new fad and a new book to read. I suspect his
Fifth Generation book is equally inexpensive.

Jones, R. M., Laird, J. E., Nielsen, P. E. Coulter, K. J., Kenny, P. G.,
& Koss, F., (1999). Automated intelligent pilots for combat flight
simulation. AI Magazine, 20(1), 27-41.

Paolo
--
Lisp Propulsion Laboratory log - http://www.paoloamoroso.it/log

We market and sell a very comprehensive AI system for real-time
control of mineral processing plants.  We're as busy as we ever have
been.   Lynn

    KnowledgeScape? advanced process control software integrates
fuzzy logic, neural
    networks, genetic algorithms, statistical process control
techniques and client server
    architecture to provide an easy to use, on line, real time, global
optimization solution to
    the process control industry.

Notice how little credit the rules-based heritage gets.  This is pretty
ubiquitous in my experience; the rules part of it all doesn't get you
very far at all, but you have to have it around for the regulatory or
business logic parts where non-compliance is just not allowed.

Another interesting example of this was the recommendation engine that
I created for Musicmatch (now Yahoo! Music).  The first use of the
system was to link web pages and recommend music.  This worked well
with raw statistical AI as the output.  The next use, however, was to
program radio stations.  The statistical relatedness measures could
produce nice listenable radio stations after you tweaked them a bit by
putting in some ad hoc measures to encourage variety, but there was no
way that they would meet the requirements of the compulsory licenses
under the DMCA.

Thus, we hacked on a generic sort of rule engine that augmented the
statistical scoring with large penalties to prevent the random search
from violating the law.

The general principle is that rules alone will get you compliance with
some nasty little details, but the bits that can reasonably be called
"expertise" are best captured by other means.

Is this correct?  Does it apply only to large systems?

If I recall correctly from my readings, classic (i.e. pre-AI winter)
expert systems were considered large when they contained more than a
few thousand rules.  But systems with hundreds of thousand rules have
been mentioned in this thread.

Paolo
--
Lisp Propulsion Laboratory log - http://www.paoloamoroso.it/log

Having worked on systems with thousands of rules in an academic setting
I would run, not walk from any project that tried to field a large
rule-based system in any practical setting.

Take as an example a large financial transaction processing company
that I had a hand in working with once.  They had monthly updates to
their procedures manuals.  They also had a team of 7 "knowledge"
engineers who were charged with reducing this material to a logical
form.  The knowledge team never came close to being up to date.
Generally, logical updates were so late that by the time they became
available, they were old news and had often been superceded by
additional changes.

This was the most successful instance of a traditional AI project that
I have ever encountered personally.  It didn't exactly blow apart.  It
just never had any positive net value and wound up being completely
replaced.  As it turned out, a text retrieval system could give people
access to the information they needed with a minimally complex system.

Contrast this with a system that I built over a decade ago.  The task
was to look at radar returns and determine when birds were about to
land on a cooling pond next to a power plant.  Since the cooling plant
had high levels of dissolved salts and such, the birds have to warned
off of landing on the pond.  The system used standard image processing
techniques, some minor learning systems and about 5 rules for
increasing variety in response.  The same system has been running
successfully without change for the last 11 years.

Anyway, my guiding principle has always been to avoid asking for
trouble.  I avoid building large rule systems if I possibly can.

Our system allows for limitless neural networks to be defined to model
the processes we control.  New vector acquisition, training and
predicting all happen concurrently.  The built in genetic algorithms
are used to answer the general question " given where I am at what set
point changes should I make to the process to improve where I'm going
to be in the near future"?  The results of the neural network
predictions can be used in the rules, by the GA optimizers, trended
etc.  Very flexible and powerful.

Rules are the core I would say.  Regards, Lynn

On Thu, 10 Mar 2005 01:47:29 GMT, Paolo Amoroso <amor...@mclink.it>
wrote:

We, at KnowledgeScape,  primarily work in the mineral processing
industry.  Here our systems can contain hundreds of rules, many neural
network models with GA optimization.  Rules, both crisp and fuzzy
rules certainly are the core however.  

We don't run into any corporate compliance issues that require rules
versus other technologies.  Rules are certainly understandable
however.

Regards, Lynn

On Thu, 10 Mar 2005 01:43:32 GMT, "Ted Dunning"

In fact, the combination of apparent simplicity and actual
impenetrability is very dangerous.

Each of these systems receives upto 2000 new process data points at
least every minute.  Process data comes from sensors, instruments,
actuators , analyzers, real-time images, etc.

Our systems run in the largest minerals plants throughout the world
and are well known to increase the yield of the plant in many cases of
well over $100,000 per day.

Many of our systems go untouched after installation but for many more
we provide quarterly software updates to as well as plant visits to
monitor, turn, tweak and provide additional training.  As an
interesting aside the project in the Philippines took six months to
implement which was extremely fast for the state of the industry at
that time.  Last fall we installed a larger and more complex version
of our system in a plant adjacent to the rain forest of northern
Brazil in a total of two weeks.