The wildest hypothesis yet...
Jorn Barger
site will have discovered, long since, that I've been holding out on
this group, with regard to the full extent of my heresies against
conventional AI thinking...
New comp.ai readers should be warned that I'm a comp.ai *persona
non grata*, shunned for breaking the code of silence, by speaking freely
about my (negative as well as positive) experiences at Roger Schank's
ILS of NWU. Mark Kantrowitz, for example, has refused to include any
pointer to my ftp-site in his comp.ai faq (even while he plunders my
research for his report on RACTER).
But another part of my 'heresy' is to have claimed to have found a
simple partial solution to the story-indexing problem, a simple
'data topology' that should soon allow Roget's Thesaurus to be re-sorted
*in a principled way* for the first time.
I'd had this crazy delusion that the world of a.i. was eager for new
ideas in this area! But I've since been well disabused of that massive
blunder... <;^/
So, for a lot of readers, the following, greater heresy will further
cut my already-bloodied throat...
I realized in 1980 or so that an economical story-thesaurus would have
to be sorted *so as to form, itsel* an odd sort of 'super story
sequence'. The way the brain works, I believe, such a 'fusion' would
just be a *vastly* more efficient way to store any arbitrary list of
story plots. (Think of the 'File' menu on a Macintosh-- you couldn't
dream of putting 'Close Window' *above* 'Open...' could you? Wouldn't
it work better, too, if they put 'Print' between 'Save' and 'Close...'?)
Some years later, I successfully arranged 3000 quotations on romantic
love into an elegant chronological inventory of the progress of love,
from loneliness thru courtship to relationship and bereavement, some
500 gradations of experience in all, pleasant and logical, a sort of
literature-based textbook of this obscure subfield of psychology.
And I have further proposed that one possible global sorting of *all*
storyplots might form a mythic cycle, following an 'everyman' from a
heavenly bliss of gratifications, down via the various sorts of human
'fall', into a hell of suffering, and back to 'heaven' thru effort or
abstinence or creativity.
The astonshing discovery that followed, in the mid 80's, was that James
Joyce's Finnegans Wake, undeniably the most problematic literary work
in history, whose composition followed immediately upon that of the
most admired novel of all time (Ulysses, by near-universal consensus)...
*that FW also* told a story of an everyman's fall and reconstitution,
and was baldly declared by Joyce to be a universal history, even
*mathematical*... but that no one had yet cracked its overall code
(being that it's written in an all-encompassing language almost
entirely invented by JJ!)...
But it was only in 1991 that I gained access to facsimiles of Joyce's
(untranscribed) earliest notes and drafts for FW, and discovered that
their elucidation had barely been begun... largely because no one had
any idea what to look for, among the superabundance of clues... no one
had even a *testable hypothesis* about how Joyce had laid out the overall
groundwork for that book.
I spent the better part of 1992 wrestling this mass of confusion into
a reliable chronological order, and dumbly pondering the significance
of some 10,000 notes like:
B3.13 "the son's life repeats the father's. He does not see it. Make the
reader see it"
B3.14 "What is it? Chorus of females -- Love!"
B3.14 "It wasn't anything he said exactly or did or anything still it was
smthg (love)"
B3.14-15 "ritual forms in art litany (catalogue) curse psalms A + B
rosary mass chanted gospel"
B3.15 "Yr stinking putrid soul"
B3.16 "Trist wounded at Lansdowne Rd"
B3.16 "Is - moustaches of beer"
Gradually, I began to notice that these notes tended to lean on
certain traceable *motifs*-- the umbrella motif, the poetry motif, the
reading motif, the performance motif, the generations motif, etc etc
etc-- and that, when recognized, these identical motifs could always
be traced back to Ulysses, and the surviving Ulysses notes.
Joyce claimed *in Ulysses* to have sorted the world into 18 chapters,
each with its own themes and motifs and style, all reduced by a sort
of transformation-by-metaphor into the most ordinary imaginable
experiences of one Leopold Bloom, over the course of one fairly
ordinary day: June 16, 1904.
Joyce was reported to have had doubts whether he had 'oversystematized'
Ulysses, and critics have seen FW as a near-chaos by comparison...
yet Joyce claimed he could justify every letter of the later book,
that it was indeed "mathematical"... and in fact he spent a week in
1929 explaining the structure to James Stephens in great detail, in
the hopes that Stephens would complete the book if Joyce were to fail,
himself. (In a letter some years later Stephens praised JJ as "a most
subtil man...")
The first three 'vignettes' written by Joyce in 1923 depict the fallen
King Roderick O'Conor, abandoned by his followers, getting as drunk as
he can afford on their leavings, after hosting their free feast; then
the infatuated couple Tristan and Isolde committing to adultery in
their self-projected heaven, Isolde mentally cursing a reduplicated
image of King Roderick, here renamed King Mark, to fit that character
into the T&I mythos. Finally, Joyce depicted Saint Kevin retreating
into sainted purity, only by cutting himself off from all human
relationships.
The first two of these ended up placed at the halfway point of the
finished text, the third ended up very near the end. (FW is
written as a circle, the last sentence unfinished except by
rebeginning the book!)
I'm convinced that Joyce's researches while writing Ulysses led him
to see these three images as extremes in the human life-drama, and
more particularly as *the three fundamental human relationships with
motives*-- suffering want, gratification, and abstemiousness or denial.
But *further than that* I'm still unable to see with any confidence.
Ulysses, I've come to realize, must also be a comparatively unexplored
puzzle-book... just by examining Joyce's use of compound words in
Ulysses, I've spotted over 400 motifs that he seems to have
systematically explored and sorted (and this listing is highly
preliminary, but brilliantly rich, imho):
THINGS
tool: weapon fuel mirror comb razor rope soap toy makeup nail tack
weapon: shield sword rifle bomb cannon
container: crate valise cup bottle bucket dish box envelope sack pocket
instrument: bell flute lute piano voice drum trumpet
clothes: boots trousers shirt shoe button coat skirt dress knickers stockings
accessories: hat watch stick hanky jewel coin note stamp glove glasses
pocket ring
furniture: bed chair table stove desk drawer lamp candle
vehicle: coffin bicycle horse boat carriage train wagon tram trapeze?
plants: tree flower rose grass weed [etc]
food: apple biscuit sweets jam honey milk meat lemon potato bread egg
tea coffee cabbage nut carrot olive orange pea peach salt pepper plum
rice sandwich sausage soup spices sugar wheat mushroom melon
food tools: spoon knife fork cup plate dish pot pan
intoxicant: whiskey beer wine brandy tobacco cigarette snuff coffee tea
opium bhang
medicine: pill ointment poison
bodyparts: head hair eye ear brain mouth teeth tongue nose arm wrist
hand finger blood bone foot toe bottom cunt cock lung chest breast
heart muscle navel neck stomach shoulder waist wings shell
waste: soot ash dirt dust corpse
bodywaste: shit piss snot vomit menses flatus tear
PLACES
home grave heaven hell moon sun stars village city farm park beach ocean
prison river quay street mountain bog [etc]
building: church pub hospital school library shop outhouse warehouse
theater concerthall bank pawnshop morgue courtroom parliament brothel [etc]
room: stairs hall roof wall window door fireplace tunnel maze plaster
chimney bath? [etc]
PERSONS
roles: asleep drunk lusty ill wounded dead ghost angel god pregnant baby
child student bride mother father widow [etc]
occupations: priest whore beggar serf servant waiter milkwoman cook maid
messenger shepherd butcher midwife nurse engineer detective driver
policeman soldier sentry mayor lawyer moneylender landlord druggist [etc]
activities: party meal game court sleep touch kiss fuck wank test race
prize circus opera vow dance sing play recite read write talk yell walk
war weave wedding wash yawn exchange calculate debate tax [etc]
emotions: terror love [etc]
animals: bear bull cat cow deer dog donkey elephant fish horse monkey
oxen pig rabbit rat sheep snail spider tiger toad weasel worm [etc]
birds: canary dove goose hawk hen rooster parrot peacock pigeon raven
sparrow swan wren
MODALITY: letter book card heresy ink coin note music newspaper paint
photo play poem printing telegraph telephone television ticket statue
gravestone wave story secret theory [etc]
symbols: cross alphabet number punctuation name shadow flag [etc]
colors: white black brown grey red orange yellow green blue violet pink
gold rainbow shadow
?: nature night rain thunder weather wind
stuff: air wood water fire metal fog peat coal gas ice stone powder sand
shell hide wool mud
For each of these, I suggest, a pattern should be discernable in
Ulysses, showing how Joyce tried to exhaustively inventory the range
of possible story-complication using each of these motifs, and then
sorted the results across the 18 chapters, as with the motif 'mirror'
in the following instances:
Telemachus:
1.2 Mulligan's shavingkit (symbolizing vanity/ appearances?)
1.17 used as lid of latherbowl
1.121 talks to self in mirror
1.130 signals "tidings" (betraying Ireland to foreign invaders?)
1.135,138 stolen from servant, cracked
1.136 reflects SD's image, critically
1.143 (denies Caliban his image, infuriatingly)
1.243 sea as mirror
Nestor:
2.9 history as "shattered glass"
2.159 mocking mirrors of Averroes and Maimonides diluting (?) Aristotle
Proteus:
3.137 little SD bowed to self in mirror
3.249 shattered glass again
Calypso:
4.288 Milly "my lookingglass from night to morning"
4.293 Milly finds Goodwin's mirror in hat
4.531 Molly uses handglass because mirror is in shadow, rubs it against
"bub" (Here's where LB says "wouldn't pan out somehow" but see Penelope
below)
Lotuseaters:
5.335 sunglasses of missionary?
5.429 whispering gallery? (sound-mirror)
Hades:
6.31 "crazy glasses" in carriage doorframe?
6.76 "Me in his eyes." of lost little Rudy
6.478 "murderer's image in eye of the murdered"
Eolus:
7.21, 206, 683 'reflected' letters and words
Lestrygonians: ??? anyone?
8.297 Farrell stares through heavystringed glass?
Scylla:
9.221 Mrs D's mirror "sheeted" (how Scylla?)
Wandering:
10.644 auctionroom lacquey views self in chalked mirror
10.742 Kernan preens self in hairdresser's window's sloping mirror
10.1027 long John Fanning approaches self in mirror
Sirens:
11.214, 421 mirror behind bar reflects barmaids
11.691 LB imagines MB greeting BB
11.722 barber spoke to Bloom direct though Bloom spoke to barber's
reflection
11.1046 Bronzemaid gazes sideways at mirror conscious of best side of face
Cyclops:
12.349 Dignam from other side views previous life as seen in a glass darkly
(cf "wouldn't pan out"?)
Nausikaa:
13.162 Gerty inspects new dress
13.192 practices crying nicely
13.919 Milly dries hanky on mirror to save ironing (hanky motif too)
13.920 Bloom sees mirror as good place for advertising to women
13.1138 Starting fires with glass
13.1260 Bloom looks into pool
Oxen:
14.1044 Bloom sees past self via mirror in mirror
14.1060 mirror breathed on, vision recedes
14.1160 Lynch's girl checks image in pocketmirror after meeting Conmee
Circe:
15.145 Bloom distorted by convex and concave mirrors
15.2053, 2303 Kitty looks at self in mirror
15.2991 Bloom crossdressed before mirror
15.3821 LB and SD see WS in mirror
15.3907 SD carny barking men to see in mirror "every positions"
15.4058 Morning Hours catch the sun in mocking mirrors (cf Nestor)
Eumeus:
16.486 sailor says glass boggles natives (cf Lotus)
Ithaca:
17.239 SD dislikes "aqueous substances of glass and crystal" (cf gem motif
thruout, esp Circe)
17.516 LB exercises before mirror
17.1183 LB and SD see selves reflected in other's face
17.1342 LB sees reflections in mantlemarriagegifts
Penelope:
18.414 MB uses hanglass for powdering because mirror doesn't give you the
expression (This reminds me of one of my favorite Iris Murdoch images
(from Nuns and Soldiers) of a mean woman's face softening when she looks
unselfconsciously into a mirror.)
18.647 MB sees youngself transformed by love
18.1155 Dr Collins' gilt mirrors
I think very likely, if some of these seem unconnected to their
chapter, the fault is in our limited grasp of Joyce's design... and
that before we can go much further with FW, these riddles need to be
mostly solved, for all 100+ categories in the Joycean 'ontology'
above...
=----------=- ,!. --=----=----=----=----=----=----=----=----=----=----=
Jorn Barger j't Anon-ftp to ftp.mcs.com in mcsnet.users/jorn for:
<:^)^:< K=-=:: -=-> Finnegans Wake, artificial intelligence, Ascii-TV,
.::.:.::.. "=i.: [-' fractal-thicket indexing, semantic-topology theory,
jo...@mcs.com /;:":.\ DecentWrite, MiniTech, nant/nart, flame theory &c!
=----------= ;}' '(, -=----=----=----=----=----=----=----=----=----=----=
Mark Kantrowitz
>ILS of NWU. Mark Kantrowitz, for example, has refused to include any
>pointer to my ftp-site in his comp.ai faq (even while he plunders my
>research for his report on RACTER).
The AI FAQ is mainly concerned with issues of a factual nature. I
received a request from Jorn Barger on 30-AUG-93 to add a pointer to
his ftp site to the FAQ. This FTP location contained at the time Mr.
Barger's "ILS memoirs", "Racter FAQ", and other materials.
I retrieved a copy of Mr. Barger's "ILS memoirs" and found them to be
highly inflamatory and libelous. At best they were opinion, and not on
a topic which would normally be added to the AI FAQ.
With respect to Mr. Barger's Racter FAQ, most of the factual content
was already present in the AI FAQ, having been added on 3-SEP-92 at
the request of J. A. Durieux. Some information from Mr. Barger's
Racter FAQ (e.g., pricing, machines on which it runs) was added to the
AI FAQ after verifying it by a telephone call to the vendor. A few
errors and omissions in Mr. Barger's FAQ were indicated to him at the
time, which he apparently corrected. The rest of Mr. Barger's Racter
FAQ consisted of opinion (albeit less vitriolic than his polemic
against Schank).
It was therefore my feeling that Mr. Barger's ftp site did not merit
an entry in the AI FAQ.
--mark
Jorn Barger
>I retrieved a copy of Mr. Barger's "ILS memoirs" and found them to be
>highly inflamatory and libelous. At best they were opinion, and not on
>It was therefore my feeling that Mr. Barger's ftp site did not merit
>an entry in the AI FAQ.
I will post the lot, and I encourage anyone who disagrees with Mark
to let him know. My feeling is that ILS is *dangerously dishonest*
and it's my *reponsibility* to make that known. Anyway, science without
inflammatory ideas is dead science...
Here's a scratchy first draft of an alternative to Mark's FAQ, that
I hope to re-post regularly, enlarging it and including a more-compact
set of pointers to things I consider most useful.
Comp.ai.imho FAQ
(Version 0, seriously incomplete)
- Why comp.ai.imho?
The reason for this comp.ai.imho ("in my [not particularly ;^] humble
opinion") FAQ is that AI (artificial intelligence) is too important to
leave to the experts! The experts are too concerned with outcompeting
rivals for status and grants, and this has led to a proliferation of
obfuscatory jargon, and the suppression of innovation... *imho*.
When I try to picture the *real* future successes of this field of science,
I'm not thinking merely of more effective business software! Much more
urgently, I'm looking for a profound revolution in human self-
understanding, to be swiftly followed by a parallel revolution in human
self-government... so I've got no patience with academic politics and
grantsmanship!
Admittedly, I don't have any degree in a.i., and there are many areas
where my ignorance is near-total... but I'm perfectly willing to add
any of these to the faq, later, if someone will trouble to convince me
of their usefulness.
- The layout of this faq, and the shape of ai's past
This faq will try to embed a full range of useful net-pointers (as any
faq should) within a *readable* historical narrative of AI's history.
Briefly:
AI's long pre-history begins with divination, then proceeds thru Aristotle
to Leibnitz, Roget, and Polti-- all looking for neat, universal sortings of
the *full range* of human experience. The primary tools of this era were
the abstraction hierarchy and the concept of orthogonal dimensions...
imho.
With the invention of the programmable digital computer, such *neat
systematizations* began to promise a whole new level of payback. One
conceptual hurdle to be crossed, though, was the transition from arithmetic
computation to symbolic computation, most notably via the invention of the
LISP language. LISP's dramatic (imho) history will be explored.
An early goal of AI research was automated translation of natural language.
One line of research has focused on parsing the grammatical (syntactic)
structure of sentences (basically a pattern-matching problem), another on
representing the meanings, or semantics (that bigger problem that dates
back to Aristotle). Emacs and SGML are useful tools for thinking about
these problems. Current grammar-checkers and text-adventure games
adequately represent the state of the art in these areas... imho.
AI is prone to collective 'manias', regarding 'magical' hardware solutions
like parallel processing, LISP machines, neural nets, etc. ***TANSTAAFL***
...imho. It's also prone to being poisoned by ego and greed, because of
the combination of low standards and high stakes... again, imho.
Expert systems are AI's greatest success story, but so far their
construction has normally been a one-off affair-- each one must be built
from scratch by a labor-intensive process of 'knowledge extraction'. The
CYC project is trying to fix this by building a huge, universal expert
system by 'brute force'. From a programmer's point-of-view, though, the
laying out of the data structures is the important thing-- the algorithms
are comparatively trivial. And while CYC is the most ambitious attack yet
on the 'Aristotle problem', it offers as yet few breakthrus on the problem
of representing human mental (and emotional) states... (imho).
The natural way to represent these mental states, and their laws, is in the
form of *stories*. The Aristotle-problem turns out to require a universal
inventory of human stories, following the groundwork laid out by Polti and,
surprisingly, James Joyce. A new data topology that unites the abstraction
hierarchy with the idea of orthogonal dimensions, promises a new direction
for story representation.... imho.
- The prehistory of AI
Strangely enough, the first recorded human attack on the problems of AI
came from the *fortune tellers*. If their systems for generating
predictions had not been capable of generating certain ranges of human
events, then obviously they could never correctly predict them! Beginning
before 1000 BCE, astrologers were already exploring an especially rich
(though arbitrary) system of planetary relationships, trying to map them
onto human experiences. The I Ching, slightly later, explored the 64
precise permutations of a six-bit binary system, as well as the three-bit
half-words they contained. The Kabalah and Tarot offered simpler systems
around 1000 AD, tied more directly to particular human meanings like virtue
and vice.
(I expect I'll turn up some Net resources on these, eventually. There's an
I Ching for Unix somewhere, I'm sure...)
Aristotle made a much more *grounded* assault on the range of human
meanings, around 300 BCE. The middle ages saw Raymond Lull playing
combinatorial games, leading eventually to Leibnitz's (1646 - 1716) dream
of a purely rational world, where all concepts will have been encoded as
mathematical formulae, and philosophical disputes will be met with the cry,
"Let us calculate!"
Various text archives around the Net carry these classics.
See Frances Yates, "The Universal Theater"(?)
Giambattista Vico in his "New Science" (17??) was the first to anticipate a
universal dictionary of concepts, realized in 18?? with Peter Mark Roget's
Thesaurus. The Dewey Decimal System (19??) and Library of Congress
Classification (19??) are two later evolutions, but all of these are
plagued by redundancies and ambiguities. Two net-specific proposals are
the Usenet hierarchy itself, and "Joel's Hierarchical Subject Index"
(JHSI).
When you implement a hierarchy in computer memory, it's cheap and easy to
allow a single element to be 'multiply indexed' at more than one location
in the hierarchy... but even this minor tweak comes only slowly to human
thinking-habits.
Roget's classification, along with his wordlists, can be ftp'd from:
The Usenet system can be examined via:
To access JHSI, use a WWW browser such as NCSA's Mosaic or Lynx. The URL
is: http://www.cen.uiuc.edu/~jj9544/index.html
- LISP and symbolic computation
Programming computers is really hard. Designing algorithms and data
structures requires such a difficult sort of analytic thought that it's a
wonder anyone can do it well at all. The main barrier to AI is the need to
innovate new ways of thinking about programming-- such as object-oriented
design, and declarative code. (Jargon is the greatest enemy of the required
clarity of thought.)
The LISP language was invented by John McCarthy in 1958, to simplify the
handling of arbitrary lists of arbitrary data objects. LISP builds
(almost) everything out of "cons cells", an elegant structure consisting of
two memory locations, and nothing else. These may hold pointers to other
cons cells, or to numbers-- and these numbers may represent quantities, or
*qualities* or concepts.
The basic task of a LISP program is usually to wander along chains of cons
cells, pointer to pointer to pointer, looking for some pattern in the other
half of the cell. LISP represents these chains very elegantly by enclosing
them in parentheses:
(pattern1 pattern2 pattern3 ... )
LISP is cleverly designed to allow a great deal of flexibility in program
modification, and for this reason it is much favored by symbolic AI
researchers. The early successes of expert systems led to a sudden influx
of venture capital into companies developing dedicated LISP hardware,
starting around 1980, a bubble that was to collapse by 1988 as expert-
system development moved on to C, and competitive LISP compilers became
available for general-purpose microcomputers.
GNU Emacs is a fully programmable word processor written in E-LISP, and
makes a fine development platform for many sorts of text-oriented AI
research.
John McCarthy can usually be found on rec.arts.books.
A free LISP for DOS machines is...
Gnu Emacs is available...
- Natural language translation, natural language processing
The naive view of language translation naturally expects that a *dictionary
of word-substitutions* ought to take us the best part of the way to
comprehensible translated text. In fact, though, any brief test of this
idea shows it to be massively undermined by two difficulties: 1) idiomatic
phrases that don't work in literal translation, and 2) the large percentage
of words that allow several entirely different meanings. AI researchers
pursued this line between 1956 and 1966, when the ALPAC report killed all
government funding for translation research.
A more sophisticated approach focuses on 'parsing' the syntactic structure
of each sentence, as a way of disambiguating shades of meaning by
eliminating (at least) those implying an impossible part-of-speech. This
school of thought continues to try to add more and more complex algorithms
for finding more and more subtle syntactic patterns... but less work has
been done trying to collect any huge database of the patterns themselves,
probably because the latter task de-emphasises the element of 'programmer
macho' (a factor that steers research directions much more than it ought!).
The minimalist parsers familiar from Infocom-style text adventure games
offer about as much grammatical sophistication as one may expect from
algorithms alone. (The poverty of parsing can be easily seen by exploring
the pathetic grammar-checkers offered, eg, with Microsoft Word!) Several
toolkits for adventure-game development are available on the Net, allowing
one to experiment with parsers and their limitations.
Emacs, with its facility for "grepping" complex patterns, is a useful tool
for parsing experiments. Griswold's text languages, SNOBOL and Icon, are
also useful. Another direction is offered by SGML, TEI, and HTML, three
related projects involving the addition of a layer of 'markup' within text
documents. The Text Encoding Initiative (TEI) has been working out
detailed conventions for marking up various classes of literary text, using
the Structured Generalized Markup Language (SGML). The HyTime Markup
Language (HTML) is an extension of SGML to support hypertext linkages
within and between documents, and has gained a great success via the World
Wide Web project.
Speech-understanding research was funded by DARPA until 1976, when it
became clear that no quick solution was emerging. The state of the art is
still limited-vocabulary-spoken-by-a-single-user, and probably can't do
better until improved language-understanding allows the software to *predict*
which words are likeliest. Handwriting recognition has done a little
better, since the time-path of the stylus can now be tracked, but again the
big improvements depend on word-prediction.
Natural language *generation* is largely the domain of ELIZA (1966,
Weizenbaum and Colby) and RACTER (198?, Etter and Chamberlain). The annual
Loebner competition has been won the last two years by Joe Weintraub ().
The remarkable thing about these efforts is their uncanny successes at
mimicking intelligence, despite absurdly primitive knowledgebases.
A complex parser project called SNEPS is ongoing at the University of
Buffalo....
The text-adventure archives are at:
TADS is the most popular platform.
For emacs, see above.
Icon is available at:
comp.text.sgml
TEI-L
TEI-L archives
comp.infosystems.www
ELIZA is avaiable at the AI Attic...
For a FAQ on RACTER, try ftp.mcs.com in /mcsnet.users/jorn/racterfaq.ai
- Hardware (& other) manias: Parallelism, neural nets, LISP machines, etc.
Speed increases in hardware are great, but they're not AI. Parallelism and
neural nets allow some new implementation strategies, but don't begin to
solve the "Aristotle problem".
Neural nets timeline:
1959: Frank Rosenblatt introduces Perceptron
1969: Minsky & Papert's "Perceptrons" kills funding for neural net research
1975: Cooper & Erlbaum found Nestor to develop neural net technology
1982: John Hopfield resuscitates neural nets
These hardware manias are one form of a more general problem plaguing AI,
caused by the unfortunate combination of very high stakes (especially DARPA
grant money), and a very immature domain in which bold bluffing can take
you far...
- Expert systems, Prolog, and CYC
The history of AI has shown a gratifying series of successes in the realm
of expert systems, all thru the 60s, 70s, and 80s.
The idea of expert systems is that expertise involves *logical* thinking,
and can be modelled by enumerating lists of logical propositions and
performing logical transformations upon them. However, contemplation of
the data structures and algorithms involved reveals that all the expertise
is in the data structures, and the 'logic' is entirely trivial. So once
again, the Aristotle-problem is the real hurdle to expert-system
development-- how do you concisely represent knowledge in a computer
memory?
The Japanese Fifth Generation Project between 1982 and 1992 failed in its
goals largely because it focused on logic, eg by choosing Prolog over LISP
as its implementation language (a choice that may have been simple anti-
Americanism, since Prolog was developed by the French).
Doug Lenat's CYC Project, begun in 1984, uses logic, too, but emphasizes
the Aristotle-problem, trying to enumerate tens of millions of common-sense
facts that will 'add up to' logical intelligence. Lenat's goal is a system
that can understand and speak ordinary language, and detect violations of
common sense as readily as humans can.
Lenat has revised his estimate of the total number of 'rules' required for
this upward by a factor of ten, and extended the time needed by another ten
years. It bothers me a lot that the sort of thing being added apparently
includes rules like, "A creature with two arms probably has two legs."
This seems out-of-control to me...
CYC's ontology includes abstractions like: ...
CYC's data objects include slots for: ...
Lenat, at some point, asked John McCarthy to take a shot at enumerating the
laws of human emotion. I don't know where this led, but McCarthy could
have done worse than to start with Andrew Ortony's dimensional analysis of
emotions. Ortony's classes of emotion: ...
- Understanding human behavior via *stories*
The basic data structures in AI might be the rule, the frame, and the
script... but from the programmer's point-of-view these are really
interchangeable-- just arrangements of pointers in memory-space. So the
real challenge is to find an interpretation for these pointers that best
matches the way human thinking works.
Case-based reasoning tries to identify nodes with 'cases'-- often actual
realworld events that could (theoretically) be described to any infinite
level of detail without being exhausted. A case is thus approximately
equal to a *story*. But this sort of thinking, again, comes very hard to
conventional computer hackers. If a story is infinitely complex, for
example, how can it be indexed as *more similar* to some stories than to
others?
In 1900, a French literary critic named Georges Polti published an analysis
of literary plots, entitled "The 36 Dramatic Situations." The 36, which
Polti also subdivided further (citing particular plays and novels that
embodied each variant) were: ...
A simple analysis that Polti included for each was an enumeration of the
basic 'elements' needed for the plot, eg. for Pursuit: ...
Folklorists Vladimir Proppp and Stith Thompson offered alternate approaches
to the problem, but Polti's remains the most useful for AI researchers.
Abelson et al introduced the concept of scripts in "Scripts, Plans, Goals,
and Behavior" in 1976. That book also recapped a primitive vocabulary
called Conceptual Dependency (CD) Notation, consisting of the following
verbs: ...
Later work in this Abelsonian school proposed a clumsy structure called
MOPS, then later a mulligan-stew called the Universal Indexing Frame.
...
- Bibliography, etc.
The following timeline is based on "The Brain Makers" by H.P. Newquist:
1953: Shannon gives Minsky and McCarthy summer jobs at Bell Labs
1956: Rockefeller funds M&M's AI conference at Dartmouth
1956: CIA funds GAT machine-translation project
1958: McCarthy creates first LISP
1959: M&M establish MIT AI Lab
1959: Frank Rosenblatt introduces Perceptron
1962: McCarthy moves to Stanford, creates Stanford AI Lab in '63
1963: ARPA gives $2 million grant to MIT AI Lab
1965: Feigenbaum takes over SAIL; Noftsker takes over MIT AI Lab
1965: Feigenbaum and Lederberg begin DENDRAL expert system project
1966: Weizenbaum and Colby create ELIZA
1966: Donald Michie founds Edinburgh AI lab
1966: ALPAC report kills funding for machine translation
1967: Greenblatt's MacHack defeats Hubert Deyfus at chess
1969: Minsky & Papert's "Perceptrons" kills funding for neural net research
1969: Kubrick's "2001" introduces AI to mass audience
1970: Terry Winograd's SHRDLU, minor NLP success
1970: Colmerauer creates PROLOG
1972: DARPA cancels funding for robotics at Stanford (Shakey)
1973: Lighthill report kills AI funding in UK
1973: LOGO funding scandal: Minsky & Papert turn MIT lab over to Winston
1974: Edward Shortliffe's thesis on MYCIN
1975: Larry Harris found Artificial Intelligence Corp. to sell NLP
1975: Cooper & Erlbaum found Nestor to develop neural net technology
1976: DARPA cancels funding for speech understanding research
1976: Greenblatt creates first LISP machine, "CONS"
1978: Xerox steals BBN's hackers to build LISP machine
1978: SRI's PROSPECTOR discovers molybdenum vein
1979: Schank founds Cognitive Systems, insists on BMW "for safety"
1980: First AAAI conference at Stanford
1980: Prototype of Dipmeter Advisor
1980: McDermott's XCON for configuring VAX systems
1980: Greenblatt & Jacobson found LMI; Noftsker starts Symbolics
1981: Teknowledge founded by Feigenbaum.
1981: Kazuhiro Fuchi announces Japanese Fifth Generation Project
1982: John Hopfield resuscitates neural nets
1983: Feigenbaum's 5th Generation book sounds alarm, guest on Merv Griffin
1983: Symbolics' new-machine hype drives LMI into arms of TI
1983: MCC consortium formed under Bobby Ray Inman
1983: Symbolics saved by Howard Cannon's hacking protypes of 3600
1983: IntelliGenetics markets KEE
1983: DARPA's Stategic Computing Initiative commits $600 million over 5 yrs
1984: Gold Hill creates Golden Common LISP
1984: Bruce Gras creates K:Base for Shearson, then leaves it orphaned
1984: Noftsker (briefly) replaced by Kulp at Symbolics
1984: Austin AAAI conference launches AI into financial spotlight
1984: AI firms find sales and marketing people seriously clueless
1984: European Community starts ESPRIT program
1984: Doug Lenat begins CYC project at MCC
1984: Perez & Rapaport start Neuron Data, selling Nexpert for the Mac
1984: Phil Cooper founds Palladian, spends money like no one else
1984: TI wins MIT contract for LISP machines away from Symbolics
1984: GM puts $4 million into Teknowledge
1984-86: Corporations invest some $50 million in AI startups
1985: Xerox wins $20 million contract for LISP machines, later cancelled
1985: Palladian releases Financial Adviser, despite tons of bugs
1985: UCLA IJCAI conference wallows in marketing excesses
1985: GM and Campbell's Soup find expert systems don't need LISP machines
1985: Teknowledge abandons LISP and PROLOG for C
1986: Teknowledge goes public, amid wild optimism
1986: Borland offers Turbo PROLOG for $99
1986: Paperback Software offers VP Expert for $99
1986: Thinking Machines, Inc introduces Connection Machine
1986: Neural net startup companies appear
1986: IBM enters AI fray at AAAI, with a LISP, a PROLOG, and an ES shell
1986: PICON ES group leaves LMI and starts Gensym
1987: "AI Winter" sets in
1987: Bottom drops out of LISP-machine market due to saturation
1987: LMI files for bankruptcy, other bankruptcies and layoffs follow
1987: Palladian's software doesn't work, Phil Cooper resigns
1987: George Lucas's Pixar signs deal with Symbolics
1988: Symbolics fires Noftsker and Sear, replacement team flounders
1988: TI announces microExplorer- a Mac with a LISP chip
1988: Minneapolis AAAI is the last hurrah for the party atmosphere
1988: AI revenues peak at $1 billion
1988: Beleaguered Teknowledge merges with American Cimflex, to no avail
1988: The 386 chip brings PC speeds into competition with LISP machines
1988: Object-oriented languages are 'in'
1988: Schank forced to resign from Yale and Cognitive Systems
1989: Palladian ceases operation
1989: Coral sells out to Apple
1989: Schank reappears at Northwestern, funded by Andersen Consulting
1990: MacArthur Foundation gives Richard Stallman $240,000 genius grant
1990: AICorp goes public
1990: Gold Hill closes doors
1990: AAAI in Boston resembles a wake (as did 1989 AAAI in Detroit)
1991: KnowledgeWare reneges on offer to buy IntelliCorp
1992: Japanese Fifth Generation Project ends with a whimper
1992: Japanese Real World Computing Project begins with big money
1993: Symbolics files for bankruptcy
1993: Kurzweil AI goes public
1985-present: many other expert-systems success stories
Jorn Barger
Who's munging the hacker ethic?
by Jorn Barger
I see a frequent pattern on Usenet, where someone posts a longish, thoughtful
piece, and one or two small parts of it turn into a long, meandering threads,
while the rest gets ignored, or quickly forgotten in the flurry. And more
often than not, unhappily, the thread-seized-upon is not the one I would have
liked to see talked thru.
So (in part) with ps...@aurora.alaska.edu, back on May 1, with "Where did the
hacker ethic go?" He quotes Dr Dobbs (with some anguish), and then comments:
> [Dr D:] "There's nothing wrong with Microsoft setting the standards for the
> computer industry. The industry NEEDS an IBM for the 90's."
> [pstlb:] Where has the hacker ethic gone? ...the hacker ethic, which tells
> us to value the free distribution of information and yield to the hands-
> on imperative?
> [...] Why is it that, if a person like myself posts messages to
> Usenet on how to get into the little nooks, crannies, and idiosyncrasies
> of a computer system, they are not given any useful information by those
> who know, just a badmouthing (or are completely ignored)?
It's this subtopic that I would have liked to see spun out farther. Some
sort of hacker ethic *ought* to be visible on all the newsgroups in the sci
and comp hierarchies, turning them into lively ongoing seminars *on every
topic under their charters*. This is where Usenet promises miracles of
efficiency. But it's not consistently happening. Why?
I've complained before that the norm on comp.ai is:
A-asks-question, then B-replies-with-article-references.
Other variants include:
A-asks-naiveseeming-question, then everyone-ignores-A; or
A-asks-naiveseeming-question, then B-attacks-A-for-exposing-naivete.
It seems quite clear to me that the most effective way to improve one's
understanding, in *any* domain, is to try and explain its most difficult
ideas in the clearest, simplest, most intuitive language possible, *at every
opportunity*. And this corresponds perfectly to the hacker's eternal search
for crisper code-- if you relax your standards so far as to allow a
programmer 'status-points' for obscure code, they'll never be motivated to
make the huge effort necessary to streamline that code...
So I think anyone who wants to nurture the ailing hacker ethic, ought to make
a serious effort to *trickle down* some of their expertise, on a regular
basis, in their home newsgroups. (One measure of a newgroup's health, then,
is the 'meatiness' of its FAQ-- is it packed with information, or just with
pointers and meta-information?) And the evidence of many healthy newsgroups
suggests that it's not really that nobody lives by the hacker ethic anymore--
there's still plenty of willing souls, if the zeitgeist were sympathetic.
But there's a seriously malign force that resists this populism, that is
profoundly *threatened* by it, and that's the *power-status-territoriality*
motive. I see this taking two main forms:
1) more mature sciences tend to become rigidified around their 'known'
principles, and actively squelch innovative thought as *failing to conform to
what's known*, while
2) 'immature' sciences *proliferate jargon* as, effectively, a union-card,
and may attack even those who understand the ideas, if they haven't been
properly initiated into the jargon-club.
One drastic consequence of this is that the hypothesis-generation phase in
many sciences seems to have retreated underground, and those sciences must be
paying dearly for it-- like an economy where avarice leads to far more saving
than investing, and the GNP plummets. Usenet newsgroups ought to serve, in
part, as giant, *protected* brainstorming sessions, where the agreed goal is
to maximize cross-fertilization. The interests of science are not served by
the attitude that *any* branch of science is a closed book, viewing opposing
ideas as dangerous threats to the status quo!!!
Cross-fertilization of distant domains is one of the richest sources of new
insights, but this is exactly what's most penalized by jargon-barriers. How
many thinkers have the luxury to earn union-cards in a dozen different
fields? Truly, *none*. (Another aspect of the hacker ethic, that the
Dictionary definition somewhat underemphasizes, is the incredible
productivity-gains that come with *enthusiasm*. Someone who says they're too
busy to hack anymore has *lost it*, and we all pay a price for such
diminished productivity.)
Businessmen playing it safe is part of the problem, but AI is still too
uncommercial for this to be of major relevance. In AI, a much greater
villain is the publish-or-perish or get-grants-or-perish mentality. It
appears to me that AI is so immature it's still operating largely on a basis
of political allegiances and deference-to-allies. So the normal career path
is to stake out a niche with the approval of one's clique, and generate
clouds of jargon until tenure-time, with journal-acceptance being mainly a
function of staying well-connected within your tribe.
This game must necessarily find the populist strategy seriously threatening,
because one's stable territorial niche could be cannibalized in an instant,
and with it all one's imagined security re papers and grantmoney, if a
simplification reveals your niche to be redundant or illusory.
And in AI, at least as I experienced it at ILS, this 'phobic imperative'
reaches the point where everyone's bread is entirely buttered on a side that
*assumes* no indexing breakthrus will be possible within our foreseeable
lifetimes, because ***that would just be too inconvenient***...
So research projects are emasculated upfront by a pre-Columbian fear of
"falling off the representational edge". And anyone who shows even a
perfectly ordinary scientific optimism, a simple faith in the universe's
comprehensibility, becomes the victim of every sort of vindictive 'esteem
voodoo'...
And because of the huge sums of money at stake in AI, the esteem-voodoo
sometimes verges on the sociopathic, as the many who've crossed swords with
Roger Schank will *privately* confess...
=----------=- ,!. --=----=----=----=----=----=----=----=----=----=----=
Jorn Barger j't Anon-ftp to genesis.mcs.com in mcsnet.users/jorn for:
Jorn Barger
at Roger Schank's Institute for the Learning Sciences of Northwestern
University, when Schank fired me at the end of 1992, *for asking for
a forum in which to present an original idea to the group*.
It was posted first, one chapter at a time, to rec.arts.int-fiction,
in late January 1993. It was reposted in approximately this form to
comp.ai in early February. My intention in writing it was to subvert
any attempt Schank might make to blacklist me, by explaining my side
of the story in a lively, open and *honest* way.
Although I was trying to aim my explanations for interactive fiction
amateurs, I think in many places the writing is still way too complex.
Sorry.]
====================================================================
"Was: Barger@ILS"
(memoirs of an a.i. hacker)
by Jorn Barger
====================================================================
Chapter 1: CD Notation
I didn't really hear about Roger Schank until 1987, from a student of his,
Kris Hammond, who had just been hired out of Yale by the CS department of
the University of Chicago. I phoned Kris out of the blue after hearing
him on the radio talking about his planner "CHEF", wanting to ask him
about planning within videogames. He referred me to Abelson and Schank's
"Scripts, Plans, Goals and Understanding" (locally pronounced spuh-GOO).
It's definitely my personal favorite among Roger's books, because it
wrestles unapologetically with the absurdly difficult question of whether
a small finite set of symbols can represent the full breadth of human
experience.
I was dreaming of a videogame that went beyond the take-drop-use-fight
cliches of adventure games. Schank's "CD notation" offers a set of verbs
that may be boiled down to:
give-take-make-move thing
give-take-make idea
insert thing into thing (ingest, but also enter)
extract thing from thing (expel, or exit)
These are inarguably *general*, but somewhere in the generalization
process all the interesting story content got bleached out: an 'insert'
into a body may be food or drugs or poison or penis or semen or scalpel or
Jarvik heart or shiv or bullet or midget-sub or gerbil... and even *ideas*
might be seen as insertions into the head...!
So in '87 Hammond turned me on to A&S's SPG&U. And I was delighted,
because everything I'd been sampling on the bookstore shelves, AI-wise,
seemed just depressingly unconnected to my practical goals. One tiny
exception, in an anthology on 'the Frame Problem', was a synopsis of the
"histories approach" to that problem: in modelling the world, you can
reduce the amount of background detail you have to keep track of, if you
can analyse out a finite set of *qualitatively distinct life-histories*
that the system can embody. Eg, a charged particle aimed towards a small
target may be: a) refracted, b) reflected, c) absorbed, or d)
transmitted-without-change. Each of these stories has an infinite range
of mathematically distinct instances, which might be described to any
ridiculous degree of detail/precision, but the most useful, practical,
tractable, compact summary of any particle-interaction is just to classify
it, in two bits, as one of these four 'histories'.
So who's compiling the catalog of *human* histories? What's it look like,
already? For interactive fiction, you want to know, eg, when an elf meets
an orc, what different ways might the meeting go? Fight, communicate,
gift, theft, etc etc etc... but who knows what shape an exhaustive
inventory of such histories, broad enough to support true interactive
*literature*, will assume?
As regards *planning*, I got the sense from Hammond that not a whole lot
of progress had been made since the 1960's game-tree-exhaustive-
combinatorial-search approaches. Hammond called his CHEF a 'case-based'
planner, effectively a *histories* approach to planning: pick an old plan
from the set of known plans and twiddle its variables until they fit the
current task. But he admitted that CHEF's successes were *real* limited.
And then after a couple of visits Kris got too busy to go further, and I
wandered off on my own path (compiling a histories-based analysis of
romantic love, actually!) until in 1989 I heard that Schank was moving
from Yale to Northwestern, and mailed him my application for employment...
Chapter 2: Ortony on Emotions
Very likely if I hadn't gotten in on the ground floor of ILS the way I
did, months before it officially existed, my job-application would have
disappeared in the first cut, because I have no degree, and all my
experience had been in videogame conversions. In their first months ILS
was doing a lot of hiring, with not a lot of advance public notice. So I
was in the right place at the right time... but I have to give Schank
credit for taking a chance on me.
In my approach letter I was totally outspoken about my sense of Schank's
work as 'on target' in a unique way... which surely helped my chances!
What I came to understand in those first weeks was that Schank's "scruffy
school" of AI saw themselves as a fairly lonely island of story-content-
savvy realism in a desolate sea of "straight", math-and-logic-oriented AI.
(I wonder now if there aren't other factors contributing to that
communications gap. Be aware that I see myself as a post-Schank Schankian
-- I consider that Schank himself is Schankian AI's worst enemy! I'm even
inclined to call it *Abelsonian* AI, after that much more *honorable*
man...)
At that time (Sept 1989), ILS had about 50 people-- faculty, grads,
programmers, and admin. Schank's PR-image for ILS has been that they are
"trying to fix the schools" by building educational software that better
fits the natural ways students learn. Most of ILS's financing comes from
corporate sponsors like Arthur Andersen accounting, who spend such huge
sums on training their employees that they can afford to risk a few
million more on fairly basic research.
The first encouraging connection I made at ILS was with Andrew Ortony,
who'd just published (with Clore and Collins) "The Cognitive Structure of
Emotions" (about $12 paper from Cambridge U.P.). I love typologies, and
the emotions-typology in this book is certainly the only one I've ever
seen that shows real, careful analytic thinking, identifying several
distinctly 'orthogonal' dimensions. Ortony et al. (aka OCC) sort emotions
into three superclasses-- event-oriented, person-oriented, and
thing-oriented:
Emotions about things: liking, disliking.
Emotions about persons: approving, disapproving.
about self: pride, shame.
about others: admiration, reproach.
Emotions about events for self: pleasing, displeasing.
for other: gloating, pity, resentment, happy-for.
about events in the future: hope, fear.
realized (positive): satisfaction, relief.
realized (negative): disappointment, fears-confirmed.
(This last is a nice instance of a 'new' emotion predicted by theory, like
the positron in particle physics.)
Emotions about another person's role in events: gratitude, anger.
about self's role: 'gratification', remorse.
We were conscious of the narrow gap between this theory, and (in
particular) Chris Crawford's "Trust and Betrayal: the legacy of Siboot",
built around a little society of creatures who approve and disapprove of
each other's actions. (Siboot was also held up around ILS as an enviable
implementation of the inverse-parser concept: you can build sentences in
that game from elegant iconic menus. Crawford is selling the source for
$150 but I haven't seen it yet. You can get his address from the r.a.i-f
FAQ.) Ortony's student Clark Elliott has since implemented a microworld
of hot-tempered taxidrivers using this analysis. You can order his thesis
as an ILS tech report-- see below.
So here, clearly, was one little piece towards the 'inventory of human
histories' that I was after. Ortony wanted to follow up this 'cognitive'
theory of emotions with a look at the 'affective' side: given that what
emotions we *feel* depends on such factors as approval and anticipation,
was a similar analysis possible of *how one acts* in consequence of each
emotion? We were looking at categories like what sounds and movements
you'd tend to make, where your thoughts would be directed, etc. But at
this point we drifted apart, and I'm not sure where that work stands now.
I don't think Ortony has taken the step of asking, for each emotional
category, what are the usual 'human histories' it plays a role in, which
is where I'd like to see him go.
Andrew sees himself as a psychologist, which normally sets off danger
alarms for me-- one alternative way of posing the 'inventory of human
histories' question, it seems to me, is to ask: what will be the
section-headings in the *ideal* psychology text of the future? I consider
that the current cluster of paradigms that pass for 'scientific'
psychology and so dominate the psych texts, are hopelessly bogged down in
jargon and speciously imitative 'scientistic' methodology, and anybody who
tries to conform to those standards is just wasting good brainpower. { :^)
I expect to see that whole realm cannibalized from without by
survival-of-the-fittest among interactive fictions-- any other sort of
'lab-work' in psychology is hopelessly premature. You can't *measure*
behavior if you don't yet have a model of its dimensions!
(Imagine an IF-development environment sophisticated enough that you
could, eg, feed in the ethical codes of every sort of human philosophy or
religion, letting you explore the stories that result when they interact!
Now *that* I'd be proud to call psychology!)
Chapter 3: LISP
In our first weeks at ILS, the new hires all got a short'n'sweet overview
of Lisp programming from Chris Riesbeck, and a nice shiny Macintosh each,
with Macintosh Allegro Common Lisp (MACL) to start experimenting on.
Anyone who gripes about the profligate way Lisp uses parentheses is
completely missing the point. Parentheses are just the simplest possible
way of depicting tree-structures. For instance, the tree:
A
B
1
a
b
2
C
is topologically equivalent to: ((A)(((a)(b))(2))(C)). Lisp requires
exactly as many parentheses as it needs to define such trees, and no more.
(G. Spencer-Brown's "Laws of Form" (1969, out-of-print-- an
underappreciated masterpiece) formalizes this level of abstraction very
elegantly, in the domain of Boolean algebra.)
Using this uniform notation system for both data and program, as Lisp
does, allows one to decompose more-complex structures into simpler ones,
and to take advantage of structural similarities between units of
different overall complexity.
MACL is a big comfy chair of a programming environment, very quick for
prototyping, very easy to maintain and modify if used wisely, but pretty
much of a bytes-and-hertz hog. (SIMMS and speeds being what they are,
though, these days, it's quite an excellent choice for lots of sorts of
exploratory development.)
The latest version of MACL uses the CLOS (CEE-loss) object system. I
think it might not be too radical to divide the history-to-date of AI into
two periods, the first characterized by exploration of the concept of
abstraction hierarchy (c500 BC to c1980), the second by "object-oriented
programming," where the program's vocabulary of 'verbs' is distributed
across an abstraction hierarchy of types (and their instances), with
more-specialized nodes inheriting verbs from the generalizations above
them. Perhaps the next era will arrive via a neat solution to the CLOS
meta-object-protocol problem, analysing the most-elegant internal
mechanics for connecting types, instances, and code-methods.
I'm chasing after the idea that we might view code-segments as *the
histories (or stories) of their parameter/argument/variables*, so that the
"+" story is one of the usual, useful stories you'd want to tell about two
integers, and "show-view" is ditto for a data-object and a user, and
"edit-hierarchy" ditto for a programmer and a knowledgebase. In MACL's
CLOS, all the interface objects like windows and menus end up in the same
overarching hierarchy with all the data objects one chooses to represent
(eg persons, places, things), so using the story-history metaphor for both
looks like potentially a neat 'win'.
Here's Doug Lenat on Lisp vs Prolog (this is from a book commissioned by
Texas Instruments and distributed by Radio Shack, a combo about as tasty
as a 9-volt battery shorted across the tongue ;^)
Q: How does Prolog differ from Lisp?
Lenat: There has been a constant dream in AI, by a large fraction of
people in the field for 25 years, of the form: there really ought to be
some way of formalizing and axiomatizing human thought and reason. But
time and time again, all attempts at axiomatizing things have led people
to trivialize them, to the point where they no longer apply to what they
were originally modelled after in the real world.
There are a lot of people in the field who want to be *sure*, who want
to believe that they can get absolutely precise, logically guaranteeable
models that are close to what is going on in the real world. If you
believe that, then the kinds of operations you want as primitives are
*logical* operations, those involved simply in *theorem proving*. Those
are the sort of operations that are present in Prolog.
Q: Why is Prolog used more in Europe than in the U.S.?
Lenat: In most European countries [and elsewhere- jb], you have very rigid
hierarchies of 'ancient' professors, and then younger professors, and then
research associates-- and then it filters down about seven levels to the
people actually writing the programs. It is the people at the top who
decide what research is going to get done, not the people at the bottom
who have experience with what is actually happening.
The people at the top-- who [may] want to believe in a nice, simple,
mathematical, axiomatizable universe-- basically determine the kind of
research that is going to get done. The experiences that would lead them
to change their minds are simply not occurring to them, they are occurring
to the people at the bottom, who have no say.
Q: Is Prolog used in the Japanese 5th Generation Project for the same
reason?
Lenat: In Japan, they use Prolog mainly because it is not an American
language; it adds to their national spirit and pride to be removed from
what is going on in America. But if you look real hard, what the Japanese
have done is to build Lisp-like functions on top of Prolog so that by now
it is hard to tell what language they are using. They would have probably
been about two years ahead if they had used Lisp to start with instead of
Prolog.
from "Understanding Artificial Intelligence" by H.C. Mishkoff, 1986.
[Although I put down this book, if you don't judge it by its cover its
signal-to-noise (S/N) ratio is totally admirable, and my stickerprice
shows an unbelievable $3.95 in 1986 dollars. If this were, say, a British
import, it would be way cool, but as it is I'm always having to apologize
for giving it shelfspace.]
[Someone on r.a.i-f took issue with Lenat's attack on Prolog, and thought
he may have outgrown that opinion. Anybody know? It seems clear to me
that you can rewrite Prolog in Lisp a lot easier than vice-versa...]
-------------------------------------------------------------------
Bonus riddle for British-style-crossword fans (answer a proper name):
1 Down (5 letters): Contrary pride at heart of abrupt 'railroad'.
-------------------------------------------------------------------
Chapter 4: Case Based Reasoning
[This chapter, and #6, are fairly technical. I will be proposing some
simpler ways of looking at this material in later chapters, but at this
point I'm trying to retrace my steps at ILS.]
If you enumerate all the interesting categories of 'human histories' on
indexcards, all the stories of emotion, every plausible configuration of
events you'll ever need in an interactive fiction (elf meets orc, villain
plots counterattack, boy kisses girl), and lay them out on the floor (of a
*gym*, figure... it's gonna be a lot of cards!), so that as far as
possible the most similar ones are closest together... what's the overall
pattern?
If you then stretch lengths of string from each card to all its most
similar neighbors, and also to some distant neighbors that are maybe very
different, but still similar in some important ways (analogous,
contradictory-- all the ways our real, human brains seems to link
stories)... is there a simple pattern to the crosslinks?
This, in fact, is the central problem of CBR, one Schank's been pursuing
his whole career.
Long before 1989, it seems, the Yale school's focus of interest had
shifted from SPG&U to Schank's next book "Dynamic Memory". DynaMem is
credited with opening up the whole field of CBR, introducing the model of
an indexed network of storylike cases. (The 'dynamic' part-- creating a
memory that revises itself-- is still a dream. Just creating a static
case-based memory structure turns out to be a plenty big problem.)
DynaMem suggested three sorts of longdistance link between indexcard-
stories (which stories it lumpily termed Memory Organization Packets or
MOPs):
1) Story X1 is one *scene* within more-complex story Y1;
2) Story X2 is just like the more-obvious story Y2, but with a different,
unexpected ending (an "expectation failure"); and
3) Story X3 is a simplified version of story Y3.
#1 seems totally useful for IF-- it will always be most economical to
analyse out all elements shared by two or more stories, and store them
separately.
#2 was suggested by real-life observation of the sorts of stories that
we're naturally *reminded of* in the course of the day-- we'll always
remember the story of the time when things went haywire, not least so we
can make sure *that* doesn't happen again! A planning algorithm in IF
must similarly consider all the ways the planned story could go wrong, and
even if it's not case-based, it will still have to use something
*equivalent* to links like these.
#3 is the abst-spec link of traditional abstraction hierarchies, but its
application to *stories* is not at all simple. One definition of
simplification might be to remove 'scenes' from a story, one by one, to
create more general stories, so that Y3 = X3 + X1. (Story X3 is boy-talks-
to-girl, Scene X1 is girl-winks-at-boy, Story Y3 is girl-winks-at-boy-
who's-talking-to-her)
DynaMem also speculated about a possibly-perpendicular hierarchy of
"Thematic Organization Packets" that account for occasions when a story in
one domain reminds us metaphorically of a story from a totally different
domain-- her wink was like a *door swinging open*.... So stories X and Y,
from different domains, may both point to an indexcard that presents a
common abstraction of their themes: in this case, *access-level-
increased*, where Door = Heart's emotional locks. (We will look more
closely at these TOPs in a future chapter.)
The first case-based domain I was assigned to turned out to be
meteorology, *weather*. We had (or thought we had) some money from the
Navy to write training software for their weather-school. We were just
going to make a modest casebase of weather stories, from balmy summer
afternoon to hurricane tsunami. Students would look at a weather map and
abstract out its features (by hand), and the tutor would retrieve closely
matching cases.
Weather stories can be expressed pretty simply in terms of temperature
trend, wind trend, cloudcover trend, precipitation trend, etc. Each of
these 'slots' needs only a handful of possible fillers (the fillers for
the slot TempTrend might be rising, falling and steady), and a way to
search for the best match among the known cases. But before we really got
started on the retrieval problem, the funding dried up... though in
retrospect I can ask myself, how could we have built the network of
weather histories as a CBR *storybase*?
The concept of 'scene' is easily generalized to the much-mushier
'feature', so 'cloudy' might be a scene within low-black-cumulonimbus-
followed-by-thunder-and-torrential-rain. It would be quite natural to
present the student with a weather situation *along with the various ways
it might go next*: "Remember that time it was sunny and clear, and then a
minute later all hell broke loose? Better take your raincoat-- it might
happen again..."
In theory, 'simpler' weather-stories might treat one or two of the trend-
dimensions (eg, temp and cloudcover) and ignore the others: more complex
stories could then be created additively. (Some combinations would never
occur in real life-- bald spots on the meteorological 'gym floor'.)
Alternatively, a 'simpler' story might follow *all* the trends, but for a
shorter span of storytime, or in less finely-resolved detail. So the
story "overcast" is a simplification of "overcast, then clearing later",
and also of "overcast with a couple of places where the sun peeked
through".
Choosing one of these classes of simplification is choosing an 'indexing
scheme', effectively determining *the sequence of menus* that should be
presented to a human user trying to navigate the storybase. So in the
latter case, at any point you'd be offered a menu of scenes-that-might-
happen-next, and in the former case a menu of other-features-that-apply-in-
the-same-moment. There's a human-factors element to this, in that you
must avoid overlong menus without resorting to confusingly abstract menu-
items.
The 'indexing problem' is subtly different from the representation
problem: you can sufficiently *represent* a weather story, but still not
know how best to index it within a tree of menus so you can find it when
you need it. SPG&U was mainly about representation, DynaMem shifted the
focus to the general theory of indexing.
[Was this too 'heady'? Let me know if you found it useful. If you need
more examples, check out Dynamem, or "Tell Me a Story". For programmming
mechanics, look at "Inside Case-Based Reasoning" by Riesbeck and Schank.]
Chapter 5: Random images, parsers, buttons
Standing back from the last chapter, it seems obviously to boil down to
one elegant question: what does it mean for one story to be *simpler*
than another? (Any part of a story *must* be simpler than the whole, is a
partial answer.)
One might visualize the network of human histories as concentric circles
(or spheres), with a central point of *zero story content*, the "null
story", each ring adding a single quantum (!) of story content. Reallife
instances will necessarily have infinite content (where, exactly, *was*
that grain of dust between the ridges of your left indexfingertip?), so
*instances* must stand as far from the null-content origin, as the sphere
of stars is from our starry stares....
But can these content 'atoms' be enumerated? Are they finite? What are
the toplevel headings of our story-inventory-textbook-tree? (Well, here's
my answer: first-- person place thing motive, second-- person place thing
motive again, and third-- the set of all simple relationships between
whatever you choose for those first two. So if you choose 'person' and
'thing' you'll be offered makes-acquires-uses-breaks-disposes. But the
last time I said that publicly, I got viciously terminated for my trouble,
so I'm still a bit *skittish*...:^( ) ((Crosswordpuzzle clue hint: ILS has
the not-invented-here syndrome real bad, where 'here' = between Roger's
Rs!))
Back to 1989, or 1990, some random images:
There was a party I went to where someone described Schank's methodology
as 'mere' introspection, to which I said, yea-verily. How else are you
supposed to see what's going on inside you? ***Refusing to introspect
doesn't make you objective-- coming to terms with your self-deluding,
self-denying impulses makes you objective!*** So great literature is most
often highly objective (that's what makes it great), but not at all
distant or cold.
Almost everything I can say about *parsers at ILS* is hearsay, because I
almost never saw one-- ILS philosophy is (correctly, I think) that the
parser problem is too hard to bother with until the CBR-inventory-
of-human-stories problem is solved. According to legend, Roger's old
company, Cognitive Systems, managed to create only a single AI product
that was actually used: an email router for a bank, called ATRANS. I
heard ATRANS described as a *case-based* parser, because it worked by
exhaustively annotating every turn of phrase this bank-office-email-
router might need to understand. (This illustrates the Schankian
principle of AI as the ability to handle realworld-scale casebases without
drowning. See Schank's ILS tech report, "Where's the AI?" ...or on second
thought, my paraphrase is probably adequate... :^)
Every so often on comp.ai, someone will inquire after an Abelsonian { ;^)
CD-parser, expecting that ILS (or someone) must have something they've
been maintaining and evolving since the 70s. But CD is acknowledged at
ILS as hopelessly unwieldy, and the 'next thing' has yet to arrive.
(Charles Martin's Direct Memory Access Parser DMAP was the theoretical
rage for a month or two-- I was almost reassigned to try and code it-- but
it's also wholly dependent on the (expected) invention of an elegant
CBR-memory to parse directly into!)
[End of parsers part.]
A random poetic image from 1989, for armchair freudians and sex-magic
mystics: once I'd established my competence, I was given a big raise and
moved to a better office, by Schank himself. (The symbolism of chairs and
offices at ILS is an adventure unto itself...) The night *before* I got
this seal of approval, I had a dream that Roger *invited me into his
bed*.... (praise dog, I woke up! ;^)
Which reminds me of another story: at a certain point I was assigned to
index some randomly chosen stories (on video-- see next chapter), which
happened to be a top ILS faculty member reminiscing about Roger's
*chair fetish*. He told this story of Roger's spending a term in
Switzerland, at some outfit where the director had the only 'good' chair
in the building, so every morning Roger would steal it from the director's
office, and every evening (!?) the director would claim it back. I
resigned the challenge, because the indexings I was coming up with were
operatic-prima-donna, or rock-star-with-M&Ms (see a.f.u FAQ again, I
think, for the M&Ms). I see now I coulda gone with bully-forces-other-
short-end-of-stick (which can be made to sound flattering if you rephrase
'bully' to 'macho stud').
Also, the big raise had a hidden cloudlining (an eternal ILS verity): I
was told it would be retroactive to September, and celebrated by buying an
expensive piece of fine art (a huge, pretty Calman Shemi wall hanging, if
you didn't wanna know). When the raise came, the retroactive-to-September
part turned out to be just 5%, with the other 30 or 40% starting from then
on....
"Button theory" shows Schank at his most *bozoic*-- my unconscious mental
image is of a brightly colored clownsuit with a row of fluffy pompoms down
the front, the clown baffled by just what they're for.... Actual ILS
software, currently consuming sponsors' funds, greets the user with a row
of Schankian button-icons, the most conspicuous of which eternally drools
"HUH?"
Button theory started from a seminar where a list of usual-educational-
software-control-commands was compiled, with the admirable idea that ILS
software would uniformly use a single panel of buttons for these uniform
commands. The list included:
more/ less detail
what now?
why?
review
too slow/ fast
jump ahead
change tasks
The whole list, after repeated boilings-down, was still around 20
commands, and the proposed panels of icons overwhelmed the remaining
screen area, while boggling one with their obscurity.
I had a hunch that there was an elegant design solution possible, and
cooked up a panel of 20-or-so tiny 16-by-16-pixel Mac icons, unified by
the (pc!) theme of Kid Button and her/his bike trip, following a natural
chronology from preparation to starting out to taking a break to returning
home to reviewing the day, with an 'uphill strain' icon and a 'downhill
out-of-control' icon, etc etc etc. It was completely ignored by Schank,
but the design principle of "The Menu Is A Story" stayed with me-- even
the Macintosh 'File' menu tells a chronological Open-Close-Quit story (by
this principle, it ought to be arranged Open-Save-Print-Close-Quit).
This principle was also visible in the natural shape that my 1987-88
*inventory of histories of romantic love* had taken. I read every love
poem in every anthology in every library and bookstore I could get to (can
you guess? it was unrequited ;^/ ), and compiled the best images into a
frame that went from loneliness thru courtship thru failed courtship,
successful courtship, relationship and breakup. I drafted similar
structures for business psych (starting-a-business to bankruptcy,
sending-a-resume to termination) and several others. But there are no
anthologies of *brilliantly evoked* business stories, so these latter
frameworks remain relatively unfilled-in! Ditto, interestingly, for
relationship-problem stories. I was reduced to self-help books! (And then
there was the mortifying foul-up where I returned one of these to the
wrong library branch and had to spend *literally months* arguing about
whether I owed them for "Love Stinks: true tales of jerk-ass boyfriends"
or some such... ;^) One interesting result of the romance research was
the vivid demonstration that human love-histories (!) are identical
whether you're talking ancient Egypt, aborigine, medieval China, Swahili,
or Palookaville...
I also suggested around ILS, once or twice, resigned, that the buttonpanel
be implemented as a *tool palette*, so that you could point to any part of
the screen with, eg, the Huh? tool, an idea I see from Computer Gaming
World is now common in point-and-click adventure interfaces, where Huh?
will likely appear as a magnifying-glass cursor/icon, or an eye. The
latest issue of CGW also mentions a forthcoming game where each object has
its own set of verbs-- a boombox has insert-tape, play-tape, etc--
(object-oriented adventuring!), and another with an MIT-developed
pop-up-right-under-your-current-mouse-position diamond-of-icons, and a
really cool sounding trick where you can "take movies" of what you see
and *play them later* for others, as a way to talk without typing. (Wish
I'd thought of *that*! What an IF-challenge, to build puzzles around that
*new form of competence*!) The ILS design-stratum had a consistent scorn
for the emerging conventions of Mac interface, and things generally had to
be implemented so compleat idiots (like captains of industry) wouldn't be
embarrassed by them-- so tool palettes were effectively out of reach.
Chapter 6: Ask Tom (AI in SuperCard!)
So after the weather project was cancelled, I got juggled around for a
week or two and then offered this swat-team 'one month' easy-showpiece
top-priority project already named "Ask Tom", a passive hypervideo
'browser' for exploring *a storybase of videoclips* of a top Arthur
Andersen trust-banking expert, telling accounting-consulting 'war
stories'.
Early on in the project, I bemoaned my sense of hypocrisy that all we were
doing was making a *really conventional* textbook of trust-banking, with
the cross-references hardwired in, but Larry Birnbaum explained, in a
fatherly talk that was the turningpoint in my reconciliation to ILS's
homely-looking ambitions, that the general problem of *how to build a
coherent outline for *any* arbitrary domain of knowledge* was as important
a question as any in AI. Yes, Larry said, Roget (the thesaurus-ist) was
doing AI, as is, after a fashion, *any* bright popularizer trying to make
a complex domain accessible to the general reader. Looking back on Ask
Tom, I now see pretty much *every* important abstraction of AI
programming, hardwired into... SuperCard.
(SuperCard, for you non-Mac readers, is a more-powerful version of
HyperCard, the breakthru 'paint'-an-application application. It's
*really* easy to use, and so, on the surface, seems beneath a
professional-AI-macho-stud's contempt.)
There was also something wildly disproportionate about spending hundreds
of thousands of dollars putting high-quality video into an application,
when all the video showed was an accountant's talking head! (The S/N here
seems about a googol times worse than an ascii transcription of his words.
;^) But I actually feel this was a sort of heroic pragmatism, that if
video-based educational software is worth exploring at all, it's gonna
cost astronomical sums and deliver very little at first. You might as
well start with talking heads, because filming the trust bank itself would
be a *lot* more expensive!
The semi-flashy 'gimmick' that made "Ask Tom" special was that Roger
wanted us to come up with a simple, consistent view of the storybase
where, after seeing a storyclip, one would always be presented with the
same 'meta-menu' of followup clips, or rather followup-clip *types*. The
theoretical rationalization was to be derived from an old paper of Roger's
on "cognitive associative categories", which looked at the sorts of ways
an ordinary human conversation can change course. (You can stay on the
topic, or shift in the direction of some subtopic, etc.) So one of our
tasks was to compile an inclusive list of all the sorts of followup a
browser might enjoy, and then boil it down to a short-list that would look
graceful on the SuperCard screen!
We all watched hours of accounting-video so we'd be able to 'ground' our
speculative proposals in concrete examples. And the final list of
"CAC-links" (cogn-asso-cate, above) included just eight: background,
results, context, examples, warnings, opportunities, alternatives, and
indicators. So after seeing Tom talk about the difference between small
and large banks, say, you might be offered followups that talked about why
some banks grow and others don't, or examples of each, or the implications
of bank-size for a trust consultant, what to look out for if it's a small
bank rather than a large one, etc.
As this list was distilled from the much-longer list, categories were
necessarily merged, and the most general label chosen to include them
both, so these final eight labels were necessarily rather vague, and it's
not at all clear to me that they're really *useful*, compared to just
listing the same followup stories in order of interestingness, say. And
worse, a whole class of Schankian CACs got ignored, because there was no
way to include them simply: Schank's original article focussed most on
links of the type: same-person, same-object, same-theme, etc... but our
spec just didn't allow the complexity of enumerating the persons, objects,
and themes in a story, so that you'd be able to choose the one you wanted
to followup via...
The screen display was designed on a spec that consisted of Roger waving
his hands to suggest a network of nodes in a black 3-D outer space,
twisting parallaxically as you moved from node to node. With SuperCard's
limited animation capabilities, we re-imaged this as a SuperCard 'card'
for each storyclip, represented with a central story-card surrounded by
eight peripheral *pseudo-3D piles* of story-cards, one for each of the
eight CAC-links, forming a 'lotus' (or 'crown of thorns', on bad-fridays).
The parallax-twisting ended up looking more like Vegas poker-dealing....
This was the first ILS project that required the function of "media
indexer," which is now an official ILS job-title, borne with diffident
pride by dozens of ILS employees. (As clerical jobs go, it's definitely on
the interesting end of the scale, or would be if project-design
hierarchies at ILS were a little more enlightened.) All the links from
story to story in the Ask-Tom casebase had to be 'hand-crafted' or
hardwired, which ultimately meant looking at every possible pair of clips
and asking whether either would make an interesting followup to the other,
and which of the eight CAC-links it made most sense under.
So the network as a whole was just a stack of screens, almost identical,
where the CAC-card-piles were implemented as buttons that 'moved' the top
card of that pile to center screen. The only programming task was writing
'meta-code' that made these standardized screens *self-generating* (given
the card titles and what-was-linked-to-what)....
I said earlier that I felt, at first, we were just writing a not-very-
special hypertextbook, but the textbook-outline aspect of Ask Tom was
completely perpendicular to the CAC-browser. Since the CAC-browser *had*
to be perfectly homogeneous, with exactly the same sort of links no matter
where in the network you stood, there was a need for an external
'orienting-structure', if nothing else so that you didn't always have to
begin at the same point in the network.
This we called the 'zoomer' as opposed to the 'browser', suggesting
vertical versus horizontal movement, and implemented finally as a very
*witty* hierarchical sequence of graphical 'maps' of trust-banking
meta-concepts: the toplevel screen identified the four main 'players' in
the trust-banking-consulting game: Andersen-as-an-institution, the
individual Andersen consultant (who was taken to be the user of the
storybase), the client bank, and the banking industry-as-a-whole. You
could click on any of these, or on the colored bands connecting them,
which bands represented the *relationships* between the basic players.
Clicking on one of these screen-elements led down a level to a screen that
might depict, eg, the sequence of steps usually experienced in one of the
toplevel *relationships* (the Andersen-client relationship 'expanded' into
Andersen-courts-client, client-communicates-need, Andersen-proposes-
solution, etc), or the internal structure of one of the players (the
bank's organizational hierarchy, say). Clicking on any part of one of
these displays led to a 'themes' screen that sketched the most important
themes that Tom's storytelling had pointed up, within that area, and
offered a few choice stories as startingpoints for browsing. This was the
most ad-hoc design decision, and grew (partly) out of my insistence that
the way to deal with large casebases is to write each storytopic on an
indexcard, and sort them into piles!!! So my cardpile categories became
the themes under the subdivisions of the toplevel 'maps'.
It was these graphical-conceptual 'maps' that became the focus of interest
in the once-Tom-was-done project, a followup *tool* for building
generalized 'Ask systems'. We did some really nuts-and-bolts AI-thinking
at this point, about how to take any heap of stories, and carve from it
*with the aid of a software tool*, a system comparable to Ask Tom. One of
my favorite generalizations from that period (to give a taste of the level
of abstraction we were trying to come to terms with) was the idea of a
'sorting task' ...in the course of any sort of intellectual-creative
effort, there will be phases where what you're doing is taking object X
(and Y and Z) and looking at a row of 'buckets', one or more of which you
might want to drop it into. A tool to make this easier will always need a
"???- Try again later" bucket that commits you to no particular
bucket-assignment, and allows you to reopen the matter later, for
reconsideration.
-------------------------------------------------------------------------
As a startingpoint for catching up on Roger's work, I like "Tell Me a
Story" best. "The Connoisseur's Guide to the Mind" covers similar ground
but is less likeable.
The following ILS tech reports are relevant to this story in one way or
another. They're free to academics, and just a few dollars each for the
rest of us. Requests go to Elizabeth Brasher-Brown, 1890 Maple Avenue,
Evanston, IL 60201, or by email at: br...@ils.nwu.edu
Elliott: The Affective Reasoner (the TaxiWorld thesis)
Kolodner & Jona: Case-Based Reasoning: an overview
Schank, Ferguson, Birnbaum, Barger, & Greising: Ask Tom
Schank & Osgood et al: A Content Theory of Memory Indexing (UIF)
Schank & Fano: A Thematic Hierarchy for Indexing Stories
Jorn Barger
An Overview of 'Fractal Thicket' Indexing
Jorn Barger
This document will attempt to offer a self-contained tutorial on the
semantic indexing problem in AI, and my own 'fractal thicket' proposal for
its solution.
The indexing problem in AI
When physicists talk about a grand, unified, "theory of everything," all
they're really asking for is a precise description of strings and spinors
and quarks. This level of analysis will not be immediately useful to
average people and their ordinary problems! So it's fair to ask if there
might be other valid approaches to a 'theory of everything', that give
central place instead to the sorts of problems that humans really consider
most important and useful. A library's reference section always includes
plenty of books that offer provisional solutions to this problem--
encyclopedias, thesauri, dictionaries of quotations... even libraries' own
classification schemes, like the Dewey Decimal and Library of Congress
schemes.
These are all 'theories of everything' of a very different sort. (You
can't usefully index library books according to the flavors of their
quarks!) But in general, the reference books that need such theories fall
back instead on lists of alphabetized keywords, because every other scheme
ever proposed has been fraught with (even worse) redundancies and
uncertainties. (Prototypical example: books of quotations sorted into
themes like Angels, Animals, Baseball, Capitalism, Conformity, Democracy,
Enlightenment, etc...)
Attempts in AI to automate translation between natural languages have
foundered on these same shoals-- to translate a word you have to know which
of several possible meanings is the relevant one in the given case, and to
disambiguate those meanings, you *need* an overarching classification of all
the things words *can* mean. Roget proposed an elaborate scheme almost two
centuries ago, tagging 1000 categories of synonyms, but this is far too
ad-hoc to build AI with, and very little progress has been made since,
towards improving it.
And I think there's an extremely odd paradox at work here-- we may know how
to distinguish a million shades of meanings in our words, and have no
problem recognizing when a word is being misused, but it's damnably hard to
articulate or classify exactly *what it is we know* that allows these
subtle discriminations. Doug Lenat's CYC project is trying to nibble the
problem away by exhaustively annotating every concept in the world, but
they've barely scratched the surface of the huge domain of psychological
nuances-- feelings and emotions and desires, and even actions. Human
beings are just *bad* at this sort of self-knowledge, universally, though
it's the most important element in every sort of communication.
So we have this huge, messy indexing-problem, *of vital importance*, and
we're so handicapped by this (somewhat neurotic) blindspot in our self-
knowledge that we can barely even think or talk about it.
Where I've come from
I've spent the last 20 years trying to find new strategies for getting
beyond that blindspot. My greatest successes have involved analysis of the
range of themes expressed in *literature*, because the literary tradition
has always been concerned with precise description of psychological states.
These researches have been given some added 'shape' by focusing on the
immediate goal of programming a video-game, of the interactive-fiction
genre, that would make use of some (yet-to-be-discovered) carefully
articulated theory of story-plots and human emotions. (Prototype: George
Polti's "36 Dramatic Situations" See rec.arts.int-fiction for more FAQs.)
I spent the last three years (1989-92) learning the ins and outs of AI
hacking at a lab where programming was seen as "inherently uninteresting,"
and programmers ignored at the bottom of a status hierarchy that included
senior faculty, faculty, senior grad students, etc, all outranking us lowly
coders, who were however the only ones who really had a grasp of how the
programs worked! Consequently, I found myself grappling with the most
difficult problems in AI, fairly alone, and when in 1991 I found what
seemed like a promising solution, I was outlawed from implementing or even
presenting it, and then fired when I publicly drew attention to my plight.
Sigh...
The fractal-thicket indexing topology
My proposal includes a new data-topology, a way of interpreting that
topology *semantically*, and a preliminary, *incomplete* set of semantic
fillers for the first, experimental implementations to start from.
AI has been leaning heavily, all along, on the ancient and venerable
*hierarchy* or tree topology. More general topics are put at the top of
the hierarchy (the root of the tree) and specialized step-by-step as you
move down the hierarchy (up the tree ;^).
As an AI programmer, though, I was frequently confronted with hierarchy
branches like 'Plans' or 'Events' that had 10 zillion different immediate
specializations, or more often were left entirely unspecialized, with the
footnote "to be filled in Real Soon Now".
Other branches, though, like 'Things', were much more tractable. So I
conceived the idea that those *ill-behaved* hierarchy-branches were
inherently *complex* rather than simple, in that they could be analysed
(broken down) into *sets* of the simpler elements, with the most obvious
toplevel categories being person, place, and thing.
So 'emigration' can be analysed as including, at least, one person and two
places, and it should be indexed so that if you request all person-place-
place material, emigration is included near the top of the list!
But I hated the idea of having to search thru all ten zillion complex
items, looking for matches to some pattern. I wanted an indexing scheme
that allowed you to step right *to* exactly those right items. And what
gradually came clear to me was that the *general* topology to allow this,
has to place a *self-similar sub-hierarchy* under each node on the simple
hierarchy, trees within trees within trees.
So if the toplevel of the tree offers person, place, and thing, then each
of those will offer, along with their normal specializations, an 'added-
element subtree', also beginning with person/place/thing. And when you
navigate down two levels to the node 'person-then-place' you'll find all
the person-place material (and also perhaps indentically at the node place-
then-person).
And I call this a "fractal thicket", because its branching factor is so
dense. And the first part of my proposal is to just contemplate and
investigate this topology, ignoring all details of the particular labels on
the simple hierarchy. (Take *any* hierarchy, and do the thought experiment
of mapping its elements over itself, to see what new semantic classes
emerge when you consider your categories in pairs.)
There's a strong parallel here to mapping a simple 1-D array across itself
repeatedly, creating a multi-dimensional array. Fractal thickets work just
the same way, mapping trees across trees.
It may seem like this is a perfect recipe for an uncontrolled combinatorial
explosion, instantly clogging memory with an only-sparsely-meaningful
structure. But imagine that each branch might be a simple array of
pointers to similar arrays, an eight-element 'cons' cell, if you like,
and realize you don't have to instantiate a
branch on a subtree if it's meaningless, so the *worst-case* overhead is
really just the size of that array times the number of steps it is deep,
which is not bad at all. And even better, you can assume that each of
those steps probably stored useful data of its own, info about that
slightly-less-specialized set of simple elements, so there's really almost
*no* wasted overhead, and the dense combinatorial explosion will expand
exactly as far as needed, and no further. If it's got a few orders of
magnitude above a million elements, so be it. That's the best we can hope
for, for natural language!
Fractal-thicket semantics
The immediate semantic challenge that 'falls out' of this topology is to
consider your toplevel simple elements in pairs and trios, etc, and ask
what sorts of *relationships* do these types of elements usually display?
So persons and things may demand:
person makes thing
person acquires thing
person uses thing
person maintains thing
person destroys thing
person disposes of thing
The simple traditional person-place-thing categories now suddenly generate
a rich set of relationship-primitives, which bear a striking resemblance to
the classic verb-sets used in adventure games (still the state-of-the-art,
for interactive fiction).
It seems heuristically promising to sort each such set of primitives into a
simple, universal *chronology* of its own, as in this example. (Thought
experiment: Again, take your favorite hierarchy, and ask: for each pair of
elements, how do relationships between these two classes usually begin?
What then happens next? How do such relationships usually end?)
A natural next step is to take these relationships in pairs, and link them
with temporal-causal connectives like
person goesTo place
then
person acquires thing
person uses thing1
enables
person acquires thing2
A user-interface for such an indexing scheme should offer first the top
level of the simple hierarchy, which can then be navigated to any degree of
specialization, *and then* an identical toplevel menu for the first
"complication", which can again be specialized, and then a *choice* of
another complication *or* a menu of relationships for the current complex
set, and then a way to sequence these relationships, etc.
Preliminary semantic fillers
Here's a hierarchy of simple elements. It needs lots of work.
element
motive
hunger
safety
sex
esteem
family
self-expression
thing
food
tool
container
vehicle
clothing
weapon
bodypart
waste
place
inside
home
livingroom
kitchen (etc)
school
office
outside
yard
street (etc)
person
gender
age
species
role?
modality
emotion
belief
etc
Person, place, and thing here are quite familiar, and are, as well,
universal in adventure games. Motive and modality are original proposals
that I think promise great payback, but must require persistence and
discipline to explore. Many interesting human psychological (and *social*)
phenomena involve an exchange or tradeoff relationship between two motives.
Here's where the 'blindspot' kicks in with a vengeance!
What I'm calling modalities normally involve a relationship between a
person *and another relationship*. So "person desiresThat (person acquires
thing)" can be parsed in the same way as "person believes (person isAt
place)".
I've tested this system on the last 150 categories in Roget (figuring they
were the hardest) and the results are impressive but need polish. (Write
me if you can't wait, I guess ;^)
Some basic two-element relationships
person thing: makes acquires uses maintains destroys disposes
person place: bornAt movesTo occupies movesFrom diesAt
person motive: suffers gratifies indulges abstains denies
relationship relationship: causes enables precedes becomes
person relationship (ie modality): imagines wants believes fears believes-
deserves
person person (usually with a 3rd element): begets kinTo marries harms
helps conflictsWith cooperatesWith communicatesWith hasStatusOver
motive motive: tradeoff exchange conflict substitute
thing thing: hasPart attachedTo
person skill: discovers learns practices uses forgets
Three-element relationships are most interesting when two of the elements
are persons, like person-gives-thing-to-person.
[comp.ai, April 1994]
A more-technical look at fractal-thicket indexing
Apparently, once upon a time, there was general agreement among AI
professionals that 'the representation problem' was a key-- perhaps
even *the* key-- to building intelligent machines.
For a machine to reason about the world, everyone agreed, it needed
representations of all those circumstances-in-the-world that were
relevant to the domain of its reasoning...
But something like a superstitious awe now seems to have fallen over
the AI community, almost everywhere seeming to back off from the
representation problem as *just too difficult for human thought* (the
CYC project being the notable exception).
Speaking personally, my sense is that there's really been only one
'conceptual tool' deployed in this battle so far: the abstraction
hierarchy. And my contention is that the *topology* of the hierarchy
is just too simple to handle the problem, and more advanced topologies
are going to be required.
Consider an auto accident. Two vehicles, their passengers, a place,
a time... you might imagine the representation taking the form of
a a police report, with pre-defined slots for these predictable
classes of filler.
But can the completed report be squeezed into a general hierarchy,
in anything like a logical manner?
The concept of 'car' can, easily, via some path like physicalObject->
manmadeObject->vehicle->automobile. The concepts of place and time
can, too, eg: country->state->city->street.
But topologically, a *hierarchy* at this point can offer only a
flat list of pointers, from each of these nodes to *all* the
relevant accident reports (not to mention every other sort of
indexable knowledge we might want the knowledgebase to include).
So retrieval becomes a tedious process of *search*, and parallel
processing starts to seem like a win, simply because search is
so damned slow...
What I'm proposing, with the fractal-thicket concept, is that
we distinguish *simple* entities, which can be neatly handled by
a hierarchy topology, from *complex* ones which must be analysed
according to the simple entities they comprise.
As trivial as this sounds, it immediately calls for a topology that
can cope with, in the simplest case, any arbitrary *pair* of
simple hierarchy elements, assigning such pairs to *some predictable
location* relative to the simple hierarchy.
Stripping away the semantics, for now, let's picture a perfectly
behaved hierarchy with a branching factor of eight, where individual
elements can be represented like:
6.3.1.5
meaning: start from the root, take the 6th branch, then the 3rd, etc.
Fractal thicket indexing would place *at each such node* an
alternative *class* of branches that offer a self-similar image of the
basic hierarchy, so that to find a complex entity composed of
"6.3.1.5" and "2.7.4.1" you would trace the path "6.3.1.5/2.7.4.1"
(or, alternatively, "2.7.4.1/6.3.1.5").
The "/" operator is entirely unlike the "." operator in its
semantics. Where "." means "has a specialization", "/" means "has
relationships to".
So, to explore these new semantics, one should logically begin by
generating all the binary relationships of the form:
1/1
1/2
1/3
...
2/1
...
8/7
8/8
and then reflecting on the sorts of *real world* relationships
captured by these simplest new forms.
While the process should work identically, no matter what
interpretation is assigned to 1 through 8 (etc), a simple 'starter
set' of concepts might include person, place, and thing, and the
simple relationships might look like:
person thing: makes acquires maintains changes disposes destroys
If person is "1" and thing is "2", then *all* these relationships will
be lumped, at first, under the path "1/2"-- person/thing
relationships. (We may hope that at some point the relationships
themselves can also be assigned regular numbers, and a third operator
introduced so that "1/2*5" might indicate person-disposes-thing.)
An automatic consequence of this is that a relationship like
2.1.5/7.4.1
will *inherit* semantics from
2/7
and
2.1/7.4
For example, car-collides-car will inherit from vehicle-collides-vehicle
and from thing-collides-thing.
Rather than multiply-indexing 1/2 under 2/1, we can introduce a
permutation rule that tells us 2/1 will *always* be found at 1/2
(essentially 'alphabetizing' the elements of the complex entity).
And three-element (etc) relationships are also available, with the
same math:
1/1/2 might imply person-gives-thing-to-person.
This all implies, as well, a new way of looking at complexity in
general: *Every* real event is inevitably infinitely complex,
analysable into any number of constituent parts (an atom being a
thing, too!). But for the purposes of knowledge representation, the
trick is to freeze the analysis at whatever *simplest possible* level
can still captures the information desired.
One can go through Roget's, and ask for each category, how many
persons does this concept require? How many things? (Etc.) And
there emerges, in consequence, a new sorting of concepts from simpler
to more complex...
Does it explode? No! (It's not a bomb, it's a *reactor*... ;^/
=----------=- ,!. --=----=----=----=----=----=----=----=----=----=----=
Jorn Barger j't Anon-ftp to ftp.mcs.com in mcsnet.users/jorn for:
Michael Hucka
> I will post the lot, and I encourage anyone who disagrees with Mark to
> let him know. My feeling is that ILS is *dangerously dishonest* and it's
> my *reponsibility* to make that known. Anyway, science without
> inflammatory ideas is dead science...
> Here's a scratchy first draft of an alternative to Mark's FAQ, that I
> hope to re-post regularly, enlarging it and including a more-compact set
> of pointers to things I consider most useful.
Speaking as a fairly disinterested observer of your personal vendetta, I
would just like to comment that Mark Kantrowitz's FAQ is the best FAQ I've
ever seen. While you have some interesting ideas in your proposed
alternative, the S/N ratio is far too low. And incidentally, although I've
never read Newquist's _The Brain Makers_, any "timeline" of AI that doesn't
even mention Newell and Simon has got something seriously wrong with it.
--
-- Mike Hucka
University of Michigan, Ann Arbor, Mich.
Jorn Barger
[...]
> you have some interesting ideas in your proposed alternative
Thanks!!!
j