interesting reading part 1
Rusty Wright
COM. I've exceprted the relevant articles.
ART COM (electronic) Magazine #43 and #44 (Nov & Dec '90) were devoted to
Interactive Fiction. These two back issues are available by e-mail. Send
requests to d...@hpsemc.cup.hp.com, with the one-word Subject: ArtCom.
Requests are processed by a program; append a personal message if you wish.
/\ ___ ___ ___ ___ __ __
/__\ /__/ / / / / / / /
/ \/ \ / /__ /__/ / / /
-----------------------------------------------------------------
DECEMBER 1990 NUMBER 44 VOLUME 10 NUMBER 10
-----------------------------------------------------------------
Welcome to ART COM, an online magazine forum dedicated to the
interface of contemporary art and new communication technologies.
You are invited to send information for possible inclusion. We are
especially interested in options that can be acted upon: including
conferences, exhibitions, and publications. Proposals for guest
edited issues are also encouraged. Send submissions to:
well!artc...@uunet.uu.net
artc...@well.sf.ca.us
Back issues of ART COM can be accessed on the Art Com Electronic
Network (ACEN) on the Whole Earth 'Lectronic Link (WELL),
available through the CompuServe Packet Network and PC Pursuit.
To access the Art Com Electronic Network on the WELL,
enter g acen at the Ok: prompt. The Art Com Electronic
Network is also accessible on USENET as alt.artcom.
For access information, send email to: artc...@well.sf.ca.us.
*Guest Editor: Abbe Don
*Executive Editor: Carl Eugene Loeffler
*Editor: Anna Couey
*Systems: Fred Truck and Gil MinaMora
ART COM projects include:
ART COM MAGAZINE, an electronic forum dedicated to contemporary art
and new communication technologies.
ART COM ELECTRONIC NETWORK (ACEN), an electronic network dedicated to
contemporary art, featuring publications, online art galleries, art
information database, and bulletin boards.
ART COM SOFTWARE, international distributors of interactive video and
computer art.
ART COM TELEVISION, international distributors of innovative video to
broadcast television and cultural presenters.
CONTEMPORARY ARTS PRESS, publishers and distributors of books on
contemporary art, specializing in postmodernism, video, computer
and performance art.
ART COM, P.O. Box 193123 Rincon Center, San Francisco, CA, 94119-3123, USA.
WELL E-MAIL: artcomtv TEL: 415.431.7524 FAX: 415.431.7841
-------------- A R T C O M / DECEMBER 1990 / VOL. 10(10) / #44 --------------
COMPUTATIONAL DRAMA IN OZ
Joseph Bates
Dr. Joseph Bates (joseph...@wizard.oz.cs.cmu.edu) is director of
the Oz project at School of Computer Science, Carnegie Mellon
University. He was co host of the Workshop on Interactive Fiction and
Synthetic Realities at the AAAI-90 conference. (Abbe Don)
From working notes of AAAI-90 Workshop on Interactive Fiction and
Synthetic Realities, Boston, July 1990. These are informal notes
prepared as background material for a lecture given at the AAAI
workshop.
The Oz project at the CMU School of Computer Science is developing
technology for high quality interactive fiction. Our goal is to
provide users with the experience of living in a dramatically
interesting simulated world that includes simulated people.
A variety of researchers on human interfaces are studying virtual or
artificial realities. These are computer simulated interactive visual
environments that people experience as real. Most existing research
concerns issues close to the interface, that is, how to take an
underlying simulated world and present it in a convincing fashion.
The Oz group plans to use this interface technology as it develops,
but our work is on creating rich, deeply modeled underlying worlds.
Thus, we study the simulations behind the interface, which we call the
deep structure of virtual reality.
Our work applies existing artificial intelligence technology to the
problem of building dramatic worlds. These worlds are composed of a
(simulated) physical environment, intelligent/emotional agents which
live in the world, a user interface and theory of presentation to let
one or more humans interact with the world and its agents, and a
computational theory of drama which plans and controls the overall
flow of events in the world. There are clear applications of such
simulations to entertainment (interactive fantasy experiences) and to
training (e.g., improving interpersonal skills in business). Also,
since we think art develops as new media develop, we hope our work
will be the basis for one of the first sophisticated knowledge based
art forms, using computers as the underlying medium.
We believe these simulations, which today require engineering
workstations, will run on future consumer electronics products that
integrate video and audio with RISC engines, large DRAMs, and digital
signal processors. Interactive fiction was the most popular home
software in the U.S. in the early 1980's, despite an extremely low
level of technical sophistication. We suspect that by taking
advantage of known technologies, interactive fiction can develop into
a popular and long lasting art form. We think it can serve as a
primary motivation for people to own the personal digital systems of
the middle 1990's.
AN OVERVIEW OF OZ
As preparation for the workshop lecture, this document presents an
overview of the Oz Project. Our efforts can be partitioned into six
areas: physical world simulation, the minds of simulated characters,
the user interface with its theory of presentation, theories of drama,
the world building environment, and the artistic use of the system,
each of which is described below.
Oz is built in Common Lisp, with substantial use of the Common Lisp
Object System. The system incorporates large amounts of code from
other Lisp based AI projects. We develop Oz on Mach/Unix
workstations, but do not presently rely on anything beyond Lisp.
The Oz group at CMU includes Bates, five Computer Science graduate
students, an undergraduate, and a gradually growing collection of
users from the English and Drama departments and elsewhere in the CMU
community. We are assisted, especially on dramatic theory, by Brenda
Laurel.
With Brenda and with Margaret Kelso, of the CMU Drama department, we
have started studying real life interactive improvisations. These can
be viewed as simulations of Oz, and can tell us both about the
inherent nature of this art form and about how to build computer based
IF (interactive fiction) systems.
PHYSICAL WORLD SIMULATION
The Oz physical world simulator provides a commonsense model of the
physical world. It is an abstract model, thus many aspects of the
real world are omitted. Unlike existing interactive fiction systems,
our emphasis in not on manipulating objects in the world, but on
character and plot. Thus, while we are applying object oriented
techniques to flexibly model the world, and while these models could
ultimately become quite rich, our requirement is only to provide
enough of a physical reality to let authors construct interesting
characters and stories.
THE MINDS OF CHARACTERS
These are the minds of the (non-human) agents that populate worlds.
One of the claims of the Oz project is that the mental architectures
and real world knowledge bases that have been developed in AI over the
last 15 years, while perhaps still too weak for real robots, are well
suited to the demands of interactive fiction. Our goal is to draw on
the best of these existing systems, such as work from Yale, CMU, and
Microelectronics and Computer Technology Corporation (MCC), and
interface them to Oz as frameworks for the minds of agents. Once
these frameworks have been established, the builders of worlds will
use them to construct individual characters.
We are currently working on two frameworks: a goal driven reactive
planner called HAP and the Prodigy planner. We are developing HAP
within the Oz group based on work at MIT by Agre and Chapman and on
work at Yale by Firby. Prodigy is a planner/learner being developed
at CMU by Jaime Carbonell's machine learning group. We expect to
extend both systems using ideas of Wilensky, Dyer, Carbonell, and
others to provide some level of social awareness and emotion, in
addition to rudimentary intelligence.
In addition, we are interested in exploring other systems. The
developers of Soar (particularly Allen Newell at CMU and John Laird at
University of Michigan) and CYC (Doug Lenat at MCC) have expressed
interest in connecting their systems to Oz. We are hopeful that this
will occur during the next few years.
USER INTERFACE AND THEORY OF PRESENTATION
The user interface connects human agents to the simulated world. For
the immediate future, we expect this connection will be via natural
language text. We have been using software from the CMU Center for
Machine Translation to generate text, producing both the narrative
description of the world and the textual "speech" of computer modeled
agents. We are now developing Glinda, our own generator, with careful
attention to the PENMAN work at ISI.
In our application, parsing is easier than generation. At present we
use a general purpose bottom up parser with a simple grammar and ad
hoc semantic and pragmatic analysis. We are considering using instead
a word based parser, such as the DYPAR parser originally developed by
Roger Schank's group at Yale. These parsers seem appropriate for
processing short, syntactically limited, possibly ill-formed input,
which is typical in interactive fiction.
We have started studying ways to "tune" the natural language
generation to provide subtle emotional influence on the human player.
In theatre and cinema, extra-semantic influences such as music,
lighting, point of view, zooms, and film editing play a significant
role in determining viewer reaction. The artistic technique developed
in these areas is crucial to their respective media. Our report
"Towards a Theory of Narrative for Interactive Fiction" describes
results of our initial efforts to find analogous technique for
interactive fiction. We are pursuing this research, with the goal of
having Oz adjust its style of output to suit the varying dramatic
content of the story. Hovy's work on PAULINE is directly relevant to
our efforts.
As individual character architectures develop, they may well bring
their own mechanisms for natural language processing. Where these
mechanisms improve the quality of characters, we will probably use
them in place of Glinda.
Oz presently uses a text interface for two reasons. First, an
argument can be made that text based IF is a valid form in its own
right, allowing certain kinds of artistic technique, such as
narrative, that cannot easily be applied in a VR setting. Second, we
feel that our research effort and computing capacity is best spent now
on characters, natural language, and dramatic theory. However, as
virtual reality interface technology matures and as we develop
efficient implementations for inhabited dramatic worlds, we plan to
investigate ways of replacing the text interface with facilities for
speech, animation, and gestures. We hope this work will be in
collaboration with researchers studying each of these areas. We have
discussed, but not taken, such steps with Andy Witkin's animation
research group.
COMPUTATIONAL DRAMA
Oz worlds are intended not only to be realistic, but to be
interesting. Often this means giving people the feelings that come
with good stories, feelings that arise in part from the structure of
plot, such as complication, climax, and resolution.
We understand how this can be achieved for static text: an author
carefully constructs the text to convey the structure. However, in
interactive fiction we do not write out the whole text in advance, and
we don't know in advance the detailed sequence of events a reader will
experience. Thus, the Oz system must dynamically, and subtly, adjust
the behavior of the world and its characters to provide experiences
with the desired dramatic structure. This means developing and
implementing a computational theory of drama, and using it to guide
the behavior of worlds.
We see several approaches to developing such a theory. The simplest
comes from having the author express a partial order on the
significant events of the story (an "abstract plot graph"), explicitly
representing that partial order in the system, and using it to drive
the character goals and narrative decisions in the rest of the system.
This approach would leave almost all the dramatic theory in the mind
of the author, with the plot graph serving as a kind of partially
ordered program to be executed by the system.
A richer approach is to develop a library of abstract plot units and
then, as the interaction proceeds, rapidly search abstract plot space
for controllable paths that have the desired dramatic structure. We
can view this as a kind of abstract adversary search, where we define
a set of abstract operators, means for mapping operators into concrete
moves, means for recognizing the abstract effects of the users moves,
and an evaluation function on event histories (ie, stories) that lets
us recognize sequences with "good" dramatic structure.
We are working toward implementations of both of these approaches.
The former appears to be a relatively easy way to provide dramatic
control signals for experiments with the rest of Oz; the latter is a
self-contained long term research goal. In both of these efforts, we
are drawing on Brenda Laurel's studies of computational versions of
Aristotle's Poetics.
(editor's note: the workshop lecture discussed these matters in
greater detail).
WORLD BUILDING ENVIRONMENT
Once we gather and integrate available AI technologies, thus making
dramatic worlds possible, we need to provide some means for "normal
people" to construct such worlds. For interactive fiction to develop
as an art, many artists must explore it, because their feedback is
crucial to guide the technology toward artistically desirable goals.
While these artists may be computationally inclined, they do not need
to be experienced Lisp/AI programmers.
Designing the right tools for a general purpose Oz authoring
environment can only come after we have some experience building
individual Oz worlds. However, we know that building Oz worlds will
be a kind of programming. It will involve creating, accumulating, and
reusing large numbers of world parts, such as physical objects and
settings, parts of minds (planners, plans, kinds of social knowledge),
sets of linguistic rules, and components of narrative and dramatic
theories.
The Oz "parts" libraries, similar perhaps to the backlots of Hollywood
studios, will be large and varied. The libraries will contain
mechanisms for modifying and building objects (meta-knowledge) as well
as individual objects of special value. We believe that the overall
library structure and the processes for building it may be similar to
those of the NuPrl system. NuPrl is an interactive environment for
mathematicians and programmers to use in semi-automatically creating
large bodies of explanations and procedures. It is one of the first
and most successful systems of its kind, and has spawned related
research in the U.S., Japan, and Europe. Bates was one of the leaders
of the PRL project, and based on that experience has designed a
successor system, called MetaPrl, which is planned as the basis for
the Oz authoring environment.
USE OF THE SYSTEM
It is very important that the technical efforts of building Oz be
guided by the needs of artists building worlds. We have already
involved several people from the CMU English, Drama, and other
departments in this process. We intend to have the population of Oz
users grow as the system develops, by teaching courses to the full CMU
community and by making Oz available to the research communities
outside of CMU. This widespread use is necessary in practice, since
it will require great effort to construct a substantial library of
world parts and the effort must be distributed over a large base of
developers. But we believe it is even more necessary in principle, to
learn the potential of interactive fiction as a new art form and to
guide the development of Oz toward reaching that potential.