AI Seminar
Thomas Bartold
From: "Edmund H. Durfee" <dur...@umich.edu>
To: <ai-se...@eecs.umich.edu>
Subject: AI Seminar: Francis Quek
Date: Friday, September 29, 2000 12:23 AM
The next AI seminar is being presented by an alum of the AI lab:
AI SEMINAR
October 3, 2000, 4pm, 175 ATL Building
Multimodal Discourse: Gesture, Speech and Gaze
Francis Quek
Vision Interfaces and System Laboratory
Computer Science & Engineering Department
Wright State University
Human discourse is an active process of converting thoughts into
speech, gesture, and gaze activity. Grounded on the psycholinguistic
foundations on the production of such multimodal 'conversational-acts'
(as opposed to the mono-dimensional speech-act), we addresses the
interpretation of gesture, speech, and gaze in the context of
discourse management. We investigate the cues afforded by each
mode of interaction and the algorithms necessary to detect and
extract them; study the spatial and temporal relationships among
these cues and associate them with topical units in discourse;
study the interactions of gesture, speech and gaze in discourse
segmentation; and a multimedia database system that integrates
these elements into a coherent whole. Our approach involves
experiments designed to discover and quantify cues in the various
modalities, and their relation with respect to discourse management;
the development of computational algorithms to detect and recognize
such cues; and the integration of these cues into a cogent discourse
management system.
We present psycholinguistic phenomena that are detected by our analysis.
The understanding of how such phenomena are detectable from video and
audio signal, and the determination of the kinds of computable cues that
support such analysis are the first steps toward the bridging the
signal-sense gap in multi-modal interaction. Among these are cues for
semantic segmentation and organization, cross-modal temporal integration,
and the significance of 'hold tension release'.
We have assembled a strong interdisciplinary team comprising
psycholinguistic, machine vision and signal processing researchers to
address the holistic nature of discourse and language itself. This
permits us to base our research squarely on the realities of human
communication in spontaneous discourse across a wide range of pragmatic
conditions. Technology is being developed that have significant impact
on natural language discourse analysis, human-computer interaction
systems, neuropathological studies (Parkinson's Disease and Left/Right
Hemisphere Damage) and discourse and video databases. Another
significant outcome of this research is to introduce computational and
quantitative rigor to the psycholinguistic study of discourse production.
This represents a model of collaborative research between the fields of
engineering and cognitive science.
Biography:
Francis Quek is currently an Associate Professor in the Department of
Computer Science and Engineering at the Wright State University. He has
formerly been affiliated with the University of Illinois at Chicago,
the University of Michigan Artificial Intelligence Laboratory, the
Environmental Research Institute of Michigan (ERIM) and Hewlett-Packard
Human Input Division. Francis received both his B.S.E. summa cum laude
(1984) and M.S.E. (1984) in electrical engineering from the University of
Michigan in two years. He completed his Ph.D. C.S.E. at the same
university in 1990. He also has a Technician's Diploma in Electronics
and Communications Engineering from the Singapore Polytechnic (1978),
and briefly attended Oregon State University in 1982. Francis is a member
of the IEEE and ACM.
He is director of the Vision Interfaces and Systems Laboratory (VISLab)
which he established for computer vision, medical imaging, vision-based
interaction, and human-computer interaction research. He performs
research in multimodal verbal/non-verbal interaction, vision-based
interaction, facial modeling, multimedia databases, medical imaging,
collaboration technology, computer vision, human computer interaction,
and computer graphics.
Francis is the Principal Investigator of several prestigious National
Science Foundation grants in gesture, speech, and gaze research and of a
Whitaker Foundation grant in neurovascular extraction in medical brain
images. He leads a team of researchers in a multi-million dollar NSF-KDI
project spanning multiple-disciplines, institutions, and countries to
understand the communicative realities of multimodal interaction. He is
also P.I. on an NSF/STIMULATE grant.
++++++++++++++
Edmund H Durfee
Professor of Electrical Engineering and Computer Science
(also Professor of Information)
Director, Artificial Intelligence Laboratory
University of Michigan
Ann Arbor MI 48109
dur...@umich.edu, 734-936-1563 (voice), 734-763-1260 (fax)
http://ai.eecs.umich.edu/people/durfee
--
CSEG Web Page: http://www.eecs.umich.edu/CSEG
See the CSEG web page for unsubscription instructions.
Please support CSEG by buying your soda from CSEG vending machines.