Will Mind Children Make People Obsolete?
Paul D. Fernhout
graduate student at SUNY Stony Brook, which I just now scanned and OCR'd as
I continue to slim down and recycle my paper archives. I thought I'd post it
here, and Bryan feel free to forward it to one of your Extropian lists. :-)
I'm BCCing this to Hans, too, just for his records.
I'm amazed though, with the Singularity ideas (unknown to me as a term in
1992) and the short time horizon for the Singularity as now discussed, how
fast this is all happening. Rather than the 5000 to 500,000 years mentioned
below, it seems more like we will be deeply confronting these ethical and
practical issues of coexistence with various forms of
human-equivalent-or-beyond machine intelligence within the next fifty years
(or even the next twenty). I think Han's insight of AI being stuck at one
MIPS for decades helps explain the sudden increasing explosion in
possibilities, allowing a lot of old ideas to become practically feasible,
even though there is still much we don't know about intelligence:
http://www.frc.ri.cmu.edu/~hpm/project.archive/robot.papers/2000/Cerebrum.html
--Paul Fernhout
===========================================
Will Mind Children Make People Obsolete?
by Paul D. Fernhout
Written around 1992
Are humans obsolete (or soon to be so)? That question has
been rolling around in my mind since I spent nine months as a
visitor to Hans Moravec's Mobile Robot Lab at Carnegie Mellon
University in 1986. Dr. Moravec is the author of "Mind
Children", a speculative book about the future of humans and
machines. He was writing it while I was there at the lab.
A conversation with one of the staff researchers one day
really shook me. This researcher said quite forcefully that
in fifty years or so humans will be obsolete, replaced by
intelligent space faring robots. What scared me is that he
thought this was a good thing, was working his hardest to
realize this vision, and had the technical skills to have a
chance of success. This idea is central to Moravec's book.
One could try to dismiss Moravec's work as marginal or
science fiction. One could also point out difficulties in
some of the underlying assumptions especially regarding
evolution. One might also quibble with the time scale,
suggesting 5000 or 500,000 years as the time this development
will likely take. In the end, those criticisms aren't
productive. One must wrestle directly with the central
concept: that there is such a thing as human obsolescence,
that we are creating our only children in our machines, and
that this is a good thing.
This question has troubled me for six years. Should we
create such progeny if this means our basic human form is to
be discarded? Alternatively, should we devote our every
waking hour to preventing this? One must consider the
general loss of humanity involved; our mind children may be
called intelligent but they won't think or feel like humans
if for no other reason than the problems they face in
survival are not the same. One must also consider that the
new designs might be unstable in survival in the long term,
and so we might embody our minds in machines that quickly go
extinct for unexpected reasons. The machines might also be
hostile to humans, trying to exterminate them as in the movie
"Terminator". The combination, extermination of humans and
then rapid extinction, is particularly worrisome.
Nanotechnology has the same problem - an unstable grey ooze
might overtake all and then evaporate. The fact that the
defense department has been a major funder of robotics and
artificial intelligence work (especially Moravec's) lends
this scenario a bit more likelihood.
The resolution of this question in my mind has come from an
unexpected source: Hydra. Hydra are typically five
millimeter long organisms growing in freshwater ponds and
streams. They generally have around six tentacles used to
sting prey like the water flea, Daphnia, and push them
through their mouth to be digested in a central cavity. They
have two layers of cells and a simple neural net. Some even
have symbiotic algae to help provide food and oxygen. They
are one of the simplest multi-cellular animals and have been
around at least tens of millions of years. They are
extraordinarily well adapted to life in their environment.
Yet they have no brain.
The Hydra's superior adaptedness to its environment is for
many reasons. It is a good size to allow it to replicate
quickly when food is available and then disperse by floating.
It can function well over a variety of sizes; it can also
shrink if no food is available. Hydra may well be immortal;
they may not be programmed to die like humans or many other
more complex organisms. Its neural network is complex enough
to allow it to feed, yet doesn't require much energy to
maintain (unlike a brain). It has what it needs to survive
and replicate, with perhaps a little extra useful for
evolving. In terms of efficiency, they might be considered
far superior to humans. There is more to survival and
adaptation than intelligence. For the hydra, a big brain and
increased intelligence might even be a handicap by making it
more difficult to reproduce.
It is easy to see humans as on top of some pyramid of
intelligence or complexity. It is easy then to see
intelligent machines as somehow above us in this pyramid.
What of the hydra? Even if intelligent machines exterminate
humanity, what of the hydra? Short of the total reworking of
the earth's hydrosphere, I would suggest hydra will still be
around in a hundred million years. They would be difficult
to kill even with targeted pesticides given their rapid
potential for evolution. For example, when exposed to high
levels of cadmium in the lab, they rapidly evolved better and
better ways to deal with the problem.
Why should intelligent machines bother to exterminate hydra
at a great cost, unless they were totally paranoid about
life? More likely they would not, because they would have
things to do they might consider more important or more
interesting like expansion into space. If the machines
don't care about exterminating hydra, will they exterminate
other creatures? Algae, roaches, sparrows and badgers might
do quite well under their benign neglect.
The attempted extermination of humans by machines is a more
open question since humans and intelligent machines might
have a greater overlap in terms of niche. A few people like
Moravec say they will go willingly when the time comes,
especially if some part of their consciousness is preserved
in machine form. They will accept the notions of efficiency
and obsolescence as valid ways to make the decision to live
or die.
The rest of us will fight the process of assimilation for the
same reason we fight cultural assimilation and thoughts of
suicide. The groups that didn't feel this way changed until
they did feel this way or didn't exist. We will treasure our
unique ways of life and fight to preserve them. We are
comfortable with how we live and want to maintain our way of
life regardless of seemingly better alternatives. After all,
better is defined only within a cultural context. If our
culture says it is better than other cultures, then it is.
The only difficulty is all the competing contexts a human is
in, like personal, family, nation, religion, biological, and
job-related contexts. So defining better is tricky. It
would probably be safe to say most of these contexts would
not define the extinction of humanity as a good thing
regardless of whether any mind children were produced.
So we will fight the machines to stay alive if we must.
Imagine the worst case scenario in the year 10,000 A.D.
Machines then might be to us as we are to hydra. They might
have a vast ability to think, feel, desire, create, predict,
and sense. Humans would be creating makeshift ecologies
amidst a vast interstellar machine civilization in the same
way cockroaches live in human cities. The machines might try
to keep the human population down for practical reasons
because people accidentally disrupt things in the same way
rats gnaw through telephone cables. In general, the machines
really can't be bothered usually because they have more
interesting things to do than squash people, so human
population will grow as machine civilization advances. Where
people go, so go the hydra, roaches, rats, and if resources
permit, even the white rhino. This is the worst case
scenario I can envision. It's not "Terminator" but a parody
of a termite inspection commercial. "Do you have humans
disrupting your data streams? Call 555-555-555-TERMI-MAN."
The best case is that humans develop machines like
intelligent O'Neill space habitats capable of self
replication that actively preserve human and other life.
These systems might incorporate something like Asimov's three
laws of robotics. These laws will not be so simple as
Asimov's linguistical constructions, and they will involve
preserving certain desired patterns within certain limits.
Of course, the laws can not prevent the evolution of rogue
machines that want to survive and reproduce for their own
sake, sometimes at odds with human goals. Humans will first
try to exterminate these like we try to exterminate roaches.
After that proves unsuccessful, they may strike some
compromise with the rogues in the same way nations can
negotiate treaties. In the vastness of space both will
coexist.
In a strange way, both the best case and worst case scenario
end the same. Humans will survive and prosper. They will
continually design and construct machines to help them
preserve desirable habitats for humans and other forms of
life - even if the parts come from other machines they don't
understand. They will coexist with self-replicating machine
intellects which have their own goals.
Intelligence is not adaptiveness. There are many ways of
generating adaptive behavior with many different types of
brains. Among humans there is a wide range of thinking
styles and behaviors. Super-intelligent self replicating
machines will be another form of life. They will have their
own unique problems and own unique ways of thinking. They
will also evolve into a multiplicity of different designs the
same way biological life has. They will also have a variety
of goals - like humans. Some will exterminate people; others
will keep them as pets; others will claim people have a right
to exist as free creatures.
Intelligent machines will start off as child species of the
human species. This analogy quickly breaks down. Human
individuals have a well defined maximal life span of 120
years; the human species has no such limit. Evolution has
many examples of species spinning off other species and still
persisting, often longer than the newer child species.
To continue the parent-child analogy, don't we humans have
many children, all different and mostly not what we expected?
We love them despite their not being like us or what we
wanted them to be. Sometimes we love them all the more
because of that. Loving a child doesn't mean you love
yourself any less or have any less wish to live as full a
life as you can.
[As a personal note, I abandoned robotics and artificial life
research in 1988. I presented a computer simulation I had
developed of not very intelligent yet very vicious self
replicating and self repairing robots at an AI conference
that year. I had spoken on how easy it was to make vicious
robots and how much harder it would be to make cooperative
ones. Someone from the Army cosponsoring the workshop
literally patted me on the back afterward and told me to keep
up the good work. It was then my doubts got much more
serious and I stopped working on the code. I saw the worst:
the military's short-sighted encouragement of development of
vicious killing machines.
In response I detoured into environmentalism, organic
agriculture, socially responsible business, urban
revitalization, decentralization, self renewal, solar energy,
and a bunch of other humanistic ventures. It is only
recently that I began to consider if that person with the
Army might have been encouraging the future development of
cooperative robots suggested by my final view graph.
My current thinking on the subject makes me more accepting of
artificial life work again and I may get back into the field
in various ways. My goal is to develop that more positive
future of self-replicating intelligent space habitats. A
more direct approach may produce less intermediate human
suffering than the negative scenario of letting such things
happen on their own. At the very least, it is a fun project
with many potential positive spinoffs.]
Bryan Bishop
> Here is another essay I wrote around 1992 while an Ecology and
> Evolution graduate student at SUNY Stony Brook, which I just now
> scanned and OCR'd as I continue to slim down and recycle my paper
> archives. I thought I'd post it here, and Bryan feel free to forward
> it to one of your Extropian lists. :-) I'm BCCing this to Hans, too,
> just for his records.
Paul, it's pretty weak. :( You're not wrong, though. I was trying to
separate some of the bullshit in transhuman discussions on my critique
of the Wikipedia article: http://heybryan.org/transhumanism_def.html
When you bring up the concept of 'obsoletism', for example, that's
really a meaningless term to me; there are various examples that others
have been employing for a while now, such as those who say the ai will
want goo to destroy humans and convert our bodies into carbon-carbon
diamondoid structures, there are the others that say that an ai
domination scenario will result in peace, love and harmony for all on
earth with the ability to predict our every whims, blah blah blah, it's
just a load of crap, and the amount of separation of subjects and
different competing concepts in the readership is difficult enough.
((Also, the extropians have been thinking about this for the past few
decades, so you're missing the gravy train methinks. Good times.))
- Bryan
________________________________________
http://heybryan.org/
Engineers: http://heybryan.org/exp.html
irc.freenode.net #hplusroadmap
Paul D. Fernhout
yesterday, so I could post there perhaps someday if I feel inclined (after
getting a feel for it reading it for a while).
I'll also say that thinking about things on a continuum -- plain human,
transhuman and/or networked plain human, and machine AI (networked or not)
-- may lead one to different ideas than that 1992 essay which basically only
considered unaugmented human intelligence versus stand-alone machine AI.
I still think, as with some of my comments on Kurzweil, that what a lot of
the extropian and transhumanist comments that I have seen (which is not that
much) is simply a lack of a basic understanding of ecology and evolution and
their interplay, which that essay made a start at addressing. There is a
vast academic literature related to those themes which provides many
surprising and sometimes counterintuitive examples of evolution in action.
In looking at the future, some reading about the past (e.g. almost any good
related book by Stephen J. Gould)
http://en.wikipedia.org/wiki/Stephen_Jay_Gould
might do many interested in the field some good.
--Paul Fernhout
Bryan Bishop
<pdfer...@kurtz-fernhout.com> wrote:
> I still think, as with some of my comments on Kurzweil, that what a lot of
> the extropian and transhumanist comments that I have seen (which is not that
> much) is simply a lack of a basic understanding of ecology and evolution and
> their interplay, which that essay made a start at addressing. There is a
> vast academic literature related to those themes which provides many
> surprising and sometimes counterintuitive examples of evolution in action.
> In looking at the future, some reading about the past (e.g. almost any good
> related book by Stephen J. Gould)
> http://en.wikipedia.org/wiki/Stephen_Jay_Gould
> might do many interested in the field some good.
Your original critiques of Kurzweil do better addressing the topics at
hand. :-)
- Bryan
Paul D. Fernhout
--Paul Fernhout
Paul D. Fernhout
Wikipedia page I read today:
http://en.wikipedia.org/wiki/Posthumanism
"Posthumanism is sometimes used as a synonym for an ideology of technology
known as "transhumanism" because it affirms the possibility and desirability
of achieving a "posthuman future" in purely evolutionary terms. However,
posthumanists in the humanities and the arts are critical of transhumanism,
in part, because they argue that it incorporates and extends many of the
flaws of Enlightenment humanism, namely scientific imperialism and
perfectibilism"
I think that "posthuman" versus "transhuman" distinction is a good one to
keep in mind. That 1992 essay is more along the lines of posthumanism, again
quoting from the posthumanism Wikipedia article:
"Posthumanism mainly differentiates from classical humanism in that it
restores the stature that had been made of humanity to one of many natural
species. According to this claim, humans have no inherent rights to destroy
nature or set themselves above it in ethical considerations a priori. Human
knowledge is also reduced to a less controlling position, previously seen as
the defining aspect of the world. The limitations and fallibility of human
intelligence are confessed, even though it does not imply abandoning the
rational tradition of humanism."
And maybe that is also why posthuman humans (even augmented ones) becoming
like roaches in some vast machine civilization is not that appealing a
sentiment to transhumanists? :-) I'm not saying that is the future, it is
just a thing to think about to at the very least keep us humble about the
machines we do create. But it also is a theme I see echoed in Iain Bankes'
Culture novels, where the humans are in some ways kept as pets (it's not
completely clear -- there may be aspects of Asimov's Three Laws or Robotics
in the design of the machine "Minds" as well).
By the way, the word "obsolete" was picked specifically to be controversial,
in part because that seemed the tone that some conversations took way back
when (and was part of the tone of the Mind Children book). Here is an
example from 1993:
"The cinema of human obsolescence - science fiction films and society -
Column"
http://findarticles.com/p/articles/mi_m1272/is_/ai_13387205
"In the best retelling of the Frankenstein story to date, "Blade Runner"
also deals with the marginalization of humanity itself in the age of the
cybernetic revolution and supranational corporatism. The killer cyborg Roy
Batty (brilliantly portrayed by Rutger Hauer) ultimately is more
compassionate and "human" than his creators or pursuers. Indeed, the film
raises questions about whether or not the human soul can survive
media/computers/drugs/implants/transplants and the other components of the
civilization that makes individual identity seem obsolete. One can not help
but notice how the ideas of "Blade Runner" and the subsequent cinema jibe
with the decade's academic and public debates about the death of the subject
and the end of history. What separates this film, however, from much of the
cinema that picked up its ideas is the hardly sanguine tone about the
movie's conclusions: humanity here clearly is threatened, looking back
nostalgically on its history and struggling in the hyperalienating
post-modern moment to keep a handhold to survive."
I agree "obsolescence" is not the most productive way to look at the
situation of different or overlapping ecological niches for species with
different types or quantities of intelligence. Is the approximately half a
billion years old Horseshoe Crab obsolete? Horshoe crabs still seem to be
going strong even though they suffer from this obvious defect:
http://en.wikipedia.org/wiki/Horseshoe_crab
"Every year approximately 10% of the horseshoe crab breeding population
dies when rough surf flips the creatures onto their backs, a position from
which they often cannot right themselves. In response, the ERDG launched a
"Just Flip 'Em" campaign, in the hopes that beachgoers will simply turn the
crabs back over."
Similarly, will it be their be a campaign of compassionate (or not) machine
AIs to help humans break out of tight pleasure loops they can lock
themselves into via food or drugs or pleasure center drouds? :-)
"Droud -- A wire providing current directly to the pleasure center of the
brain."
http://www.technovelgy.com/ct/content.asp?Bnum=207
See also:
"The Pleasure Trap: Mastering the Hidden Force That Undermines Health &
Happiness" by Douglas J. Lisle
http://www.amazon.com/Pleasure-Trap-Mastering-Undermines-Happiness/dp/1570671508
"This startling new book focuses on a problem that permeates modern life:
that the abundance and ease of 21st century living is a mixed blessing. The
authors offer unique insights into the movtivational factors that make us
susceptible to dietary and lifestyle excesses, and present ways to restore
the biological processes designed by nature to keep us running at maxium
efficiency and vitality. ... A wake-up call to even the most health
conscious people, The Pleasure Trap boldy challenges conventional wisdom
about sickness and unhappiness in today's contemporary culture, and offers
groundbreaking solutions for achieving change. Authors Douglas Lisel, Ph.D.,
and Alan Goldhamer, D.C., provide a fascinating new perspective on how
modern life can turn so many smart, savvy people into the unwitting
saboteurs of their own well-being. Inspired by stunning original research,
comprehensive clinical studies, and their successes with thousands of
patients, the authors construct a new paradigm for the psychology of health,
offering fresh hope for anyone stuck in a self-destructive rut. Integrating
principals of evolutionary biology with trailblazing, proactive strategies
for wellness, they argue that people who are chronically overweight, sick
and ailing, or junk food junkies aren't that way because they're lazy,
undisciplined, or stuck with bad genes. The authors reveal that most are
victims of a dilemma that harkens back to our prehistoric past-"the Pleasure
Trap." Drs. Lisle and Goldhamer then call upon their clinical experience,
scientific investigations, and a recent revoution of understanding in human
motivational psychology to provide you with solutions for the challenges of
keeping on a healthful course-and how to make the most of your life."
--Paul Fernhout
Bryan Bishop
> As I reflect on your comment some more, I saw a potential link to
> this Wikipedia page I read today:
> http://en.wikipedia.org/wiki/Posthumanism
<snip>
> And maybe that is also why posthuman humans (even augmented ones)
> becoming like roaches in some vast machine civilization is not that
> appealing a sentiment to transhumanists? :-) I'm not saying that is
I was not saying it's not an appealing sentiment, but rather that the
idea space has been explored to a more extreme degree in other places.
Hell, I spent my summer working on a project geared towards documenting
factors contributing to this Uncertain Future -- and yes, that was then
name of the project. There's all sorts of dooms day scenarios, let's
imagine an axis of nanotechnological disaster, and then you get to
throw up points re: military disasters, grey goo disasters, paperclip
ai taking over ai for grey gooage purposes, all the way to other more
extreme basement universe nanotechnological backwash disasters, and the
list goes on and on.
+ Bryan