An interesting poser?
Suzan Humphrey NASA/GSFC/HEASARC 301.286.4682
Can the 3 laws (and even the Zeroth) be circumvented in such a manner:
A robot fires a loaded weapon at an empty space, which, by the time the
bullet arrives, will have a person in it (the space... one would guess the
bullet would then be in the human, not verse visa). Supposedly, since s/he
/it did not fire at a human, and cannot beat the bullet to the human, s/he/
it is not at fault... or is s/he/it?
Anyway, we had a long discussion on this that came to no viable conclusions
: I am not expert.
If someone, anyone, could not only explain why not (according to the direct
wording of the rules), and let me know where to find the arguments... or
tell me how woefully misguided I am, I would appreciate it.
My access to the net is at best intermittent, but I get e-mail at work.
Hump...@lheavx.gsfc.nasa.gov. I'd love to hear the answers, and I will be
checking back....
Sue
danny burstein
>Sue
that's exactly the type of thought experiment the Good Doctor references
in his writings. For example, one robot can shoot an arrow, or another can
poison a glass of milk....
makes for quite interesting reading.
--
----------------------------------
dan...@panix.com adds: all the usual disclaimers regarding liability,
intelligence, accuracy apply. spelling disclaimer is doubled.
and...
while science is pretty sure that the sun will rise in the east tomorrow,
we're not quite so sure it'll be shining.... (i.e. where are the neutrinos?)
Ed Seiler
hump...@heagip.gsfc.nasa.gov (Suzan Humphrey NASA/GSFC/HEASARC
301.286.4682) wrote:
> My husband asked a questions maybe you more learned individuals can answer.
>
> Can the 3 laws (and even the Zeroth) be circumvented in such a manner:
> A robot fires a loaded weapon at an empty space, which, by the time the
> bullet arrives, will have a person in it (the space... one would guess the
> bullet would then be in the human, not verse visa). Supposedly, since s/he
> /it did not fire at a human, and cannot beat the bullet to the human, s/he/
> it is not at fault... or is s/he/it?
It seems to me that the robot should be able to calculate the likelyhood
that a human will be hit by the bullet and act accordingly. In your
example, if the person is visible at the time the weapon is to be fired,
and is moving at a steady (predictable) speed, the robot should not fire at
a point ahead of the person such that the bullet will hit the person. But
if the robot had no knowledge that the person was coming, and was told to
fire the weapon at the precise spot and instant, the robot could manage to
kill with the weapon. In other words, I think a positronic robot is capable
of involuntary manslaughter.
In between lies a grey area, which is why a positronic brain is so
complicated. For any task that a robot does in the presence of humans,
there is some probability that harm could come to humans or itself as a
result, and some tasks have a greater danger than others. A properly tuned
positronic brain should balk at performing highly dangerous tasks (such as
shooting bullets calculated to miss by microscopic amounts), but not at
tasks below some threshold of danger. If robots could not perform any tasks
with non-zero risks, they wouldn't be as useful, so any interesting world
in which robots exist has such risks.
--
Ed Seiler
sei...@nibbles.gsfc.nasa.gov
"If puns are outlawed, only outlaws will have puns."
Tom O Breton
> A robot fires a loaded weapon at an empty space, which, by the time the
> bullet arrives, will have a person in it
Nah, that's exactly the sort of thing Asimov would have considered.
That's why he says a robot may not CAUSE harm to a human being, rather
than simply "A robot may not harm a human being." I'm sure he considered
indirect actions while phrasing that. A tribute to his brilliance.
One thing he doesn't mention (As far as I recall) is the certainty that
an action would cause harm.
Could a robot do something that does not cross a threshhold of probable
harm, and then repeat it so frequently that it's likely to happen?
Could a robot, say, sabotage a victim's car so that in the unlikely
event of a minor fender-bender, certain death ensues to the occupants?
(And then every day undo the sabotage and re-sabotage it, so that no
*single* act "causes" the death of humans.)
Or is it linked to results, predictable or not, so that if a robot
"causes" a hurricane via the "butterfly effect", it's brain burns out?
In that case, it would seem that all robots must self-destruct as soon
as they understand this.
Tom
--
The Tom spreads its huge, scaly wings and soars into the sky...
(t...@world.std.com, TomB...@delphi.com)
Robert Huss
|> > Can the 3 laws (and even the Zeroth) be circumvented in such a manner:
|> > A robot fires a loaded weapon at an empty space, which, by the time the
|> > bullet arrives, will have a person in it
|>
|> Nah, that's exactly the sort of thing Asimov would have considered.
|>
|> That's why he says a robot may not CAUSE harm to a human being, rather
|> than simply "A robot may not harm a human being." I'm sure he considered
|> indirect actions while phrasing that. A tribute to his brilliance.
|>
|> One thing he doesn't mention (As far as I recall) is the certainty that
|> an action would cause harm.
|>
|> Could a robot do something that does not cross a threshhold of probable
|> harm, and then repeat it so frequently that it's likely to happen?
|>
|> Could a robot, say, sabotage a victim's car so that in the unlikely
|> event of a minor fender-bender, certain death ensues to the occupants?
|> (And then every day undo the sabotage and re-sabotage it, so that no
|> *single* act "causes" the death of humans.)
|>
Besides the actual statements of the 3 laws, Asimov
makes it clear that the whole structure of the positronic brain
has the laws incorporated into it. Therefore, even the concept
of injury to humans causes the robot discomfort.
Without some weakening in this integration, a robot should
be incapable of skimming the edges of the verbal definitions.
An example of a weakened first law robot is in "Little Lost Robot"
I think.
Bob.
.
harr...@force.ssd.lmsc.lockheed.com
It must be pointed out that the 3 Laws are verbalizations of rules encoded in
the positronic brain; the actual rules are actually much more complex and
complete. Another poster mentioned "Lost Little Robot" a story in one of the
collections of robot stories (maybe "I, Robot"?) in which a robot had been
programmed with a weakened 1st law, such that it could initiate an action that
could cause harm to a human, such as dropping a weight such that it would fall
on someone. The robot wasn't causing harm directly, but indirectly.
This was found not to be a good idea (for robots to be able to do this.)
Most robots, could be tricked into causing harm to a human, but their
positronic brains would suffer damage as a result.
Tom Harrison
Usual disclaimer regarding my employer applies.
Andy Bensky
of Gaia may have developed out of the earth colony Nemesis? Nemesis, or rather
the moon that was colonized, was the home of a form of life that already existed
as a sort of collective consciousness with the planet itself. That consciousnes
seemed to exist at an emotional level which was capable of tying into certain
humans that were sensitive to it. It also seemed to be integrated into the water
which made up a large part of the planet itself, including the water in the
atmosphere.
If this were really the building blocks of Gaia it would explain Daneel's inference
about life forms other than humanity. Since one was found it is possible that others
would also be discovered. It may have been shere luck that the one found first was
of a nature to be symbiotically compatable with human consciousness. Next time
we might not be so lucky.
This connection seemd very clear when I first read Nemesis, which was a few years
ago. I can't remember all the details now, but the way they described the union
between the planet/creature and the little girl and other sensitive people that went
to the planet seemed very much Gaian.
anyone have any thoughts on that hypothesis?
andy
Jason W Solinsky
|>
|> Can the 3 laws (and even the Zeroth) be circumvented in such a manner:
|> A robot fires a loaded weapon at an empty space, which, by the time the
|> bullet arrives, will have a person in it (the space... one would guess the
|> bullet would then be in the human, not verse visa). Supposedly, since s/he
|> /it did not fire at a human, and cannot beat the bullet to the human, s/he/
|> it is not at fault... or is s/he/it?
|>
|> Anyway, we had a long discussion on this that came to no viable conclusions
|> : I am not expert.
|>
|> If someone, anyone, could not only explain why not (according to the direct
|> wording of the rules), and let me know where to find the arguments... or
|> tell me how woefully misguided I am, I would appreciate it.
It depends entirely upon how intelligent and well informed the robot is. A three
law robot which is stupid could easily be ordered to do this. Any robot that has
any reason to believe that this might result in the death of a human would avoid
this. If the robot were unaware of the circumstances of the humans arrival until
after the bullet was fired, this might well work even if the robot was highly
intelligent (if they had transporters for example).
A robot with the zeroth law will happily kill a human being if it is the only
way to advance what is in the best interests of humanity. Hittler could have
made excellent use of zeroth law equiped robots by giving them a good background
in Eugenics (sp?).
Jason W. Solinsky
Paul
>|> My husband asked a questions maybe you more learned individuals can answer.
>|>
>|> Can the 3 laws (and even the Zeroth) be circumvented in such a manner:
>|> A robot fires a loaded weapon at an empty space, which, by the time the
>|> bullet arrives, will have a person in it (the space... one would guess the
>|> bullet would then be in the human, not verse visa). Supposedly, since s/he
>|> /it did not fire at a human, and cannot beat the bullet to the human, s/he/
>|> it is not at fault... or is s/he/it?
>|>
>|> Anyway, we had a long discussion on this that came to no viable conclusions
>|> : I am not expert.
>|>
>|> If someone, anyone, could not only explain why not (according to the direct
>|> wording of the rules), and let me know where to find the arguments... or
>|> tell me how woefully misguided I am, I would appreciate it.
Since I've exhausted my re-reading of Asimov's best books I started on the
Lucky start books I have, Asimov's attempts at space opera, readable but not
that good, anyway I digress, the point is, in the "Rings of Saturn" and
probably in the rest of the series, I only have 2, there are robots used by
the Sirian enemies who do kill humans. Apparently the Sirians are very good
at "convincing" the robots that their actions will not cause harm. For
instance, they are told that they can destroy Earth ships because there are
no humans aboard and even though they may think they detect humans aboard
this is just a trick by the earth people to stop robots destroying their
ships.
--
Paul Richards, University of Wales, College Cardiff
Internet: pa...@isl.cf.ac.uk
JANET: RICHA...@CARDIFF.AC.UK
John Walter Biles
> A robot fires a loaded weapon at an empty space, which, by the time the
> bullet arrives, will have a person in it (the space... one would guess
the
> bullet would then be in the human, not verse visa). Supposedly, since
s/he
> /it did not fire at a human, and cannot beat the bullet to the human,
s/he/
> it is not at fault... or is s/he/it?
>
> Anyway, we had a long discussion on this that came to no viable
conclusions
> : I am not expert.
2 cases:
1. Yes, you can circumvent the three laws--if the Robot does not know that
a human will enter that space. The first law does not prevent robots from
shooting empty space after all.
2. No--If the robot has enough data to know that someone will be there,
it won't shoot. If it witnesses the person being hit, it will suffer some
degree of mental damage, varying in part by how much the person was hurt.
Similar dodges of the first law occurs in The Caves of Steel and The Naked
Sun, but to say more would spoil the story if you haven't read it.
A robot can be tricked into violating the laws, but if it finds out, its
positronic brain will be damaged or ruined. The robot stories are rife
with examples of this. An interesting variation occurs in one story where
someone "proves" they are not a robot by physically assaulting another
human. However, since the suspect in question appears to be human,
someone raises the question of whether the target of his assault might
also have been a robot(in which case the first law would not be violated).
The Red Ghost, Kalganeese Pirate
joh...@wam.umd.edu
Kenneth M. Webb
>
>A robot fires a loaded weapon at an empty space, which, by the time the
>bullet arrives, will have a person in it (the space... one would guess the
>bullet would then be in the human, not verse visa). Supposedly, since s/he
>/it did not fire at a human, and cannot beat the bullet to the human, s/he/
>it is not at fault... or is s/he/it?
>
enjoy reading Dr. Asimov's books and short stories about robots. I'll
take a stab at this one. Asimov's robots were programmed using the three
laws of robotics as a guideline. These laws, as far as I can remember,
were not meant to be enforced by officials. As in "The Robots of Dawn," a
robot was forced to break one of the laws of robotics. This resulted in
what was called Roblock (Robot Block). This was his penalty for breaking
one of the laws.
Next, I have to ask the question, Is it possible for a human to
jump in front of a gun that has just been fired? I know people are shot
by accident with guns often, but not very often. Also, why would a robot
be firing a loaded gun into empty space? Please, take no offense to my
questions, none is intended. These are just a few questions that popped
into my mind when I read your post.
Mike Kuras
>A robot fires a loaded weapon at an empty space, which, by the time the
>bullet arrives, will have a person in it (the space... one would guess the
>bullet would then be in the human, not verse visa). Supposedly, since s/he
>/it did not fire at a human, and cannot beat the bullet to the human, s/he/
>it is not at fault... or is s/he/it?
I'd have to admit, that's a very interesting question. While I doubt
anyone here can answer the question (beyond the realm of speculation), the
closest I've come to this question is Asimov's short story "Little Lost
Robot."
That's just one example of Asimov's storys that is kinda long winded, but
contains one tiny spark that one returns to for years thereafter because
it is so interesting.
Ukjent person
>From: t...@world.std.com (Tom O Breton)
>Subject: Re: An interesting poser?
>Date: Tue, 14 Sep 1993 03:12:51 GMT
Neil Dobson
>In article <CDBq9...@world.std.com> t...@world.std.com (Tom O Breton) writes:
>>From: t...@world.std.com (Tom O Breton)
>>Subject: Re: An interesting poser?
>>Date: Tue, 14 Sep 1993 03:12:51 GMT
>>> Can the 3 laws (and even the Zeroth) be circumvented in such a manner:
.........
>>Or is it linked to results, predictable or not, so that if a robot
>>"causes" a hurricane via the "butterfly effect", it's brain burns out?
>>In that case, it would seem that all robots must self-destruct as soon
>>as they understand this.
Isn't this what essentially happened to Giskard? His actiions were for
the good of Humanity, but in the short term damaged them and so his brain
burned out?
Paul
>In article <CDBq9...@world.std.com> t...@world.std.com (Tom O Breton) writes:
>>From: t...@world.std.com (Tom O Breton)
>>Subject: Re: An interesting poser?
>>Date: Tue, 14 Sep 1993 03:12:51 GMT
>>> Can the 3 laws (and even the Zeroth) be circumvented in such a manner:
>>> A robot fires a loaded weapon at an empty space, which, by the time the
>>> bullet arrives, will have a person in it
>>
>>Nah, that's exactly the sort of thing Asimov would have considered.
>>
>>That's why he says a robot may not CAUSE harm to a human being, rather
>>than simply "A robot may not harm a human being." I'm sure he considered
>>indirect actions while phrasing that. A tribute to his brilliance.
>>
>>One thing he doesn't mention (As far as I recall) is the certainty that
>>an action would cause harm.
I've mentioned this before.....
In the Rings of Saturn, one of the David "Lucky" Starr series, his robots
are a little less stringently defined. In particular, there are scenes
where discusions take place about how the enemy, the Sirians, can use
robots to harm humans. One example, David's friend poses the question
that robots can't harm humans and he replies, ahh, but they can be told
to imprison humans in a ship with no air supply and then launch that
ship.
They specifically make the point that the sirians are far more adept at
circumventing the three laws through clever phraseology than the earth
people are.
Ben A. Ostrowsky
This brings to mind "Little Lost Robot", in which the First Law had been
modified. Robots insisted on "rescuing" humans from radiation despite the
extremely low risks involved.
The new First Law read, "No robot may harm a human being."
Asimov explained in the story that such a robot could drop a heavy object upon
a man, knowing that it could save him before the object struck him. But the
Modified First Law would allow the robot to stand idly by while the man was
crushed.
I wonder how far a loose interpretation could go? Could a robot launch many
nuclear rockets, knowing that it could later disarm the warheads, and then
allow the destruction of Earth?
It's something to chew on, anyway.
Ben
Graham Mark Davison
Not as far as I understand it. I reread Robots and Empire over the
weekend. Giskard inactivates (is that the term?) because he is not really
sure whether the destruction of the Earth _will_ be a benefit for
humanity. Daneel was convinced of this, but Mandamus, the Spacer who set
off the intensifier was convinced that it would harm the Settlers. Since
Giskard was not sure whether he had allowed humanity to die or not, he
inactivated.
------------------------------------------------------------------------------
Graham Davison * "I suppose it will all make sense when we grow up."
umda...@cc.umanitoba.ca * - Calvin & Hobbes
Computer Engineering III *
University of Manitoba *
CANADA *
Eric D Broberg-1
>>the good of Humanity, but in the short term damaged them and so his brain
>>burned out?
>Not as far as I understand it. I reread Robots and Empire over the
>weekend. Giskard inactivates (is that the term?) because he is not really
>sure whether the destruction of the Earth _will_ be a benefit for
>humanity. Daneel was convinced of this, but Mandamus, the Spacer who set
>off the intensifier was convinced that it would harm the Settlers. Since
>Giskard was not sure whether he had allowed humanity to die or not, he
>inactivated.
Are you sure Daneel was convinced that the destruction of the
Earth was a benefit to humanity? As I recall, he had no say in this matter
and Daneel never even brought up the topic in the book. It was Giskard who
made the decision and Giskard who accepted the consequences. Daneel was
prevented from interfering and therefore was unable to affect the
situation protecting him from becoming inactive.
One thing I never really understood about Robots and Empire. If I remember
correctly (correct me if I'm wrong), throughout the book Giskard
constantly states that he cannot accept the validity of the zeroth law
"One may not injure humanity or through inaction allow humanity to come to
harm". How is it that Giskard roblocks because he is uncertain that he
harmed humanity which is the zeroth law, the law that Giskard could not
accept.
Nirvana
--
Nirvana is inner peace
Harro de Jong
> Earth was a benefit to humanity? As I recall, he had no say in this matter
> and Daneel never even brought up the topic in the book. It was Giskard who
> made the decision and Giskard who accepted the consequences. Daneel was
> prevented from interfering and therefore was unable to affect the
> situation protecting him from becoming inactive.
>
> One thing I never really understood about Robots and Empire. If I remember
> correctly (correct me if I'm wrong), throughout the book Giskard
> constantly states that he cannot accept the validity of the zeroth law
> "One may not injure humanity or through inaction allow humanity to come to
> harm". How is it that Giskard roblocks because he is uncertain that he
> harmed humanity which is the zeroth law, the law that Giskard could not
> accept.
>
> Nirvana
It seems quite logical to me. Giskard roblocks because he cannot accept
the zeroth law, and has to obey the first law instead. By his decision he
has caused direct harm to a lot of people, which is a big enough violation of
the first law to force him to inactivate. If he had been more able to accept
the zeroth law, he could have survived his decision.
Harro de Jong H.C.d...@research.ptt.nl
Graham Mark Davison
[stuff deleted]
>>Not as far as I understand it. I reread Robots and Empire over the
>>weekend. Giskard inactivates (is that the term?) because he is not really
>>sure whether the destruction of the Earth _will_ be a benefit for
>>humanity. Daneel was convinced of this, but Mandamus, the Spacer who set
>>off the intensifier was convinced that it would harm the Settlers. Since
>>Giskard was not sure whether he had allowed humanity to die or not, he
>>inactivated.
> Are you sure Daneel was convinced that the destruction of the
>Earth was a benefit to humanity? As I recall, he had no say in this matter
>and Daneel never even brought up the topic in the book. It was Giskard who
>made the decision and Giskard who accepted the consequences. Daneel was
>prevented from interfering and therefore was unable to affect the
>situation protecting him from becoming inactive.
> One thing I never really understood about Robots and Empire. If I remember
>correctly (correct me if I'm wrong), throughout the book Giskard
>constantly states that he cannot accept the validity of the zeroth law
>"One may not injure humanity or through inaction allow humanity to come to
>harm". How is it that Giskard roblocks because he is uncertain that he
>harmed humanity which is the zeroth law, the law that Giskard could not
>accept.
Sure I'm sure. Daneel and Giskard had discussed it and come to that
conclusion. (I'm not sure exactly where in the story.) Also, while
Giskard claimed that he couldn't accept the Zeroth Law, he did to a
certain extent. When D and G were going to see the Energy Minister, G
used the Zeroth Law to justify his adjustment of people. He had to adjust
people quickly, without studying them, and this could ahve caused harm.
He never completely accepted (or at least at that point) the law, which
was shown by his trouble walking.
OK, then why did Giskard "roblock" if it wasn't because of the Zeroth Law,
I don't think he harmed anyone at Three Mile Island. Mandamus wasn'teven
touched, and the other guy (can't remember his name, I can't believe this!!)
was only put to sleep(I think, am I wrong??), or at least not damaged.
As a final proof of Giskard's accepting the Zeroth Law, and the reason for
his inactivation, he _says_ it's because he isn't sure if the destruction
of the Earth will benefit humanity. He _syas_ it. I don't think he was
trying to make Daneel happy 8).
------------------------------------------------------------------------------
Graham Davison * "I suppose it will all make sense when we grow up."
umda...@cc.umanitoba.ca * - Calvin & Hobbes
Computer Engineering III *
Universtiy of Manitoba *
CANADA *
Lionel Tun
>correctly (correct me if I'm wrong), throughout the book Giskard
>constantly states that he cannot accept the validity of the zeroth law
>"One may not injure humanity or through inaction allow humanity to come to
>harm". How is it that Giskard roblocks because he is uncertain that he
>harmed humanity which is the zeroth law, the law that Giskard could not
>accept.
Well, in that case you fall back onto Giskard inactivating because
he struck down a human in cold blood.
--
________ Lionel Tun, lio...@cs.city.ac.uk ________
/ /_ __/\ Computer Vision Group /\ \__ _\
/___/_/_/\/ City University, London EC1V 0HB \ \___\_\_\
\___\_\_\/ 071-477 8000 ext 3889 \/___/_/_/
Eric D Broberg-1
>> One thing I never really understood about Robots and Empire. If I remember
>>correctly (correct me if I'm wrong), throughout the book Giskard
>>constantly states that he cannot accept the validity of the zeroth law
>>"One may not injure humanity or through inaction allow humanity to come to
>>harm". How is it that Giskard roblocks because he is uncertain that he
>>harmed humanity which is the zeroth law, the law that Giskard could not
>>accept.
>Well, in that case you fall back onto Giskard inactivating because
>he struck down a human in cold blood.
Struck down a human in cold blood? Yes he did make a human unconscious,
but he didn't kill anybody. He did harm a human (in terms of
mental damage) but I don't understand how he was able to violate the first
law like that, unless he did in fact accept the zeroth law, something which he
was unable to do throughout the novel.
--
Nirva'na (-va-n. 1) in Buddhism, attainment of union with the
divine by the conquest of personal desires and passions. 2) a band. 3) My
alias
Harro de Jong
>(stuff deleted)
> "nirvana" wrote
>
> Struck down a human in cold blood? Yes he did make a human unconscious,
> but he didn't kill anybody. He did harm a human (in terms of
> mental damage) but I don't understand how he was able to violate the first
> law like that, unless he did in fact accept the zeroth law, something which he
> was unable to do throughout the novel.
He tried to accept the zeroth law (or had accepted it to a certain degree),
enabling him to harm humans. But he didn't fully accept it , hance his
inactivation. I think Giskard knew that he would have to pay the price
for harming humans (ie his actions would lead to his inactivation),
but he still did it because of the zeroth law.
(do robots have a soul? in that case you could say Giskard accepted
the zeroth law with his mind, but not with his soul :-) )
Harro de Jong H.C.d...@research.ptt.nl
Lance Shaw
>
>>(stuff deleted)
>He tried to accept the zeroth law (or had accepted it to a certain degree),
>enabling him to harm humans. But he didn't fully accept it , hance his
>inactivation. I think Giskard knew that he would have to pay the price
>for harming humans (ie his actions would lead to his inactivation),
>but he still did it because of the zeroth law.
>(do robots have a soul? in that case you could say Giskard accepted
>the zeroth law with his mind, but not with his soul :-) )
>
>Harro de Jong H.C.d...@research.ptt.nl
The problem was he couldn't accept it in his mind. His brain had the three
laws built in to it to the point it couldn't function without them. Any
violation would cause him to cease to function.
They had discussed that because Daniel was designed to be more human like he
was much closer to be able to accept it. Even though they realize that there
should be a zeroth law neither one of them could really accept it. Daniel
struggles with the first law getting in the way all the rest of his exsistence.
Graham Mark Davison
I dunno, but it seems to me like Giskard _did_ accept the zeroth law.
Read the last page of the book. He says something along the lines of "I'm
not sure if I have doomed humanity or made it stronger." This uncertainty
is what "killed" him, IMHO.
Keith Law
I agree 100%. The impression I had when I read the book was that he did every-
thing he could to accept the law, and acted in accordance with it knowing that
his action was correct BUT could kill him for his own non-acceptance.
It turned out he made the ultimate sacrifice.