ARMS-D-Request@XX.LCS.MIT.EDU (Moderator) (09/24/86)
Arms-Discussion Digest Wednesday, September 24, 1986 3:51PM
Volume 7, Issue 19
Today's Topics:
Administrivia
Towards a definition of "autonomous" weapons
SDI
Nova/Frontline program tonight (Sept.23)
indiscriminate weapons
re: autonomous weapons
Correction
Autonomous Weapons
Comment on an editorial comment
----------------------------------------------------------------------
Date: Wed, 24 Sep 1986 08:45 EDT
From: LIN@XX.LCS.MIT.EDU
Subject: Administrivia
==>> Someone please help:
Date: Wednesday, 24 September 1986 05:38-EDT
From: MAILER-DAEMON%thales at Sun.COM (Mail Delivery Subsystem)
To: <ARMS-D-Request>
Re: Returned mail: Can't create output
----- Transcript of session follows -----
550 /usr/titan/toma/inbox... Can't create output
------------------------------
Date: Tue, 23 Sep 86 18:16:46 PDT
From: Clifford Johnson <GA.CJJ@Forsythe.Stanford.Edu>
Subject: Towards a definition of "autonomous" weapons
> I don't understand what it means for a weapon to "take a decision".
> Clearly you don't intend to include a depth charge set to explode at a
> certain depth, and yet a depth charge could "decide" to explode at 100
> feet given certain input.
My concept of "decision" embraces *all* manner of conditional
executions, delimited expressly by the customary law that recognizes
certain "special" decisions require human participation. I know of
no precedent suggesting that "aiming" at a properly comprehended
target requires such discretion, which is the logical mechanism that
eliminates depth charges from my definition.
> What should be the role of the human being in
> war? I would think the most basic function is to decide what targets
> should be attacked.
The question I'm concerned with is "What should/could be the role of
law in precluding the unconscionable automation of war?" With the
basic thrust on target selection I agree, but to include it I would
amend my definition by referencing "fatal consequences"
(generalization of "targets"):
AN "AUTONOMOUS WEAPON" IS DEFINED TO BE ANY WEAPONS SYSTEM WHICH
IS DE FACTO PREPROGRAMMED TO TAKE DECISIONS WHICH, DUE TO THEIR
POTENTIALLY FATAL CONSEQUENCES, UNDER THE LAW OF NATIONS REQUIRE
THE EXERCISE OF HUMAN DISCRETION AT THE TIME THEY ARE TAKEN.
N.B. The "de facto" is important to exclude the excuse that a
human might "override" the weapons' decision, when, in practice,
he/she wouldn't be competent to do so.
> But then I note what a recent contributor said -- MINES are autonomous
> weapons, and I don't want to get rid of mines either, since I regard
> mines as a defensive weapon par excellence. Do I add mobility to the
> definition? I don't know.
It seems such confusions are inevitable when seeking a
primarily technological definition of "autonomous" weapons. My
"trick" is to reverse the analysis, placing the burden upon the
human judgment that some kinds of decisions are unconsionably
automated. The nature of and context of the particular decision
becomes the focus. This isn't a cop-out, but would build upon
established military/international custom. As I have intimated, it
would seek precedents where particular decisions would "initiate" or
"escalate" a conflict, or be beyond the ordinary authority of the
officer operating the system.
To: ARMS-D@XX.LCS.MIT.EDU
------------------------------
Date: Tue, 23 Sep 1986 22:48 EDT
From: LIN@XX.LCS.MIT.EDU
Subject: SDI
DA -- ignore this note, unless you want to respond. I'm CC-ing you
just for your information, and I respect your wish to not engage a new
argument.
From: Dale.Amon at h.cs.cmu.edu
Well we may not have the equations for anti-missile defense down yet,
but my engineering sense tells me it can be done, and that it is worth
doing.
How are we to judge your engineering sense? In particular, how can we
tell that you are not confusing feasibility with desirability?
I can't accept a world in which defense means threatening to immolate
millions of Russian men, women and children who have little say in the
aggressive policies of 'their' government. There has got to be a better
way to protect our right to be left alone, and it is worth trying to
make it real.
Again, the desirability of something does not make it feasible, as any
proponent of perpetual motion or immortality will tell you.
I would like to mention several hopeful signs towards feasibility of
SDI.
1) The phase conjugate laser techniques appear to have the
potential to track a target, correct for beam diffraction
and to focus the full energy of the main laser on the target
without requiring computing. The weapon need not even be
physically pointed since the phase conjugate effect can
generate an off-axis beam.
Phase conjugation does NOT correct for diffraction -- beams continue
to spread. That is a fundamental physical law. PC does allow
correction for atmospheric disturbances caused by density
fluctuations.
3) The computing problem has been discussed by the Eastport
panel and found to be tractable if modern techniques of
modularity are applied rather than the monolithic and
certain to fail approach castigated by Parnas.The Eastport
panel had doubts that the DOD and it's contractors could
adjust to these techniques, but I would say that a great
deal of the Software Engineering ideas that have gone into
ADA are moves in the correct direction.
The Eastport panel did not "find to be tractable" the computing
problem. They merely asserted it. It is true that most people agree
that IF a solution is to be found, it will require revolutions in DoD
software contracting.
------------------------------
Date: 1986 September 23 21:47:52 PST (=GMT-8hr)
From: Robert Elton Maas <REM%IMSSS@SU-AI.ARPA>
Subject:Nova/Frontline program tonight (Sept.23)
Tonight there was a 2-hour Nova/Frontline "special" program on TV
channel 9 (KQED, SF) dealing with SDI (Strategic Defense Initiative).
It covered a lot of good points, including some not previously covered
on ARMS-D, but I have three specific remarks.
My major disappointment was the short discussion of software to
implement the overall battle operation of SDI. The narrator said
there's been a lot of debate whether such a program could ever be
written, and made to work absolutely correctly the very first it is
ever used, and presumably the only time. But it failed to give any of
the arguments pro and con, it just dropped it at that and went onto
another subject. I think the total time for software was about two minutes.
One item completely left out was my argument about the transition
period from here to a full system being unstable, first-strike
optimal. The narrator said that SDI may be counterproductive (not his
word, but synomym) if it is anything less than a massive 100%
effective system. But the narrator failed to point out that even if we
embark on such a system there must be a transition period during which
we have a partial system which he has already said would be
destabilizing and undesirable. He seems to say it's all or nothing,
but fails to point out that to get from here to there we must pass
through some middle ground.
One of Reagan's later speeches (after the original announcement) to
some small group gave a really good description of where SDI is
heading (in Reagan's vision). We want to render thermonuclear weapons
impotent and obsolete, STARTING WITH ICBMS. (emphasis mine) Note that
he effectively answers or charges that even if ICBMs are stopped it
leaves all the other modes of delivery unstopped. He wants to stop
ICBMs now, and later move ahead against other delivery methods. That
seems to me quite reasonable (assuming the whole idea works at all and
isn't destabilizing of course). I think it would be a straw man for us
to say that Reagan plans to stop ICBMs only. We should admit that
stopping ICBMs is only stage one of Reagan's plan.
Now some additional ideas provoked by my watching this program...
Perhaps the way SDI should work is we admit it can stop only ICBMs in
the first major stage of implementation, and use that as an incentive
for the USSR to shift much of its ICBM stockpile away from land-based
ICBMs toward the other modes (planes, submarines, cruise missiles). We
could perhaps render ICBMs not exactly impotent but less
cost-effective than other delivery modes, and negotiate a treaty
whereby we phase out both multiple warhead missiles (any
basing/delivery mode) and most land-based ICBMs while decreasing the
total number of warheads although perhaps not the total number of
launch vehicles. Perhaps that way of using SDI could be stabilizing
instead of destabilizing. (rebuttal to this brand-new idea strongly encouraged)
Also I was impressed by Teller's arguments against some forms of SDI
proposed by Reagan et al that involve thermonuclear warheads in space
to run X-ray lasers. (I forgot the details, sigh.)
Also, the rapid-refire railgun demonstrated near the end of the
program would seem to have civilian applications in drilling tunnels
through montains. Wheras a laser vaporizes the rock, wasting a lot of
energy on heating it to thousands of degrees, the railgun would merely
shatter the rock and knock the pieces loose until they miagrated out
the entrance or pulverise the rock to dust and blow the dust out the
entrance due to the small amount of rock actually vaporized that
created a net wind outward from the hit point. It would thus seem
drilling by railgun would be more efficient than drilling by laser
even if the basic efficiencies were comparable. On the other hand,
this research with nicely formed cubes of metal would NOT seem to be
useful for moving large quantities of raw materials surface of the
moon due to the careful machining of the carefully chosen metal
pellets. (rebuttal or discussion of peaceful aspects/spinoff of
SDI-funded research, railgun or other?)
------------------------------
Date: 24 Sep 86 11:20:26 bst
From: S.WILSON%UK.AC.EDINBURGH@ac.uk
Subject: indiscriminate weapons
Volume 7, Issue 16
Date: Mon, 22 Sep 86 18:43:21 PDT
From: Clifford Johnson <GA.CJJ@Forsythe.Stanford.Edu>
Subject: Towards an effective defintion of "autonomous" weapons
There's great difficulty in defining "autonomous weapons" so as to
separate some element that seems intuitively "horrible" about
robot-decided death...
The problem is, of course, that many presently "acceptable" weapons
already indiscrminately-discriminate targets,... Is there an element
exclusive to computerized weapons that is meaningful?
I don't have an answer, but feel the answer must be yes... First,
weapon control systems that may automatically target-select among
options based upon a utility function (point score) that weighs killing
people against destroying hardware would seem especially unconscionable.
I don't remember what the current situation with regard to the deployment
of the neutron bomb (sorry, enhanced radiation weapon) is, but doesn't that
'automatically' discriminate against human beings rather than hardware?
Sam Wilson, ERCC, University of Edinburgh, Scotland
------------------------------
Date: Wed, 24 Sep 86 10:40:02 EDT
From: Bruce Nevin <bnevin@cch.bbn.com>
Subject: re: autonomous weapons
Mines are also offensive weapons, as demonstrated not long ago
in Nicaragua. Part of the inappropriateness of mines, antipersonnel
weapons, etc., is precisely that they do .not. select targets with
any real discrimination.
A related issue: I was a conscientious objector because joining a
military organization would contract me to obey orders given by
people not in touch with their consequences, and to give orders
to whose consequences I would not be exposed, yet whose intent
would be destruction of life and livelihood. The ability to
discriminate right use of means was divorced from the deployment
of means.
On a deeper level, the dilemma of how to introduce intelligent
discrimination into the conduct of war is that war is not an
intelligent thing to do. Put another way, for whatever problems
war is intended to solve there are always more intelligent solutions,
solutions that make more appropriate use of our God-given powers of
discrimination and choice.
Bruce Nevin
bn@cch.bbn.com
(This is my own personal communication, and in no way expresses or
implies anything about the opinions of my employer, its clients, etc.)
------------------------------
Date: Wed, 24 Sep 86 08:51 PDT
From: "Morton Jim"@LLL-MFE.ARPA
Subject: Correction
> ------------------------------
>
> Date: 23 Sep 86 16:26-EST
> From: sam mccracken <oth104%BOSTONU.bitnet@WISCVM.arpa>
> Subject: Re: TV news erratum: LLL -> Sandia Labs
>
> -----
> Sandia does highly classified research and is next next to a substantial
> nuclear warhead depot.
>
> ------------------------------
Half wrong. Half right. Sandia does do classified research
but is not near a nuclear warhead depot. It is next to
Lawrence Livermore National Lab, which is not a weapon depot.
Jim Morton
lll-mfe
------------------------------
Date: Wednesday, 24 September 1986 11:01-EDT
From: Henry Thompson <hthompso%eusip.edinburgh.ac.uk at Cs.Ucl.AC.UK>
To: ARMS-D
Re: Autonomous Weapons
In our briefing of the British Ministry of Defense on (among other things)
computational flaws in SDI, we at Edinburgh Computing and Social
Responsibility found it helpful to emphasise a distinction between fully
automated decision-making systems which functioned to prosecute an existing
war, and such systems which could function to start one. As several
postings have observed, the former already exist in a number of forms,
but it is the latter (exemplified by SDI) which are particularly worrying.
Perhaps we could start by trying for a more restricted definition focussed
on autonomous systems with war-initiating potential.
I also enter a qualified disagreement with Lin's disinclination to call what
a depth-fused mine does a decision. A longer posting on the essential
nature of autonomous systems will follow at some point.
Henry Thompson
Edinburgh CSR
------------------------------
From: Eugene miya <eugene@ames-aurora.arpa>
Date: 24 Sep 1986 1012-PDT (Wednesday)
Subject: Comment on an editorial comment
> [Many say there is every reason to believe we would simply export our
> reluctance to build peace (homeostasis in higher-level unity). Others,
> Carl Sagan and various astronauts for example, speak eloquently of the
> transformative power of seeing Earth whole from space. How can the foot
> argue with the hand?]
I think the above is from Herb.
[Nope. Not me. It might have been sent to me as moderator, and I
could have forwarded it, though. Herb]
Some physicists and generals argue that we should demonstrate the
"amesome destructive power" of nuclear weapons by publicly exploding
one every few years, for politicans, and that the amount of released
radiation (has to be a surface shot) would be worth avoiding nuclear war.
I personally think it will have an opposite effect for some. Foot stepping
on nail.
From the Rock of Ages Home for Retired Hackers:
--eugene miya
NASA Ames Research Center
eugene@ames-aurora.ARPA
"You trust the `reply' command with all those different mailers out there?"
{hplabs,hao,nike,ihnp4,decwrl,allegra,tektronix,menlo70}!ames!aurora!eugene
------------------------------
End of Arms-Discussion Digest
*****************************