RISKS@SRI-CSL.ARPA (RISKS FORUM, Peter G. Neumann, Coordinator) (01/10/86)
RISKS-LIST: RISKS-FORUM Digest, Friday, 10 Jan 1986 Volume 1 : Issue 38
FORUM ON RISKS TO THE PUBLIC IN COMPUTER SYSTEMS
sponsored by the ACM, Peter G. Neumann, moderator
Contents:
Ad-hominem SDI discussion (Mike McLaughlin [and Peter Neumann])
Men in the loop (Martin J. Moore)
Failure probabilities in decision chains (Jim Miller) [also in SOFT-ENG]
Testing SDI (Karl Kluge, Robert Goldman)
Summing Up on SDI (Jim McGrath)
NOTE: More Lin-McGrath discussions were submitted to RISKS, but are not
included here. They appear in ARMS-D, and are also available in
SRI-CSL:<RISKS>RISKS-1.38x.
The RISKS Forum is moderated. Contributions should be relevant, sound, in good
taste, objective, coherent, concise, nonrepetitious. Diversity is welcome.
(Contributions to RISKS@SRI-CSL.ARPA, Requests to RISKS-Request@SRI-CSL.ARPA.)
(Back issues Vol 1 Issue n stored in SRI-CSL:<RISKS>RISKS-1.n.)
----------------------------------------------------------------------
Date: Wed, 8 Jan 86 05:48:49 est
From: mikemcl@nrl-csr (Mike McLaughlin)
To: risks@sri-csl.ARPA
Subject: Ad-hominem SDI discussion
I note with dismay the ad-hominem phraseology that is appearing in the
SDI discussions. It is understandable but unacceptable. The issue is
too important. I request that the participants review their inputs
prior to submission, and edit out phraseology that is not relevant to
the issue. It is not fair to expect the forum's moderator to be a
censor. - Mike McLaughlin [Thanks. PGN]
[I have received various complaints about some of the recent SDI
verbiage -- its quality, accuracy, relevance to RISKS, what right has
RISKS to distribute SRI discussion when it should be in ARMS-D, etc.
In general, I have to hope that our contributors exert some good sense.
Some of the nit-picking should clearly be resolved privately. In some
cases I might request two antagonists get together and write a single
position statement to which they both agree -- but that seldom works.
So, at this point I would like to elevate the quality of SDI
discussion -- but not to stifle meaningful discussion that is really
RISKS-related. SDI remains one of the most important issues
confronting us, and open <but informed> discussion of the computer
risks therein is clearly vital.
This issue omits a bunch of further McGrath-Lin discussions, but
I point out above they can be found in ARMS-D or in the <RISKS>
directory. PGN]
----------------------------------------------------------------------
Date: 0 0 00:00:00 CDT
From: "MARTIN J. MOORE" <mooremj@eglin-vax>
Subject: Men in the loop
To: "risks" <risks@sri-csl>
Talking about men in the loop for SDI:
> From: scott@rochester.arpa
> I find it remarkable that this suggestion is taken seriously. Time scales
> simply do not permit it.
There are at least several minutes in which to respond. While this is
certainly too short for any chain-of-command processing, I believe (for
reasons which are detailed below) that it is plenty of time for a trained
individual, responsible for a single battlestation (or a jointly commanded
small group of battlestations) to respond.
> As has been pointed out by previous contributors, human intervention during
> boost phase is out of the question, since it's all over in a minute or two.
For SLBMs, or missiles aimed at Europe from the western Soviet Union, I'll
grant the point. But for land-launched missiles aimed at the US, I estimate
at least a five-minute boost. If this is wrong, someone please correct me!
> But even for later phases of missile flight, it is ridiculous to expect
> competent, cool-headed behavior from human operators who 1) have presumably
> been watching their consoles for years with no action, ...
No real action...but plenty of simulated action designed to train that
operator to make the correct response almost automatically. For several years
I had the privilege(?) of observing the training of Range Safety Officers at
Cape Canaveral. These officers monitor the launch of a missile in real time,
and if the missile endangers a populated area, they destroy it. Of course, if
that missile is a Space Shuttle, they would either destroy the Shuttle and its
flight crew...or they would fail to destroy it and possibly kill several
thousand bystanders. I would not like to have that job! One of the RSOs is
selected as the Training Officer on a rotating basis. He sets up launch
simulations, realistic in every detail, to train his fellow officers. In
addition to the missile going awry, various equipment failures, communication
outages, etc., are thrown at the officers during some simulations; other
simulations are completely nominal. Some of the simulations I've seen were
much, much rougher than the worst actual launch. On the very rare occasions
when a flight termination has been necessary, it has been handled cooly and
correctly.
I'll save someone the trouble of pointing out that there has never yet been
a termination of a manned launch. This is true. But I believe, based on
my observations, that even in this case the Range Safety Officers would make
the correct decision -- and make it fast.
> ... 2) have the fate of the world in their hands, ...
This will slow them down? I think it would make them most conscious of the
need for timely action. If you are suggesting that the responsibility would
be too great, and the operator would freeze up...I suppose it's possible, but
I doubt such a person would be in the job in the first place.
> ... and 3) have only a few minutes in which to make their decisions.
Ten minutes is a long time. In the early stages of a missile launch from
the Cape, the missile is perhaps ten seconds from impact in Cocoa Beach.
There, the men in the loop have worked very well; when they've had to throw
the switches, they've done it well before those seconds ticked away. No
missiles in Cocoa Beach yet! (Lest someone mention the famous incident early
in the Cape's history when a missile impacted in the Banana River just
offshore from a trailer park -- that missile did not carry a destruct
device...it was the last missile launched at the Cape not to carry one.)
Generally speaking, until the day when a computer system can be made 100%
reliable (not just 99.999...9%), I strongly believe that *any* system capable
of causing harm to human life must have a man in the loop. He may do no more
than press a YES/NO switch when the computer selects an action, but he *must*
be there. This transcends SDI or any other weapons system...it applies
generally!
These views are strictly my own and do not necessarily agree with those of my
employer or any of its customers.
Martin J. Moore
mooremj@eglin-vax.arpa
----------------------------------------------------------------------
Date: Wed 8 Jan 86 15:37:24-CST
From: Jim Miller <HI.JMILLER@MCC.ARPA>
Subject: Re: Failure probabilities in decision chains
To: wmartin@BRL.ARPA
cc: Soft-Eng@MIT-XX.ARPA, Risks@SRI-CSL.ARPA, walsh@ALMSA-1.ARPA
US-Mail: MCC, Human Interface Program, 9430 Research, Austin TX 78759
Phone: 512-834-3342
Well, if you assume that each decision has a 90% likelihood of being
correct (or, a 10% chance of being wrong), and that all of the five
decisions are independent of each other, and all that other
statistical stuff, then the probability that all five decisions will
be correct is .9^5, or .59. However, how you would go about
determining in a real-life situation what the actual probabilities of
those decisions being correct really are, or whether the decisions are
really independent, is beyond me. This sounds like one of those
statistics that people love to quote, regardless of whether it has any
validity.
Jim Miller
MCC Human Interface
------------------------------
Date: 7 Jan 1986 22:41-EST
From: Karl.Kluge@G.CS.CMU.EDU
To: Risks@sri-csl.arpa
Subject: Testing: Differences between SDI and other systems.
Message-Id: <505539701/kck@G.CS.CMU.EDU>
> Date: Mon 6 Jan 86 18:24:44-PST
> From: Jim McGrath <J.JPM@LOTS-A>
>
> You are, of course, correct. The problem is that your points could
> also be (and are) made about any complex weapons systems (or indeed,
> any complex system at all). It is NEVER possible to fully test ANY
> system until it is actually used in battle (and even then it can fail
> in future battles).
1) The consequences of failure of an SDI system are orders of magnitude
greater than the consequences of failure of a "normal" weapons system.
We damn well should be orders of magnitude more confident in it.
2) Which is really irrelevant, since the function of the SDI is to enhance
deterrence, not replace it. The SDI system doesn't have to work, it just
has to create reasonable worry that it might work in the mind of a potent-
ial attacker. This makes it different from most systems, which are built
to be used. If the SDI system ever has to be used then it has failed,
which means...
(* rest of message on ARMS-D. Karl *) [THANKS... PGN]
My opinion, of course, in no way reflects the opinion of anyone I'm
associated with. Karl
------------------------------
Date: 8 Jan 86 (Wed) 15:26:18 EST
From: Robert Goldman <rpg%brown.csnet@CSNET-RELAY.ARPA>
To: risks@sri-csl.arpa
Subject: Testing SDI
In RISKS-1.34 Jim McGrath discusses testing the SDI. A common objection to
the SDI is that it could not realistically be tested.
McGrath correctly points out that this is true of any modern weapon system.
I agree with this basic point, but of course, some weapons systems are
easier to test than others. For example, Mr. McGrath points out
it took WWI for the Germans to realize that the
lessons of the 1880's concerning rapid infantry fire (and thus the
rise of infantry over calvary) did not take artillery development
adequately into account
This is correct, but dodges the point that at that time, modern artillery,
rifles and machine guns HAD all been fired in anger. The almost universal
inability to profit from the experience was due to institutional failures,
rather than lack of raw data.
I need hardly point out that ICBMs and SLBMs have NOT been tested in wartime
conditions, and that we have reason to believe that there won't be a second
chance to correct any mistakes.
Mr. McGrath goes on to suggest that the SDI might be tested against meteor
storms. I question this for three reasons:
1. As I understand it, the particle-beam weapons of the SDI are not
intended to destroy warheads outright, but rather prevent them from reaching
their targets and detonating. How would one judge that a meteorite's fusing
and guidance mechanisms had been destroyed?
2. Meteorites are not human-made objects which are designed with an eye to
penetrating enemy defenses: they do not drop chaff, employ electronic
counter-measures, etc.
3. Meteorites have a wholly different flight pattern. As I understand it,
ICBMs have a boost phase, a cruising phase and a re-entry phase. Doesn't a
missile's detectability and vulnerability depend on which of these phases it
is in?
Robert Goldman
[I deleted a paragraph on what might happen "if the SDI accidentally
(or on purpose) shot down a Soviet reconnaissance satellite?" PGN]
------------------------------
Date: Thu 9 Jan 86 22:45:57-PST
From: Jim McGrath <J.JPM@Epic>
Subject: Summing Up on SDI
To: "risks@sri-csl"@Sushi
Reply-to: mcgrath%mit-oz@mit-mc.arpa
[Reminder: I have omitted a bunch of McGrath-Lin messages.
If you wish to read them, see aove. PGN]
From all these messages, I've come to two conclusions. First, that on
the whole, I tend to feel that SDI is a more complex system (mainly
because of its mission size) than existing ones, such as Aegis. But I
could be wrong (either way). And I do not believe that it is "orders
of magnitude" more complex than existing systems. Thus it would seem
that it could be built to existing system performance standards.
The second question is whether these standards are "adequate," and if
not can we improve upon them? It is clear that existing systems are
not being tested as fully as possible (Herb Lin's report on the Aegis
tests makes them appear to be a joke). I've already pointed out that
substantial testing of mid-course and terminal phases, far more
extensive testing than any of our other systems have received, can be
carried out. Thus, provided we have a proper commitment, even our
current testing technology can be better applied (with better results)
than we have done up until now.
To a large extent the standard you require depends upon your
definition of the mission of the system. Clearly the system was never
designed to make OUR nuclear weapons obsolete (just the Soviet's). So
under any mission we would probably retain a force sufficient to
destroy the USSR, and so can always fall back on MAD. Any reasonable
performance level would protect our weapons to a significant extent
(and if not, you can always keep a sub force). So the real question
is whether it can protect cities.
I would tend to doubt it, under a full and unimpeded Soviet attack.
But there are many scenarios (from accidental launch to a limited
(decapitation or counterforce) strike to a second strike (the first
perhaps going to Europe and/or China)) where it quite possibly could.
In any event, I am certainly not sure enough to either commit to SDI
deployment nor to terminate research. Since all SDI is at the moment
is research, I have no problem with the existing program. Still,
knowing how programs have a tendency to outlive their usefulness, I
think strong scrutiny is appropriate. But not mindless opposition.
Jim
------------------------------
End of RISKS-FORUM Digest
************************
-------