ecl@mtgzz.UUCP (e.c.leeper) (02/03/86)
This is being reposted from mod.risks. I have given a summary of loss-of-life
incidents only at the beginning for some perspective on the Challenger
accident. By the reasoning that many people/Luddites are giving we should be
banning pacemakers, microwaves, anti-theft devices, weather buoys, all
computers and software, robots (so much for unmanned missions!), autopilots,
medical computers, and of course cars and cigarettes. Needless to say,
I think this reasoning is a load of horse puckey!
Evelyn C. Leeper
...ihnp4!mtgzz!leeper
*****************************************************************************
*[Summary: loss of life incidents only *
*-------------------------- SYSTEM + ENVIRONMENT ---------------------------*
*!S Arthritis-therapy microwaves set pacemaker to 214, killed patient *
*!S Retail-store anti-theft device reset pacemaker, man died. *
*!$ Deaths of 3 lobstermen in storm not predicted by NWS; unrepaired buoy *
*------------------------------- SOFTWARE ----------------------------------*
*!$ 1983 Col. River flood, faulty data/model?; 6 deaths *
*-------------------------- HARDWARE/SOFTWARE ------------------------------*
*! Michigan man killed by robotic die-casting machinery *
*! Japanese mechanic killed by malfunctioning Kawasaki robot *
*!? Chinese computer builder electrocuted by his smart computer. *
* (This is from the WEEKLY WORLD NEWS, a trash rag if ever there was one*
* so I suspect it's not even true. -ecl) *
*-------- COMPUTER AS CATALYST, HUMAN FRAILTIES, OR UNKNOWN CAUSES ---------*
*!!$ KAL 007 shot down, killing 269; autopilot left on wrong frequency *
*!!$ Air New Zealand crashed; computer error detected/fixed, pilots not told*
*! Woman killed daughter, tried to kill son and self; "computer error" *
* blamed for false report of their all having an incurable disease *
*!$$ Shuttle Challenger explosion, 7 killed. Cause not yet known. *
*****************************************************************************
SOME COMPUTER-RELATED DISASTERS AND OTHER EGREGIOUS HORRORS
Compiled by Peter G. Neumann (31 January 1986)
The following list is drawn largely from back issues of ACM SIGSOFT Software
Engineering Notes [SEN], references to which are cited as (SEN vol no), where
vol 11 = 1986. Some incidents are well documented, others need further study.
Please send corrections/additions+refs to PGNeumann, SRI International, BN168,
Menlo Park CA 94025, phone 415-859-2375, Neumann@SRI-CSL.ARPA.
Legend: ! = Loss of Life; * = Potentially Life-Critical;
$ = Loss of Money/Equipment; S = Security/Privacy/Integrity Flaw
-------------------------- SYSTEM + ENVIRONMENT ------------------------------
!S Arthritis-therapy microwaves set pacemaker to 214, killed patient (SEN 5 1)
!S Retail-store anti-theft device reset pacemaker, man died. (SEN 10 2, 11 1)
*S Auto speed changed by interference from CB transmitter (SEN 11 1)
*S Failed heart-shocking devices due to faulty battery packs (SEN 10 3)
*$ Three Mile Island PA, now recognized as very close to meltdown (SEN 4 2)
*$ Crystal River FL reactor (Feb 1980) (Science 207 3/28/80 1445-48, SEN 10 3)
!$ Deaths of 3 lobstermen in storm not predicted by National Weather Service --
3 mos unrepaired weather buoy; $1.25M award (SEN 10 5) [NY Times 13 Aug 85]
** SAC/NORAD: 50 false alerts in 1979 (SEN 5 3), incl. a simulated attack whose
outputs accidentally triggered a live scramble [9 Nov 1979] (SEN 5 3);
** BMEWS at Thule detected rising moon as incoming missiles [5 Oct 1960]
(SEN 8 3). See E.C. Berkeley, The Computer Revolution, pp. 175-177, 1962.
** Returning space junk detected as missiles. Daniel Ford, The Button, p. 85
** WWMCCS false alarms triggered scrams [3-6 Jun 1980] (SEN 5 3, Ford pp 78-84)
** DSP East satellite sensors overloaded by Siberian gas-field fire (Ford p 62)
** 747SP (China Air.) autopilot tried to hold at 41,000 ft after engine failed,
other engines died in stall, plane lost 32,000 feet [19 Feb 85] (SEN 10 2)
** 767 (UA 310 to Denver) four minutes without engines [August 1983] (SEN 8 5)
* F18 missile thrust while clamped, plane lost 20,000 feet (SEN 8 5)
* Mercury astronauts forced into manual reentry (SEN 8 3)
* Cosmic rays halve shuttle Challenger comm for 14 hours [8 Oct 84] (SEN 10 1)
* Frigate George Philip fired missile in opposite direction (SEN 8 5)
$ Hurricane Gloria in NY closes Midwest Stock Exchange (SEN 11 1)
$S Debit card copying easy despite encryption (DC Metro, SF BART, etc.)
$S Microwave phone calls easily interceptable; portable phones spoofable
$S Sputnik frequencies triggered garage-door openers
------------------------------- SOFTWARE ------------------------------------
!$ 1983 Colorado River flood, faulty data/model? Too much water held back
prior to spring thaws; 6 deaths, $ millions damage [NY Times 4 Jul 1983]
*$ Mariner 1: Atlas booster launch failure DO 100 I=1.10 (not 1,10) (SEN 8 5)
*$ Mariner 18: aborted due to missing NOT in program (SEN 5 2)
*$ F18: plane crashed due to missing exception condition, pilot OK (SEN 6 2)
*$ F14 off aircraft carrier into North Sea; due to software? (SEN 8 3)
*$ F14 lost to uncontrollable spin, traced to tactical software (SEN 9 5)
*$ El Dorado brake computer bug caused recall of all El Dorados (SEN 4 4)
$$ Viking had a misaligned antenna due to a faulty code patch (SEN 9 5)
$$ First Space Shuttle backup launch-computer synch problem (SEN 6 5 [Garman])
* Second Space Shuttle operational simulation: tight loop upon cancellation of
an attempted abort; required manual override (SEN 7 1)
* Second Shuttle simulation: bug found in jettisoning an SRB (SEN 8 3)
*$ Delays of two Discovery shuttle launches due to backup computer outage
[most recently 25 Aug 85] (SEN 10 5) [NY Times 26 August 1985]
* Shuttle STS-6 bugs in live Dual Mission software prevented aborts (SEN 11 1)
* Gemini V 100mi landing err, prog ignored orbital motion around sun (SEN 9 1)
* F16 simulation: plane flipped over whenever it crossed equator (SEN 5 2)
* F16 simulation: upside-down F16 deadlock over left vs. right roll (SEN 9 5)
* Nuclear reactor design: bug in Shock II model/program (SEN 4 2)
* Reactor overheating, low-oil indicator; two-fault coincidence (SEN 8 5)
* SF BART train doors sometimes open on long legs between stations (SEN 8 5)
$ IRS reprogramming delays; interest paid on over 1,150,000 refunds (SEN 10 3)
$ $32 BILLION overdraft at Bank of New York (prog counter overflow) (SEN 11 1)
*S Numerous system intrusions and penetrations; implanted Trojan horses; 414s;
intrusions to TRW Credit Information Service, British Telecom's Prestel,
Santa Clara prison data system (inmate altered release date) (SEN 10 1).
Computerized time-bomb inserted by programmer (for extortion?) (10 3)
PC Graphics program Trojan horse (ArfArf) wiped out users' files (SEN 10 5)
*$ Union Carbide leak (135 injuries) exacerbated by program not handling
aldicarb oxime plus operator error [NY Times 14 and 24 Aug 85] (SEN 10 5)
* Multipatient monitoring system recalled; mixed up patients (SEN 11 1)
* Pacemaker locked up when being adjusted by doctor (SEN 11 1)
* Diagnostic lab instrument misprogrammed (SEN 11 1)
S Chernenko at MOSKVAX: network mail hoax [1 April 1984] (SEN 9 4)
S VMS tape backup SW trashed disc directories dumped in image mode (SEN 8 5)
*$ C&P computer crashes 44,000 DC phones (SEN 1 1)
$ 1979 AT&T program bug downed phone service to Greece for months (SEN 10 3)
$ Demo NatComm thank-you mailing mistitled supporters [NY Times, 16 Dec 1984]
$ Slow responses in Bankwire interface SW resulted in double posting of tens
of $millions, with interest losses (SEN 10 5)
$ Program bug permitted auto-teller overdrafts in Washington State (SEN 10 3)
- Quebec election prediction gave loser big win [1981] (SEN 10 2, p. 25-26)
- Other election problems including mid-stream corrections (HW/SW) (SEN 10 3)
- SW vendor rigs elections? (David Burnham, NY Times front page, 29 July 1985)
- Alaskan DMV program bug jails driver [Computerworld 15 Apr 85] (SEN 10 3)
- Vancouver Stock Index lost 574 points over 22 months -- roundoff (SEN 9 1)
- Gobbling of legitimate automatic teller cards (SEN 9 2, another SEN 10 5)
-------------------------- HARDWARE/SOFTWARE ---------------------------------
! Michigan man killed by robotic die-casting machinery (SEN 10 2, 11 1)
! Japanese mechanic killed by malfunctioning Kawasaki robot (SEN 10 1, 10 3)
[Electronic Engineering Times, 21 December 1981]
! Chinese computer builder electrocuted by his smart computer. (WWN headline:
"Jealous Computer Zaps its Creator" after he built newer one!!) (SEN 10 1)
* FAA Air Traffic Control: many computer system outages (e.g., SEN 5 3)
* ARPANET ground to a complete halt [27 Oct 1980] (SEN 6 1 [Rosen])
*$ Ford Mark VII wiring fires: flaw in computerized air suspension (SEN 10 3)
$S Harrah's $1.7 Million payoff scam -- Trojan horse chip (SEN 8 5)
$ Great Northeast power blackout due to threshold set-too-low being exceeded
$ Power blackout of 10 Western states, propagated error [2 Oct 1984] (SEN 9 5)
$ NY Stock Exch. halted for 41 minutes; drum channel errors killed primary
and backup computer systems [24 Feb 72]
- SF Muni Metro: Ghost Train reappeared, forcing manual operation (SEN 8 3)
*$ Computer-controlled turntable for huge set ground "Grind" to halt (SEN 10 2)
*$ 8080 control system dropped bits and boulders from 80 ft conveyor (SEN 10 2)
S 1984 Rose Bowl hoax, scoreboard takeover ("Cal Tech vs. MIT") (SEN 9 2)
-------- COMPUTER AS CATALYST, HUMAN FRAILTIES, OR UNKNOWN CAUSES -------------
!!$ Korean Airlines 007 shot down [1 Sept 1983], killing 269; autopilot left on
HDG 246 rather than INERTIAL NAV? (NYReview 25 Apr 85, SEN 9 1, SEN 10 3)
!!$ Air New Zealand crashed into mountain [28 Nov 1979]; computer course data
error had been detected and fixed, but pilots not informed (SEN 6 3 & 6 5)
! Woman killed daughter, tried to kill son and self; "computer error" blamed
for false report of their all having an incurable disease (SEN 10 3)
* Unarmed Soviet missile crashed in Finland. Wrong flight path? (SEN 10 2)
*$ South Pacific Airlines, 200 aboard, 500 mi off course near USSR [6 Oct 1984]
*S San Francisco Public Defender's database accessible to police (SEN 10 2)
* Various cases of false arrest due to computer database use (SEN 10 3, 11 1)
* Avionics failed, design used digitized copier-distorted curves (SEN 10 5)
$ .5M transaction became $500M, due to "000" convention; $200M lost (SEN 10 3)
$ Possible fraud on reinsurance -- message time stamp faked??? (SEN 10 5)
$ N-step reinsurance cycle; SW checked only N=1 and 2 (SEN 10 5)
* FAA Air Traffic Control: many near-misses not reported (SEN 10 3)
!$$ Shuttle Challenger explosion, 7 killed. Cause not yet known. [29 Jan 86]
---------------- ILLUSTRATIVE OF POTENTIAL FUTURE PROBLEMS -------------------
*S Many known/past security flaws in computer operating systems and application
programs. Discovery of new flaws running way ahead of their elimination.
* Expert systems in critical environments: unpredictability if (unknowingly)
outside of range of competence, e.g., incompleteness of rule base. StarWars
$S Embezzlements, e.g., Muhammed Ali swindle [$23.2 Million], Security Pacific
[$10.2 Million], City National Beverly Hills CA [$1.1 Million, 23 Mar 1979]
[These were only marginally computer-related, but suggestive. Others
are known, but not publically acknowledged.]
---------------------- REFUTATION OF EARLIER REPORT --------------------------
* "Exocet missile not on expected-missile list, detected as friend" (SEN 8 3)
[see Sheffield sinking, reported in New Scientist 97, p. 353, 2/10/83];
Officially denied by British Minister of Defence Peter Blaker
[New Scientist, vol 97, page 502, 24 Feb 83]. Rather, sinking abetted by
defensive equipment being turned off to reduce communication interference?