[net.columbia] Technology Failures

ecl@mtgzz.UUCP (e.c.leeper) (02/03/86)

This is being reposted from mod.risks.  I have given a summary of loss-of-life
incidents only at the beginning for some perspective on the Challenger
accident.  By the reasoning that many people/Luddites are giving we should be
banning pacemakers, microwaves, anti-theft devices, weather buoys, all
computers and software, robots (so much for unmanned missions!), autopilots,
medical computers, and of course cars and cigarettes.  Needless to say,
I think this reasoning is a load of horse puckey!

						Evelyn C. Leeper
						...ihnp4!mtgzz!leeper

*****************************************************************************
*[Summary: loss of life incidents only                                      *
*-------------------------- SYSTEM + ENVIRONMENT ---------------------------*
*!S Arthritis-therapy microwaves set pacemaker to 214, killed patient       *
*!S Retail-store anti-theft device reset pacemaker, man died.               *
*!$ Deaths of 3 lobstermen in storm not predicted by NWS; unrepaired buoy   *
*------------------------------- SOFTWARE ----------------------------------*
*!$ 1983 Col. River flood, faulty data/model?; 6 deaths                     *
*-------------------------- HARDWARE/SOFTWARE ------------------------------*
*!  Michigan man killed by robotic die-casting machinery                    *
*!  Japanese mechanic killed by malfunctioning Kawasaki robot               *
*!? Chinese computer builder electrocuted by his smart computer.            *
*     (This is from the WEEKLY WORLD NEWS, a trash rag if ever there was one*
*     so I suspect it's not even true. -ecl)                                *
*-------- COMPUTER AS CATALYST, HUMAN FRAILTIES, OR UNKNOWN CAUSES ---------*
*!!$ KAL 007 shot down, killing 269; autopilot left on wrong frequency      *
*!!$ Air New Zealand crashed; computer error detected/fixed, pilots not told*
*!  Woman killed daughter, tried to kill son and self; "computer error"     *
*    blamed for false report of their all having an incurable disease       *
*!$$ Shuttle Challenger explosion, 7 killed.  Cause not yet known.          *
*****************************************************************************

       SOME COMPUTER-RELATED DISASTERS AND OTHER EGREGIOUS HORRORS
             Compiled by Peter G. Neumann (31 January 1986)

The following list is drawn largely from back issues of ACM SIGSOFT Software
Engineering Notes [SEN], references to which are cited as (SEN vol no), where 
vol 11 = 1986.  Some incidents are well documented, others need further study.
Please send corrections/additions+refs to PGNeumann, SRI International, BN168, 
Menlo Park CA 94025, phone 415-859-2375, Neumann@SRI-CSL.ARPA.

Legend: ! = Loss of Life; * = Potentially Life-Critical; 
        $ = Loss of Money/Equipment; S = Security/Privacy/Integrity Flaw

-------------------------- SYSTEM + ENVIRONMENT ------------------------------
!S Arthritis-therapy microwaves set pacemaker to 214, killed patient (SEN 5 1)
!S Retail-store anti-theft device reset pacemaker, man died.  (SEN 10 2, 11 1)
*S Auto speed changed by interference from CB transmitter (SEN 11 1)
*S Failed heart-shocking devices due to faulty battery packs (SEN 10 3)
*$ Three Mile Island PA, now recognized as very close to meltdown (SEN 4 2)
*$ Crystal River FL reactor (Feb 1980) (Science 207 3/28/80 1445-48, SEN 10 3)
!$ Deaths of 3 lobstermen in storm not predicted by National Weather Service --
    3 mos unrepaired weather buoy; $1.25M award (SEN 10 5) [NY Times 13 Aug 85]
** SAC/NORAD: 50 false alerts in 1979 (SEN 5 3), incl. a simulated attack whose
    outputs accidentally triggered a live scramble [9 Nov 1979] (SEN 5 3);
** BMEWS at Thule detected rising moon as incoming missiles [5 Oct 1960] 
    (SEN 8 3).  See E.C. Berkeley, The Computer Revolution, pp. 175-177, 1962.
** Returning space junk detected as missiles.  Daniel Ford, The Button, p. 85
** WWMCCS false alarms triggered scrams [3-6 Jun 1980] (SEN 5 3, Ford pp 78-84)
** DSP East satellite sensors overloaded by Siberian gas-field fire (Ford p 62)
** 747SP (China Air.) autopilot tried to hold at 41,000 ft after engine failed,
    other engines died in stall, plane lost 32,000 feet [19 Feb 85] (SEN 10 2)
** 767 (UA 310 to Denver) four minutes without engines [August 1983] (SEN 8 5)
*  F18 missile thrust while clamped, plane lost 20,000 feet (SEN 8 5)	
*  Mercury astronauts forced into manual reentry (SEN 8 3)
*  Cosmic rays halve shuttle Challenger comm for 14 hours [8 Oct 84] (SEN 10 1)
*  Frigate George Philip fired missile in opposite direction (SEN 8 5)
$  Hurricane Gloria in NY closes Midwest Stock Exchange (SEN 11 1)
$S Debit card copying easy despite encryption (DC Metro, SF BART, etc.)
$S Microwave phone calls easily interceptable; portable phones spoofable
$S Sputnik frequencies triggered garage-door openers

------------------------------- SOFTWARE ------------------------------------	
!$ 1983 Colorado River flood, faulty data/model? Too much water held back
   prior to spring thaws; 6 deaths, $ millions damage [NY Times 4 Jul 1983]
*$ Mariner 1: Atlas booster launch failure DO 100 I=1.10 (not 1,10) (SEN 8 5)
*$ Mariner 18: aborted due to missing NOT in program (SEN 5 2)
*$ F18: plane crashed due to missing exception condition, pilot OK (SEN 6 2)
*$ F14 off aircraft carrier into North Sea; due to software? (SEN 8 3) 
*$ F14 lost to uncontrollable spin, traced to tactical software (SEN 9 5)
*$ El Dorado brake computer bug caused recall of all El Dorados (SEN 4 4)
$$ Viking had a misaligned antenna due to a faulty code patch (SEN 9 5)
$$ First Space Shuttle backup launch-computer synch problem (SEN 6 5 [Garman])
*  Second Space Shuttle operational simulation: tight loop upon cancellation of
    an attempted abort; required manual override (SEN 7 1)
*  Second Shuttle simulation: bug found in jettisoning an SRB (SEN 8 3)
*$ Delays of two Discovery shuttle launches due to backup computer outage 
    [most recently 25 Aug 85] (SEN 10 5) [NY Times 26 August 1985]  
*  Shuttle STS-6 bugs in live Dual Mission software prevented aborts (SEN 11 1)
*  Gemini V 100mi landing err, prog ignored orbital motion around sun (SEN 9 1)
*  F16 simulation: plane flipped over whenever it crossed equator (SEN 5 2)
*  F16 simulation: upside-down F16 deadlock over left vs. right roll (SEN 9 5)
*  Nuclear reactor design: bug in Shock II model/program (SEN 4 2)
*  Reactor overheating, low-oil indicator; two-fault coincidence (SEN 8 5)
*  SF BART train doors sometimes open on long legs between stations (SEN 8 5)
$  IRS reprogramming delays; interest paid on over 1,150,000 refunds (SEN 10 3)
$  $32 BILLION overdraft at Bank of New York (prog counter overflow) (SEN 11 1)
*S Numerous system intrusions and penetrations; implanted Trojan horses; 414s;
    intrusions to TRW Credit Information Service, British Telecom's Prestel,
    Santa Clara prison data system (inmate altered release date) (SEN 10 1).
    Computerized time-bomb inserted by programmer (for extortion?) (10 3)
    PC Graphics program Trojan horse (ArfArf) wiped out users' files (SEN 10 5)
*$ Union Carbide leak (135 injuries) exacerbated by program not handling
    aldicarb oxime plus operator error [NY Times 14 and 24 Aug 85] (SEN 10 5)
*  Multipatient monitoring system recalled; mixed up patients (SEN 11 1)
*  Pacemaker locked up when being adjusted by doctor (SEN 11 1)
*  Diagnostic lab instrument misprogrammed (SEN 11 1)
 S Chernenko at MOSKVAX: network mail hoax [1 April 1984] (SEN 9 4)
 S VMS tape backup SW trashed disc directories dumped in image mode (SEN 8 5)
*$ C&P computer crashes 44,000 DC phones (SEN 1 1)
$  1979 AT&T program bug downed phone service to Greece for months (SEN 10 3)
$  Demo NatComm thank-you mailing mistitled supporters [NY Times, 16 Dec 1984]
$  Slow responses in Bankwire interface SW resulted in double posting of tens 
    of $millions, with interest losses (SEN 10 5)
$  Program bug permitted auto-teller overdrafts in Washington State (SEN 10 3)
 - Quebec election prediction gave loser big win [1981] (SEN 10 2, p. 25-26)
 - Other election problems including mid-stream corrections (HW/SW) (SEN 10 3)
 - SW vendor rigs elections? (David Burnham, NY Times front page, 29 July 1985)
 - Alaskan DMV program bug jails driver [Computerworld 15 Apr 85] (SEN 10 3)
 - Vancouver Stock Index lost 574 points over 22 months -- roundoff (SEN 9 1) 
 - Gobbling of legitimate automatic teller cards (SEN 9 2, another SEN 10 5)

-------------------------- HARDWARE/SOFTWARE ---------------------------------
!  Michigan man killed by robotic die-casting machinery (SEN 10 2, 11 1)
!  Japanese mechanic killed by malfunctioning Kawasaki robot (SEN 10 1, 10 3)
    [Electronic Engineering Times, 21 December 1981]
!  Chinese computer builder electrocuted by his smart computer. (WWN headline: 
   "Jealous Computer Zaps its Creator" after he built newer one!!)  (SEN 10 1)
*  FAA Air Traffic Control: many computer system outages (e.g., SEN 5 3)
*  ARPANET ground to a complete halt [27 Oct 1980] (SEN 6 1 [Rosen])
*$ Ford Mark VII wiring fires: flaw in computerized air suspension (SEN 10 3)
$S Harrah's $1.7 Million payoff scam -- Trojan horse chip (SEN 8 5) 
$  Great Northeast power blackout due to threshold set-too-low being exceeded
$  Power blackout of 10 Western states, propagated error [2 Oct 1984] (SEN 9 5)
$  NY Stock Exch. halted for 41 minutes; drum channel errors killed primary
    and backup computer systems [24 Feb 72]
 - SF Muni Metro: Ghost Train reappeared, forcing manual operation (SEN 8 3)
*$ Computer-controlled turntable for huge set ground "Grind" to halt (SEN 10 2)
*$ 8080 control system dropped bits and boulders from 80 ft conveyor (SEN 10 2)
 S 1984 Rose Bowl hoax, scoreboard takeover ("Cal Tech vs. MIT") (SEN 9 2)

-------- COMPUTER AS CATALYST, HUMAN FRAILTIES, OR UNKNOWN CAUSES -------------
!!$ Korean Airlines 007 shot down [1 Sept 1983], killing 269; autopilot left on
    HDG 246 rather than INERTIAL NAV? (NYReview 25 Apr 85, SEN 9 1, SEN 10 3)
!!$ Air New Zealand crashed into mountain [28 Nov 1979]; computer course data
    error had been detected and fixed, but pilots not informed (SEN 6 3 & 6 5)
!  Woman killed daughter, tried to kill son and self; "computer error" blamed 
    for false report of their all having an incurable disease (SEN 10 3)
*  Unarmed Soviet missile crashed in Finland.  Wrong flight path? (SEN 10 2)
*$ South Pacific Airlines, 200 aboard, 500 mi off course near USSR [6 Oct 1984]
*S San Francisco Public Defender's database accessible to police (SEN 10 2)
*  Various cases of false arrest due to computer database use (SEN 10 3, 11 1)
*  Avionics failed, design used digitized copier-distorted curves (SEN 10 5)
$  .5M transaction became $500M, due to "000" convention; $200M lost (SEN 10 3)
$  Possible fraud on reinsurance -- message time stamp faked??? (SEN 10 5)
$  N-step reinsurance cycle; SW checked only N=1 and 2 (SEN 10 5)
*  FAA Air Traffic Control: many near-misses not reported (SEN 10 3)
!$$ Shuttle Challenger explosion, 7 killed.  Cause not yet known. [29 Jan 86]

---------------- ILLUSTRATIVE OF POTENTIAL FUTURE PROBLEMS -------------------
*S Many known/past security flaws in computer operating systems and application
    programs.  Discovery of new flaws running way ahead of their elimination.  
*  Expert systems in critical environments: unpredictability if (unknowingly) 
    outside of range of competence, e.g., incompleteness of rule base. StarWars
$S Embezzlements, e.g., Muhammed Ali swindle [$23.2 Million], Security Pacific 
    [$10.2 Million], City National Beverly Hills CA [$1.1 Million, 23 Mar 1979]
    [These were only marginally computer-related, but suggestive.  Others
    are known, but not publically acknowledged.]

---------------------- REFUTATION OF EARLIER REPORT --------------------------
* "Exocet missile not on expected-missile list, detected as friend" (SEN 8 3)
   [see Sheffield sinking, reported in New Scientist 97, p. 353, 2/10/83]; 
   Officially denied by British Minister of Defence Peter Blaker
   [New Scientist, vol 97, page 502, 24 Feb 83].  Rather, sinking abetted by
   defensive equipment being turned off to reduce communication interference?