RISKS@CSL.SRI.COM (RISKS FORUM DIGEST, Peter G. Neumann -- Coordinator) (09/22/86)
RISKS-LIST: RISKS-FORUM DIGEST, Sunday, 21 September 1986 Volume 3 : Issue 61
FORUM ON RISKS TO THE PUBLIC IN COMPUTER SYSTEMS
ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator
Contents:
Computers and Ethics (Robert Reed)
Autonomous weapons (Wayne Throop)
Simulation risk (Rob Horn)
Viking software (James Tomayko)
Risks of passwords on networks (Bruce)
More on digital jets; Sanity checks (Eugene Miya)
[I'm back from almost 3 weeks away, trying to keep up remotely with an
enormous flood of contributions. The burden on our readers must have
been intense as well. Let's sharpen up a little... PGN]
The RISKS Forum is moderated. Contributions should be relevant, sound, in good
taste, objective, coherent, concise, nonrepetitious. Diversity is welcome.
(Contributions to RISKS@CSL.SRI.COM, Requests to RISKS-Request@CSL.SRI.COM)
(Back issues Vol i Issue j available in CSL.SRI.COM:<RISKS>RISKS-i.j.
Summary Contents in MAXj for each i; Vol 1: RISKS-1.46; Vol 2: RISKS-2.57.)
----------------------------------------------------------------------
To: "RISKS FORUM, Peter G. Neumann -- Coordinator" <RISKS@CSL.SRI.COM>
Subject: Computers and Ethics
Date: 19 Sep 86 13:36:54 PDT (Fri)
From: bobr%zeus.tek.csnet@CSNET-RELAY.ARPA
In RISKS-3.54 Mark S. Day writes:
> ...people will read mail and data files stored on a timesharing system, even
> though it's unacceptable to rifle through people's desks. [...]
It occurs to me that each of these suggested mechanisms can be interpreted
in different ways which may provide new insights into the problem.
Novelty. Social conditioning aside, the thrill of adventure in a new
environment leads many people to explore the system in a
quest for new understanding about it. It is perhaps easier
to lay the moral questions aside when caught in the fervor
of covering new ground. In fact the thrill is enhanced by
doing something slightly larcenous.
Distance. Certainly the distance between people is greater, but the
distance between private pathways is shorter.
Psychologically, I feel closer to your portion of the file
system than I do to the contents of your desk drawers.
Especially if working in an environment where limited
sharing of files is part of the norm, the sense of
territorial lines is less distinct within such an electronic
medium
There is a third aspect which is related to the thrill factor, and that is
the threat of being caught. If I am found in your office with my hand in
your desk, the evidence is pretty compelling and not easy to hide. Within a
computer system, we are all little "virtual people", moving silently around
the directory tree, and so much less likely to arouse suspicions, so even
when ethical considerations are present, the concern about getting caught is
lessened by the nature of the medium.
Robert Reed, Tektronix CAE Systems Division, bobr@zeus.TEK
------------------------------
Date: Fri, 19 Sep 86 16:46:17 edt
From: rti-sel!dg_rtp!throopw%mcnc.csnet@CSNET-RELAY.ARPA
Subject: Autonomous weapons
Apparently-To: mcnc!csl.sri.com!risks
> eugene@AMES-NAS.ARPA (Eugene Miya)
> Most recently, another poster brought up the issue of autonmous weapons.
It is worth pointing out that we are *currently* using autonomous weapons
and they are *not* smart enough to distinguish signs of surrender. Give up?
I'm talking about, for example, hand grenades or landmines. These are
autonomous (after being thrown or burried) and their mission (guided by a
particularly simple "computer") is to saturate their environment with
shrapnel after a suitable delay. Bombs with proximity fuses, self-guided
missiles, and so on, where there is "intelligence" in the weapon and a
significant time delay between the decision to deploy and the weapon's
effective discharge can all be considered cases of "autonomous weapons". We
are (in this view) simply trying to make the beasties smarter, so that they
eventually *will* be able to recognize signs of surrender or cease-fire or
other cases of cessation of hostilities. (Picture land-mines getting up and
"trooping" back to an armory after the war is over... )
Perhaps this is more appropos to one of the "arms" lists, but I think it is
worth noting that we are allowing some *very* simple "computers" to be in
charge of some *very* powerful weapons right now. It is an interesting
question to ask if we really *want* to make the weapons smarter. But I
don't think it is a question of whether to use autonomous weapons at all...
we're already using them.
Wayne Throop <the-known-world>!mcnc!rti-sel!dg_rtp!throopw
------------------------------
Date: Sat, 20 Sep 86 16:11:42 edt
From: harvard!wanginst!infinet!rhorn@seismo.CSS.GOV (Rob Horn)
To: risks-request@CSL.SRI.COM
Subject: Simulation risk
One kind of risk that I have not seen discussed here is the problems posed
by using computer simulation models that are not adequate. In particular I
am refering to situations where due to either insufficient computer
resources, or insufficient mathematical analysis, the really accurate model
results are not available. Usually more primitive, inaccurate model results
are available and being used by the ideologues on both sides of an issue.
This places the responsible scientists and engineers in a difficult
situation. How do you say ``I don't know yet'' and how do you deal with
making recommendations in the absence of adequate information.
I can think of two such situations that have major public decision-making
impact.
The first is the ``nuclear winter'' situation. I remember many years ago
reading the sensitivity analysis of the one-dimensional and two-dimensional
climate models to solar input. They were hyper-sensitive, with variations
on the order of measurement error causing massive climate change. It was
not until recently (1982) that the vectorized Climate Model was analyzed and
shown to be reasonably well behaved. And even it has some contentious
approximations. This model requires 15 hours on a CRAY-1 to analyze one
situation for one season.
When the nuclear winter stories came out I had my doubts. Where did these
people find a solid month (12 seasons x 4(?) test cases) of CRAY time? Had
they used one of the hyper-sensitive 1 or 2-dimensional models. What would
the accurate models find? And how should I respond when I knew that it
would probably be a year or more before that much CRAY time and post-
simulation analysis could be finished? (Fortunately I only had to handle
party conversation with people who knew that I had worked in that field.)
The same kind of problem occured in the ozone layer issues during the mid
70's. The more accurate model had two extremely severe problems: 1) it was
unconditionally unstable when phrased as a finite difference problem or
exceedingly temperamental when phrased as an implicit differencing problem.
2) It involved solving extremely stiff differential equations. In this case
the official answer given was ``we don't know. It will take several years
of mathematical research effort to make this problem tractable. The real
answer is anyone's guess. The published model answers are meaningless.'' A
truthful answer but of little value to decision makers. (There was a brute
force throw-computers-at-it solution. Estimated run-time on a CRAY was
about 1,000 years per simulated year. Easier to wait and see what
happened.)
How often are we placed in a situation where the inaccurate computer
simulation is available, but the accurate simulation unavailable?
What is an appropriate way to deal with this problem?
Rob Horn
UUCP: ...{decvax, seismo!harvard}!wanginst!infinet!rhorn
Snail: Infinet, 40 High St., North Andover, MA
------------------------------
Date: Sunday, 21 September 1986 09:25:25 EDT
From: James.Tomayko@sei.cmu.edu
To: risks@csl.sri.com
Subject: Viking software
The Viking software load for the lander was 18K words stored on plated wire
memory. The Martin Marietta team decided to use a 'software first' approach
to the development of the flight load. This meant a careful examination of
the requirements, a serious memory estimate, and then commitment by the
project team to stay within that memory estimate. The software was developed
on an emulator that used microcoded instructions to simulate the
as-yet-unpurchased computer. Sources for this are a Rome Air Development
Center report that studied software development, later summarized in a book
by Robert L. Glass. The Viking software documents for the orbiter, developed
by JPL, are so good I use them as examples of tracability in my current
software engineering courses.
------------------------------
Date: 20 SEP 86 14:57-EST
From: BRUCE%UC780.BITNET@WISCVM.WISC.EDU
To: RISKS@CSL.SRI.COM
Subject: Risks of passwords on networks
A few thoughts about networks which ask for passwords to send files. Take a
computer network with three computers. Call them computer A, B, and C.
Computer user on A wants to send a file to their account on C through
computer B. No problem, we invoke the command to send files, supply it with
a password (and maybe a username at computer C) and off the files go. But,
on computer B, there is a "smart" systems programmer who monitors all
network traffic through his/her node. How interesting... A file copy
operation with a user name/password attached.
The point? Just a password is not a good solution. Maybe one
would need to encrypt the packets through the network (so that
intermediate nodes couldn't read them).
Bruce
------------------------------
Date: Sat, 20 Sep 86 11:40:44 pdt
From: eugene@AMES-NAS.ARPA (Eugene Miya)
To: risks@sri-csl.ARPA
Subject: More on digital jets; Sanity checks
Talk about timing:
In the latest copy of IEEE Spectrum (why didn't anyone else post this?)
%A Cary R. Spitzer
%Z NASA LaRC, Hampton, VA
%T All-digital Jets are Taking Off
%J IEEE Spectrum
%V 23
%N 9
%D September 1986
%P 51-56
%X Covers F-14D, F-16[CD], A-3[012]) airbus, 7J7, MD-11, and
other 1st and emerging 2nd generation digital systems.
Has good basic references.
Added note. I will be contacting some old Viking friends for a further
detailed description and references as requested (probably next Tu. or We
when they come up here).
On Sanity checks:
I had a similar incident in a Silicon Valley Mexician restaurant
which I reported in a early RISK to the pocket book. This issue
has appeared other news groups like mod.comp-soc on the USENET.
I offer the following reference:
%A Jon L. Bentley
%Z ATT BL (research!)
%T The Envelope is Back
%J Communications of the ACM
%S Programming Pearls
%V 29
%N 3
%D March 1986
%P 176-182
%K rules of thumb, cost, orders of magnitude, quick calculations,
Litle's Law
%X JLB's principles include:
Familiarity with numbers
Willingness to experiment [actively, discussing this one with Denning]
Discipline in checking answers
Mathematics, when you need it
He also gives the "Back of the Envelope" column in the
American Journal of Physics as good reading.
I am reminded of a quote by Eric Shipton, an early English Mt. Everest veteran
who died recently: (paraphased) Never go on an expedition which you can't
plan on the back of an envelope. I know this is how spaceflight is
frequently done.
--eugene miya
NASA ARC
------------------------------
End of RISKS-FORUM Digest
************************
-------