nigam@MITRE.MITRE.ORG (Alok C. Nigam) (07/11/89)
Software Engineering Digest Monday, 10 Jul 1989
Volume 6 : Issue 29
Today's Topics:
Re: code reviews (3) (U)
Code reviews: a suggested approach (U)
CompEuro Call for Papers
Call for Position Papers
Leonardo project (U)
------------------------------------------------------------
Date: 20 Jun 89 16:42:41 GMT
From: Mark Paulk <mcp@sei.cmu.edu>
Organization: Carnegie-Mellon University, SEI, Pgh, Pa
Subject: Re: code reviews (U)
The following references have some interesting data and insights on the
role and power of inspections. Unfortunately it is difficult to get
hard data on the effectiveness of methods such as inspections, but there
has been some work published which is much more pertinent than my
opinion would be... Pertinent quotes follow each reference.
Let me also emphasize correct definition of terms! Inspections, reviews,
walkthroughs, etc., are frequently overloaded terms. For the "official"
definitions, see IEEE Standard 1028 (or Fagan's original paper for that
matter). There are distinct differences between the concepts; the most
formal is inspection. I use peer review as the all inclusive term
because it is not defined in 1028, but some people use it in the hierarchy
desk check
peer review
walkthrough
inspection
as meaning a 1-1 review with A peer (as opposed to a group of peers).
Review, by itself, covers a wide range of sins. Much of this discussion
has been oriented at technical reviews, i.e., CDR, PDR class reviews,
which are NOT what the original poster was asking about.
@b([ACKE89]) A.F. Ackerman, L.S. Buchwald, and F.H. Lewski,
"Software Inspections: An Effective Verification Process,"
IEEE Software, Vol. 6, No. 3, May, 1989, pp. 31-36.
Software inspections have been found to be superior to reviews
and walkthroughs, p. 31
collection and analysis of data for immediate and long-term
process improvement, p. 32
inspections improve quality and productivity, p. 34
inspections give project management more definite and more dependable
milestones than less formal review processes, p. 35
@b([GILB88]) T. Gilb,
PRINCIPLES OF SOFTWARE ENGINEERING MANAGEMENT,
Addison-Wesley, Reading, MA, 1988.
The inspection method is the most effective quality control method for
software specification documentation we know about. p. 68
Testing is a maximum of 50 to 55% effective at defect identification and
removal for a single test process. p. 221
Inspection is about 80% (+/- 20%) effective in removing existing defects.
p. 221
The average is five hours saved for every hour invested in inspection.
p. 221
Inspected programs were ten times cheaper to maintain than ... similar
non-inspected programs. p. 221
High level inspections (requirements and design specification) were the
most powerful things they could do. p. 244
@b([GRAD87]) R.B. Grady and D.L. Caswell,
SOFTWARE METRICS: ESTABLISHING A COMPANY-WIDE PROGRAM,
Prentice-Hall, Englewood Cliffs, NJ, 1987.
The five "modern programming practices" which had the strongest correlation
with productivity were top-down design, modular design, design reviews, code
inspections, and quality assurance programs. p. 20
Projects of highest productivity are among those with the lowest defect
densities. p. 140
@b([MYER88]) W. Myers,
"Shuttle code achieves very low error rate,"
IEEE Software, Vol. 5, No. 5, September, 1988, pp. 93-95.
500K Space Shuttle source code with 0.11 errors/KSLOC credited by IBM/FSD
to process definition, rigorous inspection of work products across the
process, independent software verification, defect-cause analysis, knowledge
engineering, expert systems, specialized tools, and "good old value gained
from lessons learned." (Barbara Kolkhorst) Reduced from 2 errors/KSLOC,
versus 8-10 errors/KSLOC for industry.
Effort/time spent on software reconfiguration reduced by 50%.
About 85% of major errors discovered in inspection.
------------------------------
Date: 21 Jun 89 20:24:10 GMT
From: Jerry Leichter <leichter@yale-zoo.arpa>
Organization: Yale Computer Science Department, New Haven, Connecticut, USA
Subject: Re: code reviews (U)
Many years ago, in wilder times, I and two friends spent a fair amount of
time, ahem, breaking into our college computer. We generally worked on hacks
cooperatively.
One day, one of us developed a very simple program. He decided that it was
simple enough, and so clearly correct, that he could just try it out.
Well, needless to say, the program had a completely obvious bug which one of
us spotted instantly on looking at the code. The effect of the bug was to
cause a system crash leaving fairly obvious pointers to the culprit. The
author had run the thing twice...the first crash "must have been a coinci-
dence". (Back in those days, the average time between crashes was a couple
of hours, so this was not a completely absurd thought.) We were VERY lucky
not to get caught.
The rule we learned from this was: Never hack alone. From there on out, none
of us would run a new hack program unless one of the others had checked it out
first. We had no further incidents of this particular sort.
Years later, I worked for an organization with a policy of code reviews. We
actually had multiple levels of review. There were informal design reviews.
In practice, most people hated those and they were not very useful - they
tended to bring out the worst in those people who insisted on doing things in
particular standard ways which were usually irrelevant.
On the other hand, we also had a very strict CODE review policy - we called it
"releasing". Any particular implementation project, whether new code or a
patch to existing code, required two people, a coder and a releaser. The
coder wrote and tested the code, and handed it to the releaser. The releaser
was responsible for checking that the code met its specs, was written to our
internal standards (which were not very strict; what they required was all
very simple and reasonable), was reasonably documented, and so on. More gene-
rally, the releaser was expected to understand the code as well as the coder
did. If he found problems, he could at his disgression fix them himself or
require the coder to fix them.
One very important element of this system was that there were no separate
groups of coders and releasers - everyone did both jobs. If you released my
code today, I might release yours tomorrow. This avoided the typical con-
flicts you often see between development and QA groups. (Yes, it can have
disadvantages, too.) It also provided some check to ensure that people didn't
try to cut corners on either job - though of course that happened. (One per-
son who worked for me for a while viewed his releaser's role as one of finding
and fixing all the bugs. He was known to hand over code that wouldn't com-
pile. Needless to say, he was seen as a problem....) However, in my experi-
ence the system worked quite well, and the code quality that resulted was
quite high. (Sorry, I can't quantify any of that...but I'd certainly use such
a system again.)
What did doing this cost? It's hard to be exact because people varied. A
good manager had to adjust for the individuals involved. For most people, I'd
estimate release time at about half of pure coding time - where pure coding
time did NOT include design or debugging. For the "problem" person I mention-
ed above, I'd have to drop coding time and increase debugging and (even more
so) release time; the total would end up somewhat higher. But that rarely
happened. Since except for small, isolated projects pure coding time wasn't
likely to be more than a third or so of the total time, release time might
add about 15% to the total - and it could often be used to fill in otherwise
dead spots on schedules, so in real terms its contribution to total completion
time might be quite small. On the other hand, it also provided you with a
second expert on the particular piece of code involved. The training to do
that would have cost you some time, too.
The group in which I saw this done was not very large (50 people perhaps in
total) and was probably unusually talented, since management made it a point
to hire the brightest people they could, regardless of experience. How well
this kind of approach would generalize to other groups, I can't really say.
(I also believe that the group eventually stopped doing this when pressure
from above to get stuff done faster became much greater. There's never time
to do it right, but always time to do it again....)
------------------------------
Date: 21 Jun 89 21:04:28 GMT
From: "Thomas.J.Tom.McClory" <hubcap!ncrcae!ncrlnk!emdeng!tmcclory@gatech.edu>
Organization: E&M Dayton, NCR Corporation, Dayton, Ohio
Subject: Re: code reviews (U)
I will throw in references for the "Design & Code Inspection Process",
a specific variation of the general peer review methods.
M. E. Fagan, "Design and Code Inspections to Reduce Errors in
Program Development," _IBM Systems Journal_, Vol. 15, No. 3, 1976,
pp. 182-211.
Combines both a "how to" of the process as
with the results of using the process
several IBM products.
M. E. Fagan, "Advances in Inspection,", _IEEE Transactions on
Software Engineering_, July 1986, pp. 744-751.
A follow-on work to the paper above that refines
the process description, and documents additional
successes using the process.
J. H. Dobbins, "Inspections as an Up-Front Quality Technique," in the
_Handbook of Software Quality Assurance_, G. G. Schulmeyer and J.
I. McManus, eds., Van Nostrand Reinhold, New York, 1987, pp. 137-177.
A case study of inspections as used by the IBM
Federal Systems Division. Describes how the process
was implemented and the process measurements that
the Q.A. staff implemented. Describes how statistical
process control can be applied to software development.
A. Frank Ackerman, L. S. Buchwald, and F. H. Lewski, "Software
Inspections: An Effective Verification Process," _IEEE Software_,
May, 1989, pp. 31-36.
A good overview description. The two Fagan papers are
better how-to descriptions, but this is a good
place to start if you don't know about inspections.
------------------------------
Date: 26 Jun 89 12:30:34 GMT
From: Charles Weir <mcvax!ukc!stl!idec!marlow!cweir@uunet.uu.net>
Organization: Reuters Ltd PLC, Marlow, Bucks, England
Subject: Code reviews: a suggested approach (U)
Suggested approach for a Code Walkthrough
Following the discussion about code reviews, here is a document I put
together a little while back. I should be interested to find out how
it compares with other people's experiences.
We had been using code walkthroughs to find bugs in code where
testing was proving unsuccessful. Sometimes they were very
successful, sometimes not. This note attempts do analyse what leads
to a successful review.
Our targets for a walkthrough are:-
* No Bugs.
* No indeterminate situations.
* Maintainable and understandable code (as for as convenient) .
A bug is a straightforward point where under some situation the code
will not work as the programmer intended. An "indeterminate
situation" is where the code behaviour is unpredictable (technically
this is a bug too).
How we go about it
------------------
We used the following guidelines:-
A code walkthrough is constructive. The walker needs to avoid
attacking or threatening the programmer either directly or indirectly.
The walker must focus attention on a particular problem, a particular
line of code or element. We avoid issues of approach and style ("I
wouldn't do it like this..."), or unnecessary performance enhancement
("It'll be faster if...").
We tackle problems using questions rather than statements:-
* How Is this function defined?
* How does this work?
* What parameters are being passed here?
* When is this called and by what?
* I think I may see a problem here. Am I right? (N.b. Quite
often the answer will be no)
The process of questioning the code is more effective than a straight
challenge. It is non-threatening. And it encourages the programmer
to look at his/her code from a different perspective.
The inspections should not last too long. The mental effort
required is similar to University entrance examinations, so
over-stretching here oneself does nothing for anyone. A sensible
maximum is two hours for a session or less. It is much better to
inspect a body of code bit by bit in a number of short inspections
over a week or two, than all at once.
Further Suggestions (particularly for C code reviews)
-----------------------------------------------------
Bug hunting too is partly a matter of habitat. It is important to
look at the places where bugs live. We look mostly at:-
* Buffer, string, and arrays. Could there be overruns? Zero
termination of strings? "strlen" etc.
* Error Handling: Is an error ignored? Does the handling
mechanism work?
* Statuses: Are there states (combinations of statuses and flags
etc.) which can occur but are not handled? Or which are handled
but could never occur?
* Double use: Same data structure used by conflicting processes.
Flags and status variables used for two purposes.
* Comments: Misleading or out-of-date? It is often worth
covering up comments when inspecting the code.
For the inspection to be effective, it is not necessary for the
reviewer fully to understand the code and the design. The
questioning process uses the programmer's knowledge to supply that
information.
It is important to keep a sense of perspective. In C, for example,
it is important to replace magic numbers with "#defines" for
maintenance purposes: This is not the same as a bug. Function
headers and comments generally are necessary for future maintenance:
Again these are not bugs. And we shouldn't treat them as such.
The GOTCHA trap is a classic: It's easy to communicate the sense
of victory at finding a difficult bug in a way which threatens the
programmer ("GOTCHA"). Again we use questions to communicate,
rather than statements. A sense of humour is important too. A laugh
both defuses the tension, and makes the process more fun.
------------------------------
Date: Sun, 25 Jun 89 09:32 IDT
From: GIDEON@ILNCRD.BITNET
Subject: CompEuro Call for Papers
CompEuro 90 ISRAEL, 7-9 MAY 1990
Call for Papers
CompEuro The goal of CompEuro conferences is to provide an annual
Conferences international forum for computer experts to discuss latest
----------- results on computer technologies, systems, applications
and their interaction. CompEuro conferences are sponsored
by the IEEE Computer Society and IEEE Region 8 comprising
Europe, Africa and the Near East.
Theme CompEuro 90 will focus on the systems engineering aspects
----- of complex computerized systems and their software. Most
modern computerized systems are multi-level and
multi-computer systems including hardware, software and
communications subsystems. Their engineering requires
expertise in those basic areas, a wide knowledge and
analytical insight of the application domains together
with systems integration skills and experience.
The purpose of the conference is to promote the state of the art of
computerized systems engineering and to present methods, approaches and tools
needed in the development of these complex systems. The presentation of
industrial development case studies will be encouraged. The conference theme
is therefore defined as:
COMPUTER SYSTEMS AND SOFTWARE ENGINEERING
Systems Engineering Aspects of Complex Computerized Systems
This theme will be expanded to discuss the following topics:
, 1. Computer Systems Engineering and Embedded Computed Systems
* Systems analysis
* System and computer architectures and their evaluation
* Mutual impact and trade-offs systems and computer
architectures and software
* Computer communications
* Development and testing methods
* Integration of large computerized systems
2. Software Engineering
* Software system analysis
* Requirements specification and top-level design
* Software system architectures and their evaluation
* Formal methods in software development
* Development and testing methods
3. VLSI Systems
* High performance, multiprocessing systems
* VLSI impact on performance and on system integration
4. Rapid Prototyping of Systems and Software
* Rapid prototyping techniques
* Rapid prototyping laboratories and tools
5. Artificial Intelligence Techniques
* Incorporation of Al techniques in systems implementations
* Use of Al techniques in the development process
6. Development Environment
* Architecture of development environments
* Tools for systems engineering
* Tools for software engineering
* Hardware-Software integration issues
7. Management of the Development Process and Resources
* Project management
* Cost estimating techniques
* Quality assurance and configuration management
* Training of industrial personnel in industry and academia
Tutorial Tutorial seminars to introduce and to provide technical
-------- updata on key topics covered by ComEuro 90 are scheduled
for Monday May 7, 1990.
Letter of In order to structure the program early, the program committee
Intent requests prospective authors for letter of intent giving title,
-------- topic group and address or notification by reply card as soon
as possible, but no later than June 15, 1989.
Author 1000-1500 word summaries due: September 1, 1989 The conference
Schedules Notification of acceptance: November 1, 1989 language is
--------- Camera-ready copies due: February 1, 1990 is in English
Technical A technical exhibition is planned to complement the conference
Exhibition papers, it will include CAS2E - Computer Aided Systems and
---------- Software Engineering Environements and Tools.
Conference Conference Chairman: J. Raviv, IBM, Israel
Committee Technical Program Chairmen: J.Z. Lavi, IAI, Israel;
--------- W.B. Proebster, IBM, W. Germany
Technical Topics Chairmen:
Topic 1: A.O. Ward, British Aerospace, England;
M. Winokur, IAI, Israel
Topic 2: K. Jackson, Systems Designers, England;
S. Koenig, Tadiran, Israel
Topic 3: Z. Barzilai, IBM, USA;
U. Wieser, Intel, Israel
Topic 4: H. Zullighoven, GMD, Germany;
J. Cohen, Rafael, Israel
Topic 5: A. van Lambsweerde, University of Namur, Belgium;
J. Rosenschein, Hebrew University, Israel
Topic 6: M. Lehman, Imperial College, England;
P. Rogoway, Motorola, Israel
Topic 7: P. Ein-Dor, Tel-Aviv University, Israel;
H.G. Sol, Delft Technical University, Holland
Tutorial Chairman: A. Yehudai, Tel Aviv University, Israel
Technical Exhibition Chairman: Y. Zeira, ATL, Israel
Conference Secretariat, c/0 ORTRA Ltd. 2 Koffman St. POB 50432,
Tel-Aviv 61500, ISRAEL. Tel. 972-3-664825, Fax. 972-3-660952, Tlx. 361142 ORTR
Official Carrier El-Al, ISRAEL AIRLINES
------------------------------
Date: Fri, 30 Jun 89 15:04:06 EDT
From: rh@mbunix (Rich Hilliard)
Subject: CALL FOR POSITION PAPERS
Call for Position Papers
------------------------
Informal Workshop on Ada Support for Object-Oriented Programming
----------------------------------------------------------------
September 11-13, 1989
Hosted by The MITRE Corporation
Program Committee: Christine Braun, CONTEL Technology Center
Richard F. Hilliard II, The MITRE Corporation
Steven D. Litvintchouk, The MITRE Corporation
Robert F. Mathis, CONTEL Technology Center
The process is now under way to define requirements for Ada 9X, the
projected revision to the Ada language standard. This workshop will
focus specifically on whether, and to what extent, Ada 9X should offer
improved support for object-oriented programming and related
development methodologies.
Topics of interest include, but are not limited to:
--------------------------------------------------
1. Object-oriented development paradigms, programming styles, and
related life cycle issues:
* Abstract and concrete classes
* Inheritance
* Delegation
* Polymorphism
* Dynamic binding
* Object-oriented concurrency
* Persistent objects
* Analysis, design, verification and validation
* Support tools and environments
2. Support for object-oriented programming to be provided by Ada 9X
language features:
* Types and subtypes
* Packages
* Bodies and subunits
* Tasking
* Generics
* Program library
* New Ada 9X language features not currently in Ada
* Related compilation and runtime issues
The workshop will consist of 3 days of analysis of position papers,
and followup discussions. The workshop will be held at:
The Ramada Inn
15 Middlesex Canal Park Road
Woburn, MA 01801
617-935-8760
(Arrangements at the Ramada can be made for a special workshop rate of
$72/night which includes taxes, gratuities and full American
breakfast. When making reservations, mention that this is a MITRE
hosted function.)
The workshop will be limited to 25 participants. Prospective
participants should submit a position paper of no more than 3000 words
to:
Richard F. Hilliard II
The MITRE Corporation
Mail Stop A156
Burlington Road
Bedford, MA 01730
rh@mbunix.mitre.org
...{att,decvax,genrad,ll-xn,philabs,utzoo}!linus!rh
If possible, actual programming examples should be included as part of
the position paper. Electronic mail submission of position papers is
preferred. Because of limited accommodation, it is preferred that
only the principal author of a selected position paper attend the
workshop. For co-authored papers the principal author should
therefore be designated. A collection of selected position papers
will be prepared. The results of the workshop will be reported at a
future SIGAda meeting. Participants are encouraged to publish
expanded versions of their position papers elsewhere.
There will be a workshop fee of $150 per participant, to cover expenses.
Important Dates: July 21, 1989: Position papers due.
August 15, 1989: Notification of acceptance
and distribution of position
papers.
September 11-13, 1989: Workshop
------------------------------
Date: 29 Jun 89 19:02:29 GMT
From: David Lamb <jarvis.csri.toronto.edu!qucis!dalamb@rutgers.edu>
Organization: Queen's University, Kingston Ontario
Subject: Leonardo project (U)
Can anyone point me to articles in the open literature on the Leonardo
software design assistance project at MCC (Microelectronics and
Computer Technology Corporation)? I have a brief summary that seems to
say most information is restricted to shareholders, and would like to
know what (if any) material is available to the general public.
------------------------------
End of Software Engineering Digest
**********************************