PAAAAAR@CALSTATE.BITNET (10/09/89)
QUICK REVIEW "The Errors of \TEX(TM)" by Donald Knuth Software - Practice and Experience, Volume 17, No. 7, July 1989, pp607-685. This is an interesting paper. Donald Knuth has spent the last 10 years developing what has grown to be a 14,000 statement Pascal Program (\TEX) which allows the user to markup a text for printing. >From March 1977 through September 88 he recorded every change he made to both the original system(written in SAIL) and the later one ( written using WEB). He as prepared this List for publication and prefixed his comments. His humility is to be commended. His data fits much of what Frederick P. Brooks said in the "Mythical Man Month" in 1975 For example Brooks said, "Plan to throw one away". Knuth says: > It soon became clear what ought to be done: > the original version of [\TEX and METAFONT] should be > scrapped, [when we had ] enough user experience. DETAILS: (*else search for OPINION *) Here are some more detailed comments and quotes: I started a "BugLog" in 1971. It was replaced by an Lab Note Book two years later. They both taught me much. I still have them and re-read them. He has 15 categories of change. 9 of these are Bugs and 6 are Design Enhancements. They range from "A for Algorithm" through to "S for Surprise" with some gaps in the alphabet. He noted that only 1.4% of the errors were associated with goto statements - and concludes: >"Every feature of a programming language can be harmful if it is misused." His original design was reviewed by others and implemented by 2 graduate students while he was outside the country - when he returned he took over the implementation and conludes: >"Important policy decisions had to be made at every turn[...]. The designer of > a new kind of system must participate in the implementation[...]." >"The designer should also write the first user manual" Later he makes it clear that he believes that it is the newness of the \TEX idea that lead to the need for the designer to be involved in implementation and documentation. In the original version there were 500 errors in 4.5K statements. The commonest types of change where: I: Improvement of Interactive User Interface G: Generalization Q: Quest for Quality = better looking output F: Forgotten Function C: Clean up for Clarity and Consistency B: Blunder and Botch - (what I call a "Brain Parity Failure") In the later version (300 errors) the commonest types of errors were C, G, I (as above) R: Robustness Reinforcement. (Note. I could be wrong with these lists since the figures and tables are (deliberately?) not statistical summaries) It is clear that for 10 years he thought that \TEX was 90% to 100% complete. He had 850 entries in May 87, and 865 entries in Sept 88 however - the new ones being mainly Surprises. He pays people who report problems - and doubles the prize each year - this seems a little overconfident. He describes how he selected test data - he uses the words "Torture Test". He notes that a burst of changes resulted from his presenting the code in class. The addendum, to the paper is the complete list of 850+ errors (which I did not read in detail). His conclusion is that keeping a log of changes (a BugLog) has taught him to accept and enjoy his errors. OPINION The paper is a comfort to me. It indicates that Software Engineering (SE) has something to offer the programmer and computer scientist. There is no mention of SE techniques. However Knuth's data supports three or four pieces of SE lore. He is an intelligent programmer using structured programming and complex tools - and making mistakes which SE aims to avoid. (\TEX is a symbol for a trademark of the American Mathematics Society) Dr. Richard J. Botting, Department of Computer Science, 5500 State University Parkway, San Bernardino CA 92407 "where smog of LA, is blown away, and the sun shines bright all the day"! PAAAAAR@CCS.CSUSCC.CALSTATE paaaaar@calstate.bitnet<docdick> PAAAAAR%CALSTATE.BITNET@CUNYVM.CUNY.EDU voice:714-880-5326 (on odd thursdays) Disclaimer:These are my own opinions, nobody else wants to be responsible.
bks@alfa.berkeley.edu (Brad Sherman) (10/10/89)
In article <8910091549.AA17481@mitre.arpa> PAAAAAR@CALSTATE.BITNET writes: >QUICK REVIEW >"The Errors of \TEX(TM)" by Donald Knuth >Software - Practice and Experience, Volume 17, No. 7, July 1989, pp607-685. > ... >OPINION >The paper is a comfort to me. It indicates that Software Engineering (SE) >has something to offer the programmer and computer scientist. >There is no mention of SE techniques. However Knuth's data supports three or >four pieces of SE lore. He is an intelligent programmer using structured >programming and complex tools - and making mistakes which SE aims to avoid. > ... >Dr. Richard J. Botting, Knuth makes at least 4 claims that some might think are contradictory to current SE theory. (Paraphrasing from memory): 1) The designer should do the implementaion. 2) The designer should do the testing. 3) The designer should do the (first) user manual. 4) After design, implementation and user input, redesign and reimplement. Is this how it's done in your shop? Brad Sherman <bks@alfa.berkeley.edu> Read the article!
seibel@cgl.ucsf.edu (George Seibel) (10/10/89)
In article <1989Oct10.015725.24517@agate.berkeley.edu> bks@alfa.berkeley.edu (Brad Sherman) writes: >Knuth makes at least 4 claims that some might think are contradictory >to current SE theory. > (Paraphrasing from memory): > 1) The designer should do the implementaion. > 2) The designer should do the testing. > 3) The designer should do the (first) user manual. > 4) After design, implementation and user input, > redesign and reimplement. > >Is this how it's done in your shop? It sounds like Knuth is telling us: "If you want something done right..." In the first article of this thread, Richard Botting offers a clue to how Knuth's suggestions may have developed: "His original design was reviewed by others and implemented by 2 graduate students while he was outside the country..." So what did Knuth expect? What should anyone expect in such a situation? If you want professional quality software, it would seem appropriate to hire professionals to do the implementation, and pay them what they are worth. As for points 1-4 above, I think that it is appropriate for the designers to at least be *involved* in the implementation, testing, and documentation. Step 4 is often appropriate, and is probably done far less often than it should be. Skipping the redesign/reimplementation is usually false economy. I've seen this from both sides; after totalling up the time spent in maintainance of something that needed a complete redesign, it became painfully apparent that it would have been cheaper in the long run to do it right, and in other cases I've done rewrites that took far less time than the original design/coding effort, and paid off in far better software. George Seibel, UCSF speaking for myself only.
roberts@sunray.UUCP (Robert Stanley) (10/12/89)
In article <1989Oct10.015725.24517@agate.berkeley.edu> bks@alfa.berkeley.edu (Brad Sherman) writes: >Knuth makes at least 4 claims that some might think are contradictory >to current SE theory. > (Paraphrasing from memory): > 1) The designer should do the implementaion. > 2) The designer should do the testing. > 3) The designer should do the (first) user manual. > 4) After design, implementation and user input, > redesign and reimplement. > >Is this how it's done in your shop? If you also add in the referenced Fred Brooks quote: "Plan to throw the first one away!" 1) there is no better specification than a working prototype for communicating intent - who better than the designers to encode intent? 2) The designers should certainly test the real system, but this is not likely to be the same implementation as they created. Who better than the designers to test the system for conformance with the original intent and accuracy of results? Note, this does not rule out having testing performed by other groups as well, but speaking as a designer I would be more than somewhat upset if I didn't have a hand in testing a production-quality implementation of one of my designs! 3) Who else *but* the designer is qualified to write the first user manual, when effectively no-one else in the world knows anything about it? There are two additional points, a) we are going to throw the first one away, but the testers of it will need a first cut at user documentation, and b) I heartily endorse the theory of having a documentation specialist on the design team. I believe it was Apple, with the Macintosh documentation, who stated that if they had trouble writing the user documentation they sent the problem back to the developers... 4) Once you have it all working to everyone's satisfaction, start again from scratch, using what you have as a specification, and designing a clean and consistent architecture to support it. Quite frankly, given the enormous complexity of most of today's software, there is no other sane way to do it. When I started programming, 150 assembler statements was a largish program. When did you write one that small of late? >Read the article! Yes, very well worth it. I heartily endorse the practice of keeping a record. My own group has found that inclusion of an automated gripe facility also improves error reporting. It seems that when no change of context is required, users are more prepared to take a few moments out to record the details of a problem when it has just occurred, and on-screen context helps flesh out with details. It is quite interesting to see the consolidated gripe logs over time. Sure helps to have all the bug reports machine-readable from the outset, too. I am not a bug, I am an undocumented feature. Robert_S -- ____ ___ ___ Robert Stanley UUCP: uunet!mitel!sce!cognos!roberts | _ \ / _ \ / __| Cognos, Inc. INET: roberts%cognos.uucp@uunet.uu.net | |_> || |_| |\__ \ (Research) Voice: (613) 738-1338 x6115 |_| |_\|_| |_||___/ 45 21N 75 41W FAX: (613) 738-0002