worley@compass.UUCP (Dale Worley) (09/02/88)
Well, I agree that the Ada suite is very large compared to the suites available for other languages, but I have my doubts that it is any more thorough (relative to the number of language features to be tested). We have written on the order of 10,000 lines of Ada code here and have run into 5 or so bugs in validated compilers. Most of these seem not to be peculiar special cases of the algorithms that the compiler writers used, but rather certain requirements from the Reference Manual that were just coded up wrong. That is, if that particular sentence of the RM had been tested by the suite, the bug would have been caught. This leads me to believe that the ACVC doesn't cover the language very thoroughly. Further evidence of this is that compliance with Chapter 13 is just being added, and the statement in GCN (29 Aug 88): With the new set of tests, ACVC covers about three-quarters of the test objectives outlined in DOD's Ada implementation guidelines, Myers said. The objectives will be covered completely when the tests are revised again next year, he said. I guess what I really want to do is praise DOD for trying to develop the first genuine validation suite for testing compilers, and damn them (faintly) for not doing a comprehensive enough job of it. Dale