wood@xyzzy.UUCP (Tom Wood) (12/31/87)
I've had several inquiries concerning my findings on language
validation, verification, and test suites, so I'm posting my
current references. I've organized this according to language.
I'm still in the process of tracking down these leads, and I cannot
attest to the accruacy of this information. I have sent away for
more information on six of the C suites.
If you can add any information to this posting, please let me know.
=============================== ADA ===============================
ACVC - Ada Compiler Validation
ACEC - Suite to test Ada performance.
The University of Michigan has a suite of Ada tests and benchmarks.
Don't forget the DoD Ada validation suite! It's supposed to be one
of the most strenuous around, and is applied to all Ada compilers
before they are allowed to use the name (since Ada is a registered
trademark of the DoD, they can do that).
============================= ALGOL68 =============================
There used to be an ALGOL68 test set. This set was maintained by
Dick Grune while working at the CWI [Amsterdam]. His current
address is dick@cs.vu.nl. He can give you more details about the
suite.
================================ C ================================
SVVS - System V Verification Suite also tests the C compiler,
especially the libraries. (From AT&T)
ACE C VALIDATION SUITE - Excerpts from a press release below.
Checks the compliance of a C compiler to the ANSI X3J11
draft standard. Also tests library functions for
conformance to the X/OPEN standard or the SVID where these
differ from their draft standard counterparts.
The suite consists of over 50000 lines of source in about
600 different programs together performing over 2000 tests.
The ACE C Validation suite is priced at 15500 dutch
guilders (currently approximately $7500).
Contact Ge Gaal at info@ace.nl or ..!uunet!mcvax!ace!info
ACE Associated Computer Experts bv Phone: +31 20 646416
Van Eeghenstraat 100 Telex: 11702 (ace nl)
1071 GL Amsterdam Fax: +31 20 750389
The Netherlands
MetaWare sells a small C validation suite. $2,000
903 Pacific Ave, Suite 201
Santa Cruz, CA 95060
(408) 429-6382
The Plum Hall Validation Suite for C $10,000
Plum Hall
1 Spruce Ave.
Cardiff, NJ 08232
(609) 927-3770
The PERENNIAL Validation Suite for C Compiler Validation
PERENNIAL
4677 Old Ironsides Drive, Suite 450
Santa Clara, CA 95054
(408) 727-2255
C Compiler Torture Test - Checks a compiler against K&R. $20
The Austin Code Works
11100 Leafwood Lane
Austin, TX 78750-3409
(512) 258-0885
HCR offers a C Test Suite in various forms (50,000 tests to 350,000 tests)
HCR Corporation Phone: (416) 922-1937
130 Bloor Street West Telex: 06-218072 HCR TOR
Suite 1001 Fax: (416) 922-8397
Toronto, Ontario
Canada
M5S 1N5
Unix had SQ and CQ (Grampst) for Unix and C, but these are AT&T internal.
============================= FORTRAN =============================
Though it was not available outside Hewlett-Packard when last I
knew, I thought you might like to know about it for completeness.
I helped to design a testing suite for HP FORTRAN 77 aproximately 4
years ago. Back then there were ~6000 tests and about 750K lines
of FORTRAN code in the test suite. Inquiries could be made to the
Computer Language Lab, 11000 Wolfe Road, Cupertino, CA. There was
a smaller testing suite for Pascal.
The Federal Govt validation suites for Fortran and Cobol are not
available to the general public. The last time I heard, a single
test run cost a vendor about $15K, making it useless for those of
us who develop language-processing tools. Even the reports of the
tests may be of little use to users; I'm told they consist mainly
of pass/fail tables.
Fortran:
US Department of Commerce
National Technical Information Service
Springfield, VA
============================== PASCAL =============================
PVS - BSI Pascal Compiler Validation Service (BS 6192/ISO 7185)
This is a third party validation service. It includes
access to the validation suite by the vendor. Developed in
part by the University of Tasmania and the National
Physical Labratory. Adopted by the Federal Software
Management Support Center (part of the GSA) in 1986.
Assessment Department Phone: 0442 3111
British Standards Institution Telex: 82424
Maylands Avenue
Hemel Hempstead, Herts
HP2 4SQ
Federal Software Management Support Center
General Services Administration
Two Skyline Place, Suite 1100
5203 Leesburg Pike
Falls Church, VA 22041
Pascal:
Richard J Cichelli, President
Software Consulting Services
3162 Bath Pike
Nazareth, PA
(215) 837-8484
The University of Tasmania in Australia distributed a Pascal
Validation Suite as of 1981.
Arthur Sale of UNSW did the Pascal Validation Suite available from
ANSI (indirectly, they can tell you who to get it from).
============================== SNOBOL =============================
Algorithms in SNOBOL - Actually, this is a book, but the programs
discussed are sufficiently complex (and I mean COMPLEX)
that they are accepted as a good SNOBOL compiler test
suite.
--
Tom Wood
Data General, Research Triangle Park, NC
{the known world}!mcnc!rti!xyzzy!wood