[comp.software-eng] Interim Validation Suite Bibliography

wood@xyzzy.UUCP (Tom Wood) (12/31/87)

I've had several inquiries concerning my findings on language
validation, verification, and test suites, so I'm posting my
current references.  I've organized this according to language.

I'm still in the process of tracking down these leads, and I cannot
attest to the accruacy of this information.  I have sent away for
more information on six of the C suites.

If you can add any information to this posting, please let me know.

=============================== ADA ===============================        
        
ACVC - Ada Compiler Validation

ACEC - Suite to test Ada performance.  
        
The University of Michigan has a suite of Ada tests and benchmarks.

Don't forget the DoD Ada validation suite!  It's supposed to be one
of the most strenuous around, and is applied to all Ada compilers
before they are allowed to use the name (since Ada is a registered
trademark of the DoD, they can do that).

============================= ALGOL68 =============================        

There used to be an ALGOL68 test set.  This set was maintained by
Dick Grune while working at the CWI [Amsterdam].  His current
address is dick@cs.vu.nl. He can give you more details about the
suite.

================================ C ================================        

SVVS - System V Verification Suite also tests the C compiler,
        especially the libraries.  (From AT&T)

ACE C VALIDATION SUITE - Excerpts from a press release below.
        Checks the compliance of a C compiler to the ANSI X3J11
        draft standard.  Also tests library functions for
        conformance to the X/OPEN standard or the SVID where these
        differ from their draft standard counterparts.
        
        The suite consists of over 50000 lines of source in about
        600 different programs together performing over 2000 tests.
        The ACE C Validation suite is priced at 15500 dutch
        guilders (currently approximately $7500).

        Contact Ge Gaal at info@ace.nl or ..!uunet!mcvax!ace!info

        ACE Associated Computer Experts bv      Phone: +31 20 646416
        Van Eeghenstraat 100                    Telex: 11702 (ace nl)
        1071 GL Amsterdam                       Fax: +31 20 750389
        The Netherlands

MetaWare sells a small C validation suite.  $2,000
	903 Pacific Ave, Suite 201
	Santa Cruz, CA  95060
	(408) 429-6382

The Plum Hall Validation Suite for C  $10,000
	Plum Hall
	1 Spruce Ave.
	Cardiff, NJ  08232
	(609) 927-3770

The PERENNIAL Validation Suite for C Compiler Validation
	PERENNIAL
	4677 Old Ironsides Drive, Suite 450
	Santa Clara, CA  95054
	(408) 727-2255

C Compiler Torture Test - Checks a compiler against K&R.  $20
        The Austin Code Works
        11100 Leafwood Lane             
        Austin, TX  78750-3409
        (512) 258-0885

HCR offers a C Test Suite in various forms (50,000 tests to 350,000 tests) 
	HCR Corporation			Phone:	(416) 922-1937
	130 Bloor Street West		Telex:	06-218072 HCR TOR
	Suite 1001			Fax:	(416) 922-8397
	Toronto, Ontario
	Canada
	M5S 1N5

Unix had SQ and CQ (Grampst) for Unix and C, but these are AT&T internal.   

============================= FORTRAN =============================        

Though it was not available outside Hewlett-Packard when last I
knew, I thought you might like to know about it for completeness.
I helped to design a testing suite for HP FORTRAN 77 aproximately 4
years ago.  Back then there were ~6000 tests and about 750K lines
of FORTRAN code in the test suite.  Inquiries could be made to the
Computer Language Lab, 11000 Wolfe Road, Cupertino, CA.  There was
a smaller testing suite for Pascal.

The Federal Govt validation suites for Fortran and Cobol are not
available to the general public.  The last time I heard, a single
test run cost a vendor about $15K, making it useless for those of
us who develop language-processing tools.  Even the reports of the
tests may be of little use to users; I'm told they consist mainly
of pass/fail tables.

Fortran:
        US Department of Commerce
        National Technical Information Service
        Springfield, VA				    

============================== PASCAL =============================        

PVS - BSI Pascal Compiler Validation Service (BS 6192/ISO 7185)
        This is a third party validation service.  It includes
        access to the validation suite by the vendor.  Developed in
        part by the University of Tasmania and the National
        Physical Labratory.  Adopted by the Federal Software
        Management Support Center (part of the GSA) in 1986.

        Assessment Department           Phone: 0442 3111
        British Standards Institution   Telex: 82424
        Maylands Avenue
        Hemel Hempstead, Herts
        HP2 4SQ

        Federal Software Management Support Center
        General Services Administration
        Two Skyline Place, Suite 1100
        5203 Leesburg Pike
        Falls Church, VA  22041

Pascal:
        Richard J Cichelli, President
        Software Consulting Services
        3162 Bath Pike                
        Nazareth, PA   
        (215) 837-8484
        
The University of Tasmania in Australia distributed a Pascal
        Validation Suite as of 1981.  

Arthur Sale of UNSW did the Pascal Validation Suite available from
        ANSI (indirectly, they can tell you who to get it from).

============================== SNOBOL =============================        

Algorithms in SNOBOL - Actually, this is a book, but the programs
        discussed are sufficiently complex (and I mean COMPLEX)
        that they are accepted as a good SNOBOL compiler test
        suite.
-- 
			Tom Wood
			Data General, Research Triangle Park, NC
			{the known world}!mcnc!rti!xyzzy!wood