lmjm@doc.ic.ac.uk (Lee McLoughlin) (07/07/87)
[[RELAYED ON BEHALF OF anton@ist.co.uk - please reply directly to him.]] Can anybody help me please ? I am interested in commercially available test harnesses or coverage monitors for Ada. Idealy, what I am after is something that is reasonably portable and not too compiler specific. If anyone knows of such things, then I would be most grateful to receive details. Thank you. Anton Gibbs Imperial Software Technology E-mail: anton@ist.co.uk or anton@ist.uucp Telephone: +44 1 581 8155 Telex: 928476 istech g Surface: Imperial Software Technology Ltd., 60 Albert Court, Prince Consort Rd., London SW7 2BH ENGLAND ========================================================== ----- End of forwarded messages
eric@sarin.UUCP (Eric Beser sys admin) (07/14/87)
The one test tool that we are using is the software analysis
workstation by Northwest Instruments. It is a logic analyzer for
the software engineer.
To call it a logic analyzer may be a bit unfair. It passively sits
on the processor and does monitor the logic states of the machine.
This device connects to the IBM PC/XT or AT by a parallel port that
is memory mapped into the system. Using the Gem operating system
by Digital Research (mouse, etc) it can monitor up to two processors.
It will take the symbol table output of a linker and use the symbols
for software analysis. We are using this tool to integrate the Telesoft
Ada Compiler with a comercial operating system on a distributed
68020 platform. As many of you know, the highest area of risk in this
venture is to solve the intricate timing problems involved with the Ada
RTE, hardware and application code. Because the analysis is symbolic
you get to see what portions of code execute, how much time it takes
to get through a procedure, the percentage of time spent in the RTE,
how interrupts are being serviced and dealt with in the RTE. It provides
2167 spec code coverage to validate the test procedures.
The only requirement for use is that the compiler puts out a symbol map
with enough symbols to do the job. Northwest Instruments supports their
own format, and the Unix Namelist (COFF). We have adapted the map
output of the Telesoft Linker, our own 1750A ada compiler, and various
Jovial compilers. It works well with C too (but we don't talk about
languages that ruin you as software engineers).
We have done a comparison study between the Software Analysis Workstation
and Hewlet Packard's 64000 workstation. Both are good. NWIS beats HP
on price and in many areas on performance as well. HP does statistical
analysis. With NWIS, you can time and size a program with one execution
of the program.
Eric Beser
Westinghouse DEC
EBESER@ADA20 (arpa)
{seismo,mimsy}aplcen!cp1!sarin!eric (usenet)margono@grebyn.com (Johan Margono) (02/01/91)
I am the process of trying to locate possible testing tools to assist in
testing of Ada software. Some of my considerations are:
1. The platforms that we are using...
a. Rationals;
b. MVS on IBM 3090s running TeleSoft Ada;
c. AIX on IBM RS6000 running TeleSoft/OCS Ada.
2. The greatest need is for Test Driver generators. A tool that will
read in an Ada specification and generate an Ada program (with a
user interface) that will allow a "tester" to call all operations
and test exceptions, using his/her test cases.
3. Test Case/Scenario generation tools would be nice. Possible
methods could include branch testing, McCabe's basis path method,
and black-box testing (equivalence class partitioning and
boundary analysis).
4. Test tracking tools would be nice. Some tools assume an 'ad hoc'
method for test case generation, and show branch coverage (so you
can try a few more cases). Assuming a structured testing
approach, this tracking does not seem very valuable. However,
tracking as a support for Regression Testing would be very
helpful.
5. Also related to (4), support for black-box testing of multiple
instantiations of generics would be helpful (i.e., reuse of
testing plans and test cases).
6. Also related to (4) and (5), support for (at least) black-box
testing of modules (generic/non-generic) on multiple platforms
(rerunning all, or a subset, of the tests for each target).
7. Many of the tools I have l looked at, depend on instrumentation
of the code for test tracking, and possibly test generation.
Since, instrumentation changes the 'code,' the load module, and
thus behavior, this method is NOT preferred (it also becomes a
maintenance headache). Non-intrusive methods are preferred!
8. We are also restrained by 2167 (not A)!
I have collected information about several tools, but I am interested in
information on:
1. additional tools not listed below,
2. evaluations ("consumer reports") on any of the tools (listed or
appended),
3. (real) project experience on testing LARGE Ada systems, and the
tools and techniques that helped/hindered.
----------------------- Current Testing Tool List ------------------------
1. Rational - public domain test driver generator which uses the
Rational DIANA tree (available from the Rational Catalog). Need
to check if the test drivers are portable to other platforms.
2. Nokia - (From Finland.) TBGEN and TCMON; will run on the RAT and
PCs.
3. Software Systems Design - TestGen does not have a driver generator.
The test case generator is written in C so it won't run on the
Rational.
4. McCabe Associates - no test driver, but other nice tools. I don't
know if it will run on our platforms.
5. Intermetrics - Ada Test Support Tool (ATST) is public domain. It has a
test driver, but only runs on VAX/VMS/ACS and PC/DOC/Alsys. It
seems to only support black-box testing.
6. General Research Corporation - AdaQuest, was the Ada Test and
Verification System (ATVS). Only VAX/VMS, with a UNIX platform
later this year.
7. Glazier Electronics - The Automated Real-time Regression Testing
system (ARRT). Only on VAX, only run time testing.
8. CISI Engineering - (From France) was a research project, that was not
"productized"
--------------------- End Current Testing Tool List ---------------------
Snailmail: Gregory M. Bowen, Reuse Group
Computer Sciences Corporation
Advanced Automated System
15245 Shady Grove Road
Rockville, Maryland, 20850 USA
Phone: 301.921.3018
FAX: 301.840.8934
AdaNET ID: gbowen
Internet: margono@grebyn.com or gbowen%ceo@asv2.wvnet.edubryk@IDA.ORG (Bill Brykczynski) (02/08/91)
The following is in response to a recent request for information
regarding tools to assist in the testing of Ada software. We are just
beginning to collect such information. In the future, we may be doing
an evaluation of several of these tools. This information is not
currently not complete, but may still be of use.
We are not recommending any of these tools, we are simply collecting
information on them. If you have additional information on these or
other tools, experience with such tools, etc., please let us know.
As you can see, some of the information is skimpy.
Bill Brykczynski
Institute for Defense Analyses
1801 N. Beauregard St.
Alexandria, VA 22311
703-845-6641
bryk@ida.org
--------------
- 1 -
Alsys, Inc.
Tool: AdaTune, AdaProbe
Contact: Lori Heyman (617) 270-0030
One Burlington Business Center
67 South Bedford Street, Burlington, MA
01803-5152
Application Development System, Inc.
Tool: XPEDITER (Test Driver)
Contact: E.F. Harris
1530 Meridian Avenue, San Jose, CA 95125
Cadre Technologies
Tool: Software Analysis Workstation (SAW),
real-time performance measurement.
Tool: CodeMap, tracking tool to see which
software parts are exercised.
Tool: XRAY, simulator/debugger to execute code
on workstation (prior to target macine).
Rapid prototyping.
Tool: MDD, design phase analysis to check
performance.
Tool: SoftAnalyst, Sym Trace provides symbolic
trace of program and data flow.
Contact: Myra Linch (703) 875-8670
Convex Computer Corp.
Tool Data: Interactive performance profilers, &
Verdix development environment.
Contact: Larry Grossman (214) 497-4383
3000 Waterview Pkwy, P.O. Box 883851,
Richardson, TX 78053-5851
DDC-1
- 2 -
Tool: Symbolic Ada debugger, non-intrusive,
can input file of test data
Contact: Forest Holeman (602) 944-1883
Dynamics Research Corp.
Tool: AdaMAT
Contract: John Ragosta (508) 475-9090
60 Frontage Road, Andover, MA 01810
EVB Software Engineering Inc.
Tool: Complexity measurement
Contact: Jennifer Lot (301) 695-6960
5320 Spectrum Drive, Frederick, MD 21701
Generics Software Ltd.
Tool: AnimAda
Contact: +353-1-954012
7 Leopardstown Office Park, Foxrock,
Dublin 18, Ireland
INRIA
Tool: Mentor-Ada, measurement system
Contact: Schroeder A.
Domaine de Voluceau, B.P. 105,
Rocquencourt, 78153 Le Chesnay, France
ITT Avionics
Tool Data: Funded by STARS
Tool: UATL: Universal Ada Test Language
(Test Driver)
Contact: Ziegler J. (201) 284-5030
390 Washington Avenue, Nutley, NJ
070110-5603
- 3 -
Jackson Systems Corporation
Tool Data: data against data and program structures
to verify structure correctness.
Structures may be derived form test
data. Test data may be generated to
create templates for program testing.
"Contact:" Michael Jackson Systems Limited, London
Programming Environments
Tool: T test case generator
Description: Verifies requirements testability,
produces test cases, and analyzes
testing comprehensiveness. Inputs and
outputs are documented, flat ASCII files
to allow working with descriptions from
other tools. Designs test cases
automatically or provides help in a
computer-aided mode, for 100% functional
coverage and 100% most-probable-error-
coverage. Runs under MS-DOS, UNIX, and
VAX/VMS. Single prince copy for MS-
DOS/UNIX systems is $7000.
Contact: Robert Poston work (201) 918-0110, home
(201) 922-0968
PE Inc.
4043 State Highway 33, Tinton Falls, NJ
07753
SQL Systems International
Tool: PCMS*ADA, dependency analysis
Contact: England, 44-279-641021
Set Labs
Tool: UX-metric for estimates of errors in
- 4 -
design. Support for Ada.
Contact: Teresa Harrison (503) 289-4758
Software Research
Tool Data: Full Ada support, several environments
including Sun.
Tool: Capbak/UNIX, capture and playback
of test sessions.
Tool: EXDIFF (Extended Difference
Analyzer)
Tool: SMARTS (Software Maintenance and
Regression Testing Suite)
Tool: STCAT/C (System Test Coverage
Analysis Tool), function-call
coverage.
Tool: TCAT/C (Test Coverage Analysis
Tool)
Tool: TCAT/Path, profiles paths and
produces graph.
Tool: TDGen, test data generation
Contact: Frank Vimaggi? (415) 957-1441
Teledyne Brown Engineering
Tool Data:
Tool: ACAT: Ada Complexity Analysis Tool
Tool: TAGS/CASE2, diagnostic analyzer
provides static analysis to ensure
IORL syntax is complete and
correct, and analysis library
Contact: Mike Cario 352-8533
TeleSoft
Tool: Telegen2 (Telesoft development
environment), browser/profiler
- 5 -
Contact: Jeff Kelley (619) 457-2700
5959 Cornerstone Court West, San Diego,
CA 92121
Verilog SA
Tool Data:
Tool: LOGISCOPE, control flow testing at
unit and integration level,
currently DD-path and PP-path, some
to include LCSAJ, graphical display
of coverage with cumulative
coverage reporting, full Ada.
Tool: ASA, support for SADT IDEF0,
simluation and test case generation
for validation of requirements and
validation of system against
requirements.
Tool: GEODE, supports CCITT SDL
language, in 6 - 8 months expected
to generate Ada code.
Contact: Nandan Shah (technical), Mark Luciw,
Susan Weil (sales) (703) 354-0371 gyro@kestrel.edu (Scott Layson) (02/08/91)
I am implementing an Ada front end (more or less) and I have come across a rule in the (mostly very precise) manual that seems unclear. Perhaps someone can help me with it. It has to do with paragraphs 18 and 19 of section 4.1.3. (Please refer to the manual at this point.) Paragraph 17 talks about four kinds of named constructs whose names can be used as the prefix of an expanded name. Paragraphs 18 and 19 give additional rules about only two of those kinds of constructs, viz., subprograms and accept statements. What about block statements and loop statements? Thanks in advance, Scott Burson Gyro@Reasoning.COM