marick@m.cs.uiuc.edu (09/12/90)
This note announces the public availability of version 1.2 of a tool
for measuring the branch coverage of C programs. The tool was written
by Thomas Hoch, K. Wolfram Schafer (University of Illinois), and Brian
Marick (Motorola).
The tool is suitable for use in production environments. However, it
is unsupported and comes with no warranty. A mailing list is available
for mutual support (see below); I will read it and I may fix bugs or
documentation in my spare time.
If you are a Motorola employee, I will provide full support. Motorola
pays my salary.
Description
This tool is used to measure whether a test suite forces every branch
in a program to be taken in every possible way. If it hasn't, you may
need to write more tests. The tool is intended to be used in the
testing of large systems written in C, including the UNIX kernel and
embedded systems. Such systems have certain characteristics that
affected the design:
1. The process of compiling the system is a complex task in its own
right. The tool should require only minor and isolated changes to
this process. (If you use makefiles, usually only one needs to be
changed.)
2. Such systems often must be cross-compiled on a host and then
downloaded to the target machine, or they depend on a particular C
compiler in some other way. Therefore, adding branch coverage
instrumentation must be a source-to-source transformation.
3. Embedded systems will not be able to use ordinary file I/O. The
internal interface is designed and documented so that the tester can
easily write the code used to extract branch coverage information.
(When testing application programs, you just use the code provided
with the tool.)
Example
Suppose you have this program:
main(argc)
{
if (argc % 2 == 0)
printf("argc is even.\n");
while (argc--)
printf("argc %d\n", argc);
btool_writelog("LOG"); /* Must be added to use the tool. */
}
When you instrument it, you get this program:
main(argc)
{
if(btool_branch(0, (argc % 2 == 0)!=0))
printf("argc is even.\n");
while(btool_branch(1, (argc--)!=0))
printf("argc %d\n", argc);
btool_writelog("LOG");
}
After compiling, you would execute the program once to get:
% a.out
argc 0
% mv LOG LOG.1
% breport LOG.1
"test.c", line 3: if was taken TRUE 0, FALSE 1 times.
"test.c", line 5: while was taken TRUE 1, FALSE 1 times.
And again, to get:
% a.out 1 2 3 4 5
argc is even.
argc 5
argc 4
argc 3
argc 2
argc 1
argc 0
% mv LOG LOG.2
% breport LOG.2
"test.c", line 3: if was taken TRUE 1, FALSE 0 times.
"test.c", line 5: while was taken TRUE 6, FALSE 1 times.
You can check the branch coverage with
% bmerge LOG.1 LOG.2 | breport | bsummary
Branches taken in true and false direction 2 (100.00%)
Branches taken only in true direction 0 ( 0.00%)
Branches taken only in false direction 0 ( 0.00%)
Branches not taken at all 0 ( 0.00%)
Total number of branches 2
Cases taken 0 ( 0.00%)
Total number of cases 0
You've reached 100% branch coverage.
The tool can also be used for profiling. For example, a UNIX kernel
tester discovered a significant performance problem by noticing how
many times a while loop had been executed.
Deficiencies
1. The tool itself runs only on UNIX, although the instrumented code
may be compiled wherever you like.
2. The tool is based on the GNU C compiler and still contains the
original code generation and optimization code. This has two
practical disadvantages:
- The tool is large: 23000 512-byte blocks are needed when compiling the
distribution. The binaries take up 4500 blocks.
- The original compiler is made portable through machine and system
configuration files. These must still be used. The distribution
includes configuration files for many different machines. If there
isn't one for your machine, you may be able to use one for a similar
machine. For example, since both are derived from BSD UNIX, a tool
using a VAX configuration file can be compiled and used on a Sun. The
tool is known to run on Sun-3, Motorola SysV68, and Motorola 88K
machines.
Availability
Get the tool via anonymous FTP from cs.uiuc.edu. It is in the
directory pub/testing/btool.tar.Z and is a compressed tar file.
Mailing List
Btool@ernie.cs.uiuc.edu is a mailing list for btool users. It will be
used to distribute bug reports, bug fixes, troubleshooting hints,
documentation errata, and anything else of interest to btool users.
Mail to Btool-Request@ernie.cs.uiuc.edu will get you added to the
list. When you're added, I'll send you the latest set of 1.2
documentation updates.
Brian Marick
Motorola @ University of Illinois
marick@cs.uiuc.edu, uiucdcs!marickcox@software.org (Guy Cox) (09/12/90)
I have a coverage analyzer for C which runs under VMS. It allows full I/O from the program under test. The instrumentation overhead is extremely low as the program communicates branch "Hits" to another process which does the bean counting. It is written in Ada and would require a re-write to run on anything other than VMS (uses CLD,LIB$TPARSE,QIOWs,HELP, ,etc). If anyone is wondering why we wrote the thing in Ada to do C it's because the instructor, the editor of the OOP Journal and The Journal of Pacal Ada and Modula-2, wanted the instrumenter done for either C or Modula-2 and we had a team with defense contractor who did not want to relearn C or Modula-2 as an implementation language. We also wanted to get away with writing as few lines of Ada as we could so the VMS libraries were heavily used along with the Booch Components. It does a nice job and maintains statistics on the code under test for the individual testcases as they are run. I can make it available via FTP (soon) or a TK50. -- // //Remember; Tuesday is Soylent green day! // Guy O. Cox, Jr. Software Productivity Consortium. 2214 RockHill Rd Herndon, VA 22090 703-742-7219 cox@software.org