marick@cs.uiuc.edu (Brian Marick) (02/06/91)
As it happens, I just started writing an explanation of weak mutation testing. Here's some text. (Forgive it; it's only a first draft.) .sh 2 "Fault-based Coverage" .lp In weak mutation coverage [Howden] [Hamlet], we suppose that a program contains a particular simple kind of fault. One such fault might be using <= instead of the correct < in an expression like this: .nf if (A <= B) .fi Given this program, a weak mutation coverage system like GCT would produce a message like this: .nf "gcc.c", line 488: operator <= might be < .fi This message would be produced until the program was executed over a test case such that $(A <= B)$ has a different value than $(A < B)$. That is, we must satisfy a _test condition_ that .EQ (A <= B) != (A < B) .EN or, equivalently, that .EQ A != B .EN Notice the similarity of this requirement to the old black-box testing advice: "always check boundary conditions". .lp Suppose we execute the program and satisfy this test condition. In this case, the incorrect program (the one we're executing) will take the wrong branch. Our hope is that this incorrect program state will persist until the output, at which point we'll see it and say "Bug!". Of course, the effect might not persist. However, there's some evidence [Marick] that over 90% of such faults will be detected by weak mutation coverage. Of course, not all faults are such simple deviations from the correct program; probably a minority are [Marick]. However, the hope of weak mutation testing is that a test suite thorough enough to detect such simple faults will also detect more complicated faults; this is called the _coupling effect_ [DeMillo]. The coupling effect has been shown to hold in some special cases [Offutt], but more experiments are needed on large-scale software. .lp GCT implements two kinds of weak mutation testing. _Operator_ coverage is as already described -- we require test cases that distinguish operators from other operators. _Operand_ coverage is similar -- for any operand, such as a variable, we assume that it ought to have been some other one. That is, in .nf my_function(A) .fi we require test cases that distinguish A from B (test condition $A != B$), A from C, A from D, and so on. Since there may be a very large number of possible alternate variables, there are usually a very large number of test conditions to satisfy. The hope is that a relatively few test cases will satisfy most of them. Brian Marick Motorola @ University of Illinois marick@cs.uiuc.edu, uiucdcs!marick
marick@cs.uiuc.edu (Brian Marick) (02/07/91)
marick@cs.uiuc.edu (Brian Marick) writes: a typo. The test condition that distinguishes A<=B from A<B is not >A != B of course. It's A==B, which causes A<=B to be TRUE and A<B to be FALSE. Brian Marick Motorola @ University of Illinois marick@cs.uiuc.edu, uiucdcs!marick