[net.general] Fallacies of testing

lyang@sun.uucp (Larry Yang) (09/22/86)

Now that we're on the subject of drug testing, I thought that it would be
appropriate to share some information I have on testing in general.
Anyone who has had any training in statistics or probability will probably
be familiar with this stuff already, so you needn't read on.

Let us assume that we have this wonderful test that:

	will test positive 99% of the time if the subject has the condition
	will test negative 99% of the time if the subject doesn't have it

Let us suppose that in the population 1% of the people have the condition.

Let there be 10,000 people in our study.  Below is a chart showing
have/don't have vs. positive test/negative test:


		#have |	#don't have
              ----------------
	pos.	  99  |	  99
              ----------------
	neg.      1   |	9801
              ----------------
      total	100	9900

Note that the columns should add up to be 100/9900, since we assumed
that 1% of our population had the condition.

The interesting result is that of the people who tested positive,
99 had the condition and 99 didn't.  Thus, if a person tested positive,

***** HE/SHE HAS A 50/50 CHANCE OF HAVING THE CONDITION!! *****

What does this mean?  Basically, the moral of the story is that
no matter how good your test is, if the condition you are testing for
is sufficiently rare, you will have a very large number of
false positives (People who test positive but do not have the condition).
In medicine, doctors face this dilemma all the time.  The perform a wonderful
test for a rare disease and end up performing surgery unnecessarily
may times.

How does this relate to drug testing?  I think that it's fairly
obvious.  In this example, I chose a really good test to test for
a fairly common condition, and the result was that there was a 50%
chance of being wrong.  Drug abuse is a condition that hopefully
exists in less that 1% of the population; that would make the chance
of error even greater.

Just a random thought.....

-- 
==============================================================================
                                         /-\   /\    |
-- Larry Yang                          __|_/    |    |
   Sun Microsystems, Inc.                |      |    | /--\ /\/-\/--\
   Mountain View, CA  94043            /-|\     \    / |  | ||  ||  |
   lyang@sun.COM                       \_/ \     \__/| /--/\/|  \/--| /
   {ihnp4|hplabs|seismo}!sun!lyang                 __|/           __|/
                                                  /  /           /  /
                            (a REAL signature)    \_/            \_/
==============================================================================
  "If you don't like the news, go out and make some of your own..."
  "If you don't like the junk on usenet, post junk of yo.. <FLAME> AAAAGH!!.."
==============================================================================