[comp.cog-eng] Studies with more than 40 test users

nielsen@bellcore.com (Jakob Nielsen) (01/16/91)

Most empirical studies and user testing seems to be done with a rather small
number of test users. Indeed, I often recommend the use of small samples
myself for practical usability engineering purposes.
For some current research on the statistical characteristics of user testing
and the effect of varying the number of test users, I need access to larger
data sets with results from empirical tests of user interfaces.
Let us say that "large" = 40 subjects in the HCI field, even though this
is modest compared with other fields where one can easily find studies
with thousands of test users.
I am interested in both objective measurements (such as time or errors
when completing some task) and/or subjective measurements (user preferences
or heuristic evaluations).

If you have such data sets and might be willing to let me look at them,
I would be grateful for an email message. It does not really matter
what systems you studied, and I can let you remain anonymous or give
you credit as you like. 
--
Jakob Nielsen, Bellcore MRE-2P370, 445 South St, Morristown, NJ 07962-1910, USA
email nielsen@bellcore.com, tel. +1-201-829-4731, fax +1-201-538-9093

nielsen@bellcore.com (Jakob Nielsen) (01/22/91)

I have received a rather small number of replies to my request for data from
empirical user interface studies with 40 test users or more. Therefore, I am
still interesting in hearing about any additional such studies.

Several people told me that they originally had such data but that it was not
longer available. Unfortunately, much data seems to be deleted after the
publication of papers, Ph.D. theses and such, making it difficult to do the
kind of meta-study I am planning.
--
Jakob Nielsen, Bellcore MRE-2P370, 445 South St, Morristown, NJ 07962-1910, USA
email nielsen@bellcore.com, tel. +1-201-829-4731, fax +1-201-538-9093