MVILAIN@G.BBN.COM (Marc Vilain) (11/08/88)
BBN Science Development Program
AI Seminar Series Lecture
SHERLOCK: A COACHED PRACTICE ENVIRONMENT
FOR AN ELECTRONICS TROUBLESHOOTING JOB
Susanne P. Lajoie
Learning Research and Development Center,
University of Pittsburgh
(LAJOIE%LRDCA@Vms.Cis.Pittsburgh.Edu)
BBN Labs
10 Moulton Street
2nd floor large conference room
10:30 am, Tuesday November 15
Sherlock is a computer-based practice environment for teaching
first-term airmen avionics troubleshooting skills. Sherlock's
instructional goals were determined by a cognitive task analysis of
skill differences in this domain. The predominant instructional
strategy is to support holistic practice of troubleshooting rather
than train discrete knowledge skills. Instruction is based on complex
decision graphs of skilled and less skilled plans and actions for each
troubleshooting problem. As a trainee works through a problem Sherlock
observes the quality of decisions the trainee makes and uses that
information to provide the level of hint explicitness necessary at
particular decision points in the problem. In this way, specific
competency building is situated within the troubleshooting context and
is sharpened to the extent that satisfies each individual's needs.
Sherlock was field tested in a controlled study that compared tutored
trainees with a control group that received no extra training other than
"on-the-job" experience. Pre and post tests of verbal troubleshooting
indicated that the tutored group performed better than the control group
on post tests of troubleshooting proficiency. Not only were more
problems solved but there were several indications of emerging
competence over the course of tutoring that demonstrated that trainees
were becoming more "expert-like" in the overall troubleshooting process.
In an independent evaluation the Air Force found the Sherlock treatment
to be equivalent to 47-51 months of "on the job" experience.
Enhancements have been added to Sherlock that could increase its
effectiveness even more. An explicit articulation of expert and
student problem solving traces now exists that could facilitate the
comparison process of different levels of expertise. At the completion
of each problem trainees will be able to interrogate the trace of the
expert problem solution and see why an expert would make a particular
move as well as see the mental models used by an expert to test
different paths in the problem space.
------------------------------------
This research was made possible through the combined efforts of the
following individuals: Alan Lesgold, Jaya Bajpayee, Marilyn Bunzo,
Gary Eggan, Linda Greenberg, Debra Logan, Thomas McGinnis, Cassandra
Stanley, Arlene Weiner, Richard Wolf, and Laurie Yengo, as well as
researchers at AFHRL Brooks, and the Air Force personnel that made our
study possible.
-------