smp@sei.cmu.edu (Stan Przybylinski) (07/03/90)
Since I started monitoring this group i have been waiting for any talk of
Intel's PRO750 digital video interactive (DVI) product line. Are there any
other developers out there?
We are working on an interactive training simulation used to teach code
inspections, a software quality assurance technique. A short article is
included below.
Stan Przybylinski
Transition Manager - Advanced Learning Technologies Project
Software Engineering Institute
Carnegie Mellon University
Pittsburgh, PA 15213-3890
smp@sei.cmu.edu (412) 268-6371
-----------------------------------------------------------------------------
USING TECHNOLOGY TO TRANSITION TECHNOLOGY
By William K. McSteen
Just as interactive video games have made their way into many homes,
interactive video and artificial intelligence are making their way into
software engineering education. The Advanced Learning Technologies (ALT)
Project in the SEI Education Program is using advanced hardware and software
technology to teach code inspections.
Project leader Scott Stevens says, "By marrying intelligent tutoring systems
with interactive video, we're creating a highly interactive simulation of an
inspection, one that can't be done with traditional means." Interactive
courses can be distributed economically to a broad audience, and they can
provide a remarkably effective learning experience.
WHY CODE INSPECTION?
Code inspection is a formal review process that effectively identifies defects
in software code. Inspections can be used by many people on a software
project and are applicable during the development, testing, and maintenance
phases. Moreover, the inspection methodology and the results of its use are
well documented. Companies such as AT&T and IBM have performed and taught
code inspection for more than ten years.
Stevens says, "Code inspection is a high-payoff topic. There's enough
background there that we can say that we should be teaching the inspection
methodology because we know that it's effective."
However, code inspection (and software engineering in general) has
traditionally been difficult to teach. According to Stevens, no amount of
reading or lecturing can substitute for practical experience. Code inspection
is labor-intensive and requires instructors skilled in designing and
delivering instruction oriented toward group dynamics. Instructors are
primarily technical people, skilled in software engineering issues but not
experienced in instructional design or group leadership. Most institutions do
not have the resources necessary to develop the expertise needed to
effectively teach code inspections. The course that the ALT Project is
developing can capture this expertise and deliver it when and where it is
needed.
Teaching code inspection also highlights the presentation technology. Stevens
says, "Having the computer truly integrated with the video gives you something
greater than the sum of the two individual mediums. It's a brand new medium,
and you can teach technical material and more interpersonal material. For
inspections, we need to teach both."
A course on inspections needs to teach technical skills, such as good Ada
programming style, and interpersonal skills, such as what to do when
inspection participants are long-winded or argumentative. "We think that we
can have a realistic simulation of the group process in an inspection by
combining the computer and the video," says Stevens. "People need experience
doing code inspections, and we're going to give them that experience through
technology."
WHY INTERACTIVE VIDEODISC AND AN EXPERT SYSTEM?
The hardware technology that the project plans on using is GE/RCA's Digital
Video Interactive (DVI), which stores full-motion video, audio, graphics, and
computer data on a CD-ROM (compact disc read-only memory). The disc holds 1
hour and 12 minutes of full-motion, full-screen video; 40 hours of audio; or
600 megabytes of data.
"Our course will use about 2 hours of video, 15 hours of audio, several
thousand stills, and about 4 megabytes of computer data," Stevens says. DVI
allows some parts of the screen to display motion video while other parts
display stills, thus conserving memory.
DVI is different from interactive videodisc. Interactive videodisc stores
information in analog form, whereas DVI digitizes information, compresses it,
and stores it on a CD.
"The big difference is the capability of the medium," Stevens says. "The
digital form allows us to manipulate the images with a computer. In
traditional videodiscs, special effects must be done in post-production and
are locked in on the videodisc. With DVI, all that can be done real-time,
either under the control of the expert system or the user. There's a
tremendous amount of flexibility with the system that was not available
previously."
Combined with the interactive video technology is an intelligent tutoring
system, a type of expert system that simulates the interaction of participants
in the inspection. Typically, participants include a moderator, a reader, a
recorder or scribe, and often the producer of the code.
Stevens explains, "The expert system lets us model which person speaks, what a
person says, how he or she says it in terms of tone, who is addressed, how the
system responds to user input, and what the system shows the user. Our system
keeps track of the conversation over time and lets us create a simulation of
conversation such that at any point, the user can ask somebody a question, and
the system will respond intelligently. Any attempt at getting that type of
interaction in the past has been with text. With interactive video, the
people on the screen will speak to you."
The combination of interactive video and an intelligent tutoring system was
chosen because it seems to be the only technology that demonstrates a high
fidelity simulation of the group process. "It's not going to be enough to just
do computer-based training because you need to see the changes in people's
expressions. You need to hear the tone of their comments and interact with
people," Stevens says. "The synergy that goes on in an inspection brings
something that doesn't happen when people are sitting alone at their terminal.
We need to make it as realistic as possible."
Why not use a regular classroom? Research indicates that interactive media
produce courses that are much less expensive over their life cycle than
traditional training. "At the SEI we're providing technology-intensive
solutions to problems rather than labor-intensive solutions," Stevens says.
"The right technology can multiply development efforts many times. Studies by
IBM and the Air Force have shown that considering both development and
delivery, an interactive course over a five-year lifespan will cost five times
less than a traditional course. It costs more to develop interactive courses
compared with stand-up training, but the life-cycle costs of interactive
courses become much less.
"You develop it once, and the system can be used by thousands of people
without trainers everywhere. You can take the training to the people; you
don't have to bring the people to the training. An interactive training
course is expected to take less than half a day. You don't have to pay for
transportation and time away from work. Factoring in all these considerations,
delivering interactive video costs much less."
Army studies indicate that students learn material more fully and in less time
with interactive video, even when compared with computer-based training.
Stevens notes, "At the University of Pittsburgh, a recent study of an
intelligent tutoring system teaching electronic troubleshooting found that for
some classes of problems, people who had 20 hours on the system were
performing as well as technicians who had an average of 4 years of on-the-job
experience. It's a very high-payoff technology."
The advantages of interactive video would not be useful if the learning
technology were not accepted. Fortunately, there is a growing base of
interactive video installed in colleges, industry, and the government. Of all
U.S. organizations with 50 or more employees, 15.5% used interactive video to
deliver job-related training in 1986, up from 11.6% in 1985; 36% of all
organizations of more than 10,000 employees use interactive video.
HOW DOES THE USER INTERACT WITH THE SYSTEM?
From the beginning of the project, one of the biggest questions related to
user input. The highest fidelity simulation would require continuous speech
recognition, which isn't done on a PC-AT class machine (the target for course
delivery). At the other extreme, the simulation could be designed to only
allow simple multiple choice questions, which was unacceptable to project
members.
They found a compromise between the ideal and the unacceptable in Natural
Access, a menu-based natural language system developed by Harry Tennant of
Texas Instruments. Natural Access constructs sentences from sentence
fragments. While allowing the user freedom in constructing sentences, this
method restricts sentence syntax to a level manageable by the target machine.
Stevens explains, "You may want to say, `That's a well-written comment,' but
the system only offers, `That's a good comment.' You're saying the same thing
but just in a different form."
To determine what types of sentences fragments should be available and how to
arrange them into different menus, the project drew upon the research of
Elliot Soloway at Yale. Soloway has done extensive analyses of code
inspections, and he found that there are several main categories of
conversations in inspections, including those addressing clarity of the code,
correctness of the code, and simulation of the action of the code.
Robin Lampert, a research assistant of Soloway's, extended and tailored this
research at the SEI. She found that within each category there is a limited
number of sentence types, which the user can build to say what he or she wants
to say. Users can construct sentences by selecting the appropriate sentence
fragments from the menus and inserting references to specific objects such as
program variables, documents, or standards. For example, the sentence
This variable has an incorrect initialization.
has the basic structure of
<object> has (in)CORRECT <programming construct-aspect>
It can be pieced together by choosing items from three different menus. These
menus are constructed based on the piece of code that is inspected. (There
are several examples of Ada code that can be chosen for inspection.)
Because the affective side of a comment is important to convey, there are two
ways to adjust affect. A user can choose different sentence fragments to vary
the level of certainty (e.g., "I think" vs. "I'm positive"), and different
icons to indicate the tone of the sentence (extremely passive, passive,
assertive, extremely assertive).
Using the interactive course gives people experience in playing different
roles and in interacting with others in an inspection. The user can act as
the moderator, the reader, or the recorder. (The role of the producer of the
code is always modeled by the system.) Stevens explains, "Whichever role the
user chooses is extracted from the system and is no longer modeled. The system
keeps track of how the user acts based on what he or she says."
The simulation can be adjusted to enact different situations. Stevens says,
"The user can play the reader, and the moderator can be a rotten moderator who
lets people argue all the time. A moderator ought to step in." After the
inspection is finished, the system gives the user feedback on what went right
and wrong in the simulated inspection. By using the system, people learn the
components of an effective inspection.
WHO ARE THE POTENTIAL CUSTOMERS OF THE ALT COURSE?
Stevens envisions a broad audience for the product of the ALT Project.
Companies that have large software projects and that do inspections, such as
IBM and AT&T, could benefit from using the ALT product. So too could
government organizations that are developing software and training people in
inspections, such as the Federal Aviation Administration (FAA) and the Jet
Propulsion Laboratory (JPL). Stevens notes that JPL is teaching inspections
to a potential audience of 2,000-3,000 JPL software engineers, but using
traditional stand-up training. Finally, academic institutions are also
potential customers because many software engineering or computer science
curricula teach technical reviews. "Often it isn't done well or it takes a
lot of time out of class," comments Stevens.
Within each organization, the ALT course could be used by different types of
people in the organization. According to Stevens, there will be different
versions of the system for software engineers, managers, and trainers and
educators. "We don't see replacing trainers with the system. There will be a
track so that educators can learn about inspections and learn how to use the
system within their classes. Managers can learn how they should be using
inspections to help manage their projects; inspections provide managers with a
window into the software development process. Software engineers can learn
how to do inspections and how inspections can help them with their code or
design."
Organizations such as DEC, Electronic Systems Division of the Air Force,
MITRE, National Security Agency (NSA), and Westinghouse have expressed
interest in the project. DEC, MITRE, and NSA would like to be part of the beta
test of the product. Academic institutions are also interested; Indiana
University of Pennsylvania and Yale University volunteered to participate in
beta testing.
HOW HAVE PEOPLE REACTED TO THE PROJECT?
Stevens is pleased with the reaction of the software community to the project.
It has been presented at several conferences and symposiums, including the Ada
Software Engineering Education and Training Symposium in Denver, Colo., on
June 14-16. Stevens remarks, "People have commented that this project is the
exact kind of project that they think the SEI should be doing."
According to Dr. Ahmad Noor of NASA Langley, the ALT Project is the most
advanced project of its type. Dr. Noor has been researching all U.S.
applications of interactive video and expert systems applied to engineering
education.
Stevens attributes several reasons to Dr. Noor's statement: "There aren't
many applications in engineering. More importantly, we have a new paradigm
for creating an interactive course, a new level of realism available in
modeling interactions between people. And software development for big
projects is largely one of managing people. We feel that we have captured
lessons learned so that we can apply them more quickly in other areas such as
project management, which entails dealing with people all the time."
WHAT ARE THE PLANS FOR THE PROJECT?
Stevens says about plans, "We have a prototype; we have to make it into a real
product. We expect to have all the video and audio recorded by early 1989,
and then take the next several months to integrate it with the computer code,
refine it, and test it." Michael Christel, the software engineer on the
project, is developing all project software: the expert system, the user
interface, the window manager, and other components. Judy Chiswell is
providing instructional design for the project.
In the future, Stevens and his team may tackle developing similar systems for
teaching other subjects, including project management or requirements
specification. "Particularly for requirements specification, we're thinking
of extending it," says Stevens. "Instead of just a learning tool, we may
develop an intelligent assistant, a tool that actually helps with requirements
specification. So the next step is to embed this training in real tools and
products."
The ALT Project supports the SEI mission in several ways, according to
Stevens. "We are transitioning a software methodology with the project. What
we want to do is quickly increase the level of the state of the practice. As a
high-productivity system, we have a way of transitioning this technology to a
wide audience very quickly," he says. "Also, the project provides a
technological solution to the problem of education and training, which is one
of the biggest problems in software engineering. There just aren't enough
people out there. So, the project is applying hardware and software technology
to disseminate or transition the technical methodology of code inspection."