WALKER@SUMEX-AIM.ARPA (11/24/83)
From: Michael Walker <WALKER@SUMEX-AIM.ARPA>
Tom,
I thought you made some good points in your response to Ralph
Johnson in the AIList, but one of your claims is unsupported, important,
and quite possibly wrong. The claim I refer to is
"Expert systems can be built, debugged, and maintained more cheaply
than other complicated systems. And hence, they can be targeted at
applications for which previous technology was barely adequate."
I would be delighted if this could be shown to be true, because I
would very much like to show friends/clients in industry how to use AI to
solve their problems more cheaply.
However, there are no formal studies that compare a system built
using AI methods to one built using other methods, and no studies that have
attempted to control for other causes of differences in ease of building,
debugging, maintaining, etc. such as differences in programmer experience,
programming language, use or otherwise of structured programming techniques,
etc..
Given the lack of controlled, reproducible tests of the effectiveness
of AI methods for program development, we have fallen back on qualitative,
intuitive arguments. The same sort of arguments have been and are made for
structured programming, application generators, fourth-generation languages,
high-level languages, and ADA. While there is some truth in the various
claims about improved programmer productivity they have too often been
overblown as The Solution To All Our Problems. This is the case with
claiming AI is cheaper than any other methods.
A much more reasonable statement is that AI methods may turn out
to be cheaper / faster / otherwise better than other methods if anyone ever
actually builds an effective and economically viable expert system.
My own guess is that it is easier to develop AI systems because we
have been working in a LISP programming environment that has provided tools
like interpreted code, interactive debugging/tracing/editing, masterscope
analysis, etc.. These points were made quite nicely in Beau Shiel's recent
article in Datamation (Power Tools for Programming, I think was the title).
None of these are intrinsic to AI.
Many military and industry managers who are supporting AI work are
going to be very disillusioned in a few years when AI doesn't deliver what
has been promised. Unsupported claims about the efficacy of AI aren't going
to help. It could hurt our credibility, and thereby our funding and ability
to continue the basic research.
Mike Walker
WALKER@SUMEX-AIM.ARPADIETTERICH@SUMEX-AIM.ARPA (11/26/83)
From: Tom Dietterich <DIETTERICH@SUMEX-AIM.ARPA>
Mike,
While I would certainly welcome the kinds of controlled studies that
you sketched in your msg, I think my claim is correct and can be
supported. Virtually every expert system that has been built has been
targeted at tasks that were previously untouched by computing
technology. I claim that the reason for this is that the proper
programming methodology was needed before these tasks could be
addressed. I think the key parts of that methodology are (a) a
modular, explicit representation of knowledge, (b) careful separation
of this knowledge from the inference engine, and (c) an
expert-centered approach in which extensive interviews with experts
replace attempts by computer people to impose a normative,
mathematical theory on the domain.
Since there are virtually no cases where expert systems and
"traditional" systems have been built to perform the same task, it is
difficult to support this claim. If we look at the history of
computers in medicine, however, I think it supports my claim.
Before expert systems techniques were available, many people
had attempted to build computational tools for physicians. But these
tools suffered from the fact that they were often burdened with
normative theories and often ignored the clinical aspects of disease
diagnosis. I blame these deficiencies on the lack of an
"expert-centered" approach. These programs were also difficult to
maintain and could not produce explanations because they did not
separate domain knowledge from the inference engine.
I did not claim anywhere in my msg that expert systems techniques are
"The Solution to All Our Problems". Certainly there are problems for
which knowledge programming techniques are superior. But there are
many more for which they are too expensive, too slow, or simply
inappropriate. It would be absurd to write an operating system in
EMYCIN, for example! The programming advances that would allow
operating systems to be written and debugged easily are still
undiscovered.
You credit fancy LISP environments for making expert systems easy to
write, debug, and maintain. I would certainly agree: The development
of good systems for symbolic computing was an essential prerequisite.
However, the level of program description and interpretation in EMYCIN
is much higher than that provided by the Interlisp system. And the
"expert-centered" approach was not developed until Ted Shortliffe's
dissertation.
You make a very good point in your last paragraph:
Many military and industry managers who are supporting AI work
are going to be very disillusioned in a few years when AI
doesn't deliver what has been promised. Unsupported claims
about the efficacy of AI aren't going to help. It could hurt
our credibility, and thereby our funding and ability to
continue the basic research.
AI (at least in Japan) has "promised" speech understanding, language
translation, etc. all under the rubric of "knowledge-based systems".
Existing expert-systems techniques cannot solve these problems. We
need much more research to determine what things CAN be accomplished
with existing technology. And we need much more research to continue
the development of the technology. (I think these are much more
important research topics than comparative studies of expert-systems
technology vs. other programming techniques.)
But there is no point in minimizing our successes. My original
message was in response to an accusation that AI had no merit.
I chose what I thought was AI's most solid contribution: an improved
programming methodology for a certain class of problems.
--Tom