Barry_A_Stevens@cup.portal.com (11/27/87)
EXPERT SYSTEM SHELL SURVEY
Copyright 1987 Applied AI Systems, Inc.
We recently sent a questionnaire to 1700 users of PC-based expert
system development shells. One hundred seventy nine firms responded.
The survey was intended as a snapshot of the expert system market
during the preparation stages of a business plan. It was understood in
advance by all parties concerned that:
IT WAS NEVER MEANT TO BE AN ACADEMICALLY CORRECT SURVEY.
IT WAS ONLY TO PROVIDE SOME GENERAL INFORMATION.
If you can accept the above limits, what follows may be of interest to
you. If an imperfect survey is an abomination, you can skip the
remainder of this file.
The purpose of the survey was to educate and inform, on a gross level,
about the expert system shell marketplace. Information sought
included:
profiles of the shell users and their organizations;
general strengths and weaknesses of expert system shells;
the decision process followed by users when buying a shell;
the reasons for getting into expert systems;
expert system software in use;
job titles of people using expert system tools; and
applications that have been implemented using shells.
One hundred seventy-nine questionnaires were completed and
returned. The survey contained some questions whose answers are
confidential. We thought that some of the results, summarized for
bervity and sanitized to maintain confidentiality, might be of
interest.
WHAT CHARACTERISTICS MAKE A GOOD - AND BAD - EXPERT SYSTEM
DEVELOPMENT SHELL?
Many of the questionnaire respondents indicated that they had made
studies of multiple expert system development shells. Thirty two of
those respondents offered the following general comments about factors
that they viewed as strengths and weaknesses of those tools. A
strength was defined as a reason they would buy a tool as a
result of a product evaluation, while a weakness would cause them to
reject a tool.
Strengths
General strengths are described below, with the number of
respondents mentioning each factor shown.
The tool should be useful in many microcomputer,
minicomputer, and mainframe environments, under different
operating systems. (6)
The capability to access other programs and data should be
provided. (5)
The tool should be capable of frames and/or object
representation as well as representation by rules. (4)
Math functions and numeric and text variables should be
usable in rules (2)
Rules should be easy to structure; (2)
The product should be easy to learn. (2)
The product should be easy to use. (2)
Both editor and user interface should be in English, or
natural language;
Graphics and a good user interface should be available;
Procedural components should be available for sequencing and
interaction, including the ability to clear previous answers
and ask questions again;
The tool should handle probabilities, including fuzzy
logic;
The tool should learn by examples;
Sophisticated WHY and HOW capability should be available to
explain reasoning;
Good support and training should be available;
Good documentation should be available.
Weaknesses
General weaknesses of expert system development shells that were
identified by users are described below.
Cost of a product should be appropriate for its capabilities
and performance; (the survey indicated that users are
price sensitive, and price is a significant factor in
purchasing a product.) (12)
Special hardware requirements are a problem. The tool should
run on a standard PC or other commonly available
environment. (5)
Knowledge base size limitations are a problem. Several
available products limit the number of rules that can be
defined. (3)
Slow execution speed is a problem. Execution speed should
be such that a large number of rules can be executed in a
reasonable time. (2)
Lack of flexibility in knowledge representation and use is a
problem. (2)
THE PROCESS OF MAKING THE DECISION TO BUY
We asked about the process of decision making that went into the
purchase of a PC-based expert system shell at a price of $400. We
wanted to know who made the decision, and how it was made.
Who (by organizational unit) made the decision to buy?
INTERNAL
ORGANIZATION NUMBER % RESPONSE
Research and Development 58 31.9%
MIS/Data Processing 36 19.8%
Independent individuals 31 17.0%
Operating (line) organizations 26 14.3%
Management/staff function 19 10.4%
Advanced Planning 10 5.5%
Other 2 1.1%
TOTALS 182 100%
Who (by reporting relationship) made the decision to buy?
WHO MADE
DECISION NUMBER % RESPONSE
Me 151 88.8%
My Boss 17 10.0%
Different department 1 0.6%
My subordinate 1 0.6%
My boss's boss 0 0.0%
TOTALS 170 100%
How was the decision to buy made?
DECISION PROCESS NUMBER % RESPONSE
No formal decision process 99 40.9%
Product review and comparison 64 26.4%
Internal needs assessment 26 10.7%
Cost justification 40 16.5%
Formal review and planning process 6 2.5%
Other 7 2.9%
TOTALS 242 100%
What were the reasons for getting into expert systems?
REASON NUMBER % RESPONSE
To capture knowledge 76 16.0%
It's a training tool 71 14.9%
Part of overall corporate strategy 60 12.6%
To improve quality of work 47 9.9%
To improve quality of product 41 8.6%
To learn expert systems 41 8.6%
It's a competitive weapon 34 7.2%
It's in fashion 27 5.7%
To achieve a cost savings 23 4.8%
It's a marketing tool 22 4.6%
To provide an MIS/DP capability 19 4.0%
Other 14 2.9%
TOTALS 475 100%
What types of expert system software are installed?
We found that 76 distinct products were in use from nearly as many
vendors. It is interesting to note the categories into which these
products fell.
Lisp Based Tools:
total units installed: 80
number of products installed: 15
Prolog Based Tools:
total units installed: 120
number of products installed: 12
Development tools/shells:
total units installed: 223
number of products installed: 49
JOB TITLES OF PEOPLE USING SHELLS
We were interested in the job titles of people who had built
expert systems using shells. You may be interested as well.
Advanced Programmer/Analyst
Advanced R&D Project Engineer
Advanced Technology Group
Advisory Engineer
AI Branch Chief
Analyst
Assistant Professor
Assistant Vice President
Associate Professor
Audit Manager
CEO
Chairman of the Board
Chair, Department of Communications
Chemist
Computer Scientist
Computer Specialist
Consultant
Cost Analyst
Design specialist
Director Clinical Laboratories
Director, AI in Business
Director, Clinical Research
Director, Computation Center
Education Specialist
Engineer
Executive Consultant
Financial Services Officer
Graduate Assistant
Group Leader
Head Intelligent Systems. Lab.
Instructor
Learning Center Manager
Lecturer
Manager, R&D
Manager, Analytical Chemistry
Manager, Information Resource Management
Manager, Operations
Manager, Product Evaluation Office Automation
Manager, Proposals
Manager, Remote Sensing Lab
Managing Vice President
Materiel Operations Manager
Owner
Physician
President
Principal
Professor of AI & ES
Professor of Chemistry
Professor of Real Estate
Program Manager
Programmer
Project Coordinator
Project Engineer
Regional Manager
Research Agronomist
Research Assistant
Research Forester
Research Manager
Research Scientist
Seismic Processing Analyst
Senior Analyst
Senior Engineer
Senior Research Chemist
Senior Research Fellow
Senior System Analyst
Senior Tax Manager
Senior VP & Senior Trust Officer
Special Projects Director
Staff Machinery Engineer
Staff Research Engineer
Staff Scientist
Systems Officer
Technical Advisor
Technical Consultant
Technical Journalist
Technical Staff Member
Technology Assessor
Underwriting Director
Unit Head Conventional Safety Standards
Vice President
Wildlife Ecologist
While there are a few titles that indicate specialization in AI,
most are seen to be outside that category. Expert systems are
being built by technical professionals even with limited
computer experience.
APPLICATIONS FOR SHELLS
We asked about the applications that were either built or in
development using shells. In many cases, the reply indicated that the
nature of the application was confidential and no application name was
given. The names below are those that were listed in user
responses, with little editing and no attempt to explain.
Account business assessment
Advertising copy development
Advice on single family home purchase
Advice on stock and commodities trading
Advise nursing students on the care of patients
Advising on choices for new technology
Advisor on choosing soy bean varieties
Advisor on design of new magnetic components
Aid for financial futures traders
Aid for isolating failing chips
Aid in salmon stocking rates, species selection
Alarm management system
Analysis of simulation results in bank product planning
Analysis of soil site characteristics
Analysis of X-rays
Application sizing based on similar applications
Assist in compiling tax planning ideas
Assist in diagnosis of computer console messages
Assist in identification of rare antibodies
Assist new users of DOS
Assist service desk in troubleshooting application problems
Assistance in search for part numbers
Augment expertise of resource manager
Broker syndication planner
Call screening to interview users with application problems
Career development
Causal model of account marketing
Chemical process diagnosing and troubleshooting
Choosing a living or testamentary trust
Choosing an executor for trusts
Classification of data from satellites
Classifications of software programs
Closing and issuance assistance
Commercial loan credit analysis
Commercial loan documentation check list
Computer modeling support
Computer system configurator
Configurator for selecting, sizing, and writing parts list
Configuring programmable controller system
Conservation equipment tillage selector
Correct selection of cost codes
Cost/benefit assistant
Create standard loan documents based on characteristics
Credit control system
Crop management and irrigation simulation
Customer assistance in selecting types of investments
Customer service advisor for problem resolution
Customer water quality analysis
Data communications troubleshooting
Decision support for correct testing by auditing
Detailed analysis of hardware and software problems
Detailed design for asphalt concrete pavement
Determine correct mixture for propellant ingredients
Determining best shipping documentation and routes
Diagnose telecommunications difficulties
Diagnosis of sports related injuries
Diagnostic advisor for pulp bleaching
DP production support system
Epidemiology expert system
Equipment fault diagnosis
Equipment troubleshooting
Estimate employee's potential retirement salary
Estimating construction costs
Evaluation of commodities purchases
Evaluation of multi-family housing projects
Evaluation of stock purchases
Fault diagnosis for electronic hardware
Federal contract management
Fertilizer recommendations
Fertilizer, climate, and soil interaction
Finding phases present in super alloys
Forecast snowfall accumulation
Forecasting severe convecting weather
Futures, stocks, and options trading
Gas turbine troubleshooting
Geographic information system analysis aid
Grading of graft vs. host disease
Hardware and software selection
Hardware failure analysis
Hardware sizing assistant
Hazardous chemical ranking
Implementation planning assistant
Industrial training
Interpretation of statistical quality control data
Invention patentability expert
Irrigation and pest control management
Lime recommendation system
Line diagnosis and fault detection
Local area network selection
Machine advisor for grinding, milling, turning
Manufacturing resource planning aid
Market segmentation and positioning
Marketing advisor for process control systems
Material selection by engineers
Materials selection for specialized component parts
Medical decision making
Medical diagnosis
MIS decision support system
Mortgage credit analysis
Network operations systems diagnosis
Papaya management system
Pavement performance diagnosis
Pavement rehabilitation
PC configuration
PC Hardware and software configurator
Perform hematological diagnosis
Personal tax advisor
Pest management and soil interaction analysis
Portfolio construction
Power plant boiler tube failure identification
Problem diagnosis for local area networks
Problem diagnosis for printers on a SNA network
Product development support system
Product performance troubleshooting for salesmen
Product selection system
Production scheduling
Psychiatric interview
Quick proposal estimator
Radar mode design workstations
Rating for substandard life insurance
Real estate appraisal
Real estate site selection
Real time process control
Real time troubleshooting for wastewater process control
Recommend documentation to computer users
Relay diagnosis
Risk assessment of error or fraud in financial statements
Salary planning
Sales order analysis
Salmon diagnosis and treatment
Select pension types
Select, recommend library reference materials
Selection of non-materials in aerospace applications
Selection of solvents for chemical compounds
Service network assistant
Software development risk analysis
Software system diagnosis model
Software vendor risk analysis
Soil acidity analysis
Soil characterization and utilization
Solid waste disposal management assistant
Space shuttle payload on-orbit analysis
Strategic alternatives for a fragmented industry
Strategic marketing and planning aid
Structural damage assessment
Student financial aid eligibility
Submarine approach officer training
System to identify feasible rehabilitation strategies
System to prepare process estimates
Tactical battle management
Teaching mineral and rock identification
Telephone system configurator
Toxicity of laboratory chemicals
Training in gas turbines
Training new financial planners
Troubleshooting airplane starting systems
Underwriting assistance
Underwriting guidance for line underwriters
Weed identification
When to perform a physical audit
Applied AI Systems, Inc. and Barry Stevens may be reached at PO Box
2747, Del Mar, CA, 619-755-7231.