sweeney@cs.washington.edu (Christine Sweeney) (02/08/91)
I have the task of testing a medical ai system which is rule-based. We have written our own inference engine which does forward and backward chaining on a set of rules. The whole thing is in lisp. Does anyone know of ways to test such a system thoroughly other than writing a bunch of tests for *many* individual hypothetical cases? Can anyone recommend any books or papers on the subject of testing and verifying ai-based systems? Thank you. If I recieve enough interesting responses I will post a collection of them, Christine Sweeney sweeney@cs.washington.edu
ntm1169%dsacg3.dsac.dla.mil@dsac.dla.mil (Mott Given) (02/12/91)
From article <7516@uklirb.informatik.uni-kl.de>, by sweeney@cs.washington.edu (Christine Sweeney): > Can anyone recommend any books or papers on the subject of > testing and verifying ai-based systems? You might try looking at some of these references: AI EXPERT, Feb. 1990, "A tool for detecting expert system errors" IEEE Expert, Winter 1987, "Validating expert system performance" IEEE Expert, Spring 1988, "A software engineering tool for expert system design" IEEE Expert, June 1990, "Validating expert systems" A paper by the authors Todd Stock and others (whose title I do not have) presented at the Fourth Conference on Artificial Intelligence for Space Applications, Huntsville, AL, Nov 15-16, 1988 describes an Expert System Validation Assistant. Hall, David L.; Heinze, Daniel T. "The use of simulation techniques for expert system test and evaluation" ISA Transactions v28 no1 p19-22 '89 Culbert, Chris; Riley, Gary; Savely, Robert T. "An expert system development methodology that supports verification and validation" ISA Transactions v28 no1 p15-18 '89 "Technique for knowledge base verification" by Zhang, Du and Nguyen, Doan from the conference IEEE International Workshop on Tools for Artificial Intelligence: Architectures, Languages and Algorithms; in Fairfax, VA, on Oct 23-25, 1989 -- Mott Given @ Defense Logistics Agency ,DSAC-TMP, P.O. Box 1605, Bldg. 27 Section 1, Systems Automation Center, Columbus, OH 43216-5002 INTERNET: mgiven@dsac.dla.mil UUCP: ...osu-cis!dsac!mgiven Phone: 614-238-9431 AUTOVON: 850-9431 FAX: 614-238-3214
timm@runx.oz.au (Tim Menzies) (02/14/91)
In article <7516@uklirb.informatik.uni-kl.de> sweeney@cs.washington.edu (Christine Sweeney) writes: >I have the task of testing a medical ai system which is >rule-based. We have written our own inference engine >which does forward and backward chaining on a set of B >rules. The whole thing is in lisp. > >Does anyone know of ways to test such a system thoroughly >other than writing a bunch of tests for *many* individual >hypothetical cases? > An approach to verifying completeness and consistentcy in a rule-based expert system Suwa. M, Carlisle Scott A, Shortliffe E.H. , Technical report, Dept of Computer Science, Stanford University, August 1982.
bernus@batserver.cs.uq.oz.au (Peter Bernus) (02/20/91)
In article <7516@uklirb.informatik.uni-kl.de> sweeney@cs.washington.edu (Christine Sweeney) writes: >Does anyone know of ways to test such a system thoroughly >other than writing a bunch of tests for *many* individual >hypothetical cases? see Mark Bezem: Consistency of Rule Based Systems CWI report CS-R8736 (I think he is bezem@cwi.nl but the postal address is definitely %I Centrum voor Wiskunde en Informatica %A Kruislaan 413 %A 1098 SJ Amsterdam %C the Netherlands fax: +31-20-592.4199 ) Peter Bernus
jjacobs@uunet.uu.net (Jeffrey Jacobs) (02/26/91)
Expert System Testing, J.M. Jacobs & C. Chee, bd Systems, 357 Van Ness Way, SUite 110, Torrance, CA 90501, (213)618-8798. Ask for Christine Chee. Jeffrey M. Jacobs ConsArt Systems Inc, Technology & Management Consulting P.O. Box 3016, Manhattan Beach, CA 90266 voice: (213)376-3802, E-Mail: 76702.456@COMPUSERVE.COM