[mod.ai] Nonsense tests and the Ignorance Principle

gh%ai.toronto.edu@RELAY.CS.NET (Graeme Hirst) (10/31/86)

>A couple of years ago, on either this list or Human-Nets, there appeared
>a short multiple-choice test which was written so that one could deduce
>"best" answers based on just the form, not the content, of the questions ...

The Ignorance Principle ("Forget the content, just use the form", or "Why
clutter up the system with a lot of knowledge?") has a long and distinguished
history in AI.	In 1975, Gord McCalla and Michael Kuttner published an article
on a program that could successfully pass the written section of the British
Columbia driver's license exam.  For example:

  QUESTION:
    Must every trailer which, owing to size or construction tends to prevent
    a driving signal given by the driver of the towing vehicle from being seen
    by the driver of the overtaking vehicle be equipped with an approved
    mechanical or electrical signalling device controlled by the driver of the
    towing vehicle?
  ANSWER:
    Yes.

In fact, the program was able to answer more than half the questions just by
looking for the words "must", "should", "may", "necessary", "permissible",
"distance", and "required".

The system was not without its flaws.  For example:

  QUESTION:
    To what must the coupling device between a motor-vehicle and trailer
    be affixed?
  ANSWER:
    Yes.

This is wrong; the correct answer is "frame".  (This was an early instance of
the frame problem.)

The authors subsequently attempted a similar program for defending PhD theses,
but the results were never published.

REFERENCE

McCalla, G. and Kuttner, M.  "An extensible feature-based procedural question
  answering system to handle the written section of the British Columbia
  driver's examination".  _CSCSI/SCEIO Newsletter_ [now published as _Canadian
  Artificial Intelligence_], 1(2), February 1975, 59-67.

\\\\   Graeme Hirst    University of Toronto	Computer Science Department
////   utcsri!utai!gh  /  gh@ai.toronto.edu  /	416-978-8747