mwm@ea.UUCP (07/07/84)
#R:gloria:-29000:ea:500001:000:1095 ea!mwm Jul 7 13:25:00 1984 /***** ea:net.ai / gloria!colonel / 1:01 pm Jul 3, 1984 */ if you really want to know whether somebody is a person or a computer, you just look at him/it. -- Col. G. L. Sicherman ...seismo!rochester!rocksanne!rocksvax!sunybcs!gloria!colonel /* ---------- */ Sorry, that test only works if you want to know if somebody is a *human* or a computer - and not even reliably in that situation. The point behind the Turing test that many people seem to have missed is impartiality. You don't claim that something isn't intelligent because it's different from you (If you do, I'll sic the ACLU on you! :-). As a different example, lets consider a limited domain: calculus. If you want to know if a human understands calculus, what do you do? You give it a test. If it passes the test, then it understands calculus, and if it doesn't, then it's understanding of calculus is incomplete. Now, if a program passes the same test, shouldn't we say that it understands calculus? If not, why not - and remember, "becauses it's a computer" or "because it can't `understand'" aren't valid answers. <mike
bev@hlexa.UUCP (Beverly Dyer) (07/11/84)
Understanding? If a human passes a calculus test it means they can calculate correct answers to (some percentage of) the questions asked. If a computer does the same it means the same, but that's all.
yba@mit-athena.ARPA (Mark H Levine) (07/12/84)
If a program passes a test in calculus the best we can grant it is that it can pass tests. In the famous program ANALOGY (Bobrow's I think) the computer "passes" geometric analogy tests. It does not seem to understand either geometry or analogy outside of this limited domain of discourse. We make the same mistaken assumption about humans--that is that because you can pass a "test" you understand a subject. The Turing test was a "blind" test; in that the Colonel is wrong--someone reading this over the net or receiving a note from the bank cannot just "go look". The idea was to tell via dialog only in a blind situation (maybe even a double-blind if there are some control situations where two humans taking the Turing test face each other). The question of how to evaluate the performance of an AI system has become an important question. I am not sure that the question of "understanding" should even enter into it. In any case, let's not trivialize it. -- yba%mit-heracles@mit-mc.ARPA UUCP: decvax!mit-athena!yba
ags@pucc-i (Seaman) (07/13/84)
> If a program passes a test in calculus the best we can grant it is that > it can pass tests. > . . . > We make the same mistaken assumption about humans--that is that because > you can pass a "test" you understand a subject. Suppose the program writes a Ph.D. dissertation and passes its "orals"? Then can we say it understands its field? If not, then how can we decide that anyone understands anything? -- Dave Seaman My hovercraft is no longer full of ..!pur-ee!pucc-i:ags eels (thanks to my confused cat).
yba@mit-athena.ARPA (Mark H Levine) (07/22/84)
<Dave Seaman, re: phd thesis and orals passing> You are trivializing the point, not understanding it. The test of whether an engineer understands engineering is he can design and build things that work. I have met Ph.D.s who can write "learned" papers but cannot "do" anything concrete. If you must have a test, I'll assert that someone who can apply knowledge of a field to a new area, and can transmit that knowledge to another who was previously ignorant of said knowledge such that that person can apply the knowledge in the same way, has understanding (yes, it's recursive). Measurement of AI performance is important. It is notions like "test" that cause people to confuse "production systems" with "expert systems". You may recall that the original notion of "expert system" was "a system that solves problems the way (human) experts do". This has been reduced to rules-based production systems in many people's minds, because they think that experts solve problems by applying rules. I am not satisfied that this is true; the question several people have asked is "does a rules based system demonstrate an expert's intuition?" Afterall, it passes the "test" of applying knowledge to a problem. You can substitute "judgement", "intelligence", "understanding", or "talent" for the word intuition if you like. The question becomes rather concrete when you decide whether to allow a program to practice medicine. We have all heard of examples of accredited (human) doctors who have not been able to safely practice medicine although they passed all the qualifying "tests". In fact, there seems to be a great body of technique floating around in many disciplines; there is also a great lack of those who know what the limits of application of those techniques are (usually because they know the underlying assumptions and constraints). I greatly fear people who have become so proficient at using hammers that every problem begins to resemble a nail. I will also assert that you read my previous letter, processed the information, responded, and did all this without understanding what I meant. Now if we assume you disagree, what test would you design to see which of us was correct? (Warning: this is a hard problem). -- yba%mit-heracles@mit-mc.ARPA UUCP: decvax!mit-athena!yba