shebs%orion@CS.UTAH.EDU.UUCP (07/06/87)
I can understand Don Norman's unhappiness about the lack of scientific method in AI - from a practical point of view, the lack of well-understood criteria for validity means that refereeing of publications is unlikely to be very objective... :-( The scientific method is a two-edged sword, however. Not only does it define what is interesting, but what is uninteresting - if you can't devise a con- trolled experiment varying just a single parameter, you can't say anything about a phenomenon. A good scientist will perhaps be able to come up with a different experiment, but if stymied enough times, he/she is likely to move on to something else (at about the same time the grant money runs out :-) ). Established sciences like chemistry have an advantage in that the parameters most likely to be of interest are already known; for instance temperature, percentages of compounds, types of catalysts, and so forth. What do we have for studying intelligence? Hardly anything! Yes, I know psychologists have plenty of experimental techniques, but the quality is pretty low compared to the "hard sciences". A truly accurate psychology experiment would involve raising cloned children in a computer-controlled environment for 18 years. Even then, you're getting minute amounts of data about incredibly complex systems, with no way to know if the parameters you're varying are even relevant. There's some consolation to be gained from the history of science/technology. The established fields did not spring full-blown from some genius' head; each started out as a confused mix of engineering, science, and speculation. Most stayed that way until the late 19th or early 20th century. If you don't believe me, look at an 18th or early 19th century scientific journal (most libraries have a few). Quite amusing, in fact very similar to contemporary AI work. For instance, an article on electric eels from about 1780 featured the observations that a slave grabbing the eel got a stronger shock on the second grab, and that the shock could be felt through a wooden container. No tables or charts or voltmeter readings :-). My suggestion is to not get too worked up about scientific methods in AI. It's worth thinking about, but people in other fields have spent centuries establishing their methods, and there's no reason to suppose it will take any less for AI. stan shebs shebs@cs.utah.edu