mckee@CORWIN.CCS.NORTHEASTERN.EDU (08/21/87)
One reason is simple intellectual honesty. If AI researchers call themselves Computer Scientists (as many of them do), they're implicitly also claiming to be scientists. And to be perfectly blunt, any scientist who doesn't use the scientific method is a charlatan. I'd prefer AI to be serious science, but if you don't want to do science, I won't argue. Misrepresentation is a different matter: if it's not science, don't call it science. Another, more technical reason involves the perennial question "what is reality?", and how one verifies any answer that might be submitted. The question is important to AI not only in its "what is intelligence, really?" aspect, but also because any AI system that interacts with the real world ought to have an accurate understanding of its environment. Scientific facts are (almost by definition) the most accurate description of the universe that we have, and scientific theories the best summaries. And the reason this is so is because the scientific method is the best way we've yet discovered for making sure that facts and explanations are accurate. Besides science, the other significant field with aspirations toward understanding reality is philosophy, which has even evolved a specialized subfield, ontology, devoted to the question. Now I haven't studied ontology, not because the question is unimportant, but because I think philosophical methodology is fatally flawed, and incapable of convincing me of the substance of any conclusions that it might obtain. I'm not interested in a discussion of how philosophy has or has not lost its way since Kant wrote his "Prolegomena to Any Future Metaphysics Which Will Be Able to Come Forth as Science", but I think philosophers' methodology has kept them from being as productive of useful understanding as they could have been. The critical question in choice of methodology concerns verifiablity. I'd hate to see AI researchers cast adrift in a sea of notions by thinking that a solid intellectual structure can be built on "Philosophical Foundations", so I'm going to attempt to concisely describe a schema of the different ways a theory can be confirmed. I'm afraid I'll have to leave out a lot of details and examples, but I hope you'll be able to fill in the rest of the picture yourself. In this schema, philosophy turns out to use the weakest form of confirmation, AI as it's currently practiced uses somewhat stronger methods, and the natural sciences end up as strongest. To see how this happens, think of the subject matter of a field of study as a set of statements (observations, facts) connected by a network of reasons. The reasons can be arbitrarily long (or short) chains of inferences. What a researcher needs to do to "understand" the field is find a set of axioms and inference rules that will show the explanatory relation between any pair of observations. However, the problem is underdetermined -- there's more than one consistent set of explanations for any set of facts. At the very least, one can always say "Because!", and define a special rule for each ill-behaved pair of facts. Doing this everywhere gives your theory a very simple structure, and Occam's razor decrees that simplicity is important. If there are always multiple theories that can explain all the observed data, then one must turn to some confirmation methodology to distinguish between them, and using anything but the most powerful techniques is a waste of time and resources. They are all based on prediction -- applying explanations to facts until one has covered all the facts, then generating new "potential facts" from incompletely bound explanations. For philosophers, all that can be done is to compare predictions, since the operations of the human mind are not externally visible. Worse, the facts of experience itself are inaccessible to more than one theorist, so that the data can't be verified, only statements about it. And since Godel proved his famous incompleteness theorem, we've known that no realistic model of the world can be derived from a finite set of axioms, so there's no way of telling if any discrepancy in predictions might be cured by the addition of "just one more" axiom. [Beyond this my metamathematics doesn't go. It would be interesting to know if there's any convergence at higher degrees of metafication. I don't think so, though.] In AI, one can trace the operation of a theory that's been instantiated as a program, as long as there's sharing of source code and the hardware is the same. This gives you operational confirmation as well as implicational confirmation, since you can watch the computer's "mind" at work, pausing to examine the data, or single-step the inference engine. The points of divergence between multiple theories of the same phenomenon can thus be precisely determined. But theories summarize data, and where does the data come from? In academia, it's probably been typed in by a grad student; in industry, I guess this is one of the jobs of the knowledge engineer. In either case there's little or no standard way to tell if the data that are used represent a reliable sample from the population of possible data that could have been used. In other sciences the curriculum usually includes at least one course in statistics to give researchers a feel for sampling theory, among other topics. Statistical ignorance means that when an AI program makes an unexpected statement, you have only blind intuition and "common sense" to help decide whether the statement is an artifact of sampling error or a substantial claim. In the natural sciences, in addition to implicational and operational confirmation, you'll find external confirmation. Each relation in the theory is tested by an experiment on the phenomenon itself, often in many ways in many experiments. It's not easy to think of statements about the content of AI (as opposed to its practice or techniques) that *could* be validated this way, much less hypotheses that actually *have* been experimentally validated. Hopefully, it's my ignorance of the field that leads me to say this. The best I can think of at the moment is "all intelligent systems that interact with the physical world maintain multiple representations for much of their knowledge." To verify a hypothesis like this, one of the strategies one can use is to build synthetic intelligent systems and then look at their structure and performance, remembering that the engineering used during construction is not the scientific goal. And then, to understand the structure one would use analytic techniques, and to understand the performance one would use behaviorist techniques. (Behaviorist anti-theory can safely be ignored, but don't forget that their methodology allowed them to discover learning sets when their animals became skilled at finding solutions to new *kinds* of problems.) Another strategy is to look at the structure and behavior of the intelligent systems one finds in nature. One would use the same methods to validate the behavioral descriptions as in the synthetic case, but to study natural systems' structure one must use indirect, non-invasive means or non-human subjects, since ethical considerations forbid destructive testing of humans except in very special circumstances. However the problem here is not lack of data but lack of understanding. If I believed that more data was needed, I'd be back in the lab recording from multiple microelectrodes, or standing in line for time on a magnetic resonance imager (which can already give you sub-millimeter resolution in a 3-dimensional brain image -- why wait for magnetoencephalography which won't tell you what you want to know anyway?), instead of building and running abstract models of neural tissue. Oops, four times as many words as I had hoped for. Oh well, thanks for your attention. - George McKee College of Computer Science [sic] Northeastern University, Boston 02115 CSnet: mckee@Corwin.CCS.Northeastern.EDU Phone: (617) 437-5204 Quote of the day: "It's not what you don't know that hurts you, it's the things you know that ain't so." - Mark Twain
shebs@CS.UTAH.EDU.UUCP (08/25/87)
In article <8708240436.AA19024@ucbvax.Berkeley.EDU> mckee@CORWIN.CCS.NORTHEASTERN.EDU writes: > >[...] If AI researchers >call themselves Computer Scientists (as many of them do), they're implicitly >also claiming to be scientists. Not necessarily. "Computer Science" is an unfortunate term that should be phased out. I wasn't there when it got popular, but the timing is right for the term to have been inspired by the plethora of "sciences" that got named when the govt started handing out lots of money for science in the 60s. I prefer the term "informatics" as the best of a bad lot of alternatives. ("Datology" sounds like a subfield of history; the study of dates :-) ) >[... tutorial on scientific method omitted ...] > In AI, one can trace the operation of a theory that's been instantiated >as a program, as long as there's sharing of source code and the hardware is >the same. This gives you operational confirmation as well as implicational >confirmation, since you can watch the computer's "mind" at work, pausing >to examine the data, or single-step the inference engine. Goedel's and Turing's ghosts are looking over our shoulders. We can't do conventional science because, unlike the physical universe, the computational universe is wide open, and anything can compute anything. Minute examination of a particular program in execution tells one little more than what the programmer was thinking about when writing the program. >The points of >divergence between multiple theories of the same phenomenon can thus be >precisely determined. But theories summarize data, and where does the >data come from? In academia, it's probably been typed in by a grad student; >in industry, I guess this is one of the jobs of the knowledge engineer. >In either case there's little or no standard way to tell if the data that >are used represent a reliable sample from the population of possible data >that could have been used. In other sciences the curriculum usually includes >at least one course in statistics to give researchers a feel for sampling >theory, among other topics. Statistical ignorance means that when an AI >program makes an unexpected statement, you have only blind intuition and >"common sense" to help decide whether the statement is an artifact of sampling >error or a substantial claim. I took a course in statistics, but you don't need a course to know that sampling from a population is not meaningful, if you don't know what the population is in the first place! In the case of AI, the population is "intelligent behavior". Who among us can define *that* population precisely? If the population is more restricted, say "where native-speaking Germans place their verbs", then you're back in the Turing tarpit. A program that just says "at the end" (:-) is behaviorally as valid as something that does some complex inferences to arrive at the same conclusion. Worse, Occam's razor makes us want to prefer the simpler program, even though it won't generalize to other natural languages. When we generalize the program, the population to sample gets ill-defined again, and we're back where we started. >[...] It's not easy to think of statements about the content >of AI (as opposed to its practice or techniques) that *could* be validated >this way, much less hypotheses that actually *have* been experimentally >validated. Hopefully, it's my ignorance of the field that leads me to >say this. The best I can think of at the moment is "all intelligent systems >that interact with the physical world maintain multiple representations >for much of their knowledge." This could only be a testable hypothesis if we agreed on the definition of "intelligent system". Are gorillas intelligent because they use sign language? Are birds intelligent because they use sticks? Are thermostats intelligent? I don't believe the above hypothesis is testable. Almost the only agreement you'd get is that humans are intelligent (ah, the hubris of our species), but then you'd have to build a synthetic human, which isn't going to be possible anytime soon. Even if you did build a synthetic human, you'd get a lot of disagreement about whether it was correctly built, since the Turing Test is too slow for total verification. > - George McKee > College of Computer Science [sic] > Northeastern University, Boston 02115 AI people are generally wary of succumbing to "physics envy" and studying only that which is easily quantifiable. It's like the drunk searching under the street light because that's where it's easy to see. AI will most likely continue to be an eclectic mixture of philosophy, mathematics, informatics, and psychology. Perhaps the only problem is the name of the funding source - any chance of an "NAIF"? :-) stan shebs shebs@cs.utah.edu
mps@duke.cs.duke.EDU.UUCP (08/28/87)
In article <8708251656.AA14266@cs.utah.edu> cs.utah.edu!shebs@cs.utah.edu (Stanley Shebs) writes: > >Goedel's and Turing's ghosts are looking over our shoulders. We can't do >conventional science because, unlike the physical universe, the computational ^^^^^^^^^^^^^^^^^ >universe is wide open, and anything can compute anything. Minute examination ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >of a particular program in execution tells one little more than what the >programmer was thinking about when writing the program. > [emphasis added] Would you please explain this tantalizing remark? Surely not every formal system can compute every function (what about the ghost of Chomsky?). Are you alluding to the mutual emulatability of Turing machines? Or maybe the moral is functionalism (as philosophers use the term): that in matters computational, it's form and not matter that matters. And how does Goedel fit in? I suspect it's his completeness theorem and not his incompleteness results you have in mind. Finally, how does the third sentence follow from the second? Thanks. "Just as a vessel is a place that can be carried around, so place is a vessel that cannot be carried around." Aristotle Michael P. Smith ARPA: mps@duke.cs.duke.edu
shebs@CS.UTAH.EDU (Stanley Shebs) (09/03/87)
In article <8708281322.AA27689@duke.cs.duke.edu> duke!mps (Michael P. Smith) writes: >In article <8708251656.AA14266@cs.utah.edu> cs.utah.edu!shebs@cs.utah.edu >(Stanley Shebs) writes: >> >>Goedel's and Turing's ghosts are looking over our shoulders. We can't do >>conventional science because, unlike the physical universe, the computational > ^^^^^^^^^^^^^^^^^ >>universe is wide open, and anything can compute anything. Minute examination > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>of a particular program in execution tells one little more than what the >>programmer was thinking about when writing the program. >> > [emphasis added] > >Would you please explain this tantalizing remark? Surely not every >formal system can compute every function (what about the ghost of >Chomsky?). Are you alluding to the mutual emulatability of Turing >machines? This is basic computer science. Any formalism sufficiently powerful to compute all the computable things we know of is equivalent to a Turing machine (Church-Turing Hypothesis), and formalisms of that power are all incomplete (Goedel's Incompleteness Theorem). Incompleteness rears its ugly head when we find that our most sophisticated programs cannot be tested completely. Simpler formal systems such as CFGs are too weak to model human intelligence, although some aspects of human behavior have been asserted to be context-free (for instance, Presidents that don't learn from their predecessors :-) ). >Finally, how does the third sentence follow from the second? This is the empirical consequence of Turing equivalence. I can write Eurisko or XCON in Lisp, Forth, or IBM 1401 assembler, and they will all behave the same. Assertions about the details about a program are worthless from a theoretical point of view, details of algorithms are somewhat better, but the algorithms appearing in AI programs are either too simple (searching for instance) or too complicated to be analyzable (the abovementioned large programs). >Michael P. Smith ARPA: mps@duke.cs.duke.edu stan shebs shebs@cs.utah.edu