DIETTERICH@SUMEX-AIM.ARPA (09/28/84)
From: Tom Dietterich <DIETTERICH@SUMEX-AIM.ARPA> Reply to Shebs' other flame: "Induction...Not too hard really;" Shebs comments are very naive. Of course it isn't too hard to construct a MECHANISM that sometimes performs inductive generalizations properly. However, every mechanism developed thus far is very ad hoc. They all rely on "having the right formalism". In other words, the programmer implicitly tells the program how to generalize. The programmer communicates a set of "biasses" or preferences through the formalism. Many of us working in inductive learning suspect that general techniques will not be found until we have a THEORY that justifies our generalization mechanisms. The justification of induction appears to be impossible. Appeals to the Principle of Insufficient Reason and Occam's Razor just restate the problem without solving it. In essence, the problem is: What is rational plausible inference? When you have no knowledge about which hypothesis is more plausible, how do you decide that one hypothesis IS more plausible? A justification of inductive inference must rely on making some metaphysical assertions about the nature of the world and the nature of knowledge. A justification for Occam's razor, for example, must show why syntactic simplicity necessarily corresponds to simplicity in the real world. This can't be true for just any syntactic representation! For what representations is it true? --Tom