Dave.Touretzky@A.CS.CMU.EDU (10/16/86)
The definitive book on connectionism (as of 1986) has just been published by MIT Press. It's called "Parallel Distributed Processing: Explorations in the Microstructure of Cognition", by David E. Rumelhart, James H. McClelland, and the PDP research group. If you want to know about connectionist models, this is the book to read. It comes in two volumes, at about $45 for the set. For other connectionist material, see the proceedings of IJCAI-85 and the 1986 Cognitive Science Conference, and the January '85 issue of the journal Cognitive Science. -- Dave Touretzky PS: NO, CONNECTIONISM IS NOT THE SAME AS PERCEPTRONS. Perceptrons were single-layer learning machines, meaning they had an input layer and an output layer, with exactly one learning layer in between. No feedback paths were permitted between units -- a severe limitation. The learning algorithms were simple. Minsky and Papert wrote a well-known book showing that perceptrons couldn't do very much at all. They can't even learn the XOR function. Since they had initially been the subject of incredible amounts of hype, the fall of perceptrons left all of neural network research in deep disrepute among AI researchers for almost two decades. In contrast to perceptrons, connectionist models have unrestricted connectivity, meaning they are rich in feedback paths. They have rather sophistcated learning rules, some of which are based on statistical mechanics (the Boltzmann machine learning algorithm) or information theoretic measures (G-maximization learning). These models have been enriched by recent work in physics (e.g., Hopfield's analogy to spin glasses), computer science (simulated annealing search, invented by Kirkpatrick and adapted to neural nets by Hinton and Sejnowski), and neuroscience (work on coarse coding, fast weights, pre-synaptic facilitation, and so on.) Many connectionist models perform cognitive tasks (i.e., tasks related to symbol processing) rather than pattern recognition; perceptrons were mostly used for pattern recognition. Connectionist models can explain certain psychological phenomena that other models can't; for an example, see McClelland and Rumelhart's word recognition model. The brain is a connectionist model. It is not a perceptron. Perhaps the current interest in connectionist models is just a passing fad. Some folks are predicting that connectionism will turn out to be another spectacular flop -- Perceptrons II. At the other extreme, some feel the initial successes of ``the new connectionists'' may signal the beginning of a revolution in AI. Read the journals and decide for yourself.