richman@uxe.cso.uiuc.edu (08/03/88)
First off, I would like to thank all of the people who took time to post information on their favorite neural network books and articles in response to the following query I posted: "Could someone recommend a good introductory text which deals with Neural Networks?" The following (long) are the compendium of responses: -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Mike Richman internet: richman@uiucuxe.cso.uiuc.edu uucp: {puree,convex,uunet,...}!uiucdcs!uiucuxc!uiucuxe!richman bitnet: richman%uiucuxe.cso.uiuc.edu@interbit -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= ____________________ I took a class at Brown University from Prof. James Anderson. He's recently written a book (soon to be published) titled: "Neural Modelling Laboratory" or something like that. It's an excellent introductory text and certainly not too difficult to pick up. Of course there are the PDP (Parallel Distributed Processing Vol 1 and 2 by Rumelhart and McClealand (sp?)) books too, but they're substantially more complicated. David Hoffman ____________________ there are two good references on neural networks, both published by the MIT Press. i found them both delightful reading. 1) NeuroComputing (forgot who the editor was..collection of papers) 2) Neural Networks & Natural Intelligence (by Grossberg) have fun.... Choong ___________________________________ ____________________ Here's my two cents of worth: "Parallel Distributed Processing", Vol 1, 2, and 3 by McClelland, Rumelhart, nad the PDP Reserch Group. 1986. and the Journal of INNS (international Neural Network Society) which is called "Neural Netwok". Jen-I Pi :-) _____________________ Mike, besides the obvious choice of "Parallel Distributed Processing Vol 1" by Rummelhart and Mc Clelland, I also recommend "Self Organization and Associative Memory" by Kohonen. - Mathew Yeates _____________________ Try "Parallel distributed processing" by Rumelhart. You must already have heard about it. A very good but very new one is "Automata networks in computer science-Theory and applications". It has a lot of theoretical background on Neural nets. Your other choice is to wait until I get mine published (!!!!.....) Ah, I forgot. The authors for the second book are: Francoise Folgeman Soulie,Yves Robert, Maurice Tchuente. It's in english don't be scared. You can get the two volumes of the PDP and the second book from the Library of Computing and Information Sciences very cheap. If you become a member you get three books free and you are only obligated to by three more. Since these are in their catalogue you can get them free. This club is a McGraw Book club in case you haven't heard of it. It's cheap too. Whoever signs you up gets a book free for himself too. There is a bunch of papers if you want references. Mike, You could try the Parallel Distributed Procesing books by Rumelhart & McLelland. It is _the_ introductory text available on the subject. It's a three volume set, with vols 1 & 2 being introductory text and vol 3 which contains source code and explanatoins of all of the models in vols 1 & 2. Vols 1 & 2 comde as a set ($25.00 for paperback about $50.00 for hard), Vol 3 is paperback (@$27.50) (What a bargain!). They are available from MIT Press. Steve Dussinger _____________________ I reccommend _Parallel_Distributed_Processing_ by Rumelhart and McClelland (MIT PRESS 2 volumes, best bet is to order both of them paperback fr around $20). PDP explains basic neural network concepts, competitive learning, Boltzman Machines, Back Propogation, as well as a good number of research on NN applications. The magazine Daedelus recently had an entire issue devoted to Neural Networks ... you might wish to check that out whilst waiting for the MIT Press to send you PDP. -Thomas G. Edwards _____________________ ____________________ An EXCELLENT introductory text: Parallel Distributed Processing: Explorations in the Microstructure of Cognition edited by McClelland and Rumelhart MIT Press, 1986 and 1987 (3 volumes) An EXCELLENT introductory text: Parallel Distributed Processing: Explorations in the Microstructure of Cognition edited by McClelland and Rumelhart MIT Press, 1986 and 1987 (3 volumes) Parallel Distributed Processing, by McClelland,Rumelhart, et. al., published by the MIT Press, seems to be the book of choice for a technical introduction to Neural Netting. It's two volumes and gets very mathematical in places, but the first four chapters of Volume I are a wonderful introduction to many of the major area of PDP research going on currently. Enjoy, William R. Swanson P.S. I'm starting out in Neural Nets myself. If you get any other good references, please pass them along! _____________________ A good introductory paper on neural nets is in the April 87 issue of the IEEE ASSP MAGAZINE by Richard Lippmann called An Intoduction To Neural Nets. This paper covers six important neural net models used in pattern classification, and compares them with more traditional models. If I am correct and you are from Urbana, you should be able to get a copy in one of your engineering libraries. If not, have them send it from the main library at UIC. Hope this helps, Jeff Beer _____________________ Here's a few references: 1). "A review of artificial neural systems I: Foundations," "A review of artificial neural systems II: Paradigms, applications, and implementations," both by Patrick K. Simpson, I've got a preprint, but it was submitted to CRC Critical Reviews in Artificial Intelligence, if you can't find it, the guy's address is: General Dynamics Electronics Division P. O. Box 85310 San Diego, CA 92138 2). Parallel Distributed Processing: Explorations in the Microstructure of Cognition I. Foundations II. Psychological and Biological Models David E. Rumelhart, James L. McClelland, eds. MIT Press, 1986, especially vol. 1 3). try the physics literature on "spin glasses", the two subjects are related 4). "Nonlinear neural networks. I. General theory, and II. Information processing," by J. L. van Hemman, D. Grensing, A. Huber, R. Kuhn, Journal of Statistical Physics, Vol. 50, No. 1/2, 1988 5). "An introduction to computing with neural nets," IEEE ASSP Magazine, Vol. 4, April 1987, R. Lippman There are probably a number of textbooks and conferences proceedings in the works or already out. I know of an AIP conference on neural networks a few years ago. Just search for the title "neural net" on the computer system at a good library if you are near one. Right now the literature is pretty much scattered in computer science, electronics, biology, psychology, etc. I think you get the picture. Explore and enjoy. S. Baum _____________________ In answer to your request on the AI-list I send you the following list of references: The best introductions are: Anderson, James A., and Rosenfeld, Edward, (Eds.), Neurocomputing: A collection of classic papers. MIT press 1987. The IEEE Computer issue of March 1988 containing the following articles: - Carpenter, G., and Grossberg, S., "The ART of Adaptive Pattern Recognition by a Self-Organizing Neural Network", IEEE Computer, March 1988. - Fukushima, K., "A Neural Network for Visual Pattern Recognition", IEEE Computer, March 1988. - Graf, H.P., Jackel, D.J., Hubbard, W.E., "VLSI Implementation fo a Neural Network Model", IEEE Computer, March 1988. - Widrow, B., and Winter, R., "Neural Nets for Adaptive Filtering and Adaptive Pattern Recognition", IEEE Computer, March 1988. - Kohonen, T., "The 'Neural' Phonetic Typewriter", IEEE Computer, March 1988. T. Kohonen, Self Organization and associative memory, Springer Verlag, New York / Berlin 1984 Rumelhart, David E., and McClelland, James L., "Parallel Distributed Processing: Explorations in the micro structure of cognition". Vol 1: Foundations. MIT press 1986. McClelland, James L. and Rumelhart, David E., "Parallel Distributed Processing: Explorations in the micro structure of cognition". Vol 2: Psychological and Biological Models, MIT press 1986. M.Minsky, and S.Papert, Perceptrons: an introduction to computational geometry, MIT press 1969 Other interesting literature is: Y.S. Abu-Mostafa and D. Pslatis, Optical Neural Computers, "Scientific American, 256, 88-95, March 1987. Ackley, D.H., Hinton, G.E., and Sejnovski, T.J., "A learning algorithm for Boltzmann machines", Cognitive Science, 9, 147-169, 1985. J.S. Denker, AIP Conference Proceedings 151, Neural Networks for Computing, Snowbird Utah, AIP, 1986. Eliot, Lance. B., "Neural Networks, part 1", IEEE expert, winter 1987. Graf, H.P., and deVegvar, P., "A CMOS implementation of a neural network model", Proc. Stanford Conf. Advanced Research in VLSI, March 1987, MIT Press. Grossberg, Stephen, "Studies of mind and brain", Boston, Reidel press, 1982. S. Grossberg, The adaptive brain 1: Cognition, learning, reinforcement, and rhytm, and The adaptive brain II: vision, speech, language and motor control, Elsevier/North Holland, Amsterdam (1986). Hecht-Nielsen, Robert, "Counterpropagation Networks", Applied Optics, december 1987. Hecht-Nielsen, Robert, "Neurocomputer Applications", Proceedings IEEE international conference on neural networks - 1987. Hecht-Nielsen, Robert, "Nearest Matched Filter Classification of Spatiotemporal Patterns", Applied Optics, 26, no. 10, 1892 - 1899, 15 may 1987. Hecht-Nielsen, R., "Neurocomputing: Picking the Human Brain", IEEE Spectrum, March 1988. Hopfield, J.J., "Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proc. Natl. Acd. Sci. USA, Vol. 79, 2554-255?, april 1982. Hopfield, J.J., "Neurons with Graded response have collective properties like those of two state neurons, Proc. Natl. Acd. Sci. USA, Vol. 81, 3088-3092, may 1984. Hopfield, J.J., and Tank, D.W., "Neural computation of decisions in optimization problems", Biological Cybernetics, 52, 141-152, july 1985. C.A. Mead , Analog VLSI and Neural systems. court notes, computer science dept. california institute of technology, 1986. D.W. Tank and J.J.Hopfield, "Simple Neural Optimization Networks: An A/D converter, signal decision circuit and a lineair programming circuit. IEE Trans. Circuits Systems. CAS-33, 533-541, 1986. Rumelhart, David E. and Zipser, David, "Feature Discovery by competitive learning", Cognitive Science, 9, 75-112, 1985. Sejnovski, T.J., and Rosenberg, C.R., "Parallel Networks that learn to Pronounce English Text", Complex Systems 1 (1987) 145-168. Treleaven, P. C., "Parallel Architectures for Neurocomputers", proceedings of the European Seminar on Neural Computing, 8-9 Febr 1988, London. Treleaven, P. C., "Programming Languages for Neurocomputing", in: proceedings of the European Seminar on Neural Computing, 8-9 Febr. 1988, London. Wasserman, Philip D., and Schwartz, Tom, "Neural Networks, Part II", IEEE expert, spring 1988. Succes! Martin Kraaijveld _____________________ Many people around here use OPS5, OPS83, and KEE. If you need more info, let me know. There's also a useful book by D. Waterman, "A Guide to Expert Systems," which has an extensive catalog of expert systems, ES shells and tools. Mai-Uyen Nguyen _____________________ I have been impressed with "Perceptrons" by Pappert and Minsky, from the MIT Press. It is well written, and full of valuable insights. The preface and epilogue are perhaps particularly interesting as they provide commentaries on Neural Nets as a field of endeavor, and the place of the arguments in "Perceptrons" with respect to that field. I accept the authors' argument that, contrary to popular opinion, this book is relevant to the *current* state of neural net research, and it is a major *positive* contribution to the field. The more recent major work in this field is Parallel Distributed Processing (commonly referred to as PDP). I don't know the author. I think it's published by MIT Press. -Lindsey Spratt _____________________ Be sure to get the most recent edition of Perceptrons (Pappert and Minsky) as it has the interesting prologue and epilogue which make its discussions current. -lindsey