NEURON-Request@ti-csl.csc.ti.COM (NEURON-Digest moderator Michael Gately) (10/28/87)
NEURON Digest Wed Oct 28 09:35:59 CST 1987 Volume 2 / Issue 25 Today's Topics: Time-averaging of neural events/firings neuro sources Using Hopfield Nets to solve the Traveling Salesman Problem references code for comprehensive backprop simulator Re: Learning with Delta Rule Re: Carver Mead's book The BAM example in Byte Source of Hopfield TSP solution 1988 summer school announcement Neural Net References Announcing Neural Network Review Speech Recognition Using Connectionist Networks (UNISYS) Tech. report abstract ---------------------------------------------------------------------- Date: Fri, 16 Oct 87 08:36:52 EST From: "Peter H. Schmidt" <peter@mit-nc.mit.edu> Subject: Time-averaging of neural events/firings Is anyone out there investigating neural nets that don't use McCullogh-Pitts 1-bit-quantized neurons? In other words, is anyone investigating the possibility that all the information is *not* conveyed merely by the (high or low) frequency of neural firings? Thanks, Peter H. Schmidt peter%nc@mc.lcs.mit.edu (ARPANET) peter%nc%mc.lcs.mit.edu@cs.net.relay (CSNET) ------------------------------ Date: Tue 20 Oct 87 21:07:39-EDT From: "John C. Akbari" <AKBARI@cs.columbia.edu> Subject: neuro sources anyone have the source code for either of the following? kosko, bart. constructing an associative memory. _byte_ sept. 1987 jones, w.p. & hoskins, j. back-propagation. _byte_ oct. 1987. any help would be appreciated. John C. Akbari PaperNet 380 Riverside Drive, No. 7D New York, New York 10025 USA SoundNet 212.662.2476 (EST) ARPANET & Internet akbari@CS.COLUMBIA.EDU BITnet akbari%CS.COLUMBIA.EDU@WISCVM.WISC.EDU UUCP columbia!cs.columbia.edu!akbari ------------------------------ Date: Tue, 27 Oct 87 14:05 EST From: Fausett@radc-multics.arpa Subject: Using Hopfield Nets to solve the Traveling Salesman Problem Can anyone give me a reference to the use of Hopfield nets in solving the traveling salesman problem (TSP). Can this approach be used to solve TSP's which have local and/or global constraints? ------------------------------ Date: Thu, 15 Oct 87 14:52:09 CDT From: simpson@nosc.mil Subject: references >Date: 15 Oct 87 10:30:00 EST >From: "NRL::MAXWELL" <maxwell%nrl.decnet@nrl3.arpa> >Subject: REFERENCES > >DEAR PATRICK, > > YOUR MESSAGE WAS UNCLEAR. WAS THE PRICE ON THE REFERENCE LIST >$3.00 plus OR including $3.00 postage? > MAXWELL@NRL.ARPA ------ Dr. Maxwell, I apologize for the lack of clarity in the message. $3.00 will cover the cost of postage and handling, that is the ONLY charge. I am simply covering the cost of copies, envelopes and postage with the $3.00. Patrick K. Simpson 9605 Scranton Road Suite 500 San Diego, CA 92121 ------------------------------ Date: 19 Oct 87 19:15:51 GMT From: Andrew Hudson <PT.CS.CMU.EDU!andrew.cmu.edu!ah4h+@cs.rochester.edu> Subject: code for comprehensive backprop simulator This is in response to a query for connectionist simulator code. Within a month, one of the most comprehensive back propagation simulators will be available to the general public. Jay McClelland and David Rumelhart's third PDP publication, Exploring Parallel Distributed Processing: A Handbook of Models, Programs, and Exercises will be available from MIT Press. C source code for the complete backprop simulator, as well as others, is supplied on two MS-DOS format 5 1/4" floppy discs. The simulator, called BP, comes with the necessary files to run encoder, xor, and other problems. It supports multiple layer networks, constrained weight, and sender to receiver options. It also has nicely laid out and nicely parsed menu options for every parameter you could ever imagine. The handbook and source code can be ordered from MIT Press at the address below. The cost for both is less than $30. Why spend thousands more for second best? The MIT Press 55 Hayward Street Cambridge, MA 02142 Another version of the BP simulator which is not yet generally available to the public has been modified to take full advantage of the vector architecture of the Convex mini-supercomputer. For certain applications this gives speed increases of 30 times that of a VAX 11/780. A study is underway to see how well BP will perform on a CRAY XMP-48. - Andrew Hudson ah4h@andrew.cmu.edu.arpa Department of Psychology Carnegie Mellon 412-268-3139 Bias disclaimor: I work for Jay, I've seen the code. ------------------------------ Date: 19 Oct 87 09:37:41 PDT (Mon) From: creon@ORVILLE.ARPA Subject: Re: Learning with Delta Rule The problem is, what if you do not know the invariances before hand, and thus cannot "wire them in"? What I would like is for the net to discover the invariances. We tried and tried this, using both first order and second order (corellated) three layer nets. We had the computer randomly choose a pattern, shift it, and present it to the net. Then it would correct the net (if necessary) in the standard backprop way. We could not get the net to learn the invariaces by itself, and the net did not have the capacity to learn all possible shifts of each pattern explicity, which is not what we wanted anyway. Are there any results on non-trivial spontaneous generalization in back-propagation nets? They are good at recalling the previous input that has the minimum hamming distance from current input, but can't they do more than this? ------------------------------ Date: Thu, 22 Oct 87 09:48:01 EST From: Manoel F Tenorio <tenorio@ee.ecn.purdue.edu> Subject: Re: Carver Mead's book >> Carvey Mead's book in analog VLSI Manoj, I have talked to the publisher (Addison-Wesley), and it won't be out till the Spring. If you get your hands on the notes, I would appreciate receiving a copy. --ft. School of Electrical Engineering Purdue Univesity W. Lafayette, IN 47907 ------------------------------ Date: Fri, 23 Oct 87 13:17 N From: SCHOMAKE%HNYKUN53.BITNET@wiscvm.wisc.edu Subject: The BAM example in Byte [] Apart from the alignment dependency, which is a general characteristic of most simple neural net simulation implementations there may be more problems. I tried to build the biderectional associative memory (BAM) program from the recipe (Listing 1) and I have something that works, but...: I find the program's capacities rather disappointing, compared to e.g. (the admittedly more complex) Siloam. Recognizing more than two pairs is often difficult. The network converges alright, but it may be to a meaningless state instead of reverberating the "best matching pair". Now there are two possibilities. 1) I missed some important point while coding (I don't think so, since the simple examples with two stored 6bit pairs work alright). or: 2) The author was very lucky in selecting three pairs of character bitmaps that resulted in good recognition in his example (;-). Also, I noticed someone interpreting Figure 2 in the article as: >...in the Byte article they demonstrate correct recall of an image >corrupted by randomly flipping a number of bytes, simulating "noise"... >Greg Corson, ...seismo!iuvax!ndmath!milo They do not. Figure 2 shows the recognition process in a kind of slow motion, by randomly choosing weights that are allowed to be updated during the iteration (asynchronous recall). This randomness is not in the data, it is in the recall process itself. This tells us that the BAM does not have to be a synchronous technical machine but _could_ be a model for some kind of biological neural memory. In fact, when an association is strong, it would come up in only one to three synchronous iterations. The input pair <S>-<E> of the example is _not_ corrupted! To tell the truth, I am a little bit skeptical about BAMs. From systems theory and signal processing theory I know that you can reconstruct a single input signal from a crosscorrelation function (here: the matrix of synaptic weights) that is based on several input and output sweeps, if, and only if, the spectral contributions of the sweeps are significantly different. Adding the (I/O)-(O/I) iteration will enhance the capacities of such a system, but it will always suffer from the disability to deal with many-to-one mappings or many-to-(many-similars) mappings. Lambert Schomaker SCHOMAKE@HNYKUN53.BITNET Nijmegen, The Netherlands. Reference: Kosko, B. (1987). Constructing an Associative Memory. Byte: the Small Systems Journal, Vol. 12 (10), pp.137-144. ------------------------------ Date: Wed, 7 Oct 87 13:36:16 EST From: "Peter H. Schmidt" <peter@mit-nc.mit.edu> Subject: Source of Hopfield TSP solution The article "Computing With Neural Circuits: A Model", Science, Vol. 233, 8-8-86, pp. 625-632, by Hopfield and Tank, describes the application of a Hopfield net using graded-response neurons to TSP, and to a simple analog-binary computation. It's very readable. N.B. The circuit described doesn't "solve" the TSP in terms of finding *the* optimum solution - rather, it converges quickly to 1 of the 10^7 best solutions out of a possible ~10^30 tours in a 30 city problem, say. The advantage over conventional computational techniques is that the Hopfield net needs only 900 neurons, while a comparable time solution would require a "microcomputer having 10^4 times as many devices." (ibid., p. 632) This comparison seems a little beside the point to me. Peter H. Schmidt |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| Work: MIT 20A-002 Home: 3 Colonial Village, #3 Cambridge, MA, 02139 Arlington, MA, 02174 (617) 253-3264 (617) 646-2215 ARPANET: peter%nc@mc.lcs.mit.edu |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ------------------------------ Date: Wed 14 Oct 87 03:29:58-EDT From: Dave.Touretzky@C.CS.CMU.EDU Subject: 1988 summer school announcement THE 1988 CONNECTIONIST MODELS SUMMER SCHOOL ORGANIZER: David Touretzky ADVISORY COMMITTEE: Geoffrey Hinton, Terrence Sejnowski SPONSORS: The Sloan Foundation; AAAI; others to be announced. DATES: June 17-26, 1988 PLACE: Carnegie Mellon University, Pittsburgh, Pennsylvania PROGRAM: The summer school program is designed to introduce young neural network researchers to the latest developments in the field. There will be sessions on learning, theoretical analysis, connectionist symbol processing, speech recognition, language understanding, brain structure, and neuromorphic computer architectures. Students will have the opportunity to informally present their own research and to interact closely with some of the leaders of the field. PARTIAL LIST OF FACULTY: Yaser Abu-Mostafa (Caltech) James McClelland (Carnegie Mellon) Dana Ballard (Rochester) David Rumelhart (Stanford) Andrew Barto (U. Mass.) Terrence Sejnowski (Johns Hopkins) Gail Carpenter (Boston U.) Paul Smolensky (UC Boulder) Scott Fahlman (Carnegie Mellon) David Tank (AT&T Bell Labs) Geoffrey Hinton (Toronto) David Touretzky (Carnegie Mellon) George Lakoff (Berkeley) Alex Waibel (ATR International) Yann Le Cun (Toronto) others to be announced EXPENSES: Students are responsible for their meals and travel expenses, although some travel assistance may be available. Free dormitory space will be provided. There is no tuition charge. WHO SHOULD APPLY: The summer school's goal is to assist young researchers who have chosen to work in the area of neural computation. Participation is limited to graduate students (masters or doctoral level) who are actively involved in some aspect of neural network research. Persons who have already completed the Ph.D. are not eligible. Applicants who are not full time students will still be considered, provided that they are enrolled in a doctoral degree program. A total of 50 students will be accepted. HOW TO APPLY: By March 1, 1988, send your curriculum vitae and a copy of one relevant paper, technical report, or research proposal to: Dr. David Touretzky, Computer Science Department, Carnegie Mellon University, Pittsburgh, PA, 15213. Applicants will be notified of acceptance by April 15, 1988. ------- ------------------------------ Date: 12-OCT-1987 12:16 From: simpsonp@nosc.mil Subject: Neural Net References NEURAL NETWORK REFERENCES AVAILABLE 750+ Neural Net References, 50 page reference list. A comprehesive ANS reference list from 1938 to present, includes every major ANS researcher and their earliest puiblications. Send $3.00 to cover postage and handling to: Patrick K. Simpson Verac, Inc. 9605 Scranton Road Suite 500 San Diego, CA 92121 ------------------------------ Date: Mon, 26 Oct 87 18:00:32 EST From: Craig Will <csed-1!will@hc.dspo.gov> Subject: Announcing Neural Network Review Announcing a new publication NEURAL NETWORK REVIEW The critical review journal for the neural network community Neural Network Review is intended to provide a forum for critical analysis and commentary on topics involving neural network research, applications, and the emerging industry. A major focus of the Review will be publishing critical reviews of the neural network literature, including books, individual papers, and, in New York Review of Books style, groups of related papers. The Review will also publish general news about events in the neural network community, including conferences, funding trends, and announcements of new books, papers, courses, and other media, and new hardware and software pro- ducts. The charter issue, dated October, 1987, has just been published, and contains a review and analysis of 11 articles on neural networks published in the popular press, a report on the San Diego conference, a report on new funding initia- tives, and a variety of other information, a total of 24 pages in length. The next issue, due in January, 1988, will begin detailed reviews of the technical literature. Neural Network Review is aimed at a national audience, and will be published quarterly. It is published by the Washington Neural Network Society, a nonprofit organization based in the Washington, D.C. area. Subscriptions to Neural Network Review are $ 10.00 for 4 issues, or $ 2.50 for a single copy. International rates are slightly higher. Rates for full-time students are $5.00 for 4 issues. (Checks should be payable to the Washington Neural Network Society). Subscription orders and inquiries for information should be sent to: Neural Network Review P. O. Box 427 Dunn Loring, VA 22027 For more information on Neural Network Review, send your physical, U. S. Postal mail address in a message to will@hc.dspo.gov (Craig Will). ------------------------------ Date: Tue 27 Oct 87 20:33:41-PST From: finin@bigburd.PRC.Unisys.COM (Tim Finin) Subject: Speech Recognition Using Connectionist Networks (UNISYS) AI Seminar UNISYS Knowledge Systems Paoli Research Center Paoli PA SPEECH RECOGNITION USING CONNECTIONIST NETWORKS Raymond Watrous Siemens Corporate Research and University of Pennsylvania The thesis of this research is that connectionist networks are adequate models for the problem of acoustic phonetic speech recognition by computer. Adequacy is defined as suitably high recognition performance on a representative set of speech recognition problems. Six acoustic phonetic problems are selected and discussed in relation to a physiological theory of phonetics. It is argued that the selected tasks are sufficiently representative and difficult to constitute a reasonable test of adequacy. A connectionist network is a fine-grained parallel distributed processing configuration, in which simple processing elements are interconnected by simple links. A connectionist network model for speech recognition has been defined called the TEMPORAL FLOW MODEL. The model incorporates link propagation delay and internal feedback to express temporal relationships. It has been shown that temporal flow models can be 'trained' to perform successfully some speech recognition tasks. A method of 'learning' using techniques of numerical nonlinear optimization has been demonstrated for the minimal pair "no/go", and voiced stop consonant discrimination in the context of various vowels. Methods for extending these results to new problems are discussed. 10:00am Wednesday, November 4, 1987 Cafeteria Conference Room Unisys Paloi Research Center Route 252 and Central Ave. Paoli PA 19311 -- non-UNISYS visitors who are interested in attending should -- -- send email to finin@prc.unisys.com or call 215-648-7446 -- ------------------------------ Date: 15 Oct 87 18:22:16 GMT From: A Buggy AI Program <speedy!honavar@speedy.wisc.edu> Subject: Tech. report abstract Computer Sciences Technical Report #717, September 1987. -------------------------------------------------------- RECOGNITION CONES: A NEURONAL ARCHITECTURE FOR PERCEPTION AND LEARNING Vasant Honavar, Leonard Uhr Computer Sciences Department University of Wisconsin-Madison Madison, WI 53706. U.S.A. ABSTRACT There is currently a great deal of interest and activity in developing connectionist, neu- ronal, brain-like models, in both Artificial Intelligence and Cognitive Science. This paper specifies the main features of such systems, argues for the need for, and usefulness of struc- turing networks of neuron-like units into succes- sively larger brain-like modules, and examines "recognition cone" models of perception from this perspective, as examples of such structures. Issues addressed include architecture, information flow, and the parallel-distributed nature of pro- cessing and control in recognition cones; and their use in perception and learning. ----- Vasant Honavar honavar@speedy.wisc.edu ------------------------------ End of NEURON-Digest ********************