NEURON-Request@ti-csl.csc.ti.COM (NEURON-Digest moderator Michael Gately) (01/21/88)
NEURON Digest Wed Jan 20 18:46:14 CST 1988 Volume 3 / Issue 1 Today's Topics: Re: Genesis of language (was: Why can't my cat talk, and a bunch of others) MLNS Announcement Net Simulators Review for Science Magazine? An NN Simulator for a Simulation Class Neural computations based on timing and phase in neural nets MacBrain Hopfield Networks Mathematical Linguistics Commercial products based on neural nets? NAMING CONVENTION Stanford Adaptive Networks Colloquium Tech Report -- Connectionist Bibliography ---------------------------------------------------------------------- Date: 10 Dec 87 17:00:07 GMT From: mnetor!utzoo!henry@uunet.uu.net (Henry Spencer) Subject: Re: Genesis of language (was: Why can't my cat talk, and ...) > Apropos cell death in brains, the old saw about losing 10,000 neurons every > day is now being challenged... Also of note, the other old saw that new neurons are not formed after birth in higher animals is now known to be untrue. At least some higher animals (some types of birds) do grow new neurons at times. Last I heard, nobody yet knows how common this phenomenon is. -- Those who do not understand Unix are | Henry Spencer @ U of Toronto Zoology condemned to reinvent it, poorly. | {allegra,ihnp4,decvax,utai}!utzoo!henry ------------------------------ Date: Sun, 10 Jan 88 21:50:43 EST From: Bob Weidlich <weidlich@ludwig.scc.com> Subject: MLNS Announcement A PROPOSAL TO THE NEURAL NETWORK RESEARCH COMMUNITY TO BUILD A MULTI-MODELED LAYERED NEURAL NETWORK SIMULATOR TOOL SET (MLNS) Robert Weidlich Contel Federal Systems January 11, 1988 The technology of neural networks is in its infancy. Like all other major new technologies at that stage, the development of neural networks is slowed by many impediments along the road to realizing its potential to solve many sig- nificant real world problems. A common assumption of those on the periphery of neural network research is that the major factor holding back progress is the lack of hardware architectures designed specifically to implement neural networks. But those of us who use neural networks on a day to day basis real- ize that a much more immediate problem is the lack of sufficiently powerful neural network models. The pace of progress in the technology will be deter- mined by the evolution of existing models such as Back Propagation, Hopfield, and ART, as well as the development of completely new models. But there is yet another significant problem that inhibits the evolution of those models: lack of powerful-yet-easy-to-use, standardized, reasonably- priced toolsets. We spend months of time building our own computer simula- tors, or we spend a lot of money on the meager offerings of the marketplace; in either case we find we spend more time building implementations of the models than applying those models to our applications. And those who lack sophisticated computer programming skills are cut out altogether. I propose to the neural network research community that we initiate an endeavor to build a suite of neural network simulation tools for the public domain. The team will hopefully be composed of a cross-section of industry, academic institutions, and government, and will use computer networks, pri- marily Arpanet, as its communications medium. The tool set, hereinafter referred to as the MLNS, will ultimately implement all of the significant neural network models, and run on a broad range of computers. These are the basic goals of this endeavor. 1. Facilitate the growth and evolution of neural network technology by building a set of powerful yet simple to use neural network simula- tion tools for the research community. 2. Promote standardization in neural network tools. 3. Open up neural network technology to those with limited computer expertise by providing powerful tools with sophisticated graphical user interfaces. Open up neural network technology to those with limited budgets. 4. Since we expect neural network models to evolve rapidly, update the tools to keep up with that evolution. This announcement is a condensation of a couple of papers I have written describing this proposed effort. I describe how to get copies of those docu- ments and get involved in the project, at the end of this announcement. The MLNS tool will be distinctive in that will incorporate a layered approach to its architecture, thus allowing several levels of abstraction. In a sense, it is a really a suite of neural net tools, one operating atop the other, rather than a single tool. The upper layers enable users to build sophisti- cated applications of neural networks which provide simple user interfaces, and hide much of the complexity of the tool from the user. This tool will implement as many significant neural network models (i.e., Back Propagation, Hopfield, ART, etc.) as is feasible to build. The first release will probably cover only 3 or 4 of the more popular models. We will take an iterative approach to building the tool and we will make extensive use of rapid prototyping. I am asking for volunteers to help build the tool. We will rely on computer networks, primarily Arpanet and those networks with gateways on Arpanet, to provide our communications utility. We will need a variety of skills - pro- grammers (much of it will be written in C), neural network "experts", and reviewers. Please do not be reluctant to help out just because you feel you're not quite experienced enough; my major motivation for initiating this project is to round-out my own neural networking experience. We also need potential users who feel they have a pretty good feel for what is necessary and desirable in a good neural network tool set. The tool set will be 100% public domain; it will not be the property of, or copyrighted by my company (Contel Federal Systems) or any other organization, except for a possible future non-commercial organization that we may want to set up to support the tool set. If your are interested in getting involved as a designer, an advisor, a poten- tial user, or if you're just curious about what's going on, the next step is to download the files in which I describe this project in detail. You can do this by ftp file transfer and an anonymous user. To do that, take the follow- 1. Set up an ftp session with my host: "ftp ludwig.scc.com" (Note: this is an arpanet address. If you are on a network other than arpanet with a gateway to arpanet, you may need a modified address specification. Consult your local comm network gure if you need help.) 2. Login with the user name "anonymous" 3. Use the password "guest" 4. Download the pertinent files: "get READ.ME" (the current status of the files) "get mlns_spec.doc (the specification for the MLNS) "get mlns_prop.doc (the long version of the proposal) If for any reason you cannot download the files, then call or write me the following address: Robert Weidlich Mail Stop P/530 Contel Federal Systems 12015 Lee Jackson Highway Fairfax, Virginia 22033 (703) 359-7585 (or) (703) 359-7847 (leave a message if I am not available) ARPA: weidlich@ludwig.scc.com ------------------------------ Date: Fri, 4 Dec 87 16:06:01 EST From: futrell@corwin.ccs.northeastern.edu Subject: Net Simulators Review for Science Magazine? As you may know, Science magazine publishes reviews of software from time to time. There is some interest at Science in publishing a review of neural net simulation systems, with emphasis on ones that run on micros. In order to help Science arrange for these reviews, I need to know: 1. Candidate packages for review. These should have been on the market for a while (not a week!). They do not have to be commercial, but there has to be some confidence that the distributor will support them and answer users' questions in the future. Note that the packages are to be simulators, not applications which focus on merely using neural or connectionist learning technology underneath to solve some other problem. 2. Candidates for the reviewer. The person(s) who writes the review should be, ideally, a working scientist who has day-to-day involvement with neuroscience or cognitive science. The philosophy at Science is to have scientists write reviews for other scientists. Science mails about 170,000 issues a week and their readership is far larger than that. The thrust of the magazine is primarily biological, but that is more historical accident then by design. Please contact me, rather than Science (I am one of the advisors for their software reviews). Mail to this net or to me at (CSNet): futrelle@corwin.ccs.northeastern.edu or: (617)-437-2076 or: Robert P. Futrelle College of Computer Science 161CN Northeastern University 360 Huntington Ave. Boston, MA 02115 ------------------------------ Date: Fri, 11 Dec 87 13:04:54 EST From: "Charles S. Roberson" <csrobe@icase.arpa> Subject: An NN Simulator for a Simulation Class The study of Neural Net Simulations is starting to get some attention at William and Mary and I have been given a task -- Find 'something' from the Neural Net domain that could be discussed and simulated in a second semester simulation class. So I decided to ask those most knowledgeable in the field -- you! Some background: Simulation II is a graduate level course which has Simulation I has a prereq. Simulation I is a rather intense class which teaches the fundamentals of random number generation (from discrete and continuous distributions), fitting distributions to data, and simulating discrete events. The professor's primary area of research is Digital Image Processing, and his curiosity is piqued concerning Neural Nets. The Proposal: The professor has agreed to introduce a Neural Net simulation late in the second semester IFF there is a well written paper that would, on its own merit, provide enough of framework to allow its simulation. It its not necessary that the (graduate) students be able to understand the whole paper, but the professor must be able to digest enough of the paper to properly present it to the class. My Question: Is there such a paper that would fit these restrictions? All simulations have been of the 'Roll your own' variety (we have written everything down to our random number (Lehmer) generator), so I would imagine this philosophy will carry into the second semester. The amount of time spent on this specific simulation would probably not exceed two weeks. I know this is limiting, but the ability to perform this simulation could open up William and Mary to investigating Neual Networks more thoroughly. Thanks, Chip Roberson, Graduate Student, College of William and Mary ------------------------------------------------------------------------- Chip Roberson ARPANET: csrobe@icase.arpa 1105 London Company Way BITNET: $csrobe@wmmvs.bitnet Williamsburg, VA 23185 UUCP: ...!uunet!pyrdc!gmu90x!wmcs!csrobe ------------------------------------------------------------------------- ------------------------------ Date: Fri, 11 Dec 87 10:28:34 PST From: Kaiti Riley <kaiti%meridian@ads.arpa> Subject: Neural computations based on timing and phase in neural nets Most of the literature on neural networks emphasises spatial computation in networks by modifying weights. Another way to view neural computations is to examine the relative timing and phase in neural networks. As an example, Reichardt's original work on motion detection in fly retinae looked at the timing difference between two adjacent detectors to determine direction of motion (updated versions by van Santen and Sperling, Adelson and Bergen, Watson and Ahumada, etc.). I am currently looking for work that specifically examines dynamic timing and phasal computations in neural(like) networks. (Work in asynchronous parallel automata systems appears to also be related, but I am also in need of references on this related subject.) I will post responses to the net. Thanks in advance! ------------------------------ Date: Tue, 22 Dec 87 16:30:10 est From: ucbcad!ames.UUCP!ulowell!cg-atla!jmacdon@UCBVAX.BERKELEY.EDU Subject: MacBrain I've seen several references to a software package called MacBrain by Neuronics of Cambridge, Massachusetts. The most recent reference was on page 103 of the November issue of MacWorld magazine. None of the references to date have provided sufficient information for ordering the package or even contacting Neuronics. They are not listed in the phone directory nor is directory assistance of any assistance. If a hard pointer to Neuronics is available could some kind soul email it to me or, if Neuronics is no more, would someone with the package be willing to provide me with a copy? I would also be interested in hearing from anyone who has used MacBrain. Thanks. Jeff MacDonald jmacdon@cg-atla 617-658-0200 ext. 5406 ------------------------------ Date: Wed 30 Dec 87 13:11:38-PST From: Rakesh & <MOHAN%DWORKIN.usc.edu@oberon.usc.edu> Subject: Hopfield Networks I am using Hopfield Networks for optimization in some Computer Vision problems. I would like to know of some strategies to set the various weights. Also, has somebody experimented with non-symmetric weights and/or self-excitation (of nodes) in similar networks. Do the weights depend on the size of the problem? In other words, if a set of weights is found to work for a given problems, do the same weights work if there is change in the number of nodes by some orders of magnitude? Thanks in advance, Rakesh Mohan mohan@oberon.USC.EDU ------------------------------ Date: Thu, 7 Jan 88 12:16 N From: FLEUR%HLERUL55.BITNET@cunyvm.cuny.edu Subject: Mathematical Linguistics Dear fellow networkers, At the department of Experimental Psychology (Leyden Holland) we did a simmualtion based on 'neuronet like' principles. With our simmulation we intended to explore the field of human mental association of words. The algoritm of the program was based on Quillian's semantic memory model. This model can be adapted to neuronets when considering the semantics of a word represented by the activity of a cluster of parallel active neurons. In our view the members of the cluster can be situated 'all over the brain'. We translated mental association in exchange of 'activation' between two or more clusters. This communication we described with the aid of intertwined first-order partial differential equations concerning the change in time of total (neural) activity in a cluster connected to other clusters. We would like to contact other workers in this line of research. Greetings: Han Geurdes Erik Fleur Dep. Exp. Psy State Univ. Leyden Hooigracht 15 Leyden GEURDES@HLERUL55 ------------------------------ Date: Thu, 7 Jan 88 05:33 PST From: nesliwa%nasamail@ames.arc.nasa.gov (NANCY E. SLIWA) Subject: Commercial products based on neural nets? I've had a request for information about the existence of any commercial products based on neural net technology. Not to develop neural net applications, like HNC and Sigma neurocomputers, but actual products that use neuromimetic approaches. I've heard/read somewhere long since about two things: (1) a California-based product for processor board layout (2) a McLean, VA-based company that has been selling neural-based products since the 60's Does anyone know the specifics of these items, and/or especially any other examples? Please respond to me directly, and I'll summarize to the list. Thanks! Nancy Sliwa MS 152D NASA Langley Research Center Hampton, VA 23665-5225 804/865-3871 nesliwa%nasamail@ames.arpa or nancy@grasp.cis.upenn.edu ------------------------------ Date: Thu,17 Dec 87 16:22:49 GMT From: Julian_Dow@vme.glasgow.ac.uk Subject: NAMING CONVENTION I just got registered on the BB, and read with delight the debate on catchphrases. I too despise "NEURAL NETWORKS", but acknowledge that the term is here to stay. Why not just adopt a convention I suspect already is coming to pass: NEURONAL NETWORK for a network of neurons, i.e. in biology NEURAL NETWORK for a network of the things that electronic engineers imagine to be neurons. The difference is subtle, but unmistakable. ------------------------------ Date: Wed, 23 Dec 87 13:40:17 PST From: Mark Gluck <gluck@psych.stanford.edu> Subject: Stanford Adaptive Networks Colloquium Stanford University Interdisciplinary Colloquium Series: ADAPTIVE NETWORKS AND THEIR APPLICATIONS Co-sponsored by the Depts. of Psychology and Electrical Engineering Winter Quarter Schedule ----------------------- Jan. 12th (Tuesday, 3:15pm) Harry Klopf "The Drive-Reinforcement Neuronal Model: Wright Aeronautical Labs A Real-time Learning Mechanism for U. S. Air Force Unsupervised Learning" Feb. 9th (Tuesday, 3:15pm) Tom Landauer "Trying to Teach a Backpropogation Network\fR Bellcore to Recognize Elements of Continuous Speech." Feb. 12th (Friday, 1:15pm) Yann Le Cun "Pseudo-Newton and Other Variations of Dept. of Computer Science, Backpropogation" University of Toronto Mar. 9th (Tuesday, 3:45pm) Jeffrey Elman "Processing Language Without Symbols? Dept. of Linguistics, A Connectionist Approach" U.C., San Diego Mar. 29th (Tuesday, 3:15pm) Dan Hammerstrom "Casting Neural Networks in Silicon: Oregon Graduate Center Good News and Bad News" Additional Information ---------------------- Focus: Adaptive networks, parallel-distributed processing, connectionist models, computational neuroscience, the neurobiology of learning and memory, and neural models. Format: Tea will be served 15 minutes prior to the talks, outside the lecture hall. The talks (including discussion) last about one hour. Following each talk, there will be a reception in the fourth floor lounge of the Psychology Dept. Location: Unless otherwise noted, all talks will be held in room 380-380W, which can be reached through the lower level courtyard between the Psychology and Mathematical Sciences buildings. Technical Level: These talks will be technically oriented and are intended for persons actively working in related areas. They are not intended for the newcomer seeking general introductory material. Information: To be placed on an electronic mail distribution list for information about these and other adaptive network events in the Stanford area, send email to netlist@psych.stanford.edu. For additional information, contact Mark Gluck, Bldg. 420-316; (415) 725-2434 or email to gluck@psych.stanford.edu Program Committee: Bernard Widrow (E.E.), David Rumelhart, Misha Pavel, Mark Gluck (Psychology). ------------------------------ Date: Mon, 28 Dec 87 16:01:08 EST From: MaryAnne Fox <mf01@gte-labs.csnet> Subject: Tech Report -- Connectionist Bibliography SELECTED BIBLIOGRAPHY ON CONNECTIONISM Oliver G. Selfridge Richard S. Sutton Charles W. Anderson GTE Labs An annotated bibliography of 38 connectionist works of historical or current interest. For copies, reply to this message with your USmail address, or send to: Mary Anne Fox GTE Labs MS-44 Waltham, MA 02254 mf01@GTE-Labs.csNet ------------------------------ Date: Wed, 30 Dec 87 18:45:03 est From: Bob Allen <rba@flash.bellcore.com> Preprints available of a paper presented the the conference on Neural Information Processing Systems Stochastic Learning Networks and their Electronic Implementation Joshua Alspector, Robert B. Allen, Victor Hu, and Srinagesh Satyanarayana We describe a family of learning algorithms that operate on a recurrent, symmetrically connected, neuromorphic network that, like the Boltzmann machine, settles in the presence of noise. These networks learn by modifying synaptic connection strengths on the basis of correlations seen locally by each synapse. We describe a version of the supervised learning algorithm for a network with analog activation functions. We also demonstrate unsupervised competitive learning with this approach, where weight saturation and decay play an important role, and describe preliminary experiments in reinforcement learning, where noise is used in the search procedure. We identify the above described phenomena as elements that can unify learning techniques at a physical microscopic level. These algorithms were chosen for ease of implementation in vlsi. We have designed a CMOS test chip in 2 micron rules that can speed up the learning about a millionfold over an equivalent simulation on a VAX 11/780. The speedup is due to parallel analog computation for summing and multiplying weights and activations, and the use of physical processes for generating random noise. The components of the test chip are a noise amplifier, a neuron amplifier, and a 300 transistor adaptive synapse, each of which is separately testable. These components are also integrated into a 6 neuron and 15 synapse network. Finally, we point out techniques for reducing the area of the electronic correla- tional synapse both in technology and design and show how the algorithms we study can be implemented naturally in electronic systems. Reprints of an earlier paper "A Neuromorphic VLSI Learning System" J. Alspector and R.B. Allen, in: Advanced Reserach in VLSI, edited by P. Losleben, are also available. Contact: Bob Allen, 2A-367, Bell Communications Research, Morristown, NJ 07960, rba@bellcore.com ------------------------------ End of NEURON-Digest ********************