neuron-request@HPLMS2.HPL.HP.COM ("Neuron-Digest Moderator Peter Marvit") (11/14/90)
Neuron Digest Tuesday, 13 Nov 1990
Volume 6 : Issue 66
Today's Topics:
Re: Transputer Implementations References?
Re: Evolution
Post-docs at Institute for Advanced Study
NIPS update
preprint - associative memory in oscillating cortex
Post-NIPS workshop on Associative Memory
Proceedings -- ConnectFest 1990
Job Openings at CMU
Posting: Conf. Announcement
Send submissions, questions, address maintenance and requests for old issues to
"neuron-request@hplabs.hp.com" or "{any backbone,uunet}!hplabs!neuron-request"
Use "ftp" to get old issues from hplpm.hpl.hp.com (15.255.176.205).
------------------------------------------------------------
Subject: Re: Transputer Implementations References?
From: erik@goofy.llnl.gov (Erik Johannson)
Date: Fri, 09 Nov 90 09:39:59 -0800
One of my colleagues, Tony De Groot, has successfully implemented
backpropagation networks on Transputers, and has run the code with as
many as 64 processors. He has presented a paper at SPIE on the method,
but I can't remember the publication details. You can reach him at
degroot@icdc.llnl.gov, or at the following address:
Tony De Groot
Lawrence Livermore National Laboratory
PO Box 808, L-156
Livermore, CA 94550
Telephone: (415) 422-5496
You might be interested in a paper I recently completed on
backpropagation learning using the conjugate gradient optimization
method. You can get a copy via anonymous ftp (described below). If you
are unable to retrieve the paper, I will be happy to send you a hard
copy.
Anonymous ftp instructions:
The following commands should give you a copy of the paper; if they don't
please let me know:
ftp peabody.llnl.gov (or ftp 128.115.53.1)
Name (peabody.llnl.gov:(null)): anonymous
Password (peabody.llnl.gov:anonymous): anything
ftp> cd neural
ftp> bin
ftp> get paper.Z
ftp> quit
The password can be anything you want except a carriage return. Also,
there is no ls command available, so follow the directions exactly.
The file you receive is a post script file in compressed format. To
uncompress it, type:
uncompress paper.Z
The resulting file will be called paper. To print it, type:
lpr paper
or whatever the print command is for your particular post script printer.
The paper is 27 pages long.
If you have any questions or comments, you can reach me at the following
address:
Erik M. Johansson
Lawrence Livermore National Laboratory
PO Box 808, L-496
Livermore, CA 94550
erik@goofy.llnl.gov
erik@adams.llnl.gov
johansson@icdc.llnl.gov
Sincerely,
Erik M. Johansson
------------------------------
Subject: Re: Evolution
From: UAP001%DDOHRZ11.BITNET@CUNYVM.CUNY.EDU,
UAP001 at DDOHRZ11
Date: Tue, 13 Nov 90 16:12:31 +0700
My guess is that the proposal by Elan Moritz - that evolution is on the
threshold of producing a smarter successor to h. sapiens - is wide of the
mark. Development of the cortex probably stopped abruptly (after a spurt
of remarkable growth) about 100,000 years ago. (Give or take a few months
- - I'm easy.) Of course, nervous tissue leaves no fossils, but the major
features of the brain cavity, including asymmetries, were in place; and
the reasonable assumption is that the wetware hasn't changed much since
then. Why should it suddenly begin to change radically now? There's
certainly no evolutionary pressure to change: we're certainly not using
it to capacity. Think of what (insert name of your favorite intellectual
hero here) did with the available circuits.
It could of course be argued that such exceptions had exceptional
hardware (although there's never been convincing anatomic or histologic
evidence for this). My answer would be that not only the exception, but
also the average human, is undergoing a dramatic functional development.
Most of us are performing intellectual activities that would have been
unthinkable a century ago. Even if much of it is wasteful, and even
counter- productive, it still represents an impressive achievment; and
there's no sign of it levelling off.
And I suppose the pessimistic view is that there's a vanishingly small
chance that conditions on the earth are going to remain conducive to the
evolution of anything but a super cockroach: if, in fact, some of them
aren't already occupying congressional chairs.
------------------------------
Subject: Post-docs at Institute for Advanced Study
From: Zhaoping Li <zl@guinness.ias.edu>
Date: Fri, 09 Nov 90 13:20:54 -0500
THE INSTITUTE FOR ADVANCED STUDY
Anticipates the opening of one or more positions in the area of
mathematical biology with emphasis on neural systems and computations.
The positions are at a postdoctoral level and are for two years beginning
academic year 1991-92. Applicants should send a CV and arrange for three
letters of recommendations to be sent directly to
Dr. Joseph J. Atick,
School of Natural Sciences
Institute for Advanced Study
Princeton, NJ 08540
Applications should be received no later than January 20, 1991.
Women and minorities are encouraged to apply.
------------------------------
Subject: NIPS update
From: jose@learning.siemens.com (Steve Hanson)
Date: Mon, 12 Nov 90 06:11:18 -0500
A Friendly Reminder to REGISTER and get a room; its getting late:
IEEE Conference on
Neural Information Processing Systems
-Natural and Synthetic-
Monday, November 26 - Thursday, November 29, 1990
Sheraton Denver Tech Center
Denver, Colorado
Mail Requests For Registration Material To:
Kathie Hibbard
NIPS*90 Local Committee
Engineering Center
University of Colorado
Campus Box 425
Boulder, CO 80309-0425
hibbard@boulder.colorado.edu
------------------------------
Subject: preprint - associative memory in oscillating cortex
From: Bill Baird <baird%icsia8.Berkeley.EDU@berkeley.edu>
Date: Mon, 12 Nov 90 11:48:39 -0800
Preprint announcement: available by ftp from neuroprose
Learning with Synaptic Nonlinearities in a
Coupled Oscillator Model of Olfactory Cortex
Bill Baird
Depts. Mathematics, and Molecular and Cell Biology,
U.C.Berkeley, Berkeley, Ca. 94720
Abstract
A simple network model of olfactory cortex, which assumes only
minimal coupling justified by known anatomy, can be analytically proven
to function as an associative memory for oscillatory patterns. The
network has explicit excitatory neurons with local inhibitory interneuron
feedback that forms a set of nonlinear oscillators coupled only by long
range excitatory connections. Using a local Hebb-like learning rule for
primary and higher order synapses at the ends of the long range
connections, the system can learn to store the kinds of oscillation
amplitude and phase patterns observed in olfactory and visual cortex.
Memory capacity is N/2 oscillatory attractors, N/4 chaotic attractors in
an N node network. The network can be truely self-organizing because a
synapse can modify itself according to it's own pre and postsynaptic
activity during stimulation by an input pattern to be learned. The
neurons of the neuron pools modeled here can be viewed as operating in
the linear region of the usual sigmoidal axonal nonlinearity, and
multiple memories are stored instead by the learned {\em synaptic}
nonlinearities.
Introduction
We report recent results of work which seeks to narrow the gap
that exists between physiologically detailed network models of real
vertebrate cortical memory systems and analytically understood artificial
neural networks for associative memory. The secondary olfactory sensory
cortex known as prepyriform cortex is thought to be one of the clearest
cases of a real biological network with associative memory function.
Patterns of 40 to 80 Hz oscillation have been observed in the
large scale activity (local field potentials) of olfactory cortex and
visual neocortex, and shown to predict the olfactory and visual pattern
recognition responses of a trained animal. Similar Observations of 40 Hz
oscillation in retina, auditory cortex, motor cortex and in the EMG have
been reported. It thus appears that cortical computation in general may
occur by dynamical interaction of resonant modes, as has been thought to
be the case in the olfactory system. Given the sensitivity of neurons to
the location and arrival times of dendritic input, the sucessive volleys
of pulses that are generated by the oscillation of a neural net may be
ideal for the formation and reliable longe range transmission of the
collective activity of one cortical area to another.
The oscillation can serve a macroscopic clocking function and
entrain or ``bind" the relevant microscopic activity of disparate
cortical regions into a well defined phase coherent collective state or
``gestalt". This can overide irrelevant microscopic activity and help
produce coordinated motor output. If this view is correct, then
oscillatory network modules form the actual cortical substrate of the
diverse sensory, motor, and cognitive operations now studied in static
networks. It must ultimately be shown how those functions can be
accomplished with oscillatory dynamics.
ftp proceedure:
unix> ftp cheops.cis.ohio-state.edu
Name: anonymous
Password: neuron
ftp> cd pub/neuroprose
ftp> binary
ftp> get baird.oscmem.ps.Z
ftp> quit
unix> uncompress baird.oscmem.ps.Z
unix> lpr -P(your postscript printer) baird.oscmem.ps
For background papers, send e-mail to baird@icsi.berkeley.edu,
giving paper or e-mail address for Tex or Postscript output.
------------------------------
Subject: Post-NIPS workshop on Associative Memory
From: Jay Buckingham <jt@cns.edinburgh.ac.uk>
Date: Mon, 12 Nov 90 19:54:23 +0000
There will be a 1-day post-NIPS workshop on Associative Memory this
year. This is a short summary of what we plan to do.
NEUROBIOLOGICALLY MOTIVATED ASSOCIATIVE MEMORY
The 1990 ASSOCIATIVE MEMORY workshop will focus on theoretical issues
relevant to the understanding of neurobiologically motivated associative
memory models. We are organizing it into a set of discussions on key
issues. In these discussions the workshop participants can explain how
they have addressed the issue at hand in their work and hopefully improve
their understanding via exchange with others. What we want are
nuts-and-bolts discussions among people who have wrestled with these
topics.
Some of the key issues are:
- Architectures for associative memories, with emphasis on partially
connected, biologically motivated models, including multi-stage or
modular architectures.
- Synaptic learning rules
- Performance measures and capacity
- Information theoretic issues such as information efficiency
- Thresholding techniques
- Biological relevance of these models
(Discussing, for example, how David Marr made functional statements about
the cerebellum, hippocampus and neocortex with associative memory
models.)
This list is not exhaustive and the items are interrelated so we expect
them to come up in various contexts. We will probably begin each
discussion by formulating a few questions that the participants can
address. So come with your questions about these topics and your ways of
thinking about them.
Jay Buckingham
Cognitive Neuroscience
University of Edinburgh
2 Buccleuch Place
Edinburgh EH8 9LW, SCOTLAND
Phone: (44 or 0) 31 667 1011 ext 6302
E-mail: jt%ed.cns@nsfnet-relay.ac.uk
------------------------------
Subject: Proceedings -- ConnectFest 1990
From: David Chalmers <dave@cogsci.indiana.edu>
Date: Tue, 13 Nov 90 02:28:44 -0500
A small connectionist meeting, "ConnectFest 1990", was recently held at
Indiana University. Here are selected highlights from the official
proceedings. The full proceedings (about 3-4 times the size of this) are
available on request from dave@cogsci.indiana.edu.
The Frank Rosenblatt Memorial Award for best contribution was taken by
Tim van Gelder, who has bravely allowed his name to be attached to his
contribution at great risk to his career.
For your very own ConnectFest T-shirt (only $8 each), send e-mail to
Doug Blank, blank@copper.ucs.indiana.edu.
Dave Chalmers (dave@cogsci.indiana.edu)
Center for Research on Concepts and Cognition
Indiana University.
==========================================================================
Proceedings of ConnectFest 1990 -- Indiana University, November 3-4, 1990.
Compiled and edited by Doug Blank and Dave Chalmers.
(c) 1990, ConnectFest Publications.
----------------------------------------------------
Said the Boltzmann machine to the whore,
When his temperature reached sixty-four,
"This is such a great feeling
I've slowed down my annealing.
That way there's more time to explore."
-- Mike Gasser
----------------------------------------------------
The folks from the journal Cognition
Weren't more than a bad apparition.
All's left of Fodor
Is just a bad odor,
And nobody's missin' Pylyshyn.
-- Tim van Gelder
----------------------------------------------------
>From Bloomington there was a network,
That tried and tried to write limericks.
They didn't really rhyme,
And were short one line.
(But weren't entirely self-referential.)
-- Doug Blank
----------------------------------------------------
A couple with bad information
Got into a tight situation.
They said, "We don't see ---
It just cannot be
We only had backpropagation!"
-- Mark Weaver
----------------------------------------------------
To these still stuck in the old school,
The rule is the ultimate tool.
But connectionists know
(The others are slow)
That the rule is a tool for a fool!
-- Tim van Gelder
----------------------------------------------------
These guys Fodor and Pylyshyn
Came to town with a mission.
But they got in a jam
When faced off with RAAM
Instead they should just have gone fishin'.
-- Dave Chalmers
----------------------------------------------------
Said a network that was rather horny
To another, "This might sound too corny,
Of all of my mates,
You've got the best weights,
Let's optimize them until morning."
-- Paul Munro
----------------------------------------------------
Once quoted a young naive RAAM
"In my 3-2-3 mind I can cram
All of human cognition
With minor omission"
But it's not that much better than SPAM.
-- Devin McAuley and Gary McGraw
----------------------------------------------------
In Bloomington, geniuses gather
Working a connectionist lather
Did they solve the world's woes,
Or compound them -- who knows?
It's not a decidable matter.
-- Dave Touretzky
----------------------------------------------------
"Backprop will usually converge,"
Rumelhart and Hinton observe.
"When some people say
`Brains don't work that way'
We just smile and flip them the bird!"
-- Pete Angeline and Viet-Anh Nguyen
----------------------------------------------------
There once was a recurrent node
Who felt he was ready to explode
Without stimulation
Only self-excitation
He swelled up and then shot his load.
-- Devin McAuley and Gary McGraw
----------------------------------------------------
Pollack has made this admission
Of his neural net's true composition:
"I recursively RAAM it
With symbols, Goddamnit!
So don't pay no mind to Pylyshyn."
-- Dave Touretzky
----------------------------------------------------
Minsky and Papert were cruel
Which gave the symbolists fuel.
Connectionists waited
Till they backpropagated
And now neural networks are cool.
-- Paul Munro
------------------------------
Subject: Job Openings at CMU
From: Alex.Waibel@SPEECH2.CS.CMU.EDU
Date: Tue, 13 Nov 90 02:30:50 -0500
The School of Computer Science at Carnegie Mellon University has several
immediate positions at the level of Research Associate and Research
Programmer. These positions are research positions without teaching
obligations aimed at the development and evaluation of neural network
based speech understanding systems. We are seeking strongly motivated,
outstanding individuals who have a demonstrated background and research
experience in at least one of the following areas and an interest and
commitment to work in the others:
o Speech Recognition and Understanding (Stochastic or Connectionist)
o Neural Network Modeling and Connectionist System Design
o System design and implementation, development of connectionist
systems on fast parallel hardware, including various
available supercomputers, signal processors and neural
network chips.
Applicants for the position of Research Associate should have completed
their PhD and have a demonstrated ability to do independent innovative
research in an area listed above. Applicants for the position of
Research Programmer should have a B.S. or M.S. degree and be interested
in performing independent research and in developing innovative system
solutions.
Carnegie Mellon University is one of the leading institutions in Computer
Science and offers an exciting and challenging environment for research
in the areas listed above. The connectionist group at CMU, is one of the
most active research teams in this area in the US, and provides a
stimulating environment for innovative research. It also maintains a
lively interaction with other departments, including Psychology, the
Center for Machine Translation, and Computational Linguistics.
Interested individuals should send a complete curriculum vitae listing three
references to:
Dr. Alex Waibel
School of Computer Science
Carnegie Mellon University
Pittburgh, PA 15213
Email: waibel@cs.cmu.edu
------------------------------
Subject: Posting: Conf. Announcement
From: WANG@nbivax.nbi.dk
Date: Wed, 07 Nov 90 10:53:00 +0100
------------------------------------------------------------
INTERNATIONAL CONFERENCE
ON
NOVEL METHODS IN OPTIMIZATION
February 7 - 8, 1991
arranged by
NORDITA
Nordic Institute of Theoretical Physics
Copenhagen
and
DIKU
Department of Computer Science
University of Copenhagen
supported by funding from Nordic Initiative for Neural
Computation (NINC)
------------------------------------------------------------
In recent years there has been an increasing
interest in using neural networks, simulated annealing, and
genetics as modelling frames of reference to construct novel
search heuristics for solving hard optimization problems.
Algorithms constructed in this way, together with tabu search,
constitute promising new approaches to optimization and are
the subjects of this conference.
The aim of the conference is to bring together researchers
in classical optimization and researchers working with the
novel methods, thus enabling a fruitful exchange of information
and results. An important part of the conference will be a
tutorial presentation of both classical and new methods to
establish a common base for discussion among the participants.
Tutorial session.
-----------------
The first day of the conference will be devoted to
introductory lectures given by invited speakers. The lectures
will be on:
* Classical Optimization.
a) Laurence Wolsey, Center for Operations Research and
Econometrics, Universite de Louvain, Belgium:
Introduction to Classical Optimization: P-problems and
their solution.
b) Susan Powell, London School of Economics:
Introduction to Classical Optimization: NP-problems and
their solution.
* Neural Networks.
Carsten Peterson, Lund University, Sweden: The use of neural
networks and optimization.
* Simulated Annealing.
(Speaker to be announced later)
* Genetic Algorithms.
(Speaker to be announced later)
* Tabu Search.
(Speaker to be announced later)
* Statistical Mechanics.
Marc Mezard, Ecole Normale Superieure, Paris:
"Formal statistical mechanical methods in
optimization problems."
About the speakers:
Laurence A. Wolsey is Professor of Applied Mathematics at CORE
and is one of the leading researchers in the field of computational
mathematical programming. He received the Beale-Orchard-Hays prize
for his work in 1988, and is one of the authors of the widely used
book "Integer and Combinatorial Optimization".
Susan Powell is Lecturer in Operations Research at London School
of Economics and is well known for her work on Fortran Codes for
linear and integer programs. She has a solid background in prac-
tical problem solving through her contacts with industry and
British OR companies.
Carsten Peterson is Lecturer in Theoretical Physics at Lund
University. He is co-inventor of the deterministic Boltzmann
learning algorithm for symmetric recurrent networks and a
leader in applications of neural networks to optimization
problems.
Marc Mezard is Lecturer in Physics at the Ecole Nomale Superieure,
Paris. Together with his colleagues there and their coworkers
at the University of Rome, he pioneered the application of
methods from the statistical mechanics of random systems to
optimizaation problems.
Contributed Papers.
-------------------
The second day of the conference will be devoted to
selected half-hour contributed presentations. An abstract
of each paper submitted for presentation should be mailed
or e-mailed to:
Prof. Jens Clausen
DIKU, Dept. of Computer Science
Universitetsparken 1,
DK-2100 Copenhagen OE
Denmark.
e-mail: clausen@diku.dk
before December 15, 1990. Authors of accepted papers will be
notified before January 15, 1991. (No proceedings will
be published).
Poster Sessions.
----------------
On both seminar days there will be poster sessions. An
abstract of the poster should be mailed or e-mailed to
Prof. Jens Clausen
DIKU, Dept. of Computer Science
Universitetsparken 1,
DK-2100 Copenhagen OE
Denmark.
e-mail: clausen@diku.dk
before December 15, 1990. Authors of accepted posters will be
notified before January 15, 1991.
Registration.
-------------
The registration fee is 500 DKK (or equivalent in other
covertible currency) and covers coffee/tea and lunch both days
as well as an informal conference dinner on the evening of
February 7.
To register please fill in the form below and mail it
together with the registrations fee to the address given on
the form.
No credit cards accepted. Cheques or Eurocheques should
be payable to OPTIMIZATION CONFERENCE.
The organizing commitee must receive your registration
form January 15, 1991 the latest, and the final program will
be mailed by January 22, 1991.
Travel support for Nordic participants.
---------------------------------------
A limited amount of money from NINC is reserved for paying the
travel costs of participants from the Nordic countries,
especially younger researchers. If you would like to apply
for this support, please indicate on the registration form.
Accommodation.
--------------
The organizing commitee has reserved a certain number of
hotel rooms. Please indicate on the registration form if you
would like the conference to book one for you.
-------------------------------------------------------------
INTERNATIONAL CONFERENCE
ON
NOVEL METHODS IN OPTIMIZATION
February 7 - 8, 1991
-------------------------------------------------------------
REGISTRATION FORM
-------------------------------------------------------------
Name:_______________________________________________
Affiliation:_______________________________________________
Address:_______________________________________________
_______________________________________________
_______________________________________________
Telephone no.:_______________________________________________
e-mail:_______________________________________________
If you want the conference to reserve you a hotel room, please
indicate here for which nights:
______________________________________________________________
Nordic participants: If you want to be considered for travel
support, please indicate your needs here:
______________________________________________________________
Mail this registration form to:
John Hertz
NORDITA
Blegdamsvej 17
DK-2100 Copenhagen OE,
Denmark
For further information:
e-mail: hertz@nordita.dk
FAX: [+45] 31 38 91 57
------------------------------
End of Neuron Digest [Volume 6 Issue 66]
****************************************landman@hanami.Eng.Sun.COM (Howard A. Landman) (11/20/90)
>From: UAP001%DDOHRZ11.BITNET@CUNYVM.CUNY.EDU, >Date: Tue, 13 Nov 90 16:12:31 +0700 >Think of what (insert name of your favorite intellectual >hero here) did with the available circuits. >It could of course be argued that such exceptions had exceptional >hardware (although there's never been convincing anatomic or histologic >evidence for this). Anyone who's seen (or even read descriptions of) the brain of C.F.Gauss would have strong reason to disagree with this statement. Its cortex had about twice as many folds as that of a "normal" human brain. -- Howard A. Landman landman@eng.sun.com -or- sun!landman