neuron-request@HPLABS.HP.COM ("Neuron-Digest Moderator Peter Marvit") (05/02/89)
Neuron Digest Thursday, 27 Apr 1989
Volume 5 : Issue 20
Today's Topics:
neur-net for 3D pict.
IJCNN vs. NIPS 89
Postdoctoral positions available
NN vs biological systems
Cybernetics Discussion List
roommate for IJCNN
music and connectionism revisited
Neural "redundancy" and self-repair
Vivarium
Reinforcement technique developed
More on BP Minima
Problems of local minima in the Hopfield network.
Re: Neuron Digest V5 #16
Technical report available
network meeting announcement
Send submissions, questions, address maintenance and requests for old issues to
"neuron-request@hplabs.hp.com" or "{any backbone,uunet}!hplabs!neuron-request"
ARPANET users can get old issues via ftp from hplpm.hpl.hp.com (15.255.16.205).
------------------------------------------------------------
Subject: neur-net for 3D pict.
From: SORBELLO%IPACRES.BITNET@CUNYVM.CUNY.EDU
Date: Thu, 06 Apr 89 17:00:00 -0500
I am looking for references on neural networks for 3D image recognition.
Does anybody know somrthing about this ?
Many thanks
Filippo Sorbello
<SORBELLO@IPACRES.BITNET>
------------------------------
Subject: IJCNN vs. NIPS 89
From: rockwell.henr@Xerox.COM
Date: 11 Apr 89 15:50:00 -0500
Reply To: rockwell.henr801c@xerox.com
I can only attend one of these two conferences. I'm wondering if anyone can
give me a short description of the pluses and minuses {sp? on both} of each.
I attended last years INNS conference if you want to use that as a
reference. Thanks
Ron Rockwell
[[ Editor's Note: IJCNN appears to be the most commercially (aka
applications) oriented of the lot. The first year, the quality of the
papers was uneven. Last year, in my opinion, it was quite good, though very
large (c. 1500 people attending!). The focus tended to be computer
science-y.
NIPS tends to be academically oriented with a wider variety of attendees,
though smaller total crowd. Friends who have gone are *very* glad they did.
Technical quality was reportedly very high.
I have only been to IJCNN, as I will this year. However, I would prefer
NIPS, if it wasn't at an inconvenient time. Well, there's one man's
opinion. Other reactions? -PM ]]
------------------------------
Subject: Postdoctoral positions available
From: granger@ICS.UCI.EDU
Date: Wed, 12 Apr 89 00:38:26 -0700
COMPUTATIONAL NEUROSCIENCE PROGRAM
University of California, Irvine
The Computational Neuroscience Program at the University of California,
Irvine, has positions available for two POSTDOCTORAL RESEARCH ASSOCIATES
with strong backgrounds in the neurobiology of learning and memory,
mathematics, and computer science. We are constructing mathematical models
and computer simulations to study the computational properties of
neurobiological circuits, in a collaborative effort involving active faculty
participants from the departments of Psychobiology, Computer Science,
Anatomy, Mathematics, and Engineering. The Program has its own extensive
facilities for physiology, anatomy, biochemistry and a computer laboratory,
and long-term funding from the Office of Naval Research and the National
Science Foundation. Salaries will be commensurate with qualifications and
experience; applicants should submit their curriculum vitae, description of
research interests, and names and addresses of three references to:
Dr. Richard Granger
Computational Neuroscience Program
Bonney Center
University of California
Irvine, California 92717
------------------------------
Subject: NN vs biological systems
From: "Bruce E. Nevin" <bnevin@cch.bbn.com>
Date: Wed, 12 Apr 89 07:44:47 -0400
Of those who think neural nets are models of brain function, has anyone
looked into the role of neuropeptides in cognitive processes? (Work of Pert
at NIMH.)
And on the critical side, there is the commentary by Francis Crick in Nature
337:129-132 (12 January 1989), `The recent excitement about neural
networks'.
------------------------------
Subject: Cybernetics Discussion List
From: Eric Harnden <EHARNDEN%AUVM.BITNET@CUNYVM.CUNY.EDU>
Date: Thu, 13 Apr 89 08:58:14 -0400
Finally, got the address from a recent mailing... The subscribers to the
Neuren Digest will probably want to know about the creation of a new list,
CYBSYS-L@BINGVMB, dedicated to the discussion of Cybernetics and Systems.
It's in the startup stage now, and people are logging on and introducing
themselves. Looks good. See you there.
Eric Harnden (Ronin)
<EHARNDEN@AUVM>
The American University Physics Dept.
(202) 885-2758
------------------------------
Subject: roommate for IJCNN
From: Michael Gasser <gasser@iuvax.cs.indiana.edu>
Date: Sun, 16 Apr 89 20:18:23 -0500
Woman seeks hotel roommate for IJCNN (6/18-21). Respond to
gasser@cs.indiana.edu
or Mayumi Koide at (812) 335-6828
------------------------------
Subject: music and connectionism revisited
From: Stephen Smoliar <smoliar@vaxa.isi.edu>
Date: Wed, 19 Apr 89 14:50:35 -0800
Some time ago there was a relatively heated exchange concerned with the
application of connectionism to music research. Much of the argument
involved Jamshed Bharucha defending his work against criticism from Eliot
Handelman. At the time, my involvement was one of criticizing Bharucha for
not offering those of us "on the outside" a concise summary of just what it
was he was trying to do with connectionism.
I have now read "Music Cognition and Perceptual Facilitation: A
Connectionist Framework," a paper which Bharucha published in the Fall 1987
issue of MUSIC PERCEPTION (Volume 5, Number 1, pages 1-30). Now that I
think I have some idea of what is going on in this paper, I feel that a
summary might be valuable to the community of NEURON DIGEST readers. I am
assuming that if I have, in any way, misunderstood the nature of this work,
I shall be called to task in short order.
Bharucha appears to have a sound background as a psychologist.
Unfortunately, my own work in artificial intelligence has left me with a
(possibly distorted) view of psychology having a reputation for missing the
mark in such areas as natural language and memory due to such shortcomings
as an obsession with nonsense syllables in experimental design. Having
participated in a music seminar organized by a Psychology Department, I tend
to be suspicious of the contributions which psychologists wish to offer.
In Bharucha's case, he raised my suspicions with the very first sentence of
his abstract: "The mind internalizes persistent structural regularities in
music and recruits thoses internalized representations to facilitate
subsequent perception." As one who spends much of my non-research time
engaged in the practice of music, I am not ashamed to question this premise
x. . . particularly the bit about internalizing "persistent structural
regularities." Furthermore, I do not feel alone in my skepticism. In fact,
my feelings are reinforced by ANOTHER article which appeared in the Summer
1986 issue of MUSIC PERCEPTION (Volume 3, Number 4, pages 327-392), "Music
Theory, Phenomenology, and Modes of Perception," by David Lewin of the
Department of Music at Harvard. I fear that Bharucha's work may embody an
attempt to shoehorn the psychology of music experience into many of the
trivial fallacies which are still put forth by no end of stupid attempts to
teach "music appreciation." Lewin has taken on such attempts with great
elan, so I think there is no longer an excuse for such naivete. I have no
doubt that Bharucha has given us a well-written account of his experiments,
but I think that one can seriously question how relevant it all is.
Ultimately, this question of relevance rests on the level of abstraction
which Bharucha engages in his connectionist models. Ultimately, everything
is based on a primitive representation of a "note" as a coupling of a pitch
with a duration. I view this as analogous to trying to study language and
memory with nonsense syllables. I have a very strong intuition against this
approach to the effect that we neither hear nor remember individual notes
when we listen to music. I fear that, unless connectionism can come up with
a better approach to representation which gets away from abstractions which
have so little to do with what the mind actually hears, experimental results
will never rise above the trivial or the silly.
Bharucha has built a connectionist model in which, for example, "hearing" a
dominant chord "facilitates" hearing the next chord as a tonic. It should
be no surprise that one can build such a model. One could probably have
built it just as easily using, for example, a blackboard technology such as
has been engaged in speech recognition. Does that tell us anything about
music? In REAL music, as has been demonstrated by Heinrich Schenker and
others, a dominant-tonic relationship may be elaborated with intervening
material, just as a parenthetic remark may elaborate the flow of a sentence.
I think it is fair to ask whether or not connectionist models can take on
data more realistic than the notes of simple harmony exercises.
------------------------------
Subject: Neural "redundancy" and self-repair
From: "Dick Cavonius" <UAP001%DDOHRZ11.BITNET@CUNYVM.CUNY.EDU>
Date: Thu, 20 Apr 89 12:55:43 +0700
I've a minor comment on the recent discussion about uninjured parts of the
brain taking over the functions of damaged parts: in general, this doesn't
happen - think of any serious stroke victim; or read Luria's 'The Man with a
Shattered Mind' for a striking description. What may happen is that the
individual - human or other organism - learns behavior patterns that are
compensatory, but not identical to the lost ones.
I believe that there was also some misunderstanding about the Sur, Garraghty
& Roe article (describing a study in which visual afferents were redirected
to auditory centers). My recollection was that this described anatomical and
physiological findings, and had nothing to say about the function of the
rerouted fibers. Remarkable tho this study may be, it certainly didn't
report that the result was 'normal vision'. My guess would be that even if
neurons that normally would be part of the auditory system 'learn' to
respond in a manner that we usually associate with visual centers, their
output would continue to go to areas that couldn't process the right signals
to direct visually-guided behavior.
Dick Cavonius (uap001 at ddohrz11.bitnet)
------------------------------
Subject: Vivarium
From: dmpierce@cs.utexas.edu (Dave Pierce)
Date: Thu, 20 Apr 89 13:41:21 -0500
In Neuron Digest V5 #18, John Nagle mentioned an aquarium program by Ann
Marion and an MS thesis by Mike Travers at MIT. Does anyone have
information on how I could get my hands on these? I am doing what may be
similar work. My focus is on development rather than evolution.
More specifically, I am interested in models of sensorimotor development and
preverbal concept acquisition. I would be interested in hearing from others
who are interested in this type of learning.
- -Dave Pierce
dmpierce@cs.utexas.edu
------------------------------
Subject: Reinforcement technique developed
From: rba@flash.bellcore.com (Robert B Allen)
Date: Thu, 20 Apr 89 15:50:07 -0400
I have developed a simple but effective quasi-reinforcement technique in
which there are limited response alternatives and in which the success of
the response determines error correction. For positive results the selected
response is trained with a 1 and for negative results a 0. No other
responses are error corrected.
Consider, for instance, a DAI scenario in which one agent is watching
another agent move in a complex pattern around the corners of a square. The
task of observer is to move to where the other agent will be at the next
time interval. This is readily learned when the reinforcement technique is
applied to a temporal (Jordan) network.
This work is described in a short paper: Allen. R.B., "Developing Agent
Models with a Neural Reinforcement Technique" 1989 IEEE Systems Man and
Cybernetics Conference, and in a forthcoming technical report "Interacting
and Communicating Connectionist Agents"
------------------------------
Subject: More on BP Minima
From: john%mpl@ucsd.edu (John McInerney)
Date: Thu, 20 Apr 89 22:42:28 -0700
I will have a poster at IJCNN regarding one aspect of the the local minima
question. The paper is "Back Propagation Error Surfaces Can Have Local
Minima." We used a mixture of a "smart" brute force search and Newton's
method to search a fairly small region of the weight space for two networks,
one computing a continuous function (sine) and the other computing a Boolean
function (XOR). I think the result is somewhat interesting in that we used
a closed form for the error surface whereby we could make definitive
statements about the surface.
The results of our work show that the error surface defined by the given
nets (both had a hidden layer) and the training instances show a family of
local minima for the weight space explored by the net computing the sine
function and no minima at all for the net computing the XOR function. This
may sound a bit funny in the XOR case given most researchers experiences.
We found the 2-2-1 XOR net described in PDP-I as having a local minima had
no local minima. What happens there is that the error surface approaches an
asymptote in at least two different levels in the weight space. One
asymptote is at a large MSE while the other asymptote approaches zero.
Hence, then net can certainly behave as if it is in a local minima.
John McInerney
john%cs@ucsd.edu
------------------------------
Subject: Problems of local minima in the Hopfield network.
From: Simon Greenman <simongr%COGS.SUSSEX.AC.UK@CUNYVM.CUNY.EDU>
Date: Fri, 21 Apr 89 15:33:45 -0000
I too have a few questions about the Hopfield network, which are causing me
several headaches. Recently I have set up the Hopfield net inorder to
examine how effective it is at solving general constraint satisfaction
problems, such as the eight queens problems(place eight queens on a chess
board so that no queen can take any other queen). Great, playing around with
the network parameters(ie. gain, inhibitory value, external energy, etc.)
it is able to produce a valid solution on nearly every run. However, try it
on a sixteen queens problems and chances are that it does not produce a
valid solution.
The way I intuitively explain this result is in terms of the number of local
minima in the energy landscape. It appears that in the smaller problem there
are few local minima, and thus the system has an high a priori chance of
falling into a local energy minima representing a valid solution.
Similarly, it appears that the larger the network the more the local minima
there are and thus there is a lower probability of the network falling into
a local minima reflecting a valid solution. Similarly, this
conceptualisation would explain why in SMALL travelling salesman problems
the network often finds the best route, but in LARGER problems the resulting
route is often poor. That is, in the small networks there are a few local
minima and thus the system has a relatively high probability of falling into
the global minima, and so on..
The general question is thus: is this a good way of conceptualising the
behaviour of the network? Next question, Hopfield(1984) talks in
mathematical terms about the local minima in similar problems. He suggests
that by changing the gain scaling parameter on the I/O relations for a unit,
then this can change the number of local minima. However, I am yet to
understand why varying the number of local minima in an energy landscape by
varying the gain scaling parameter, apparently, does not allow me to get
valid solutions for the sixteen queens problems. Please, help!
Simon Greenman
School of Cognitive and Computing Sciences
Sussex University
Brighton
England.
------------------------------
Subject: Re: Neuron Digest V5 #16
From: Mike Oaksford <mike%EPISTEMI.ED.AC.UK@CUNYVM.CUNY.EDU>
Date: Tue, 25 Apr 89 15:13:54 +0100
After reading the recent debates over Fodor & Pylyshyn's (& Pinker &
Prince's critiques of Connectionism, and the call by many subscribers for
replies, we include below the abstracts from two replies we have written.
One is shortly to appear in "Cognition". We also highly reccommend Andy
Clark's (Sussex) forthcoming book which addresses many of these issues from
a more philosophical standpoint.
Autonomy, Implementation and Cognitive Architecture:
A Reply to Fodor and Pylyshyn
Nick Chater and Mike Oaksford
Centre for Cognitive Science,
University of Edinburgh
Tech Report: EUCCS/RP-27
(A shorter version of this Tech Report is to appear in "Cognition")
Abstract
In this reply we argue that the substantive dispute between Connectionist
and Classicist turns on the issue of the computational and explanatory
autonomy of cognitive and implementational levels. The Classicist believes
in autonomy, the Connectionist does not. This diagnosis is borne out in
Fodor & Pylyshyn's discussion of the `lures' of Connectionism. Although the
lures are typically taken to challenge the Classicist position Fodor &
Pylyshyn believe that they are, *in principle*, compatible with the
Classicist autonomy assumption. However, we argue that the Classicist has
yet to adequately meet this challenge, *in practice*. Moreover, we adduce
arguments concerning the nature of non-demonstrative inference, especially
default reasoning, which strongly suggest that the Classicist's task is
impossible, even in principle.
Connectionism, Classical Cognitive Science
and Experimental Psychology
Mike Oaksford, Nick Chater and Keith Stenning
Centre for Cognitive Science,
University of Edinburgh
Tech Report: EUCCS/RP-29
(A version of this Tech Report is under review by "AI & Society")
Abstract
Classical symbolic computational models of cognition are at variance with
the empirical findings in the cognitive psychology of memory and inference.
Standard symbolic computers are well suited to remembering arbitrary lists
of symbols and performing logical inferences. In contrast, human
performance on such tasks is extremely limited. Standard models do *not*
easily capture content addressable memory or context sensitive defeasible
inference, which are natural and effortless for people. We argue that
Connectionism provides a more natural framework in which to model human
cognition. In addition to capturing the gross human performance profile,
Connectionist systems seem well suited to accounting for the systematic
patterns of errors observed in the human data. We take these arguments to
counter Fodor & Pylyshyn's (1988) recent claim that Connectionism is in
principle irrelevant to psychology.
Mail requests for these TRs to:
Physical mail: Betty Hughes,
Tech Report Librarian
Centre for Cognitive Science
University of Edinburgh
2, Buccleuch Place
Edinburgh EH8 9LW
Scotland, U.K.
e-mail: betty@uk.ac.ed.epistemi
[[ Editor's note: The U.K. email address for us Yanks will be
betty@epistemi.ed.ac.uk; wouldn't you know it for folks who drive on the
"wrong" side of the street. -PM ]]
------------------------------
Subject: Technical report available
From: Melanie Mitchell <mm@cogsci.indiana.edu>
Date: Mon, 10 Apr 89 13:45:18 -0600
The following report is available from the Center for
Research on Concepts and Cognition at Indiana University:
The Role of Computational Temperature in a Computer Model
of Concepts and Analogy-Making
Melanie Mitchell and Douglas R. Hofstadter
Center For Research on Concepts and Cognition
Indiana University
Abstract
In this paper we discuss the role of computational temperature in Copycat, a
computer model of the mental mechanisms underlying human concepts and
analogy-making. Central features of Copycat's architecture are a high
degree of parallelism, fine-grained distributed processing, competition,
randomness, and an interaction of bottom-up perceptual pressures with an
associative, overlapping, and context-sensitive conceptual network. In
Copycat, computational temperature is used both to measure the amount and
quality of perceptual organization created by the program as processing
proceeds, and, reciprocally, to continuously control the degree of
randomness in the system. In this paper we will discuss the role of
temperature in two aspects of perception central to Copycat's behavior: (1)
the emergence of a "parallel terraced scan", in which many possible courses
of action are explored simultaneously, each at a speed and to a depth
proportional to moment-to-moment estimates of its promise, and (2) the
ability to restructure initial perceptions -- sometimes radically -- in
order to arrive at a deeper, more essential understanding of a situation.
We will also compare our notion of temperature to similar notions in other
computational frameworks. Finally, an example will be given of how
temperature is used in Copycat's creation of a subtle and insightful
analogy.
For copies of this report, send a request for CRCC-89-1 to
helga@cogsci.indiana.edu
or to
Helga Keller
Center for Research on Concepts and Cognition
Indiana University
Bloomington, Indiana, 47408
------------------------------
Subject: network meeting announcement
From: mike@bucasb.BU.EDU (Michael Cohen)
Date: Thu, 13 Apr 89 16:26:51 -0400
NEURAL NETWORK MODELS OF CONDITIONING AND ACTION
12th Symposium on Models of Behavior
Friday and Saturday, June 2 and 3, 1989
105 William James Hall, Harvard University
33 Kirkland Street, Cambridge, Massachusetts
PROGRAM COMMITTEE:
Michael Commons, Harvard Medical School
Stephen Grossberg, Boston University
John E.R. Staddon, Duke University
JUNE 2, 8:30AM--11:45AM
- -----------------------
Daniel L. Alkon, ``Pattern Recognition and Storage by an Artificial
Network Derived from Biological Systems''
John H. Byrne, ``Analysis and Simulation of Cellular and Network Properties
Contributing to Learning and Memory in Aplysia''
William B. Levy, ``Synaptic Modification Rules in Hippocampal Learning''
JUNE 2, 1:00PM--5:15PM
- ----------------------
Gail A. Carpenter, ``Recognition Learning by a Hierarchical ART Network
Modulated by Reinforcement Feedback''
Stephen Grossberg, ``Neural Dynamics of Reinforcement Learning, Selective
Attention, and Adaptive Timing''
Daniel S. Levine, ``Simulations of Conditioned Perseveration and Novelty
Preference from Frontal Lobe Damage''
Nestor A. Schmajuk, ``Neural Dynamics of Hippocampal Modulation of Classical
Conditioning''
JUNE 3, 8:30AM--11:45AM
- -----------------------
John W. Moore, ``Implementing Connectionist Algorithms for Classical
Conditioning in the Brain''
Russell M. Church, ``A Connectionist Model of Scalar Timing Theory''
William S. Maki, ``Connectionist Approach to Conditional Discrimination:
Learning, Short-Term Memory, and Attention''
JUNE 3, 1:00PM--5:15PM
- ----------------------
Michael L. Commons, ``Models of Acquisition and Preference''
John E.R. Staddon, ``Simple Parallel Model for Operant Learning with
Application to a Class of Inference Problems''
Alliston K. Reid, ``Computational Models of Instrumental and Scheduled
Performance''
Stephen Jose Hanson, ``Behavioral Diversity, Hypothesis Testing, and
the Stochastic Delta Rule''
Richard S. Sutton, ``Time Derivative Models of Pavlovian Reinforcement''
FOR REGISTRATION INFORMATION SEE ATTACHED OR WRITE:
Dr. Michael L. Commons
Society for Quantitative Analysis of Behavior
234 Huron Avenue
Cambridge, MA 02138
- ----------------------------------------------------------------------
- ----------------------------------------------------------------------
REGISTRATION FEE BY MAIL
(Paid by check to Society for Quantitative Analysis of Behavior)
(Postmarked by April 30, 1989)
Name: ______________________________________________
Title: _____________________________________________
Affiliation: _______________________________________
Address: ___________________________________________
Telephone(s): ______________________________________
E-mail address: ____________________________________
( ) Regular $35
( ) Full-time student $25
School ____________________________________________
Graduate Date _____________________________________
Print Faculty Name ________________________________
Faculty Signature _________________________________
PREPAID 10-COURSE CHINESE BANQUET ON JUNE 2
( ) $20 (add to pre-registration fee check)
- -----------------------------------------------------------------------------
(cut here and mail with your check to)
Dr. Michael L. Commons
Society for Quantitative Analysis of Behavior
234 Huron Avenue
Cambridge, MA 02138
REGISTRATION FEE AT THE MEETING
( ) Regular $45
( ) Full-time Student $30
(Students must show active student I.D. to receive this rate)
ON SITE REGISTRATION
5:00--8:00PM, June 1, at the RECEPTION in Room 1550, William James Hall,
33 Kirkland Street, and 7:30--8:30AM, June 2, in the LOBBY of William
James Hall.
Registration by mail before April 30, 1989 is recommended
as seating is limited
HOUSING INFORMATION
Rooms have been reserved in the name of the symposium ("Models of Behavior")
for the Friday and Saturday nights at:
Best Western Homestead Inn
220 Alewife Brook Parkway
Cambridge, MA 02138
Single: $71
Double: $80
Call (617) 491-1890 or (800) 528-1234 and ask for the Group Sales desk.
Reserve your room as soon as possible. The hotel will not hold them past
May 1. Because of Harvard and MIT graduation ceremonies, space will
fill up rapidly. Other nearby hotels:
Howard Johnson's Motor Lodge
777 Memorial Drive
Cambridge, MA 02139
(617) 492-7777
(800) 654-2000
Single: $115--$135
Double: $115--$135
Suisse Chalet
211 Concord Turnpike Parkway
Cambridge, MA 02140
(617) 661-7800
(800) 258-1980
Single: $48.70
Double: $52.70
------------------------------
End of Neurons Digest
*********************