dgross@blackbird.CalPoly.EDU (Dave Gross) (04/27/91)
I'm looking into how chaos theory might help out in the development of neural nets. If you have any information about this, or can point me to some information, please do. In connection to this, I'm looking for an E-mail address for a UC Berkeley professor of neurobiology named Walter J. Freeman. Freeman wrote an article called "The Physiology of Perception" in Scientific American a couple of months back. If you know how I might get ahold of Dr. Freeman via e-mail, please let me know.
sfp@mars.ornl.gov (Phil Spelt) (04/29/91)
In article <2818d823.3155@petunia.CalPoly.EDU> dgross@blackbird.CalPoly.EDU (Dave Gross) writes: > > > I'm looking into how chaos theory might help out in the > development of neural nets. If you have any information > about this, or can point me to some information, please > do. > You fail to specify what you mean by "help out", but if I interpret this correctly, you should look up an article by M. E. Manausa & R. C. Lacher, from FL State Univ. The article is entitled: "Chaos and the step-size dilemma in the back-prop learning algorithm". It deals with the fractal nature of convergence as a function of step size and gain in the learning algorithm of pack-prop nets on two different problems. The article appears in the proceedings of the 2nd Workshop on Neural Networks (WNN-AIND 91), held at Auburn University 11-13 February 1991. Hope this helps. ============================================================================= MIND. A mysterious form of matter secreted by the brain. Its chief activity consists in the endeavor to asscertain its own nature, the futility of the attempt being due to the fact that it has nothing but itself to know itself with. -- Ambrose Bierce ============================================================================= Phil Spelt, Cognitive Systems & Human Factors Group sfp@epm.ornl.gov ============================================================================ Any opinions expressed or implied are my own, IF I choose to own up to them. ============================================================================
ssingh@watserv1.waterloo.edu ( Ice ) (05/02/91)
[Here is a file I had laying around on my account. It is quite a mess
since it is a collection of all kinds of articles from various
sources, but the signal to noise ratio is _very_ high, I daresay.]
Hope this helps. _I_ need help with finding basins of attraction
in neural net models... <sigh>.
PS: I'd like to hear from anyone else dabbling in this area. I know
of a couple of people but the more the merrier.
Later, neuromaniacs. ;-)
Ice.
---------------snippity snip---------------------------
Here are the some references to chaos-theoretic descriptions of
the brain. They are from Neural and Brain Modelling. I also have
the Fortran files for the simulation programs contained therein.
Chay, T.R. Abnormal discharges and chaos in a neuronal model
system.
_Biological Cybernetics_. 50, 301-311
Abstract: Using the mathematical model of the pacemaker neuron
formulated by Chay, we have investigated the conditions in which
a neuron can generate chaotic signals in response to variation in
temperature, ionic compositions, chemicals, and the strength of
the depolarizing current.
Choi, M.Y., and Huberman, B.A. Dynamic Behaviour of nonlinear
networks. _Phys. Rev. A_. 28, 1204-1206.
Abstract: We study the global dynamics of nonlinear networks made
up of synchronous threshold elements. By writing a master
equation for the system, we obtain an expression for the time
dependence of its activity as a function of parameter values. We
show that with both excitatory and inhibatory couplings, a
network can display collective behaviour which can be either
multiple periodic or deterministic chaotic, a result that appears
to be quite general.
Grondin, R.O., et. al. Synchronous and Asynchronous Systems of
Threshold Elements. _Biological Cybernetics_. 49, 1-7.
Abstract: The role of synchronism in systems of threshold
elements (such as neural networks) is examined. Some important
differences between synchronous and asynchronous systems are
outlined. In particular, important restrictions on limit cycles
are found in asynchronous systems along with multi-frequency
oscillations which do not appear in synchronous systems. The
possible role of deterministic chaos in these systems is
discussed.
Guevara, M.R., Glass, L., Mackey, M.C., Shrier, A. Chaos in
Neurobiology. _IEEE Transactions on Systems, Man, and
Cybernetics_. 13, 790-798.
Abstract: Deterministic mathematical models can give rise to
complex
aperiodic ("chaotic") dynamics in the abscence of stochastic
fluctuations ("noise") in the variables or parameters of the
model or in the inputs to the system. We show that chaotic
dynamics are expected in nonlinear feedback systems possessing
time delays such as are found in recurrent inhibition and from
the periodic forcing of neural oscillators. The implications of
the possible occurrence of chaotic dynamics for experimental work
and mathematical modelling of normal and abnormal function in
neurophysiology are mentioned.
Holden, A.V., Winlow, W., and Hayden, P.G. The Induction of
Periodic
and Chaotic Activity in a Molluscan Neurone. _Biological
Cybernetics_. 43, 169-173.
Abstract: During prolonged exposure to extracellular
4-aminopyridine (4AP) the periodic activity of the somatic
membrane of an identified molluscan neurone passes from a
repetitive regular discharge of >90 mV amplitude action
potentials, through double discharges to <50 mV amplitude
oscillations. Return to standard saline causes the growth of
parabolic amplitude-modulated oscillations that develop, through
chaotic amplitude- modulated oscillations, into regular
oscillations. These effects are interpreted in terms of the
actions of 4AP on the dynamics of the membrane excitation
equations.
From: CYBSYS_L Moderator
>I don't think
>anyone is much surprised by the fact we find such phenomena in
neurobiology.
We should not blithely assume that natural systems are
describably in
terms of dynamical theory, but I know what you mean.
>The interesting question (to me) seems to be: Is chaos useful in
any sense to biological systems? Or is it that the biological
systems are such that chaos "doesn't bother" them? I mean, how
does chaos participate (if it at all) in the biological
information processing of the brain?
The evidence is very strong that chaos is both necessary and
useful for normal nervous activity. I'm mostly drawing from
Freeman here, and that is still my primary reference. His claim
is that chaos is the natural "background activity" level of
neural systems. This is evidenced by all EEG readings and his
own research specifically in the olfactory cortices or rabbits.
You can calculate the fractal dimension between 4 and 6. This is
useful to prevent "entrainment" of neural response, typified by
seizures (too regular behavior). Against this background
activity (rest state), perceptions stand out as low-dimensional
cyclic attractors.
This is complex stuff, and I'd strongly advise reading the
original by
Skarda and Freeman and the excellent series of review articles
which
followed by neurolgoists, cognitive scientists, etc. (Rene Thom
commented). I wrote a review paper on this which I could post,
but the originals are better. Bibliography follows.
Babloyantz, A, and Salazar, JM: (1985)
"Evid.ofChaoticDyn.ofBrn.Act.DuringSleep Cycle", /Physics
Letters/, v.
111A
Freeman, William J: (1972) "Waves, Pulses and the Theory of
Neural
Masses", /Progress in THeoretical Biology/, v. 2
(1975) "Mass Action in the Nervous System", Academic Press
(1987) "Simulation of Chaotic EEG Patterns", /Biological
Cybernetics/, v. 56
Freeman, William J, and Viana Di Prisco, G: (1986)
"EEG Spatial Pattern Differentiation w/Discrete Odors
Manifests
Chaos + Limit Cycle Attractors", /Brain Theory/, ed. G.
Palm,
Springer-Verlag, Berlin
Froehling, H., and et. al., : (1981) "On Determining the
Dimension of
Chaotic Flows", /Physica/, v. 3D
Grassberger, Peter, and Procaccia, I: (1983) "Measuring
Strangeness of Strange Attractors", /Physica/, v. 9D
Hebb, DO: (1949) "Organization of Behavior", Wiley
Skarda, CA, and Freeman, WJ: (1987) "How Brains Make Chaos Into
Order", /Behavioral and Brain Sciences/, v. 10
There was some discussion here earlier about references to work
concerning the "use" of chaos in neural systems. Here are some
references that were not mentioned but that might be of interest.
1) Chaos in Brain Function and the Problem of Nonstationarity: A
commentary
George J. Mpitsos, 1989, in "Brain Dynamics 2", ed: E. Basar,
T.H.
Bullock, Springer-Verlag, Berlin Heidelberg
2) Evidence for Chaos in Spike Trains of Neurons that Generate
Rhytmic
Motor Patterns
G.J. Mpitsos, R.M. Burton, Jr., H.C. Creech and S.O. Soinila,
Brain
Res. Bull., 21, 529-538, 1988
3) Connectionist Networks Learn to Transmit Chaos
G.J. Mpitsos, R.M. Burton, Jr. and H.C. Creech, Brain Res. Bull,
21
539-546, 1988
4) Variability and Chaos: Neurointegrative Principles in
Self-organization of Motor Patterns
G.J. Mpitsos, H.C. Creech, C.S. Cohan and M. Mendelson, In:
"Dynamic
Patterns in Complex Systems", ed: J.A.S. Kelso, A.J. Mandell and
M.F.
Shlesinger, WOrld Scientific Pub., 1988, 162-190
If you want reprints of these papers or pre-prints of newer
manuscripts George can be contacted at the following mail drops:
George's net address is: Mpitsos@orstate.bitnet
The Creecher's is : creechc@orstvm.bitnet
John
+-- In the Real World ----------+--- Elsewhere ---------+
|Dr. John P. Edstrom |EDSTROM@UNCAEDU Bitnet |
|Div. Neuroscience | |
|3330 Hospital Drive NW | |
|Calgary, ALberta T2N 4N1 | |
|CANADA (403) 220 4493 | |
+-------------------------------+-----------------------+
Date: Thu, 15 Mar 90 15:43:20 EST
From: aboulang@BBN.COM
Sender: aboulang@BBN.COM
Source-Info: From (or Sender) name not authenticated.
Status: R
BTW, usually wherever there is chaos, fractals are lurking
nearby. In the excitement about chaos, fractals seem to have
faded into the woodwork. Has anyone seen or done work which tries
to tie fractals, chaos, and NNs together into a biologically
plausible model? There was talk last year on the cybernetics
mailing list about calculating fractal dimensions of Purkinje
cells, but I haven't been able to send mail to those persons. Any
pointers would be appreciated.
Some people have explored the fractal basins of attraction in the
Hopfield model:
"Basins of Attraction of Neural Network Models", James Keeler,
1986
Snowbird Conference Meeting, AIP, 1986, 259-264.
--
(1ST HYPERMEDIA .SIG) ; #include <black_rain.h> ; #include <robotron.h>
"Ice" is a UW AI living at: ssingh@watserv1.[u]waterloo.{edu|cdn}/[ca]
"The human race is inefficient and therefore must be destroyed"-Eugene Jarvis
Visual component of .sig: Saito in the cafe doing some slicing in _Black_Rain_