beer@cthulhu.ces.cwru.edu (Randall D. Beer) (07/16/90)
Suppose we are given a system of N differential equations of the form
N
Ci * dyi/dt = -yi + F(sum{Wji * yi}) where F(x) = 1/(1 + e^(-x))
j=1
My question is this: Can this N dimensional dynamical system be made
to approximate the dynamics of a given M dimensional dynamical system
(M <= N) on an M dimensional subspace arbitrarily well by appropriate
adjustment of the time constants (Ci) and coupling strengths (Wij) as
N -> infinity? By "approximate the dynamics" I mean that all
trajectories on the M dimensional subspace of the N dimensional system
can be made arbitrarily close to those of the system we are
approximating. One way of expressing this is that the following term
should approach 0 for all initial conditions:
M infinity
sum{integral{(yi - di)^2 dt}}
i=1 0
where di(t) is the value of the ith state variable of the system we
are approximating at time t and yi(t) is the value of the
corresponding state variable in our approximation.
What I have in mind here is a dynamical analogue to function
approximation. If the answer to my question is yes, can any bounds be
placed on how the accuracy of the approximation will scale with N? If
the answer is no, do any such "basis systems" exist? Can the above
system at least approximate an arbitrary M dimensional attractor,
rather than the entire phase space?
Any help on these questions would be greatly appreciated.
Thanks,
R. Beer
(beer@cthulhu.ces.cwru.edu)kolen-j@neuron.cis.ohio-state.edu (john kolen) (07/17/90)
In article <1990Jul16.140630.10110@usenet.ins.cwru.edu> beer@cthulhu.ces.cwru.edu (Randall D. Beer) writes:
From: beer@cthulhu.ces.cwru.edu (Randall D. Beer)
Newsgroups: sci.math,comp.theory.dynamic-sys
Date: 16 Jul 90 14:06:30 GMT
Sender: news@usenet.ins.cwru.edu
Organization: Computer Engineering and Science/CWRU
Suppose we are given a system of N differential equations of the form
N
Ci * dyi/dt = -yi + F(sum{Wji * yi}) where F(x) = 1/(1 + e^(-x))
j=1
My question is this: Can this N dimensional dynamical system be made
to approximate the dynamics of a given M dimensional dynamical system
(M <= N) on an M dimensional subspace arbitrarily well by appropriate
adjustment of the time constants (Ci) and coupling strengths (Wij) as
N -> infinity? By "approximate the dynamics" I mean that all
trajectories on the M dimensional subspace of the N dimensional system
can be made arbitrarily close to those of the system we are
approximating. One way of expressing this is that the following term
should approach 0 for all initial conditions:
M infinity
sum{integral{(yi - di)^2 dt}}
i=1 0
where di(t) is the value of the ith state variable of the system we
are approximating at time t and yi(t) is the value of the
corresponding state variable in our approximation.
What I have in mind here is a dynamical analogue to function
approximation. If the answer to my question is yes, can any bounds be
placed on how the accuracy of the approximation will scale with N? If
the answer is no, do any such "basis systems" exist? Can the above
system at least approximate an arbitrary M dimensional attractor,
rather than the entire phase space?
Any help on these questions would be greatly appreciated.
Thanks,
R. Beer
(beer@cthulhu.ces.cwru.edu)
The answer to your question is no. The approximation dynamic described
above is limited to convex transition functions. If the dynamic is modified
as follows
N2 N1
Ci * dyi/dt = -yi + F(sum(Vij * F(sum{Wjk * yk}))),
j=1 k=1
then any dynamical system with an underlying dynamic expressable as
a Borel measurable function can be approximated within any epsilon
with finite N1 and N2. Hornik, Stinchcombe, and White proved that any
function from that class can be approximated by a system of functions of
this form
The system you are describing is known in some circles as a recurrent neural
network. These networks have been studied for DS with fixed point behavior
(Pineada), limit cycle behavoir (Pearlmutter), finite state machines (Pollack),
chaotic systems (Weigend, Huberman,Rumelhart). This list is by no means
complete, but should give you a good head start.
%A K. Hornik
%A M. Stinchcombe
%A H. White
%T Multi-layer Feedforward Networks are Universal Approximators
%J Neural Networks
(This has appeared but I don't have any more info on it)
%A F. J. Pineda
%T Generalization of Back-Propagation to Recurrent Neural Networks
%J Physical Review Letters
%D 1987
%V 59
%P 2229-2232
%A B. A. Pearlmutter
%T Learning State Space Tragectories in Recurrent Neural Networks
%J Neural Computation
%V 1
%P 263-269
%D 1989
%A J. B. Pollack
%T The Induction of Dynamical Recognizers
%R Tech Report 90-JP-Automata
%I LAIR, Ohio State University
%C Columbus, OH 43210
%D 1990
%A A. S. Weigend
%A B. A. Huberman
%A D. E. Rumelhart
%T Predicting the Future: A Connectionist Approach
%R Tech Report Stanford-PDP-90-01
%I Stanford Univeristy
%D 1990
--
John Kolen (kolen-j@cis.ohio-state.edu)|computer science - n. A field of study
Laboratory for AI Research |somewhere between numerology and
The Ohio State Univeristy |astrology, lacking the formalism of the
Columbus, Ohio 43210 (USA) |former and the popularity of the latter