xinzhi@uklirb.UUCP (Xinzhi Li AG Richter) (07/10/89)
When does a hopfield net converge to stead state? If it converges, how many steps will it take to enter the stead state? I tried to answer such problems by using methods of linear algebra (i.e. eigenvalue related methods). I always got trouble with the non-linearity caused by the threshold function. Does anyone knows any method to overcome such difficulty? Does anyone knows any theorem in this direction? I have read a assertion in a article of Lippmann (Apr. 1987 IEEE ASSP) that a hopfield net could always converge given that the net is symmetric and completely connected. But i can not get the proof of this assertion. Does anyone knows the paper proving this assertion? Which methods are used there? Thank you in advance. --------------------------------------------------- xinzhi@uklirb.uucp Xinzhi Li, university kaiserslautern, west germany
artzi@cpsvax.cps.msu.edu (Ytshak Artzi - CPS) (07/11/89)
In article <6009@uklirb.UUCP> xinzhi@uklirb.UUCP (Xinzhi Li AG Richter) writes: >When does a hopfield net converge to stead state? If it >converges, how many steps will it take to enter the stead state? I >tried to answer such problems by using methods of linear algebra >(i.e. eigenvalue related methods). I always got trouble with >the non-linearity caused by the threshold function. Does anyone knows >any method to overcome such difficulty? Does anyone knows any theorem in >this direction? Hopfield model is totally unpredictable. Moreover, it depends on the particular instance of the particular problem you try to solve, which in turn depends on the initial parameters you choose for your equations. If parameteres are not wisely (??) chosen the network WON'T converge at all. Itzik.
thomae@cslvs5.ncsu.edu (Doug Thomae) (07/12/89)
In article <3738@cps3xx.UUCP> artzi@cpsvax.UUCP (Ytshak Artzi - CPS) writes: >In article <6009@uklirb.UUCP> xinzhi@uklirb.UUCP (Xinzhi Li AG Richter) writes: >>When does a hopfield net converge to stead state? If it >>converges, how many steps will it take to enter the stead state? I >>tried to answer such problems by using methods of linear algebra >>(i.e. eigenvalue related methods). I always got trouble with >>the non-linearity caused by the threshold function. Does anyone knows >>any method to overcome such difficulty? Does anyone knows any theorem in >>this direction? > > Hopfield model is totally unpredictable. Moreover, it depends on the >particular instance of the particular problem you try to solve, which in >turn depends on the initial parameters you choose for your equations. >If parameteres are not wisely (??) chosen the network WON'T converge at >all. > > Itzik. It has been proven (in the mathematical sense ) by Hopfield and Grossberg (and perhaps others), that Hopfield networks that have 1) symmetric connections (weight from neuron i to neuron j is the same as the weight in the reverse direction) and 2) weight of connection from a neuron to itself is zero, will always converge in the sense that they will settle down and not oscillate forever. That does not mean that they will settle into the state desired by the user, just that they will settle into some state. The basic idea behind the proof is to show that a Lyaponov (sp?) function exists for the network, and then use a theorem from control theory that says that if such a function exists for a network, then the system is stable. A good text on control theory should have all the gory details. The two papers usually referenced in the neural net community for all this are: M.A. Cohen and S. Grossberg, "Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks", IEEE Transactions on Systems, Man and Cybernetics, p. 815-826, 1983 J.J. Hopfield, "Neurons with Graded Response Have Collective Computational Properties Like Those of Two-State Neurons", Proceedings of the National Academy of Sciences, Vol. 81, p. 3088-3092, May 1984 I have heard mention of more general theorem that show that a Hopfield network will also converge under some conditions when the connections are not symmetric, but I don't know the reference for it.
laird@ptolemy.arc.nasa.gov (Philip Laird) (07/12/89)
In article <3376@ncsuvx.ncsu.edu> thomae@cslvs5.UUCP (Doug Thomae) writes: >... I have heard mention of more general theorem that show that a Hopfield >network will also converge under some conditions when the connections >are not symmetric, but I don't know the reference for it. Convergence (or lack thereof) is rather well understood theoretically. Two recent references are: Bruck and Sanz, Int. Journal of Intelligent systems, 3, p. 59-75, 1988. Bruck and Goodman, IEEE Trans. on Information Theory, IT-34 (Sept. 88). -- Phil Laird, NASA Ames Research Center, Moffett Field, CA 94035 (415)-694-3362 LAIRD@PLUTO.ARC.NASA.GOV
andrew@berlioz (Lord Snooty @ The Giant Poisoned Electric Head ) (07/13/89)
In article <3376@ncsuvx.ncsu.edu>, thomae@cslvs5.ncsu.edu (Doug Thomae) writes: > [..] The basic idea behind the proof is to > show that a Lyaponov (sp?) function exists for the network, and then > use a theorem from control theory that says that if such a function > exists for a network, then the system is stable. A good text on > control theory should have all the gory details [..]. I see Lyapunov-type arguments being extensively used in the NN literature. However, I have no texts dealing with methods whereby one can perform intelligent searches for appropriate Lyapunov functions. Can anyone suggest apprpriate texts for this purpose? -- ................................................................... Andrew Palfreyman I should have been a pair of ragged claws nsc!berlioz!andrew Scuttling across the floors of silent seas ...................................................................