amp@pollux.UUCP (Ajay M. Patrikar) (06/23/89)
I was trying to solve different problems mentioned in the following reference : D.E.Rumelhart, G.E. Hinton, and R.J.Williams, "Learning Internal Representation by Error Propogation" in Parallel Distributed Processing : Explorations in the microstructure of cognition. vol. 1, MIT press 1986 I used backpropogation algorithm for solving problems such as Exclusive OR , parity problem etc. In most cases I got different convergence rates from the ones mentioned in the book. Also, quite a few times the program ran into local minima problem. This may be because of altogather different initial conditions. I was generating random numbers in the interval (-0.5, 0.5) and using them as initial weights and thresholds. Has anyone on the net ran into similar problems ? I would appreciate if someone can pass me information about 1. dependence of convergence rate on initial conditions 2. What is the general criterion for convergence. 3. performance of backpropogation on bigger problems.(no. of pattern presentations) Thanking you in advance. Ajay Patrikar uunet!dalsqnt!pollux !amp
kolen-j@toto.cis.ohio-state.edu (john kolen) (06/23/89)
In article <15559@pollux.UUCP> amp@pollux.UUCP (Ajay M. Patrikar) writes: > >I used backpropogation algorithm for solving problems such as >Exclusive OR , parity problem etc. In most cases I got >different convergence rates from the ones mentioned in the book. >Also, quite a few times the program ran into local minima problem. > >This may be because of altogather different initial conditions. >I was generating random numbers in the interval (-0.5, 0.5) and >using them as initial weights and thresholds. > >Has anyone on the net ran into similar problems ? I would appreciate >if someone can pass me information about > >1. dependence of convergence rate on initial conditions >2. What is the general criterion for convergence. >3. performance of backpropogation on bigger problems.(no. of pattern > presentations) > We ran into these problems and several others (as most other researchers have, but are reluctant to admit it). Some answers to the questions you pose appear in a recent Ohio State University Laboratory for Artificial Intelligence Research technical report "Learning in Parallel Distributed Processing Networks: Computational Complexity and Information Content". For ordering information contact LAIR Technical Report Library 217 CAE Dept. of Computer and Information Science 2036 Neil Avenue Mall Columbus, Ohio 43210-1277 -=- John Kolen (kolen-j@cis.ohio-state.edu)|computer science - n. A field of study Computer & Info. Sci. Dept. |somewhere between numerology and The Ohio State Univeristy |astrology, lacking the formalism of the Columbus, Ohio 43210 (USA) |former and the popularity of the later.