li@kiss.informatik.tu-muenchen.de.informatik.tu-muenchen.dbp.de (10/10/90)
Parity functions may be realized by NN with one hidden layer (a simple solution was given in PDP-1). It is however a hard problem to get such solution by back-propagation algorithm. I was able to train a NN with backprog and some heuristcs to realize the P_4 (i.e. the parity function of 4 bits vectors, P_2 is the XOR function). The P_5 seems, by my experience, already to be too difficult to be learned by backprop, no matter how many layers and neurons are used. Does someone know better results? Thanks. Xinzhi Li
demers@odin.ucsd.edu (David E Demers) (10/10/90)
In article <4803@tuminfo1.lan.informatik.tu-muenchen.dbp.de> li@kiss.informatik.tu-muenchen.de.informatik.tu-muenchen.dbp.de () writes: >Parity functions may be realized by NN with one hidden layer (a simple >solution was given in PDP-1). It is however a hard problem to get such >solution by back-propagation algorithm. I was able to train a NN with >backprog and some heuristcs to realize the P_4 (i.e. the parity >function of 4 bits vectors, P_2 is the XOR function). The P_5 seems, by >my experience, already to be too difficult to be learned by backprop, >no matter how many layers and neurons are used. Does someone know >better results? I am recalling Tim Ash's paper on his Dynamic Node Creation. He used it on several problems, including parity 4 and 5, I believe. It is in a recent issue of Connection Science journal. His method is essentially backprop, but with addition of nodes (or layers!) when convergence either to the training set or a test set stops. Dave
faee0ntt@zach.fit.edu ( N. Tepedelenlioglu) (10/10/90)
In article <4803@tuminfo1.lan.informatik.tu-muenchen.dbp.de> li@kiss.informatik.tu-muenchen.de.informatik.tu-muenchen.dbp.de () writes: >Parity functions may be realized by NN with one hidden layer (a simple >solution was given in PDP-1). It is however a hard problem to get such >solution by back-propagation algorithm. I was able to train a NN with >backprog and some heuristcs to realize the P_4 (i.e. the parity >function of 4 bits vectors, P_2 is the XOR function). The P_5 seems, by >my experience, already to be too difficult to be learned by backprop, The key is the number of nodes in the hidden layer. That number should be at least as big as the number of bits at the input. So if you try a net with say 8 nodes in the hidden layer I am pretty sure you will have no difficulty for the P_5 problem. > >Thanks. >Xinzhi Li Nazif. __________________________ Nazif Tepedelenlioglu faee0ntt@zach.fit.edu Dept. EE/CP Florida Institute of Technology, Melbourne, FL 32901, USA