fc174103@seas.gwu.edu (Burr) (11/25/90)
I'm using the backpropagation program that comes with the PDP books. I'm trying to train a network with 45 units (20 input, 15 hidden, and 10 output), to recognize the digits 0 thru 9. I've set up a template file that uses 132 columns by 43 lines to display appropriate info about the network while it is being trained. The display works fine and does not seem to be the thing which is hanging the network up. The network hangs when I try to train it using the ptrain or strain commands. It even hangs when I use the reset command. While these commands do not work the tall command, which just computes activations without changing the weights, does work. I'm not sure what is the problem. Is 45 units pushing the limit of this backprop program? If anyone has had experience with this program I would much appreciate any help you could offer.
clchen@babcock.cerc.wvu.wvnet.edu (Chih-Liang Chen) (12/01/90)
From article <2370@sparko.gwu.edu>, by fc174103@seas.gwu.edu (Burr): > I'm using the backpropagation program that comes with the PDP books. > I'm trying to train a network with 45 units (20 input, 15 hidden, and > 10 output), to recognize the digits 0 thru 9. I've set up a template > file that uses 132 columns by 43 lines to display appropriate info > about the network while it is being trained. The display works fine > and does not seem to be the thing which is hanging the network up. The > network hangs when I try to train it using the ptrain or strain commands. > It even hangs when I use the reset command. While these commands do not > work the tall command, which just computes activations without changing > the weights, does work. I'm not sure what is the problem. Is 45 units > pushing the limit of this backprop program? If anyone has had experience > with this program I would much appreciate any help you could offer. It does has this problem, but it does not mean the BP algorithm doesn't work. My experience shows that: 1. BP is good learning algorithm 2. training by epoch is the worst for large training patterns 3. strain is the 2nd ......... 4. ptrain, you can take chance. Your NN is a "small" one, I had 256X64X4 with 512 training patterns. It got stuck with "epoch" or "pattern" method. I solved it by "intensive training on the pattern which makes the maximum MS error". So, you can: 1. modify the bp program, or 2. write your own one. just a experience.