barronca@mtecv2.mty.itesm.mx (Ing. Elvia Patricia Barron Cano) (05/19/91)
Hello! I have problem with training of the network. I use backpropagation. The network have to learn to classify 529 patterns in 3 categories. The input is binary, there are 36 inputs, but only two are enable. First, I test whit 50 hidden units, but the total sum of squares was near 10, and to fail in five patterns. I test some times, after with more units (52, 55, 60, 80), and two hidden levels, but the results were aproximate the same. When I made I tested some rate learning and momentums. I studied the patterns and I made a network with 19 hidden units. I tested with them and 37 units but the response was bad. Why it can't find the weights? Please, email me, because i don't read so often this interesting newsgroup. Thanks, for everything. Elvia Patricia Barron Cano barronca@mtecv2.mty.itesm.mx Centro de Inteligencia Artificial Instituto Tecnologico de Monterrey Monterrey, NL, Mexico.
arms@cs.UAlberta.CA (Bill Armstrong) (05/20/91)
barronca@mtecv2.mty.itesm.mx (Ing. Elvia Patricia Barron Cano) writes: >I have problem with training of the network. I use backpropagation. >The network have to learn to classify 529 patterns in 3 categories. >The input is binary, there are 36 inputs, but only two are enable. Sorry, I don't understand the last bit, if you really mean only two input leads are enabled of the 36, meaning you only have 4 different input patterns after the 529 are projected. I shall assume you don't mean that, but rather that your inputs are restricted to 0 and 1. >First, I test whit 50 hidden units, but the total sum of squares was >near 10, and to fail in five patterns. I test some times, after with >more units (52, 55, 60, 80), and two hidden levels, but the results >were aproximate the same. When I made I tested some rate learning and >momentums. >I studied the patterns and I made a network with 19 hidden units. I >tested with them and 37 units but the response was bad. >Why it can't find the weights? I can't tell you exactly why you are having problems with backprop, but a lot of people do. It could have something to do with how close in Hamming distance some of your patterns are that are in different classes; but you would have to carefully analyse the data to find that out. Since you have binary input data, presumably what you need to compute are two boolean functions of 36 variables which separate the space {0,1}^^36 into three classes. May I suggest you try adaptive logic networks (ALNs). You can get the software via anonymous ftp from menaik.cs.ualberta.ca [129.128.4.241] in pub/atree.tar.Z I think your problem will be easily solved. You can just use the language lf, and specify your problem easily. However, if you want it to run on a PC, you will have to wait for the next version of the software. Finally, when you have trained on the data using ALNs, you can count on fast execution because of "parsimonious" (or lazy) evaluation, which refers to the fact that if you know that one input to an AND gate is 0, you don't need to do any of the computation required to obtain the other input. In more demanding cases, you could make a chip either by metallizing a gate array or, for much less money, by using a field programmable gate array. Hope this helps! Bill Armstrong -- *************************************************** Prof. William W. Armstrong, Computing Science Dept. University of Alberta; Edmonton, Alberta, Canada T6G 2H1 arms@cs.ualberta.ca Tel(403)492 2374 FAX 492 1071
demers@beowulf.ucsd.edu (David Demers) (05/20/91)
In article <arms.674682102@spedden> arms@cs.UAlberta.CA (Bill Armstrong) writes: >barronca@mtecv2.mty.itesm.mx (Ing. Elvia Patricia Barron Cano) writes: >>I have problem with training of the network. I use backpropagation. >>The network have to learn to classify 529 patterns in 3 categories. >>The input is binary, there are 36 inputs, but only two are enable. >Sorry, I don't understand the last bit, if you really mean only two >input leads are enabled of the 36, meaning you only have 4 different >input patterns after the 529 are projected. I shall assume you don't >mean that, but rather that your inputs are restricted to 0 and 1. I too was confused, but upon second reading, a reasonable interpretation might be that only two of the 36 input units are non-zero, thus 1260 (36 choose 2) total possible input patterns. >>First, I test whit 50 hidden units, but the total sum of squares was >>near 10, and to fail in five patterns. I test some times, after with >>more units (52, 55, 60, 80), and two hidden levels, but the results >>were aproximate the same. When I made I tested some rate learning and >>momentums. >>I studied the patterns and I made a network with 19 hidden units. I >>tested with them and 37 units but the response was bad. >>Why it can't find the weights? [...] >Since you have binary input data, presumably what you need to compute >are two boolean functions of 36 variables which separate the space >{0,1}^^36 into three classes. May I suggest you try adaptive logic >networks (ALNs). might be a better tool for this problem. But bp should be able to solve it if there is structure in the classifications. Try reducing the learning rate and boosting the momentum...? I'd suggest using one hidden layer but including direct connections from input to output to easily capture linearities. without knowing more about your data, I can only encourage you to use all tools available. Dave -- Dave DeMers demers@cs.ucsd.edu Computer Science & Engineering C-014 demers%cs@ucsd.bitnet UC San Diego ...!ucsd!cs!demers La Jolla, CA 92093-0114 (619) 534-8187,-0688 ddemers@UCSD
tlarson@polyslo.CalPoly.EDU (Ted E. Bear) (05/22/91)
In article <3388@mtecv2.mty.itesm.mx> barronca@mtecv2.mty.itesm.mx (Ing. Elvia Patricia Barron Cano) writes: >Hello! > >I have problem with training of the network. I use backpropagation. The network have to learn to classify 529 patterns in 3 categories. The input is binary, there are 36 inputs, but only two are enable. > ...... >I studied the patterns and I made a network with 19 hidden units. I tested with them and 37 units but the response was bad. > >Why it can't find the weights? > >Please, email me, because i don't read so often this interesting newsgroup. > I would also be interested in hearing if this problem is solved. Please post the answers or email me too. - Ted saturn!ted@polyslo.calpoly.edu tlarson@polyslo.calpoly.edu