tgk@beach.cis.ufl.edu (Tom Krajna) (09/20/89)
I'm using version 1.1 of the PDP software. When using the backprop
program (bp), the program runs until the total sum of squares (tss)
drops below a critical error level (ecrit). At this point, if a
"test all" command (tall) is executed, and you test all the patterns,
the tss is no longer the same as the one which had just caused the
program to stop running. (As a matter of fact, it seems to be the
"next" tss that would be calculated, if you ran all the patterns through
one more time, and adjusted the weights.)
Why is this a problem? Sometimes, due to the meanderings of the
gradient descent, the "next" tss may be greater than the one which
was just low enough to meet the ecrit criterion. So your net has
stopped, but statistically, it may not be a good enough fit.
Anyone have a solution or a work-around?
--
Tom Krajna UUCP: ...gatech!uflorida!beach.cis.ufl.edu!tgk
University of Florida Internet: tgk@beach.cis.ufl.edu CIS: 73267,1652
What's an analogy like? edwardm@hpcuhc.HP.COM (Edward McClanahan) (09/20/89)
Tom Krajna writes: > I'm using version 1.1 of the PDP software. When using the backprop > program (bp), the program runs until the total sum of squares (tss) > drops below a critical error level (ecrit). At this point, if a > "test all" command (tall) is executed, and you test all the patterns, > the tss is no longer the same as the one which had just caused the > program to stop running. (As a matter of fact, it seems to be the > "next" tss that would be calculated, if you ran all the patterns through > one more time, and adjusted the weights.) > Why is this a problem? Sometimes, due to the meanderings of the > gradient descent, the "next" tss may be greater than the one which > was just low enough to meet the ecrit criterion. So your net has > stopped, but statistically, it may not be a good enough fit. > Anyone have a solution or a work-around? I don't know about a "solution" or a "work-around", but this sounds like you have found a "local minima". Sometimes the steepest "decent" isn't always the best. Ed "neophite at PDP" McClanahan =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Edward McClanahan Hewlett Packard Company Mail Stop 47UE -or- edwardm%hpda@hplabs.hp.com 19447 Pruneridge Avenue Cupertino, CA 95014 Phone: (408)447-5651