alexis%yummy@GATEWAY.MITRE.ORG (11/18/88)
I have to admit some surprise that so many people got this "wrong."
Our experience is that neural nets of the PDP/backprop variety are
at their *BEST* with continueous mappings. If you just want classification
you might as well go with nearest-neighbor alg.s (or if you want the same
thing in a net try Nestor's Coulombic stuff). If you can't learn x=>sin(x)
in a couple of minutes, you've done something wrong and should check
your code (I'm assuming you thought to scale sin(x) to [0,1]).
Actually, requiring a PDP net to output 1's and 0's means your weights
must be quite large which takes alot of time and puts you way out on
the tails of the sigmoids where learning is slow and painful.
What I do for fun (?) these days is try to make nets output sin(t) {where
t is time} and other waveforms with static or "seed" wave inputs.
For those who like math, G. Cybenko (currently of U. Illinois and starting
12/10/88 of Tufts) has a very good paper "Approximation by Superpositions
of a Sigmoidal Function" where he gives a existence proof that you can
uniformly approximate any continuous function with support in the unit
hypercube. This means a NN with one hidden layer (1 up from a perceptron).
Certainly more layers generally give more compact and robust codings ...
but the theory is *finally* coming together.
Alexis Wieland .... alexis%yummy@gateway.mitre.org