rjf@beach.cis.ufl.edu (Richard Fiola) (12/21/89)
Elaborating on Dr. Fishwick's msg of 19 Dec <21539@uflorida.cis.ufl.EDU> in response to Sudahakar Yerramareddy <220700005@uxe.cso.uiuc.edu> kbesrl@uxe.cso.uiuc.edu, Sudhakar writes >I have been experimenting with back-prop neural nets for the past >few months. I find that they are only as good as polynomial >regression. Actually, I ran a back-prop neural net on some >continuous mapping problems and found that they achieved the >same performance as the `SAS' statistical package. Concerning the performance of back-propagation (bp) relative to SAS on continuous mapping problems, I think the whole point of Artificial Neural Networks (ANNs) is missed; they ARE statistical...but perform the analysis through a different paradigm. One big difference is that they are modelling with an IMPLICIT non-linear function. The reason that they may perform so well in many situations is that they are probably coming closer in those particular problems to modeling the true function through their extremely complex combination of exponentials than other statistical models (note one can easily generate the closed form of the function after the net is trained...but it will look like the equation from hell; note further that Minsky and Papert even showed that recurrent nets had a representative closed form solution). Sudhakar further writes >...how can one defend the use of neural nets as opposed to statistical >regression? Concerning justifying the use of neural nets, the point of any paradigm is to generate an accurate model. ANNs do not generate a predictive model any more than statistical packages, because they do not contain heuristics- they are search mechanisms GIVEN a model. But as a paradigm, they do provide some advantages over many statistical packages. Lippmann [Lippman, Richard P. 1987. An introduction to computing with neural nets. IEEE Transactions on Acoustics, Speech and Signal Processing, April, pp. 4-22.] goes so far as to state The ability to adapt and to continue learning is essential in areas where training data is limited and new environments are continuously encountered. Adaptation also provides a degree of robustness by compensating for minor variabilities in characteristics of processing elements. Traditional statistical techniques are not adaptive but typically process all training data simultaneously before being used with new data. Neural net classifiers are also non-parametric and make weaker assumptions concerning the shapes of underlying distributions than traditional statistical classifiers. They may thus prove to be more robust when distributions are generated by nonlinear processes and are strongly non-Gaussian. My opinion is that the advantages make using NN's attractive, but one must keep in mind that they are essentially statistical in nature. When you said that your nets performed only as well as polynomial regression, you can be assured that they were generating approximations to polynomials for the data you were providing. To assume that NN's are black boxes that do something magical or cognitive portends disaster for the field of ANN research. I find many people have a misconception about the capabilities of ANNs. There is a void of good numerical analysis concerning ANNs; I completely agree with Dr. Fishwick that much more work needs to be done, especially in that area. R J Fiola c/o Dr. Paul Fishwick CIS Department Universtiy of Florida Gainesville, FL 32611 rjf@beach.cis.ufl.edu