spackman@ohsuhcx.ohsu.edu (Dr. Kent Spackman) (04/11/90)
I'm looking for references to articles that have anything to do with the relationship between back-propagation and maximum likelihood estimation, such as is done by logistic regression. I have programmed a maximum likelihood estimator that trains a nnet by sort-of back-propagating likelihoods, resulting in a "multi-layer" logistic regression. It works well for simple problems (X-OR, etc.) involving multiple layers and reproduces the results of single-layer logistic regression. Has anyone done this before? Do you know of any references? Some background information: Logistic regression is a commonly used statistical method in medicine. It can be described as a method for maximum likelihood estimation of the weights of a single-layer feed-forward neural network. It uses the logistic transfer function just like "ordinary" back-propagation. Back-propagation does not do maximum likelihood estimation. Minimizing the sum of squared errors is not the same as maximum likelihood when using the logistic transfer function. Reply to the net or directly via email to me: spackman@ohsu.edu Kent A. Spackman Biomedical Information Communication Center Oregon Health Sciences University Portland, OR 97201
mgv@usceast.UUCP (Marco Valtorta) (04/11/90)
In article <437@ohsuhcx.ohsu.edu> spackman@ohsu.edu (Kent A. Spackman) writes: > >I'm looking for references to articles that have anything to do with >the relationship between back-propagation and maximum likelihood >estimation, such as is done by logistic regression. > >Kent A. Spackman >Biomedical Information Communication Center >Oregon Health Sciences University >Portland, OR 97201 You may want to read "A Statistical Approach to Learning and Generalization in Layered Neural Networks," by Levin, Tishby, and Solla, in the COLT '89 Proceedings, pp.245-260. The Proceedings are published by Morgan Kaufmann. Marco Valtorta usenet: ...!ncrcae!usceast!mgv Department of Computer Science internet: mgv@cs.scarolina.edu University of South Carolina tel.: (1)(803)777-4641 Columbia, SC 29208 tlx: 805038 USC U.S.A. fax: (1)(803)777-3065 usenet from Europe: ...!mcvax!uunet!ncrlnk!ncrcae!usceast!mgv