markh@csd4.csd.uwm.edu (Mark William Hopkins) (11/20/90)
Archive-name: newton-backprop/13-Nov-90 Original-posting-by: markh@csd4.csd.uwm.edu (Mark William Hopkins) Original-subject: Backpropagation with Newton's Method, and recurrence. Source code. Archive-site: csd4.csd.uwm.edu [129.89.7.4] Reposted-by: emv@ox.com (Edward Vielmetti) If you would like to test out Newton's method for backpropagation neural nets, and if you'd like to try out a relatively unknown, but efficient, algorithm for training recurrent bp. neural nets, you can obtain some demos via ftp. The programs illustrate recurrent Newton's Method backprop. on a familiar application: learning the exclusive-or function. Also, you will find a program that successfully emulates a finite state machine (a flip flop) using recurrent back prop. with "persistent activations", and a (not-so-successful) program that attempts to LEARN the flip flop. Needless to say, training backpropagation to do a flip flop is not an easy task, but bp. with recurrence and persistent activations is powerful enough to represent any finite state machine (including even an entire CPU!) ... so convergence may be possible with the right presentation strategy using nothing more than the generalized delta rule (!) Do an anonymous ftp to csd4.csd.uwm.edu, set binary mode and pluck out nn.Z from the top-level directory, uncompress it, "de-tar" it and run it on any IBM-compatible with a Quick Basic 4.5 compiler. The source has been written in such a way as to make translation to Berkeley C (using the curses package), or MicroSoft Quick C relatively easy. From command prompt: >ftp csd4.csd.uwm.edu Ftp login procedure: > Name: anonymous > Password: ident From the ftp prompt: > binary > get nn.Z > quit Back in command prompt: > uncompress nn.Z > tar -xf nn (This sequence takes about 20 to 30 seconds :) ). I'd be interested in hearing any comments on the software. -- Mark Hopkins (markh@csd4.csd.uwm.edu) ------------------------------------------------------------ Disclaimer: Everything below the dotted line is patently false.