mgj@cup.portal.com (Mark Gregory Jurik) (06/10/91)
REQUEST FOR REFERENCES TO BACKPROP UPGRADES, PART 2 One month ago, I announced that I am collecting references and suggestions (for eventual posting and inclusion in a book) on all means of improving and testing the performance of BackPropagation. A list suggestions was posted to get things started. Many new suggestions were offered since that time, thereby expanding the original list. Here s the latest list: TRAINING 1. Low bit quantization (how low can you go?) 2. Batching (optimal batch size?) 3. Momentum (fixed and adaptive) 4. Learn rate (fixed and adaptive) 5. Weight decay (fixed and adaptive) 6. Added noise to weight adjustments (fixed and adaptive) 7. Conjugate gradient searching (too much overhead?) 8. Fastprop/Quickprop (are they the same?) 9. Uniprop (does this exist?) 9. Whateverprop (anything else?) ARCHITECTURE 1. Multiple hidden layers (too much of a good thing?) 2. Sigmoidal vs.Gaussian thresholding (any others?) 4. Recurrent connectivity (instability issues?) 5. Network size (is smaller better?) 6. Complex (real and imaginary) weights (when is it useful?) PREPROCESSING 1. Kohonen layer quantization (useful for classification?) 2. Fuzzy membership representation (thermometers, etc. ...) 3. Added noise (how much is safe?) 4. Principal component decomposition (when does it help?) 5. Remove linear transformations (& add back later. Is it wise?) LABORATORY BENCHMARKS 1. N-Bit parity 2. N-M-N encoder/decoder 3. N-N-N linear channel 4. N-2-1 symmetry detection 5. 3-N-1 two out of three detection 6. 2-N-1 Intertwined spiral classification If you have more topics for the list or references to suggest: please E-mail your suggestions to mgj@cup.portal.com. The odds are you know of at least one good paper that most others are not aware. If you have material you would like me to read and consider for posting and referenceing in an upcoming book, please mail to JURIK RESEARCH, PO 2379, Aptos, CA 95001 After sufficient information has been collected, a brief synopsis of all *that has been submitted* will be posted. -- Mark Jurik, mgj@cup.portal.com