schultz@halley.est.3m.com (John C. Schultz) (02/17/91)
Thanks to everyone who responded on how to optimize input parameters
vs output values using a trained neural network.
Several suggestions were to invert the network as for example Tony
Robinson, who referred to the following paper
[1] A Linden and J Kindermann, "Inversion of Multilayer Nets", pp II425-II430,
Proceedings of the INternational Joint Conference on Neural Networks,
Washington DC, June 18-22, 1989.
I am trying to obtain this paper but currently have some doubts as to
how well you can "invert" a backprop neural network.
To my mind, there are four ways to use a trained neural network for
optimization problems.
1. exahustive search of likely input paramter combinations - brute
force but it would work for small problems. Large problems are
computationaly difficult - my (un)original idea.
2. invert the trained network. My problem with this idea is that for a
back-prop network, each node is a non-linear (tanh() in my case)
function and the concept of running this function backwards is not clearly
defined in my mind - it would seem to be highly unstable.
3. Use the trained network as the input to another network which has
one input and many outputs. The problem with this scenario for
back-prop is that back-prop learns by propogating the error from
output to input. Thus the error at the single input node would
tend to move in very large jumps and convergence seems unlikely.
4. Use an algorithmic optimizing routine which could treat the neural
network as a black box supplying inputs and searching (somehow) for
the optimial output.
I am persuing option 4) above while trying to find the reference cited
by Tony Robinson.
Thanks for the suggestions.
--
John C. Schultz EMAIL: schultz@halley.serc.3m.com
3M Company, Building 518-01-1 WRK: +1 (612) 733-4047
1865 Woodlane Drive, Dock 4, Woodbury, MN 55125
How to include the taste of Glendronach in a multi-media system?mgj@cup.portal.com (Mark Gregory Jurik) (02/19/91)
About training a network to perform the inverse of another net: Regardless of how a network is trained, if the original network does a many- to-one mapping, then the inverse will have a problem with ambiguous one-to- many mappings. If this is not the case, then in response to your statement: " 3. Use the trained network as the input to another network which has one input and many outputs. The problem with this scenario for back-prop is that back-prop learns by propogating the error from output to input. Thus the error at the single input node would tend to move in very large jumps and convergence seems unlikely. " I suggest you consider using BackPercolation instead of BackPropagation. Perc assigns each adaptive cell an output error that is not proportional to the cell's output error gradient. Training is stable and fast with nets having multiple layers. For example, 6 input parity classification was trained using Perc on a net with 6-6-6-1 configuration. It attained a total squared error < 0.0001 in only 93 epochs. Perc does not experience large jumps with either a large number of rows or a large number of cells per row, or both. I am submitting an announcement about BackPercolation to the neural-net digest. If accepted, it will be posted in a few days, I guess. -Mark Jurik, Jurik Research & Consulting, Aptos CA, 95001