"Lorien Y. Pratt" <pratt@paul.rutgers.edu> (04/18/91)
Archive-name: ai/neural-nets/pratt-phoneme/1991-04-05
Archive: cheops.cis.ohio-state.edu:/pub/neuroprose/pratt*91* [128.146.8.62]
Original-posting-by: "Lorien Y. Pratt" <pratt@paul.rutgers.edu>
Reposted-by: emv@msen.com (Edward Vielmetti, MSEN)
The following two papers are now available via FTP from the neuroprose
archives. The first is for AAAI91, so written towards an AI/Machine
learning audience. The second is for IJCNN91, so more neural
network-oriented. There is some overlap between them: the AAAI paper
reports briefly on the study describved in more detail in the IJCNN
paper. Instructions for retrieval are at the end of this message.
--Lori
#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@#@
Direct Transfer of Learned Information Among Neural Networks
To appear: Proceedings of AAAI-91
Lorien Y. Pratt and Jack Mostow and Candace A. Kamm
Abstract
A touted advantage of symbolic representations is the ease of
transferring learned information from one intelligent agent to
another. This paper investigates an analogous problem: how to use
information from one neural network to help a second network learn a
related task. Rather than translate such information into symbolic
form (in which it may not be readily expressible), we investigate the
direct transfer of information encoded as weights.
Here, we focus on how transfer can be used to address the important
problem of improving neural network learning speed. First we present
an exploratory study of the somewhat surprising effects of pre-setting
network weights on subsequent learning. Guided by hypotheses from this
study, we sped up back-propagation learning for two speech recognition
tasks. By transferring weights from smaller networks trained on
subtasks, we achieved speedups of up to an order of magnitude compared
with training starting with random weights, even taking into account
the time to train the smaller networks. We include results on how
transfer scales to a large phoneme recognition problem.
@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@%@
Improving a Phoneme Classification Neural Network
through Problem Decomposition
To appear: Proceedings of IJCNN-91
L. Y. Pratt and C. A. Kamm
Abstract
In the study of neural networks, it is important to determine whether
techniques that have been validated on smaller experimental tasks can
be scaled to larger real-world problems. In this paper we discuss how
a methodology called {\em problem decomposition} can be applied to
AP-net, a neural network for mapping acoustic spectra to phoneme
classes. The network's task is to recognize phonemes from a large
corpus of multiple-speaker, continuously-spoken sentences. We review
previous AP-net systems and present results from a decomposition study
in which smaller networks trained to recognize subsets of phonemes are
combined into a larger network for the full signal-to-phoneme mapping
task. We show that, by using this problem decomposition methodology,
comparable performance can be obtained in significantly fewer
arithmetic operations.
%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%^%
To retrieve:
unix> ftp cheops.cis.ohio-state.edu (or 128.146.8.62)
Name: anonymous
Password: neuron
ftp> cd pub/neuroprose
ftp> binary
ftp> get pratt.aaai91.ps.Z
ftp> get pratt.ijcnn91.ps.Z
ftp> quit
unix> uncompress pratt.aaai91.ps.Z pratt.ijcnn91.ps.Z
unix> lpr pratt.aaai91.ps pratt.ijcnn91.ps
-- comp.archives file verification
cheops.cis.ohio-state.edu
-rw-r--r-- 1 3169 274 95707 Apr 3 16:11 /pub/neuroprose/pratt.aaai91.ps.Z
-rw-r--r-- 1 3169 274 73013 Apr 3 16:12 /pub/neuroprose/pratt.ijcnn91.ps.Z
found pratt-phoneme ok
cheops.cis.ohio-state.edu:/pub/neuroprose/pratt*91*