french@cogsci.indiana.edu (Bob French) (04/22/91)
Archive-name: ai/neural-nets/french-forgetting/1991-03-29
Archive: cogsci.indiana.edu:/pub/french.forgetting.ps.Z [129.79.238.6]
Original-posting-by: Bob French <french@cogsci.indiana.edu>
Reposted-by: emv@msen.com (Edward Vielmetti, MSEN)
The following brief technical report is available from the Center for
Research on Concepts and Cognition at Indiana University:
USING SEMI-DISTRIBUTED REPRESENTATIONS TO OVERCOME CATASTROPHIC
FORGETTING IN CONNECTIONIST NETWORKS
Robert M. French
Center for Research on Concepts and Cognition
Indiana University
510 North Fess
Bloomington, IN 47408
e-mail: french@cogsci.indiana.edu
In connectionist networks, newly-learned information rapidly destroys
previously-learned information unless the network is continually
retrained on the old information. This behavior, known as catastrophic
forgetting, is unacceptable both for practical purposes and as a model of
mind. This paper advances the claim that catastrophic forgetting is a
direct consequence of the overlap distributed representations and can be
reduced by reducing this overlap. It is also suggested that there is an
inevitable trade-off between generalization and forgetting. A simple
algorithm is presented that allows a standard feedforward backpropagation
network to develop "semi-distributed representations", thereby
significantly reducing the problem of catastrophic forgetting.
TO OBTAIN A COPY OF THIS PAPER:
unix>ftp cogsci.indiana.edu (or ftp 129.79.238.6)
ftp>user: anonymous
ftp>passwd: ident
ftp>cd pub
ftp>binary
ftp>get french.forgetting.ps.Z
ftp>quit
unix>uncompress french.forgetting.ps.Z
unix>lpr -P(your laser printer) french.forgetting.ps
Hard copies may be requested by sending e-mail to:
french@cogsci.indiana.edu
or by writing directly to C.R.C.C. at the address indicated above.
-- comp.archives file verification
cogsci.indiana.edu
-rw-r--r-- 1 7962 0 276137 Mar 30 02:05 /pub/french.forgetting.ps.Z
found french-forgetting ok
cogsci.indiana.edu:/pub/french.forgetting.ps.Z