neuron-request@HPLABS.HP.COM ("Neuron-Digest Moderator Peter Marvit") (03/20/89)
Neuron Digest Friday, 17 Mar 1989
Volume 5 : Issue 14
Today's Topics:
Re: PDP programs not working for large nets?
Results of Image Compression Survey
Image Compression : follow-on
Re: PDP programs on the Macintosh
neural net market analysis
Neural Net Stack for the Mac
Re: Neuron Digest V5 #12
Traveling Salesmen and Rubber Bands
Mac II Neural Net Program
Need test data for a Neural Net implementation
Submission #1: Comparability of Network Efficiency
PDP volume 3: source for bug fixes?
Re: PDP volume 3: source for bug fixes?
Send submissions, questions, address maintenance and requests for old issues to
"neuron-request@hplabs.hp.com" or "{any backbone,uunet}!hplabs!neuron-request"
ARPANET users can get old issues via ftp from hplpm.hpl.hp.com (15.255.16.205).
------------------------------------------------------------
Subject: Re: PDP programs not working for large nets?
From: wlp@calmasd.Prime.COM (Walter L. Peterson, Jr.)
Organization: Prime-Calma, San Diego R&D, Object and Data Management Group
Date: Thu, 23 Feb 89 16:52:26 +0000
[[ Editor's Note: In addition to these "words of wisdom", readers might want
to check out the last submission in this Digest which is the latest
compilation of all corrections to PDP Vol. 3 (Rumelhart & McClelland's book)
and the related programs. -PM ]]
In article <2730@usceast.UUCP>, fenimore@usceast.UUCP (Fred Fenimore) writes:
>
> [stuff deleted] ...
> part of the course, we were to implement some type of project using one
> of the simulators availible. What we found with ours was that if you
> use BP or PA, then you cannot use the block commands in the .net file.
> We tried it on a Vax 11/725 and a Apollo. Both machines gave either
> a segmentation fault or out of memory error. We spent some time looking
> in the various files to see if we could find the error and to confirm
> that it was a real bug in the code or what. The semester ended with
> no results so I gave up and coded the project in C.
> ... [stuff deleted]
Since there have been several questions about porting the PDP code lately,
I'll post this rather than e-mailing it.
The obvious first question is: are you certain that you declared the
block(s) correctly? Getting things out of order could cause the program to
attempt to allocate 0 bytes. e.g. if you take the XOR.NET and give it :
%r 2 2 2 0
%r 4 1 2 2
rather than :
%r 2 2 0 2
%r 4 1 2 2
the incorrect definition of the sending level of the first block will cause
the system to attempt to allocate 0 nodes for the sending level and you will
get a run-time such as you describe. If your network definitions are
correct, then there are several other possibilities; these should be checked
anyway, since the PDP code *IS* sensitive to compiler and system
differences.
First off - the block network definitions DO work. The XOR.NET and XOR2.NET
files that are distributed with the PDP software use them for BP and there
are other network definition files that use them also. I have made networks
with over 100 nodes in 4 layers (in, out, 2 hidden), using the block
notation and have found no bugs *in the code that reads or utilizes* this
type of definition.
Note the asterix above; this emphasis indicates that I did not find bugs in
THAT part of the code, I *DID* find problems elsewhere. When I began using
the PDP code I found numerous, albeit minor, problems when I compiled,
linked and ran it using TURBO-C V2.0 under MS-DOS V3.1 .
The problem which you found seems to be the same, or close to one of the
ones which I encountered. My first attempt to run the BP program after
having re-compiled and relinked it under TURBO-C gave me the "no memory"
error. As I was using the XOR.* files that come with the code and had not
yet made any mods to the code, I knew that something was not porting
correctly. After a bit I found that the PDP code's "shells" arround calloc,
malloc and realloc allowed an input parameter of 0 to slip through; if you
try to calloc 0 bytes calloc returns NULL and the code *was* testing for
that. Having fixed that I was at least able to get started. ( Note: this
error happened soon after the copyright notice was displayed, before any
display comes up on the screen; did yours do the same ? ).
*THEN* I hit the *real* problem. I started getting Floating Point errors.
In a program that uses floats for darn near everything, that was real fun to
track down :-). ( I need to acknowledge some VERY helpful hints from Walter
Bright and Eric Raymond ). The actual problem with the PDP code when ported
to compilers and systems other than the one on which it was written ( SUN
UNIX ? ) is in the casting of floats to doubles and doubles to floats. The
culprits are at the points were there are calls to exp(x) and pow(y, x). I
don't have the code here and I don't remember off hand in what functions
these occur, but you can use grep to find them. The solution is relatively
straight forward. In those functions the return value is computed in the
return statement; change that. Add a local variable that is declared as
double, do the computations outside of the return statment, BEING VERY
CAREFUL ABOUT USING PROPER CASTING. Assign the result to the local variable
and then return the local variable . For example:
...
double foo;
...
foo = exp( < some expression > );
...
return(foo);
This simple expedient should solve your problems. Also in the functions
that use the pow(y, x) [ that is, y raised to the x ], y is ALWAYS 10, so if
your C library provides it, you might want to change this to pow10(x).
These casting problems can get nasty and can cause problems that are not
easy to track down; however, once you get them fixed the code runs just
fine. I have been able to make some rather extensive modifications to the
BP code, having gone so far as converting it to use Scott Fahlman's
"Quick-Prop" ( see "Proc. of the 1988 Connectionist Models Summer School",
Morgan-Kaufman, NY, 1988 ).
If you have the time, it might also be helpfull to convert the code from the
"old" K&R style to ANSI-C with function prototypes, but that is really not
necessary. If you have a LOT of time and you are using TURBO-C or some other
system which provides good screen IO routines, you might want to get rid of
the CURSES emulation stuff. That will eliminate some unnecessary function
calls and for long runs of large models that might help to speed things up.
Good Luck,..
Walt Peterson. Prime - Calma San Diego R&D (Object and Data Management Group)
"The opinions expressed here are my own and do not necessarily reflect those
Prime, Calma nor anyone else.
...{ucbvax|decvax}!sdcsvax!calmasd!wlp
------------------------------
Subject: Results of Image Compression Survey
From: Andrew Palfreyman <andrew@logic.NSC.COM>
Date: Tue, 28 Feb 89 06:38:24 -0800
Since some Netneters out there have expressed interest in the collation of
Image Compression papers (Digest v5.9), the received material is listed
here, with a brief review. A rather light repast (quantity, not quality)
consisted of:
1) Peter Foldiak: Adaptive network for optimal linear feature extraction.
Abstract: A network of highly interconnected linear neuron-like processing
units and a simple, local, unsupervised rule for the modification of con-
nection strengths between these units are proposed. After training the net-
work on a high (m) dimensional distribution of input vectors, the lower (n)
dimensional output becomes a projection into the subspace of the n largest
principal components (the subspace spanned by the n eigenvectors of largest
eigenvalues of the input covariance matrix) and maximize the mutual informa-
tion between the input and the output in the same way as principal component
analysis does. The purely local nature of the synaptic modification rule
(simple Hebbian and anti-Hebbian) makes the implementation of the network
easier, faster and biologically more plausible than rules depending on error
propagation.
>From sun!NSS.CS.UCL.AC.UK!PF103%phoenix.cambridge.ac.uk
Peter Foldiak
Physiological Laboratory
Downing Street
Cambridge CB2 3EG
England
2a) G.W. Cottrell, P. Munro, D. Zipser, "Image Compression By Back Propagation:
An Example of Extensional Programming", Feb. 1987, ICS Report 8702,
(to be published in N.E. Sharkey (Ed.), Models of Cognition, Norwood, N.J.)
Reprint requests to : Institute for Cognitive Science, C-105; UC San Diego,
La Jolla, CA 92093.
2b) G.W. Cottrell, "Analysis of Image Compression by Back Propagation" (cf. 2a)
3) Terence D. Sanger, "Optimal Unsupervised Learning in a Single-Layer
Linear Feedforward Neural Network", MIT AI Lab., NE43-743, Cambridge,
MA 02139. TDS@wheaties.ai.mit.edu
(Since I only got #1 electronically, the abstract is included only for #1.
I have as yet not received the full paper).
All three papers are strikingly similar. They all deal with the "encoding
problem". If there is an "odd man out", it is 2a,b), in that 3) - and I
surmise 1) - build upon it and I think subsume its results.
The Sanger 3) paper is highly germane; he seems to have defined a method
whereby maximal information preservation occurs across one layer, using
only linear elements, and a purely local update structure. The learned
matrix of weights becomes (row-wise) the eigenvectors of the input
autocorrelation. He demonstrates compression on 8-bit grey-scale photos
of about 23:1, good performance on textual segmentation, and interesting
correlations with the receptive fields found in primates, when he trains
with random noise. This paper builds upon (e.g.) the Linsker article
in Computer, 1988. It offers a new way to look at representations in hidden
units (which I've not yet seen so clearly expressed), and allows easy
selection of any desired compression level by selectively ignoring eigenvectors
of lower eigenvalue (ordering is a property of the learning algorithm), and
by the selective quantisation of the individual eigenvector outputs.
The techniques used include Oja learning, Gram-Schmidt orthogonalisation,
and the Karhunen-Loeve transformation.
Future work will include explorations of multiple layers, and of non-linear
activation functions. Highly relevant is his comparitive data with respect
to (cf. #2) self-supervised backprop, where numerous criteria show GHA
("Generalised Hebbian Algorithm") to be superior. These criteria include:
- performance in noise
- ability to selectively quantise
- training time
- dependence upon training order
- dependence upon initial weight values
- uniqueness of solution
- interpretation of solution as representing significant features
This should be provocative stuff for ardent backprop afficionados! Bear in
mind, however, that this (minimally) specifically addresses the encoding
problem. Conversely, extraction of a significance-ordered, self-uncorrelated
feature set is a quite general and useful analysis process for ANY dataset.
============================================================================
***** NOTE !! USE ONLY THESE (NOT MESSAGE HEADER) FOR E-MAIL REPLY *****
DOMAIN: andrew@logic.sc.nsc.com
ARPA: nsc!logic!andrew@sun.com
USENET: ...{amdahl,decwrl,hplabs,pyramid,sun}!nsc!logic!andrew
Andrew Palfreyman 408-721-4788 work
National Semiconductor MS D3969 408-247-0145 home
2900 Semiconductor Dr.
P.O. Box 58090 there's many a slip
Santa Clara, CA 95052-8090 'twixt cup and lip
------------------------------
Subject: Image Compression : follow-on
From: Andrew Palfreyman <andrew@logic.NSC.COM>
Date: Wed, 01 Mar 89 11:16:33 -0800
I have just received the second article from Terence Sanger on compression
using GHA (Generalised Hebbian Algorithm), and he has now explored, as
promised, performance with nonlinearities and multiple layers. Results are:
1. He achieves equivalent (or better) convergence than backprop in a
multilayer net, using an output nonlinearity of rectification at zero
threshold. Since training time will scale "at most linearly" with the
number of layers, this is "an encouragement" to apply GHA to solve
other problems (heretofore addressed chiefly by backprop systems).
2. The receptive field results for nonlinear GHA are similar to the linear
case, but include (mostly) double the number of states, since he now
obtains the complementary patterns (off <-> on interchanged).
(I find it mysterious that random noise training produces orientation-
selective receptive fields spontaneously; what's the connection between
eigenvectors of an input autocorrelation and straight lines?
Not only are these fields similar to those found in retinal cells of
cat and monkey, but, as one goes down the list in order of decreasing
eigenvalue, resemble somewhat eigenstates of wave-functions of atoms
from quantum mechanics - perhaps a coincidental isomorphism!).
3. With a 2-layer nonlinear GHA net (hidden nonlinear, output linear), one
is able to detect stereo disparity edges. He notes that "no linear
function has this property". This therefore vindicates the nonlinear
approach.
Future directions may include investigation of classes of nonlinearities
other than rectification or sigmoids, and further analysis of nonlinearity
from a mathematical standpoint.
============================================================================
***** NOTE !! USE ONLY THESE (NOT MESSAGE HEADER) FOR E-MAIL REPLY *****
DOMAIN: andrew@logic.sc.nsc.com
ARPA: nsc!logic!andrew@sun.com
USENET: ...{amdahl,decwrl,hplabs,pyramid,sun}!nsc!logic!andrew
Andrew Palfreyman 408-721-4788 work
National Semiconductor MS D3969 408-247-0145 home
2900 Semiconductor Dr.
P.O. Box 58090 there's many a slip
Santa Clara, CA 95052-8090 'twixt cup and lip
------------------------------
Subject: Re: PDP programs on the Macintosh
From: kenf1@aplcen.apl.jhu.edu (Ken Firestone)
Organization: Johns Hopkins University
Date: Thu, 02 Mar 89 05:06:58 +0000
>Has anyone out there had any experience porting the PDP stuff to the
>Macintosh? .........
Check out the latest (April) issue of the C Users Journal. They have an
article and code for a neural net program for the Mac. Don't have any more
details as it was arriving while I was departing for class tonight. Their
address is 2120 w 25th st. ste. B Lawrence, KS 66046
Hope this helps someone.
Ken Firestone
UUCP: ...!allegra!mimsy!aplcen!kenf1
ARPA: kenf1@aplcen.apl.jhu.edu
------------------------------
Subject: neural net market analysis
From: symon@lhotse.cs.unc.edu (James Symon)
Date: Fri, 03 Mar 89 15:55:37 +0000
I am looking for information about markets for neural network hardware and
software: commmercial, industrial, research, etc. I have seen postings in
the past referring to some work done for Hecht-Nielsen's company. I would
appreciate hearing from any of those posters or anyone else who can help me
with sources and strategies.
Thanks,
Jim Symon | symon@cs.unc.edu
Computer Science Dept | {uunet, decvax}!mcnc!unc!symon
Chapel Hill, NC 27599-3175 | (919) w:962-1893 h:968-1024
***Don't use "r" or my header line address***
------------------------------
Subject: Neural Net Stack for the Mac
From: RatRunner@cup.portal.com (Geoff RatRunner Crooks)
Organization: The Portal System (TM)
Date: Sat, 04 Mar 89 01:41:46 +0000
I don't know much about PDP or nets... I read this group in the hopes of
learning something about them... anyway, just last week I picked up a
HyperCard stack which claims to do neural nets... I don't know if it does
them well or not... however, it didn't bug out. If you want, I'll see about
getting you a copy. Geoff. gcrooks@unc.bitnet
------------------------------
Subject: Re: Neuron Digest V5 #12
From: gary%cs@ucsd.edu (Gary Cottrell)
Date: Tue, 07 Mar 89 22:38:58 -0800
Geoff Hinton says:
>The potential advantage of using "encoder" networks is that the code in the
>middle can be developed without any supervision.
>
>If the output and hidden units are non-linear, the codes do NOT just span the
>same subspace as the principal components. The difference between a linear
>approach like principal components and a non-linear approach is especially
>significant if there is more than one hidden layer.
This must be taken with a grain of salt. It really depends on the problem
and how it is encoded. In our work (Cottrell, Munro & Zipser, Cognitive
Science Proceedings 1987 and Cottrell, 1988 Proceedings of the workshop on
Neural Architectures for Computer Vision, AAAI), we showed that an identity
mapping network used to map patches of an image spanned the Prinicipal
Subspace when the units were linear. When they are nonlinear [-1,1]
squashing units, and the input gray scale is linearly coded as [0,1] or
[0,.85], the solution found by the network gives a noisy representation of
the first principal component (the brightness), and shortens the rest, but
is much less noisy on the lower ones. Of course, the more the input is
mapped into the linear range of the squashing function, the more "linear"
the solution discovered by the network.
In problems like the original encoder problem, where one unit is on in the
input and output, nonlinearity is necessary to solve the problem because the
principal components are degenerate (all the eigenvalues are the same). One
can imagine that for larger patch sizes, and fewer hidden units, if one were
trying to encode something like several faces, then the nonlinearities would
again be useful. I have student currently working on this idea.
This is not to dispute Geoff's point about multiple layer nets. I haven't
looked into that (although I suggested its use for learning word meanings).
Yves Chauvin has, in his 1988 PhD thesis from the UCSD Institute for
Cognitive Science. There, he uses multi-layer encoding nets to learn
abstract "word meanings". It works quite well and matches some human data.
gary cottrell 619-534-6640
Computer Science and Engineering C-014
UCSD,
La Jolla, Ca. 92093
gary%cs@ucsd.edu (ARPA)
{ucbvax,decvax,akgua,dcdwest}!sdcsvax!gary (USENET)
gcottrell@ucsd.edu (BITNET)
------------------------------
Subject: Traveling Salesmen and Rubber Bands
From: djb@flash.bellcore.com (David J Burr)
Date: Wed, 08 Mar 89 15:37:24 -0500
I have compiled some statistics on a rubber band model for finding traveling
saleman tours and reported these at ICNN-88 (D. J. Burr, An Improved Elastic
Net Method for the Traveling Saleman Problem). The cities each "pull" on
the rubber band, changing it from a circle to a complex shape which forms a
tour. Twenty five percent of the "learning" trials converged to within 2%
of the best tour length found by the Lin-Kernighan algorithm. There were
over 700 trials performed on a set of 20 random problems from 30 to 100
cities.
D. J. Burr, 2B-397
Bellcore
445 South St.
Morristown, NJ 07960
------------------------------
Subject: Mac II Neural Net Program
From: wetter@cit-vax.Caltech.Edu (Pierce T. Wetter)
Organization: California Institute of Technology
Date: Fri, 10 Mar 89 08:25:35 +0000
I have placed an initial version of a program I have been working on for
my senior thesis in the public ftp directory on csvax.caltech.edu. This
program allows you to create a network, create the patterns, and then train
the network to recognize the patterns up to some degree of error.
Only three level, delta back-prop networks with logistic activation
functions are supported (if you didn't understand that, don't worry neither
did I a week ago.)
Some caveats: The program ONLY runs on a Mac II or greater (though maybe an
SE30 might work. Need 68020 and 68881 and color ), and currently doesn't
handle memory or file errors very well. A memory error probably won't happen
since it doesn't use that much memory, but if you try you could probably get
it to die if you set the memory allocation too low under multifinder. File
errors wont cause a crash but it wont tell you that it got one either. On
the other hand it also comes with some source code to demonstrate back-prop
and training.
Have Fun,
Pierce
P.S. You will need Stuffit to decode the file.
P.P.S By the way, the program is named Thesis.sit.hqx.
wetter@tybalt.caltech.edu or wetter@csvax.caltech.edu or pwetter@caltech.bitnet
Rule to live by #32796: Act consistently in one persona until people have
characterized your reactions, then change personas.
------------------------------
Subject: Need test data for a Neural Net implementation
From: Paul Watson <paul%research4.computer-science.manchester.ac.uk@NSS.Cs.Ucl.AC.UK>
Date: Fri, 10 Mar 89 12:24:32 +0000
I am looking at various methods of implementing Neural Networks on a
parallel distributed store machine which has been built here at Manchester
as part of the Alvey FLAGSHIP project. The main problem being addressed is
how the amount of communication can be reduced.
I have come up with a few techniques that I think will be useful in Neural
Network implementation, but have very little test data to try them on. I am
therefore looking for any real (i.e. complete with weights and/or learning
data) networks. I am particularly interested in:
1) Generalised Delta Rule nets,
2) Competitive Learning nets,
3) Hopfield nets, or
4) Boltzmann nets.
If anybody could either provide or point to such data I would be very
grateful. I am willing to accept data in any format.
Thankyou in advance.
Steve Palmer e-mail: Janet: pw@uk.ac.man.cs.ux
Dept. of Computer Science Arpa/Bitnet: pw@ux.cs.man.ac.uk
The University UUCP: ...!ukc!mucs!pw
Oxford Road
Manchester M13 9PL
UK
------------------------------
Subject: Submission #1: Comparability of Network Efficiency
From: Arnfried Ossen <tub!coma!ao@hplabs.HP.COM>
Date: Sun, 12 Mar 89 18:20:23 +0100
I am a PhD candidate in CS at Tech Univ Berlin, West Germany. I am
interested in neural network scaling issues, that is, minimizing
connections, minimizing the number of units and possibly creating subsystems
that can learn separately.
The idea is to use back propagation as learning algorithm and a somewhat
generalized version of genetic algorithms to scale the network.
In Issue 7 of Neuron Digest Dave Montana <DMONTANA%COOPER@rcca.bbn.com> writes:
> [] In addition to outperforming backpropagation [using genetic algorithms]
> on the network ....
Our results indicate, that genetic algorithms cannot outperform back
propagation in feed forward networks in general.
In order to get comparable results someone should propose an environment
structure, that is, a set of input/target vectors, complex enough to allow
such optimizations AND easy to handle and understand (as the 4-2-4 etc.
encoder were for the development of the learning algorithms), together with
a couple of sentences describing statistical properties of the data.
Who can submit such a set, possibly in a wide spread standard such as
PostScript? (e.g. a set of digitalized characters, even when used for a
statistical recognition program, such as template matching.)
Arnfried
Arnfried Ossen, Technical University of Berlin <ao@coma.UUCP>
Department of Applied Computer Science <ao@db0tui62.BITNET>
28/29 Franklin St, Berlin 1/10, FR Germany
------------------------------
Subject: PDP volume 3: source for bug fixes?
From: pratt@paul.rutgers.edu (Lorien Y. Pratt)
Organization: Rutgers Univ., New Brunswick, N.J.
Date: Mon, 13 Mar 89 20:47:22 +0000
[[ Editor's note: See also the following message! -PM ]]
I am working through the ``cs'' program in PDP, volume 3, and I have found,
in the example on page 67, that pattern files are not being read in
correctly. Every entry is read as a 1.
Can anyone tell me an email address for someone I should contact about
obtaining patches for the PDP3 code in general, and this bug in particular?
I see that there's a snailmail address at MIT press, but I'd like to avoid
receiving bug fixes on paper, if at all possible!
Thanks in advance for your help,
Lorien Y. Pratt Computer Science Department
pratt@paul.rutgers.edu Rutgers University
Hill Center
(201) 932-4634 New Brunswick, NJ 08901
------------------------------
Subject: Re: PDP volume 3: source for bug fixes?
From: Marco Zagha <MARCOZ.BOLTZ.CS.CMU.EDU!marcoz@PT.CS.CMU.EDU>
Organization: Carnegie-Mellon University, CS/RI
Date: 16 Mar 89 19:21:20 +0000
[[ Editor's Note: The Digestifying software I use altered the "diff" listing
in an obvious way below (prepending "- " to any line beginning with "--").
Does anyone have the corrected software available for ftp? -PM ]]
Here is a list of bug fixed for the PDP volume 3 software from Jay
McClelland (posted with permission). DO NOT CONTACT ME WITH FURTHER
BUGS!!!! I am not maintaining the software.
== Marco (marcoz@cs.cmu.edu)
CHANGES TO PDP SOFTWARE AND HANDBOOK
For the second printing of the book "Explorations in parallel distributed
processing: A handbook of models, programs and exercises", by J. L.
McClelland and D. E. Rumelhart, a small number of changes to the PDP
software have been made, and version 1.1 of the software has been
constructed.
The changes to the software are all either bug fixes or minor cosmetic
changes. The most important fixes resolve a set of related problems with
constraints on weights in the bp program: In the old version, chaos could
result when weights were linked together if the network contained more than
100 units, or if more than 100 weights were linked together or constrained
to be positive or negative. The necessary changes are all in the file
weights.c, and are listed below.
In addition to the changes to the sources, a few errors in the handbook text
have been detected, and many of these errors have been corrected in the
second printing. The bp template file cas.tem has also been changed to
allow logging of activations to occur as described in the handbook text.
Here is a description of the changes to the source files, followed by the
results of running the unix diff program on the old and new versions. Then
a diff of the old and new versions of bp/cas.tem is provided. Following
this, the errors in the text of the handbook are described.
At the very end, fixes for new problems that have arisen with Version 1.1
are described.
DESCRIPTION OF CHANGES TO SOURCE FILES
in all .c files:
removed "(void)" cast on all calls to install_var and
install_command. (these functions are now declared as void
explicitly, see below).
bp.c:
in logistic (lines 217,220):
replaced constant 16.0 with 15.935773
(15.935773 is the number whose logistic is equal
to .99999988, which is retured if 15.935773 is exceeded)
command.c:
declared install_command as void.
in do_command:
the second argument to do_command was usually passed
as a pointer but interpreted by do_command as an integer.
This can lead to problems when pointers and integers have
different sizes. To rectify the problem, do_command now
interprets its second argument as a pointer, than converts
it explicitly to an integer, storing the result in the
integer variable Currrent_Menu, which is then used inside
the body of the do_command function.
in contin_test, do_comfile, and file_error:
cast second argument to do_command as (int *)
in do_comfile:
fixed bug that caused programs to crash
if a return was entered in response to "How many times?"
query.
command.h:
declared install_command as void.
general.c:
in emalloc and erealloc:
caused the program to exit when it runs out of
memory rather than returning a null pointer to
the caller.
main.c:
changed version number to 1.1
deleted "stuct Variable *install_var ();" declaration.
in main:
cast second argument to do_command as (int *)
both times it is called.
variable.c:
declared install_var as void and commented out return statement
in this function.
variable.h:
declared install_var as void here too.
weights.c:
in define_network (line 620):
fixed program to null constraint vectors up to
constraints[i].max rather than nunits.
above enlarge_constraints (old line 787):
added #define CON_INCR 100
in enlarge_constraints:
replaced 100 with CON_INCR throughout
old lines 789,796, replaced '=' with '==' in
conditionals.
added code for nulling fresh portions of
positive_constraints,
negative_constraints,
constraints[i].cvec,ivec
DIFF RESULTS
Here follows a diff of versions 1.0 and 1.1. This diff was performed after
all of the "(void)" casts on calls to install_var and install_command were
deleted from version 1.0. Only files with changes are listed.
The diff tells what should be done to the old version to convert it into the
new. Lines beginning with < come from version 1.0, lines beginning with >
come from version 1.1.
====================
bp.c
====================
217c217
< if (x > 16.0)
- ---
> if (x > 15.935773)
220c220
< if (x < -16.0)
- ---
> if (x < -15.935773)
====================
command.c
====================
50c50
< int current_menu;
- ---
> int *current_menu;
60c60
< Current_Menu = current_menu;
- ---
> Current_Menu = (int) current_menu;
62c62
< do_help(str,current_menu);
- ---
> do_help(str,Current_Menu);
68c68
< if (current_menu == BASEMENU) return(POP);
- ---
> if (Current_Menu == BASEMENU) return(POP);
75c75
< if ((!Command[i].menutype) || (Command[i].menutype == current_menu)) {
- ---
> if ((!Command[i].menutype) || (Command[i].menutype == Current_Menu)) {
87c87
< (Command[i].menutype == current_menu)) {
- ---
> (Command[i].menutype == Current_Menu)) {
101c101
< if( current_menu == DISPLAYMENU) {
- ---
> if( Current_Menu == DISPLAYMENU) {
104c104
< if ((Command[i].menutype == current_menu)) {
- ---
> if ((Command[i].menutype == Current_Menu)) {
123c123
< (Command[i].menutype == current_menu)) {
- ---
> (Command[i].menutype == Current_Menu)) {
260c260,264
< install_command(str, func, menu, intp)
- ---
> /* declare this as void so we don't have to cast all calls to install_command
> abh - 2/15/88
> */
>
> void install_command(str, func, menu, intp)
422c426
< if (do_command(subprompt, BASEMENU) == POP) break;
- ---
> if (do_command(subprompt, (int *) BASEMENU) == POP) break;
435,436c439,440
< int echeck;
< int nreps,i;
- ---
> int echeck,i;
> int nreps = 0;
449d452
< nreps = 1;
451c454,458
< sscanf(str,"%d",&nreps);
- ---
> echeck = 0;
> if (str) echeck = sscanf(str,"%d",&nreps);
> if (echeck == 0) {
> return(put_error("Integer argument missing in do command."));
> }
456c463
< if (do_command(Prompt, BASEMENU) == BREAK) {
- ---
> if (do_command(Prompt, (int *) BASEMENU) == BREAK) {
492c499
< if (do_command(subprompt, BASEMENU) == BREAK) break;
- ---
> if (do_command(subprompt, (int *) BASEMENU) == BREAK) break;
====================
command.h
====================
56c56
< int install_command ();
- ---
> void install_command ();
====================
general.c
====================
79c79
< if (p == 0)
- ---
> if (p == 0) {
80a81,83
> end_display();
> exit(0);
> }
94c97
< if (p == 0)
- ---
> if (p == 0) {
95a99,101
> end_display();
> exit(0);
> }
104c110
< if (p == 0)
- ---
> if (p == 0) {
105a112,114
> end_display();
> exit(0);
> }
====================
main.c
====================
29c29
< char version[10] = "1.0";
- ---
> char version[10] = "1.1";
36d35
< struct Variable *install_var ();
74c73
< do_command(Prompt, BASEMENU);
- ---
> do_command(Prompt, (int *) BASEMENU);
94c93
< do_command(Prompt, BASEMENU);
- ---
> do_command(Prompt, (int *) BASEMENU);
====================
variable.c
====================
55c55
<
- ---
> /* we replace this:
57c57,60
< install_var (s, t, varptr, max_x, max_y, menutype)
- ---
> with void to avoid the hassle of casting to void on each call
> abh - 2/15/88
> */
> void install_var (s, t, varptr, max_x, max_y, menutype)
84a88
> /*
85a90
> */
====================
variable.h
====================
31c31
< struct Variable *install_var ();
- ---
> void install_var ();
====================
weights.c
====================
620c620
< for (j = 0; j < nunits; j++) {
- ---
> for (j = 0; j < constraints[i].max; j++) {
787a788,789
> #define CON_INCR 100 /* increment in size of constriant */
>
789,790c791,794
< if (con_index = ENLARGE_POS) {
< maxpos += 100;
- ---
> int j;
>
> if (con_index == ENLARGE_POS) {
> maxpos += CON_INCR;
793c797
< (unsigned int) ((maxpos - 100) * sizeof(float *)),
- ---
> (unsigned int) ((maxpos - CON_INCR) * sizeof(float *)),
794a799,801
> for (j = maxpos - CON_INCR; j < maxpos; j++) {
> positive_constraints[j] = NULL;
> }
796,797c803,804
< else if (con_index = ENLARGE_NEG) {
< maxneg += 100;
- ---
> else if (con_index == ENLARGE_NEG) {
> maxneg += CON_INCR;
800c807
< (unsigned int) ((maxneg -100) * sizeof (float *)),
- ---
> (unsigned int) ((maxneg -CON_INCR) * sizeof (float *)),
801a809,811
> for (j = maxneg - CON_INCR; j < maxneg; j++) {
> negative_constraints[j] = NULL;
> }
804c814
< constraints[con_index].max += 100;
- ---
> constraints[con_index].max += CON_INCR;
808c818
< ((constraints[con_index].max - 100) * sizeof(float *)),
- ---
> ((constraints[con_index].max - CON_INCR) * sizeof(float *)),
814c824
< ((constraints[con_index].max - 100) * sizeof(float *)),
- ---
> ((constraints[con_index].max - CON_INCR) * sizeof(float *)),
816a827,831
> for (j = constraints[con_index].max - CON_INCR;
> j < constraints[con_index].max; j++) {
> constraints[con_index].cvec[j] = NULL;
> constraints[con_index].ivec[j] = NULL;
> }
DIFFERENCES BETWEEN OLD AND NEW VERSIONS OF bp/cas.tem
All of the changes apply only to the dlevel variable: dlevels of epochno and
tss are set to 2 and dlevels of cpname, cycleno, and output are set to 1.
Once again, the diff indicates how to change the old version into the new
version.
15,17c15,17
< epochno variable 1 $ n epochno 5 1.0
< tss floatvar 1 $ n tss 7 1.0
< cpname variable 2 $ 5 cpname -5 1.0
- ---
> epochno variable 2 $ n epochno 5 1.0
> tss floatvar 2 $ n tss 7 1.0
> cpname variable 1 $ 5 cpname -5 1.0
19c19
< cycleno variable 2 $ n cycleno 5 1.0
- ---
> cycleno variable 1 $ n cycleno 5 1.0
27c27
< output vector 2 $ n activation v 3 100.0 2 3
- ---
> output vector 1 $ n activation v 3 100.0 2 3
ERRATA BY PAGE NUMBER IN THE HANDBOOK [Those marked with * have been
corrected in the second printing]
Page 51: In Equation 2, each pair of units contributes only one time to the
first summation (w sub ij is assumed equal to w sub ji). The total goodness
is not the sum of the goodnesses of the individual units. It is best to
think of the goodness of unit i as the set of terms in the total goodness to
which the activation of unit i contribues.
*Page 67, lines 6 and 7: Two occurrences of "display/ ipattern" should read
"display/ env".
*Page 85, 2nd line from bottom: The word "two" should be "four".
Page 148, first lines 6 and 10: In both cases the word "left" should be
replaced with the word "right".
*Page 148, 12th line from bottom: The first word on the line, "weights",
should be "bias".
*Page 296, 8th line of Q.4.1.1: The word "input" should be "output".
PROBLEMS IN VERSION 1.1 OF THE SOFTWARE
The file jets.str in the cs directory uses the names 20s 30s and 40s for
three of the units. The names should be changed (by editing the file) to
in20s in30s and in40s respectively. As is these units names cannot be used
because the program interprets them as unit numbers. NOTE: This only
applies to the file jets.str in the cs directory. The file jets.str in the
iac directory is ok as is.
------------------------------
End of Neurons Digest
*********************