page%swap@Sun.COM (Bob Page) (07/13/89)
Submitted-by: jal@pan.cs.wayne.edu (Jason Leigh) Posting-number: Volume 89, Issue 168 Archive-name: applications/anncs.1 The ANNCS is a set of functions that allows quick and easy construction of neural networks. The system is very simple and as a result much can be enhanced. Eventually I may attach a graphical interface that allows you to visually create neural networks and convert the images to ANNCS Lisp function calls. This is by no means a competitor for the Rochester Connectionist Simulator but hay, what the heck, it's at least running on my favorite computer! # This is a shell archive. # Remove anything above and including the cut line. # Then run the rest of the file through 'sh'. # Unpacked files will be owned by you and have default permissions. #----cut here-----cut here-----cut here-----cut here----# #!/bin/sh # shar: SHell ARchive # Run the following text through 'sh' to create: # and.lsp # neural.lsp # neural.readme # xor.lsp # xor2.lsp # This is archive 1 of a 1-part kit. # This archive created: Wed Jul 12 19:38:16 1989 echo "extracting and.lsp" sed 's/^X//' << \SHAR_EOF > and.lsp X;;; @@FILE and.lsp X;;; Copyright (C) 1989 By Jason Leigh 5/5/89 X X;;; Redistribute this program at will. If you decide to use this program X;;; as part of some commercial package you are developing, by all means X;;; proceed, all I ask is that you give me a small amount of credit for it. X X;;; Creating an AND network using the functions in neural.lsp X X;;; The activation function is Relay and a diagram of this X;;; network can be found in: X;;; Bernard Widrow, IEEE Computer, March 1988, p.33 X X(ClearNetwork) X X;;; Select the Relay function as the output function. X(defun OutputFunction (sum threshold) X (Relay sum threshold)) X X(CreateInputLine 'i1) ;Labelled as i1 X(CreateInputLine 'i2) ;Labelled as i2 X(CreateNeuron 'n1 0) ;Labelled as n1 X ;This is the output neuron. X(CreateInputLine 'i3) ;Labelled as i3 X X(SetInputLine 'i3 1) ;Input i3 is permanently set to 1. X(ConnectInputToNeuron 'i1 'n1 1) ;Connect input 1 to neuron 1 X(ConnectInputToNeuron 'i2 'n1 1) ;Connect input 2 to neuron 1 X(ConnectInputToNeuron 'i3 'n1 -1.5) ;Connect input 3 to neuron 1 X X X;;; Now show the four outcomes of a two input AND function. X X(printf "Input: 1 1\n") X(SetInputLine 'i1 1) ; Set inputs 1 and 2 to 1 X(SetInputLine 'i2 1) X(UpdateNetwork 10 '(or1)) ; Update the system. X(printf "Output: " (NeuronOutput 'n1) "\n") X X(printf "Input: -1 1\n") X(SetInputLine 'i1 -1) ; Set inputs 1 and 2 to 1,-1 X(SetInputLine 'i2 1) X(UpdateNetwork 10 '(or1)) ; Update the system. X(printf "Output: " (NeuronOutput 'n1) "\n") X X(printf "Input: -1 -1\n") X(SetInputLine 'i1 -1) ; Set inputs 1 and 2 to -1,-1 X(SetInputLine 'i2 -1) X(UpdateNetwork 10 '(or1)) ; Update the system. X(printf "Output: " (NeuronOutput 'n1) "\n") X X(printf "Input: 1 -1\n") X(SetInputLine 'i1 1) ; Set inputs 1 and 2 to 1,-1 X(SetInputLine 'i2 -1) X(UpdateNetwork 10 '(or1)) ; Update the system. X(printf "Output: " (NeuronOutput 'n1) "\n") SHAR_EOF echo "extracting neural.lsp" sed 's/^X//' << \SHAR_EOF > neural.lsp X;;; @@FILE neural.lsp X;;; Neural Network Construction Set (Vr 1.0 : 5/5/89) X;;; Copyright (C) 1989 by Jason Leigh 5/5/89 X X;;; Comments, questions and donations may be sent to: X;;; Jason Leigh X;;; 538 Mackenzie Hall, Wayne State University, Detroit, MI 48202. X;;; email: jal@zeus.cs.wayne.edu X X;;; Redistribute this program at will. If you decide to use this program X;;; as part of some commercial package you are developing, by all means X;;; proceed, all I ask is that you give me a small amount of credit for it. X X;;; This was written as a practice session in using XLISP objects. X;;; Neurons are ideally suited to object oriented programming since X;;; each neuron must contain its own current set of attributes. X;;; See the files: and.lsp, xor.lsp and xor2.lsp for implementations of X;;; AND and XOR functions using neural networks. It is the best X;;; way to learn how to use the PUBLIC functions below. X X;;; Fool-proof code is at a minimum so if you don't use these X;;; functions just the way they are supposed to be used, you'll X;;; get a lot of error messages by XLISP. X X;;; @@SECTION printf X;;; X;;; @@REQ X;;; parameters X;;; Any LISP s-expression to be printed. Allows multiple expressions. X;;; e.g. (printf "hello there" 1 2 3 "wow \n") X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Works like printf in C or print in BASIC allowing multiple X;;; expressions to be printed on one line. Far more convenient than X;;; prin1, princ etc... X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X;;; PUBLIC X(defun printf (&rest parameters) X (dolist (item parameters) X (princ item))) X X;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; X X;;; Network is an a-list of neuron number and neuron object. X;;; PRIVATE X(setq *network* ()) X X;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; X;;; CLASS DEFINITIONS FOR NEURONS X X;;; Class of a neuron contains the instance variables: X;;; threshold, sum and list of input connections. X;;; threshold is the threshold value associated with each neuron X;;; to determine whether the neuron should fire or not. This is X;;; usually determined empirically. X;;; sum is the scalar product of all the inputs and connection strengths X;;; entering a neuron. X;;; listOfInConnections is an a-list of connections that go into a X;;; particular neuron and the connection strength associated with the X;;; connection. X X;;; PRIVATE X(setq NeuronClass (send Class :new X '(threshold sum listOfInConnections))) X X;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; X;;; METHODS X X;;; @@SECTION NeuronClass:output X;;; X;;; @@REQ X;;; None X;;; @@END-REQ X;;; X;;; @@FUNC X;;; This method X;;; calls the appropriate function to compute the output of a X;;; neuron. Uses the sum and threshold instance variables X;;; of the given object. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; There are several possible X;;; output functions that may be used. These are listed as: X;;; Relay, Linear and Sigmoid. X;;; @@END-STATUS X X(send NeuronClass :answer :output () X '((if (null listOfInConnections) sum X (OutputFunction sum threshold)))) X X;;; @@SECTION Relay X;;; X;;; @@REQ X;;; sum X;;; scalar product of all inputs and weights. X;;; threshold X;;; threshold of neuron. X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Output function of a neuron. The relay function X;;; sends a 1 if the sum >= the threshold and -1 otherwise. X;;; This conforms to the ADALINE. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X(defun Relay (sum threshold) X (if (>= sum threshold) 1 -1)) X X;;; @@SECTION Linear X;;; X;;; @@REQ X;;; sum X;;; scalar product of all inputs and weights. X;;; threshold X;;; threshold of neuron. X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Output function of a neuron. The linear function X;;; does not use the threshold but rather sends the sum directly X;;; as output; I believe this is used in the Neocognitron model. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X(defun Linear (sum threshold) X sum) X X;;; @@SECTION Sigmoid X;;; X;;; @@REQ X;;; sum X;;; scalar product of all inputs and weights. X;;; threshold X;;; threshold of neuron. X;;; X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Output function of a neuron. The sigmoidal function X;;; goes toward 0 for large -ve numbers and 1 for large +ve numbers X;;; This is suitable for the Back-propagation model where a X;;; semi-linear continuous output function is needed. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X(defun Sigmoid (sum threshold) X (/ 1.0 (+ 1.0 (exp (- 0.0 (+ (float sum) threshold)))))) X X;;; @@SECTION NeuronClass:clear X;;; X;;; @@REQ X;;; None. X;;; @@END-REQ X;;; X;;; @@FUNC X;;; This method sets the SUM instance variable of an object to zero. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X(send NeuronClass :answer :clear () '((setq sum 0))) X X;;; @@SECTION NeuronClass:setThreshold X;;; X;;; @@REQ X;;; value X;;; value to set the threshold. X;;; @@END-REQ X;;; X;;; @@FUNC X;;; This method changes the current threshold value of a X;;; neuron. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X(send NeuronClass :answer :setThreshold '(value) '((setq threshold value))) X X;;; @@SECTION NeuronClass:addConnection X;;; X;;; @@REQ X;;; neuronId X;;; Unique identifier that specifies an existing neuron. X;;; weight X;;; Connection strength. X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Adds the neuron's Id and the connection strength between the X;;; neuron and the current neuron to the current neuron's list X;;; of input connections. X;;; The list of in connections is an a-list of neuron number and X;;; weight. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X(send NeuronClass :answer :addConnection '(neuronId weight) X X '((setq listOfInConnections X (append listOfInConnections X `((,neuronId ,weight)))))) X X;;; @@SECTION NeuronClass:getConnection X;;; X;;; @@REQ X;;; srcNeuronId X;;; neuron Id X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Retrieves the neuronId and weight from the X;;; list of connections associated with a neuron object. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; X;;; @@END-STATUS X X(send NeuronClass :answer :getConnection '(srcNeuronId) X '((assoc srcNeuronId listOfInConnections))) X X;;; @@SECTION NeuronClass:removeConnection X;;; X;;; @@REQ X;;; neuronIdToRemove X;;; The neuron Id to search for in the list of input connections X;;; associated with a neuron object. X;;; destNeuronId X;;; The neuron object to remove it from. X;;; Returns T if connection to remove exists; () otherwise. X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Removes a connection between two neurons. X;;; This simply requires going to the destination neuron and removing X;;; the entry in the a-list ListOfInConnections. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X(send NeuronClass :answer :removeConnection '(neuronIdToRemove destNeuronId) X '((let* ((connection (send (GetNeuron destNeuronId) X :getConnection neuronIdToRemove)) X (remaining (Delete connection X listOfInConnections :test 'equal))) X (if (null connection) X (progn X (printf "No connection: " neuronIdToRemove "->" X destNeuronId X " exists.\n") X ()) X (progn X (setq listOfInConnections remaining) X T))))) X X;;; @@SECTION NeuronClass:setSum X;;; X;;; @@REQ X;;; value X;;; Value to set the sum. X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Set the SUM instance variable of a neuron object to a particular X;;; value. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X(send NeuronClass :answer :setSum '(value) X '((setq sum value))) X X;;; @@SECTION NeuronClass:getSum X;;; X;;; @@REQ X;;; None X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Get the SUM instance variable of a neuron object. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X(send NeuronClass :answer :getSum () X '(sum)) X X;;; @@SECTION NeuronClass:update X;;; X;;; @@REQ X;;; None. X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Method to update the sum in a neuron based on connections X;;; from inputs of other neurons. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X(send NeuronClass :answer :update () X '( X (if (not (null listOfInConnections)) X (progn X (setq sum 0) X (dolist (connection listOfInConnections) X (setq sum (+ sum X (* (NeuronOutput (car connection)) X (cadr connection))))))))) X X X;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; X;;; HIGH LEVEL INTERFACES TO METHODS X X;;; @@SECTION ConnectNeuron X;;; X;;; @@REQ X;;; srcNeuron X;;; destNeuron X;;; weight X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Form a connection between the srcNeuron and destNeuron X;;; that has a connection strength given by weight. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X;;; Connect a source neuron to a destination neuron using a weight. X;;; PUBLIC X(defun ConnectNeuron (srcNeuron destNeuron weight) X (send (GetNeuron destNeuron) :addConnection srcNeuron weight)) X X;;; @@SECTION CreateNeuron X;;; X;;; @@REQ X;;; neuronId X;;; Unique Id to give to neuron. X;;; threshold X;;; Threshold to set in the neuron. X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Create a new neuron that is identified by neuronId. X;;; If you have a friend who you think has the intelligence of a X;;; single neuron, use his/her name! X;;; X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X;;; PUBLIC X(defun CreateNeuron (neuronId threshold) X (if (GetNeuron neuronId) X (printf "Identifier: " neuronId " already exists.\n") X (progn X (setq *network* X (append *network* `((,neuronId ,(MakeNeuron))))) X (send (GetNeuron neuronId) :setThreshold threshold) X (send (GetNeuron neuronId) :clear) X neuronId))) X X;;; @@SECTION MakeNeuron X;;; X;;; @@REQ X;;; None X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Invoke the :new method to create a new neuron object of class: X;;; NeuronClass. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X;;; Make a neuron by creating an object from the neuron class. X;;; PRIVATE X(defun MakeNeuron () X (send NeuronClass :new)) X X;;; @@SECTION GetNeuron X;;; X;;; @@REQ X;;; neuronId X;;; Id of neuron X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Get a neuron from the network. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X;;; PUBLIC X(defun GetNeuron (neuronId) X (cadr (assoc neuronId *network*))) X X;;; @@SECTION Examine X;;; X;;; @@REQ X;;; neuronId X;;; Neuron Id X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Examine the inner parameters of a neuron. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X;;; Examine the inner instance variables of a neuron. X;;; PUBLIC X(defun Examine (neuronId) X (send (GetNeuron neuronId) :show) X (printf "Output: " (NeuronOutput neuronId) "\n")) X X;;; @@SECTION CreateInputLine X;;; X;;; @@REQ X;;; neuronId X;;; Give a unique name to inputs into the network. X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Actually an input line is made a neuron with no threshold X;;; so that whatever goes into the neuron comes straight back out. X;;; This saves having to declare an input class. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X;;; PUBLIC X(defun CreateInputLine (neuronId) X (CreateNeuron neuronId 0)) X X;;; @@SECTION SetInputLine X;;; X;;; @@REQ X;;; neuronId X;;; Input line Id. X;;; inputValue X;;; Value to set input value to. X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Set the output sum of a neuron to always equal some value. X;;; This disguises the fact that an input line is really a neuron X;;; with only a set output. X;;; X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X;;; PUBLIC X(defun SetInputLine (neuronId inputValue) X (send (GetNeuron neuronId) :setSum inputValue)) X X;;; @@SECTION ConnectInputToNeuron X;;; X;;; @@REQ X;;; inputNum X;;; Input line Id X;;; destNeuron X;;; Destination neuron Id X;;; weight X;;; Strength of connection X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Connect an input line to a neuron. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X;;; PUBLIC X(defun ConnectInputToNeuron (inputNum destNeuron weight) X (ConnectNeuron inputNum destNeuron weight)) X X;;; @@SECTION UpdateNeuron X;;; X;;; @@REQ X;;; neuronId X;;; neuron id. X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Update a neuron by sending update message to neuron object. X;;; This recalculates the SUM. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X;;; PRIVATE X(defun UpdateNeuron (neuronId) X (send (GetNeuron neuronId) :update)) X X;;; @@SECTION UpdateNetwork X;;; X;;; @@REQ X;;; updateLimit X;;; After how many updates through the network should X;;; results be reported. X;;; neuronList X;;; List of neuron Id's to examine when updateLimit is reached. X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Update the system by updating every neuron. This occurs X;;; repeatedly until the network outputs have stablized. X;;; Since some networks may take a long time to produce results, X;;; setting an updateLimit and neuronList will allow intermediate X;;; reporting of results while the network is updating. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X;;; PUBLIC X(defun UpdateNetwork (updateLimit neuronList) X (let ((change ()) X (old ()) X (updates 0)) X (loop X (setq change ()) X (dolist (eachNeuronInNetwork *network*) X (setq old (NeuronOutput (car eachNeuronInNetwork))) X (UpdateNeuron (car eachNeuronInNetwork)) X (if (not (equal old X (NeuronOutput (car eachNeuronInNetwork)))) X (setq change T))) X (setq updates (+ updates 1)) X (if (> updates updateLimit) X (progn X (setq updates 0) X (ReportNeurons neuronList))) X (if (null change) (return))))) X X;;; @@SECTION ReportNeurons X;;; X;;; @@REQ X;;; neuronList X;;; List of neurons to examine. X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Examine a list of neurons. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X;;; PUBLIC X(defun ReportNeurons (neuronList) X (cond ((null neuronList) T) X (T (progn X (printf "\n***** Neuron Id: " (car neuronList) "\n") X (Examine (car neuronList)) X (printf "\n") X (ReportNeurons (cdr neuronList)))))) X X;;; @@SECTION NeuronOutput X;;; X;;; @@REQ X;;; neuronId X;;; Neuron Id X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Return the output of a neuron. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X;;; PUBLIC X(defun NeuronOutput (neuronId) X (send (GetNeuron neuronId) :output)) X X;;; @@SECTION ClearNetwork X;;; X;;; @@REQ X;;; None X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Simply sets *network* to null thus losing all neurons defined so far. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X;;; Clear network of all neurons X;;; PUBLIC X(defun ClearNetwork () X (setq *network* ())) X X;;; @@SECTION ChangeWeight X;;; X;;; @@REQ X;;; srcNeuron X;;; destNeuron X;;; newWeight X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Change the connection strength from srcNeuron to destNeuron to the X;;; new value given by newWeight. This function was written so that X;;; feedback networks that learn by modifying their weights based X;;; on a comparison of target and actual results, can be designed. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X;;; PUBLIC X(defun ChangeWeight (srcNeuron destNeuron newWeight) X (if (send (GetNeuron destNeuron) X :removeConnection srcNeuron destNeuron) X (ConnectNeuron srcNeuron destNeuron newWeight))) X X X;;; @@SECTION GetWeight X;;; X;;; @@REQ X;;; srcNeuron X;;; destNeuron X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Find the connection strength between srcNeuron and destNeuron. X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X;;; PUBLIC X(defun GetWeight (srcNeuron destNeuron) X (let ((result (send X (GetNeuron destNeuron) X :getConnection srcNeuron))) X (cadr result))) X X X;;; @@SECTION GetSum X;;; X;;; @@REQ X;;; neuronId X;;; @@END-REQ X;;; X;;; @@FUNC X;;; Find the SUM of a neuron. (i.e. Net) X;;; @@END-FUNC X;;; X;;; @@STATUS X;;; Complete. X;;; @@END-STATUS X X;;; PUBLIC X X(defun GetSum (neuronId) X (send (GetNeuron neuronId) :getSum)) SHAR_EOF echo "extracting neural.readme" sed 's/^X//' << \SHAR_EOF > neural.readme X Amiga Neural Network Construction Set (Vr 1.0 : 5/5/59) X XThe following is a brief description of the files contained in Xa tiny XLISP Neural Network Construction Set (ANNCS): X XThe ANNCS is a set of functions that allows quick and easy construction Xof neural networks. The system is very simple and as a result much Xcan be enhanced. Eventually, if I have nothing better to do over the Xsummer, I will attach a graphical interface that allows you to Xvisually create neural networks and convert the images to ANNCS Lisp Xfunction calls. This is by no means a competitor for the Rochester XConnectionist Simulator but hay, what the heck, it's at least running Xon my favorite computer! X XNeural.lsp : Lisp source code for the construction set. XAnd.lsp : Implementation of the AND function using neural networks X constructed from functions in Neural.lsp X Uses Widrow's ADALINE's XXor.lsp : Implementation of the XOR function using neural networks X constructed from functions in Neural.lsp X Uses Widrow's ADALINE's XXor2.lsp : Alternate implementation of the XOR function using X neural networks constructed from functions in Neural.lsp X Uses Rumelhart's network. X XSorry there is no demonstration of learning in the Backpropagation Xmodel but it can easily be constructed using the functions in this Xconstruction set; so why don't you do it and send me a copy! X XTo load the programs, fire up XLISP and type: X X(load "Neural.lsp") X(load "And.lsp") X(load "Xor.lsp") X(load "Xor2.lsp") X X************************************************************************* XFunctions that are present in version 1.0 of ANNCS: X XRead And.lsp, Xor.lsp and Xor2.lsp for a demonstration of how the Xfunctions may be used. X X(ClearNetwork) X - Clear the network of all neurons. X X(CreateNeuron neuronId threshold) X - Create a neuron and give it a name: neuronId X Set the threshold value to: threshold. X e.g. (CreateNeuron 'myNeuron 2.3) X X(CreateInputLine inputId) X - Create an input line and give it a name: inputId. X e.g. (CreateInputLine 'input1) X X(SetInputLine inputId inputValue) X - Set the input value to the input line: inputId to the X value: inputValue. X e.g. (SetInputLine 'input1 1) X X(ConnectInputToNeuron inputId destNeuron weight) X - Connect an input line to a neuron. The weight is the X connection strength between the input line and neuron. X X(ConnectNeuron srcNeuron destNeuron weight) X - Connect two neurons together. I.e. Output of X srcNeuron feeds into input of destNeuron with a connection X strength of: weight. X X(GetNeuron neuronId) X - Get a neuron with the given Id name. This can be used to X get an input line as well. X e.g. (GetNeuron 'myNeuron) X Returns the neuron object. X X(Examine neuronId) X - Examine the contents of a neuron or input line. X e.g. (Examine 'input1) X X(ReportNeurons neuronList) X - Like examine except a list of neuron Id's are given. X e.g. (ReportNeurons '(input1 'or1)) X X(UpdateNetwork updateLimit neuronList) X - Based on the current inputs and connections, update the X entire network so that we can see what the output X should be. In some networks it may take a considerable X number of iterations before the network finally stablizes X to a result. The parameter updateLimit is an integer X that determines how many iterations the system is to X update before the list of neuron Id's in neuronList are X examined. Updating then resumes. X X X(NeuronOutput neuronId) X - Return the output of a particular neuron. X X(ChangeWeight srcNeuronId destNeuronId newWeight) X - Change the connection strength between two neurons X or an input and a neuron. I put this in so that if you X wanted to construct a backpropagation network you at least X had the means to train it. X X(GetWeight srcNeuronId destNeuronId) X - Get the connection strength between srcNeuronId and destNeuronId. X X(GetSum neuronId) X - Get the internal SUM (or NET) of a particular neuron. X X************************************************************************* XRedistribute this program at will. If you decide to use this program Xas part of some commercial package you are developing, by all means Xproceed, all I ask is that you give me a small amount of credit for it. X XSend comments, ideas, questions, improvements. donation$ to: XJason Leigh X538 Mackenzie Hall, Wayne State University, Detroit, MI 48202 Xe-mail: jal@zeus.cs.wayne.edu SHAR_EOF echo "extracting xor.lsp" sed 's/^X//' << \SHAR_EOF > xor.lsp X;;; @@FILE and.lsp X;;; Copyright (C) 1989 By Jason Leigh 5/5/89 X X;;; Redistribute this program at will. If you decide to use this program X;;; as part of some commercial package you are developing, by all means X;;; proceed, all I ask is that you give me a small amount of credit for it. X X;;; XOR created with 2 layer MADALINES. This uses the most basic X;;; AND and OR ADALINES and connects them up into the form of X;;; the XOR expressions which is: ((not a) and (b)) or ((a) and (not b)) X X(ClearNetwork) X X;;; Select the Relay function as the output function. X(defun OutputFunction (sum threshold) X (Relay sum threshold)) X X(CreateNeuron 'or1 0) ; Create a neuron for OR X(CreateInputLine 'i1) ; Create input line i1 X(CreateInputLine 'i2) ; Create input line i2 X(CreateInputLine 'unity) ; Create input line `unity' which is always 1 X(SetInputLine 'unity 1) X(CreateNeuron 'and1 0) ; Create two and neurons. X(CreateNeuron 'and2 0) X X(ConnectInputToNeuron 'i1 'and1 1) ; Make appropriate connects. X(ConnectInputToNeuron 'i2 'and1 -1) ; The best thing to do is to X(ConnectInputToNeuron 'unity 'and1 -1.5) ; get a piece of paper out X(ConnectInputToNeuron 'i1 'and2 -1) ; and draw out the connections X(ConnectInputToNeuron 'i2 'and2 1) ; to see what the network X(ConnectInputToNeuron 'unity 'and2 -1.5) ; actually looks like. X X(ConnectNeuron 'and1 'or1 1) X(ConnectNeuron 'and2 'or1 1) X(ConnectNeuron 'unity 'or1 1.5) X X X;;; Let's try out the network with the 4 possible binary combinations. X X(printf "Input: -1 -1\n") X(SetInputLine 'i1 -1) X(SetInputLine 'i2 -1) X(UpdateNetwork 10 '(or1)) X(printf "Output: " (NeuronOutput 'or1) "\n") X X(printf "Input: 1 -1\n") X(SetInputLine 'i1 1) X(SetInputLine 'i2 -1) X(UpdateNetwork 10 '(or1)) X(printf "Output: " (NeuronOutput 'or1) "\n") X X(printf "Input: -1 1\n") X(SetInputLine 'i1 -1) X(SetInputLine 'i2 1) X(UpdateNetwork 10 '(or1)) X(printf "Output: " (NeuronOutput 'or1) "\n") X X(printf "Input: 1 1\n") X(SetInputLine 'i1 1) X(SetInputLine 'i2 1) X(UpdateNetwork 10 '(or1)) X(printf "Output: " (NeuronOutput 'or1) "\n") SHAR_EOF echo "extracting xor2.lsp" sed 's/^X//' << \SHAR_EOF > xor2.lsp X;;; @@FILE xor2.lsp X;;; Copyright (C) 1989 By Jason Leigh 5/5/89 X X;;; Redistribute this program at will. If you decide to use this program X;;; as part of some commercial package you are developing, by all means X;;; proceed, all I ask is that you give me a small amount of credit for it. X X;;; Creating an XOR network using the functions in neural.lsp X X;;; The activation function is Sigmoid and a diagram of this X;;; network can be found in: X;;; Parallel Distributed Processing Volume I: Foundations X;;; By David E. Rumelhart, James L. McClelland X;;; p.331. X X(ClearNetwork) X X;;; Select the Sigmoid function as the output function. X(defun OutputFunction (sum threshold) X (Sigmoid sum threshold)) X X(CreateNeuron 'out 6.3) X(CreateNeuron 'mid 2.2) X(CreateInputLine 'in1) X(CreateInputLine 'in2) X(ConnectInputToNeuron 'in1 'mid -6.4) X(ConnectInputToNeuron 'in1 'out -4.2) X(ConnectInputToNeuron 'in2 'mid -6.4) X(ConnectInputToNeuron 'in2 'out -4.2) X(ConnectNeuron 'mid 'out -9.4) X X(printf "Input: 0 0\n") X(SetInputLine 'in1 0) X(SetInputLine 'in2 0) X(UpdateNetwork 10 '(mid out)) X(printf "Output: " (NeuronOutput 'out) "\n") X X(printf "Input: 1 0\n") X(SetInputLine 'in1 1) X(SetInputLine 'in2 0) X(UpdateNetwork 10 '(mid out)) X(printf "Output: " (NeuronOutput 'out) "\n") X X(printf "Input: 0 1\n") X(SetInputLine 'in1 0) X(SetInputLine 'in2 1) X(UpdateNetwork 10 '(mid out)) X(printf "Output: " (NeuronOutput 'out) "\n") X X(printf "Input: 1 1\n") X(SetInputLine 'in1 1) X(SetInputLine 'in2 1) X(UpdateNetwork 10 '(mid out)) X(printf "Output: " (NeuronOutput 'out) "\n") SHAR_EOF echo "End of archive 1 (of 1)" # if you want to concatenate archives, remove anything after this line exit