srh@wind.bellcore.com (stevan r harnad) (10/06/89)
I have a simple question: What capabilities of PDP systems do and do not depend on the net's actually being implemented in parallel, rather than just being serially simulated? Is it only speed and capacity parameters, or something more? Stevan Harnad Psychology Department Princeton University harnad@confidence.princeton.edu Stevan Harnad INTERNET: harnad@confidence.princeton.edu harnad@princeton.edu srh@flash.bellcore.com harnad@elbereth.rutgers.edu harnad@princeton.uucp CSNET: harnad%confidence.princeton.edu@relay.cs.net BITNET: harnad1@umass.bitnet harnad@pucc.bitnet (609)-921-7771
bill@boulder.Colorado.EDU (10/06/89)
>I have a simple question: What capabilities of PDP systems do and >do not depend on the net's actually being implemented in parallel, >rather than just being serially simulated? Is it only speed and >capacity parameters, or something more? > >Stevan Harnad (harnad@confidence.princeton.edu) A little too simple, maybe, because it isn't quite clear what it means. Let me rephrase it, and you can say whether this is something like what you meant: "We have here a black box, with a PDP device inside it and some user-interface stuff on the surface. Is it necessarily true, ignoring considerations of time and space, that the PDP device is replaceable by a serial device which from the user's viewpoint will behave identically?" If this is the question, then I think the answer is yes. Since the PDP device is a physical object, it is governed by a set of differential equations, and those equations can be solved to any desired accuracy by a serial device. (It may take a long time to solve them, though.) If the PDP system is chaotic (in the mathematical sense), then no simulation will ever be able to _duplicate_ its behavior, but the simulation will still be able to imitate it in the sense of being equally unpredictable by the user.
andrew@dtg.nsc.com (Lord Snooty @ The Giant Poisoned Electric Head ) (10/10/89)
In article <12449@boulder.Colorado.EDU>, bill@boulder.Colorado.EDU writes: > >I have a simple question: What capabilities of PDP systems do and > >do not depend on the net's actually being implemented in parallel, > >rather than just being serially simulated? Is it only speed and > >capacity parameters, or something more? > >Stevan Harnad (harnad@confidence.princeton.edu) > Since the PDP device is a physical object, it is governed by a set > of differential equations, and those equations can be solved to any > desired accuracy by a serial device. (It may take a long time to solve > them, though.) If the PDP system is chaotic (in the mathematical sense), > then no simulation will ever be able to _duplicate_ its behavior, but > the simulation will still be able to imitate it in the sense of being > equally unpredictable by the user. I'd like to add a caveat to this view: the issue of exact implementation is being glossed over. In a "real" BP NN, weight updates and weighted activation arrival times may be purely asynchronous. The "batch update" versus "per input" update paradigms also give room for difference. Although both methods are in principle simulatable in a serial fashion, I'm not quarrelling with bill's response. However, if the asynchronous case is driven differently by the simulator's random number generator w.r.t. the "real" NN, and local minima and/or instability problems exist, the performances will not match up. -- ........................................................................... Andrew Palfreyman and the 'q' is silent, andrew@dtg.nsc.com as in banana time sucks
russell@minster.york.ac.uk (10/13/89)
In article <17820@bellcore.bellcore.com> srh@wind.UUCP (Stevan Harnad) writes: > >I have a simple question: What capabilities of PDP systems do and >do not depend on the net's actually being implemented in parallel, >rather than just being serially simulated? Is it only speed and >capacity parameters, or something more? > >Stevan Harnad >harnad@confidence.princeton.edu I think this is an interesting point, but the simple answer is that there is nothing more to be gained from a parallel implementation over a serial simulation (except speed and possibly capacity - but even that depends on the parallel architecture vs. the serial machine). The reason: The only thing that can happen in a parallel system that can't be mimiced on a serial one is the occurance of simultaneous events - but is this a problem? If we consider time at a sufficiently fine granularity, then we can guarantee that any node/unit in the net will only have had one operation performed on it - there may be many nodes that actually encounter a simultaneous update or operation, but there will still have been only one per node. If our simulation on a sequential machine operates at this level of granularity, and completes all the required updates to each affected node before advancing the `clock' by one time increment, then the sequential machine will be functionally identical to the parallel implementation. The only problems occur when the simulation is run at insufficiently fine granularity, or when limiting assumptions about the number of simultaneous events are broken. Russell. ____________________________________________________________ Russell Beale, Advanced Computer Architecture Group, Dept. of Computer Science, University of York, Heslington, YORK. YO1 5DD. UK. Tel: [044] (0904) 432762 russell@uk.ac.york.minster JANET connexions russell%york.minster@cs.ucl.ac.uk ARPA connexions ..!mcvax!ukc!minster!russell UUCP connexions russell@minster.york.ac.uk eab mail ____________________________________________________________