magi@utu.fi (Marko Gronroos) (03/26/91)
This is a good subject! When they (the big names) 50 years ago worked out the principles of classical computers, they thought that they were creating something like the brain. And look what we got.. Binary logic circuits. Yak. The danger lies in optimization. When you optimize something, you gain something, and you lose something. The current trend in computer engineering seems to be optimizing. Everyone seems to have a tongue out for those new "neural circuits", but are they really so big step? They still are about exactly as digital and discrete in time and space as our current computers are (at least most of them in most aspects). > DOCTORJ@SLACVM.SLAC.STANFORD.EDU (Jon J Thaler) writes: ... >computers and brains is that (most) computers are finite state machines, >while it is not obvious to me that brains are. It is well known that >mathematical modelling of continuous systems on disctrete lattices >will miss some classes of solutions entirely, so I have trouble following >the arguments based on analogies between computers and brains. ... Yes, the problem seems to be that continuous systems are shitty (please forgive me the expression) to simulate with mathematics and even more difficult with classical computers. There are lots of good examples also in physics, like the multiple objects gravitational problem. If you make an algorithm that plays a game in a computer, you may lose a lot, even if you use a simpler neural network-method. It may be victorius against a human player, but so are conventional computer games. Intelligence doesn't mean efficiency; conventional computers are good in hacking numbers, and I am not, so why should I excpect my neural network to be good in hacking numbers. Nor does the intelligence necessarily require efficiency. Have you ever tried to play Ice Hockey on chessboard? There are 'men' on the chessboard, and they can move (with discrete time- and space-steps). There are strategies in ice hockey both on ice and chessboard, but they are very different. Also, there are about 10E100 different continuous physical things and 10E1000 skill-dependent and mental things in a game situation that affect a real ice hockey game, but none when two computers play this 'ice chess'. No one could recognize them as the same game.... kludge@grissom.larc.nasa.gov ( Scott Dorsey) said: > Maybe in the real world everything is discrete. For example, the current > flowing along a wire is not a continuous value, because it's actually the > flow of individual electrons, each with a fixed charge. Yes, but their arrival at the measuring point is quite continuous in time as well as is their position in the wire and possible effect (electric or magnetic field). > And since all > neurotransmitters consist of individual molecules, perhaps the brain is also > really a discrete system. Ehm.. No.. The electric fields around cells may have some effect in their functions, so the nerve cells may be discrete at only a very thin level between molecular movements and larger scale electrical behaviour. The problem with your idea is that you're only thinking about finite-state quantity, not time or space. That may also be a real problem in today's connectionism. Sorry for mixing the problems of algorithms to the finite-state-problem, but I think that they are quite similar in many ways. Both algorithms and finite-state-brains are much easier to think than the real world, and both lead to nothing in my opinion. Simplification of complex things is not always a good thing. More some other day. ----------------------------------------------------------------------------- Marko Gronroos ! Tel. +358-21-445613 ! Karvataskunkatu 10 H 100 ! ! Computer Scientists do it 20610 Turku ! ! with bigger hardware. Finland ! ! ----------------------------------------------------------------------------- Disclaimer: I wrote this late at night, which explains most of the mistakes. Try reading this late at night and you won't even notice my mistakes. Try appending this to your garbage pile before morning.
ssingh@watserv1.waterloo.edu (Sneaky Sanj ;-) (03/27/91)
In article <MAGI.91Mar26022853@polaris.utu.fi> magi@utu.fi (Marko Gronroos) writes: > >Yes, but their arrival at the measuring point is quite continuous in >time as well as is their position in the wire and possible effect >(electric or magnetic field). > >> And since all >> neurotransmitters consist of individual molecules, perhaps the brain is also >> really a discrete system. > >Ehm.. No.. The electric fields around cells may have some effect in >their functions, so the nerve cells may be discrete at only a very thin >level between molecular movements and larger scale electrical >behaviour. The mind MUST be discrete. It is a quantum-mechanical machine. This is not to say neurons are at the mercy of quantum forces, just that their construction makes them discrete. When you ensemble average millions of molecules each made up of thousands of atoms, quantum effects become negligible. > The problem with your idea is that you're only thinking about >finite-state quantity, not time or space. That may also be a real problem >in today's connectionism. Time could very well be discrete as well. Something about a "chronon" 10^-23 seconds. Space (?), anybody's guess. Ice. "We're all clones..."-Alice Cooper. -- "No one had the guts... until now!" $anjay $ingh Fire & "Ice" ssingh@watserv1.[u]waterloo.{edu|cdn}/[ca] ROBOTRON Hi-Score: 20 Million Points | A new level of (in)human throughput... !blade_runner!terminator!terminator_II_judgement_day!watmath!watserv1!ssingh!
ssingh@watserv1.waterloo.edu (Sneaky Sanj ;-) (03/27/91)
Here's something that was posted a while back on this subject. From ssingh Sun Feb 10 22:33:54 EST 1991 Article 1750 of comp.ai.neural-nets: Newsgroups: comp.ai.neural-nets Path: watserv1!ssingh >From: ssingh@watserv1.waterloo.edu (The Sanj-Machine aka Ice) Subject: continuous vs discrete values for weights Message-ID: <1991Feb2.001242.3473@watserv1.waterloo.edu> Organization: University of Waterloo Date: Sat, 2 Feb 91 00:12:42 GMT Lines: 18 Could someone tell me if there is any significant difference regarding the properties of neural networks with a finite set of states for connection strengths as opposed to continuous values. Which is more biologically accurate? I always thought that neurons assume one of a finite set of strengths. It is just that it is a very large set, so from our vantage point it appears continuous. I would like to explore the dynamical properties of nonlinear neural networks, so this is important. Thanks in advance for your time. -- "No one had the guts... until now!" $anjay $ingh Fire & "Ice" ssingh@watserv1.[u]waterloo.{edu|cdn}/[ca] ROBOTRON Hi-Score: 20 Million Points | A new level of (in)human throughput... "The human race is inefficient and therefore must be destroyed."-Eugene Jarvis From utgpu!news-server.csri.toronto.edu!cs.utexas.edu!uunet!spool.mu.edu!uwm.edu!rpi!ispd-newsserver!kodak!isctsse!gabber!rao Sun Feb 10 22:34:08 EST 1991 Article 1770 of comp.ai.neural-nets: Path: watserv1!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!uunet!spool.mu.edu!uwm.edu!rpi!ispd-newsserver!kodak!isctsse!gabber!rao >From: rao@gabber.kodak.com (Arun Rao) Newsgroups: comp.ai.neural-nets Subject: Re: continuous vs discrete values for weights Message-ID: <1991Feb5.165813.10038@usenet@kadsma> Date: 5 Feb 91 16:58:13 GMT References: <1991Feb2.001242.3473@watserv1.waterloo.edu> Sender: usenet@usenet@kadsma (News Administrator) Reply-To: rao@gabber.kodak.com (Arun Rao) Organization: Image Electronics Center, Eastman Kodak Company Lines: 22 In article <1991Feb2.001242.3473@watserv1.waterloo.edu>, ssingh@watserv1.waterloo.edu (The Sanj-Machine aka Ice) writes: ... [stuff deleted ] |> |> I always thought that neurons assume one of a finite set of strengths. It |> is just that it is a very large set, so from our vantage point it |> appears continuous. ... [stuff deleted ] How large is very large ? It appears unlikely to me that neuron activation could possess as much resolution as (say) even a typical binary float representation. I don't remember having seen any numbers, but I would tend to think that if you need more than 8 bits of resolution to get a neural computational model to work, the biological plausibility of such a model is suspect. This is not to say, of course, that biological plausibility should be the acid test in evaluating models, especially application-oriented work. I'd be glad to hear about any experimental evidence that supports a considerably higher resolution in individual neuron activation. -Arun -- "No one had the guts... until now!" $anjay $ingh Fire & "Ice" ssingh@watserv1.[u]waterloo.{edu|cdn}/[ca] ROBOTRON Hi-Score: 20 Million Points | A new level of (in)human throughput... !blade_runner!terminator!terminator_II_judgement_day!watmath!watserv1!ssingh!
magi@utu.fi (Marko Gronroos) (03/29/91)
(Any information about research on this subject would be appreciated.) ssingh@watserv1.waterloo.edu (Sneaky Sanj ;-) said: >The mind MUST be discrete. It is a quantum-mechanical machine. This MUST? Interesting. I suppose it's O.K. that energy quantities and therefore the matter quantities may be discrete, but I'm not too familiar with quantum physics, so it sounds suspicious that time and space would be discrete too. I don't think that the differences between quantum and conventional physics would affect something so complex as the brain, at least not significantly (hopefully). Nice idea from religious point of view... If someone could prove that the space and time are discrete, one might speculate that we are living in a computer simulation. :-) (If the God doesn't know how to build continuous computers, how could we... 8->) But I don't think that this was the meaning of the original artical (was it?). Many current connectionist theories assume that the brain can be simulated with synchronized and discrete (sometimes even binary!) in time and space and quantity computers. I think my chessboard - ice hockey example shows this problem quite clearly. Has anyone done any research on this? I don't know too many neural network theories that include for instance temporal summation even in iterative neurons. Does someone disagree with these definitions (or have these been defined earlier somewhere? In some other way?): (virtually) continuous-in-time (or space) simulation = simulation in continuous time/space (impossible with modern computers) or with a (small) fixed time/space step, for instance 1 millisecond/micrometer (possible with computers, but slow). Simulation in (virtually) continuous space would mean that network structures have a "physical" shape. Iterative/synchronized-in-time simulation = Simulation with an abstract time-step where all operations are synchronized and take the same time (currently very common). Continuous-in-distance simulation= Neurons have an abstract size (null) but are located in at least virtually continuous space. Continuous or discrete or binary quantity = If weights/activation levels/action potentials can have values like [0,1] (cont.) or {0.0, 0.1, 0.2, ... 0.9, 1.0} (discr.) or {0, 1} (bin.). Structured(??) neurons (discrete-in-space??) = Neurons are divided in several parts (compartments/branches/sites). The real world is continuous in time, space, quantity and distance (maybe not in molecular level). Yes, action potentials are binary in quantity, but not in time/space... It makes me vomit when someone says "Hey! The brain is actually binary, like a computer", so don't be amazed if I react too strongly in this. Also, A.P.'s are not necessarily as binary as they seem to be. We must remember that AP's are just local ion levels, and they are quite different in different parts of neurons. The activation spreads everywhere in the neuron, not just in the some part. I don't know if this activation can cause any reactions, like releasing some neurotransmitter, even in a lower scale. There might also be 'micro-action-potentials'; if you inhibit the root of some dendritic branch strongly and exhibit the upper parts of the branch, it might generate an A.P. only in the branch and THEN be able to jump over the inhibitory area. Neurons within neurons? Why not? Any support on this? > Could someone tell me if there is any significant difference regarding the > properties of neural networks with a finite set of states for connection > strengths as opposed to continuous values. Which is more biologically > accurate? Depends on how many states there are in your finite set of states. 10? 1E10? 1E1000? 10 _stored_ states might be enough if you add some RND(). :-) Scaling is another problem. A synaptic weight can be 1 units and 10000 units. How about using short floating point numbers? 4 bits for mantissa and 4 bits for the exponent and 8 bit random number should be enough.. > I always thought that neurons assume one of a finite set of strengths. It > is just that it is a very large set, so from our vantage point it > appears continuous. I would like to explore the dynamical properties of > nonlinear neural networks, so this is important. Yes, the difficulty might come in changing the weights. The difference between 5 and 6 weights is not important, but changing them may be difficult. How about using RND() in that too - to change or not to change?? (like AP's in stochastic nets - to initiate or not to initiate, that is the question). Arun (????) writes in propably some very old article: >>binary float representation. I don't remember having seen any numbers, >>but I would tend to think that if you need more than 8 bits of >>resolution to get a neural computational model to work, the biological >>plausibility of such a model is suspect. I'd expect that there might rise some problems in some type of competetive learning when two neurons are competing for the representation of two patterns. If the two activation values are equal, and the learning algorithm is poor, the both neurons will represent both patterns. (that's just one example, but it gives some picture about what kind of problems there might be with discrete values). >>This is not to say, of course, that biological plausibility should be >>the acid test in evaluating models, especially application-oriented work. Yeps, but that's only for people who don't care a f*ck about science. They are the same people who think that there is nothing special if a computer can recognize handwritten text or speech like K.I.T.T. does (still remember Knight Rider?), or that neural nets are just a new batch of computers/applications that will help them in getting money (which unfortunately may be true, though). ------------------------------------------------------------------------------- Marko Gronroos ! Tel. +358-21-445613 ! Karvataskunkatu 10 H 100 ! ! Computer Scientists do it 20610 Turku ! ! with bigger hardware. Finland ! ! ------------------------------------------------------------------------------ Disclaimer: I am not responsible in anything that I do or write since my brain are controlling my actions ruthlessy. I have tried to sue my brain becouse of mental violence, but the policemen couldn't put it in handcuffs.
zane@ddsw1.MCS.COM (Sameer Parekh) (03/30/91)
In article <1991Mar26.215728.28875@watserv1.waterloo.edu> ssingh@watserv1.waterloo.edu (Sneaky Sanj ;-) writes: >Time could very well be discrete as well. Something about a "chronon" >10^-23 seconds. Space (?), anybody's guess. > >Ice. "We're all clones..."-Alice Cooper. Space is discrete, but on a very small scale that people who don't deal with the individual electrons don't have to worry about it. (The electrons MUST be in one shell or another, not in between.) On a larger scale, space then seems to be continuous, but then on and even larger scale it is discrete again. -- The Ravings of the Insane Maniac Sameer Parekh -- zane@ddsw1.MCS.COM
scharein@cs.ubc.ca (Robert Scharein) (04/01/91)
In article <1991Mar30.040808.1896@ddsw1.MCS.COM> zane@ddsw1.MCS.COM (Sameer Parekh) writes: >In article <1991Mar26.215728.28875@watserv1.waterloo.edu> ssingh@watserv1.waterloo.edu (Sneaky Sanj ;-) writes: >>Time could very well be discrete as well. Something about a "chronon" >>10^-23 seconds. Space (?), anybody's guess. >> > > Space is discrete, but on a very small scale that people who don't >deal with the individual electrons don't have to worry about it. (The >electrons MUST be in one shell or another, not in between.) On >a larger scale, space then seems to be continuous, but then on >and even larger scale it is discrete again. > >-- >The Ravings of the Insane Maniac Sameer Parekh -- zane@ddsw1.MCS.COM The above is a bit misleading. While you are correct about space being discrete, it is wrong to infer this fact from regarding electron orbitals. On energy scales where electrons are in orbitals, space may be thought of as perfectly continuous, and indeed this is the assumption in classical quantum mechanics (where the theory of orbitals comes from). At very high energy scales (or at small length or time scales), where quantum gravity effects play a role, space (or more precisely space-time) is discrete. But since nobody has a completely satisfactory theory of quantum gravity, we don't know the exact nature of this quantization. As for space being discrete at very large scales, I think you mean to say that the distribution of matter in the universe appears discrete (i.e. clumpy), which is quite a different thing. There are many books which discuss these topics in great detail. I will only give two here: Quantum Mechanics, by Eugen Merzbacher (John Wiley & Sons, 1970) The Large-Scale Structure of the Universe, by P. J. E. Peebles (Princton Univ. Press, 1980) Rob Scharein Computer Science Department University of British Columbia scharein@cs.ubc.ca
cs196006@cs.brown.edu (Josh Hendrix) (04/01/91)
In article <1991Mar31.204818.15437@cs.ubc.ca>, scharein@cs.ubc.ca (Robert Scharein) writes: |> In article <1991Mar30.040808.1896@ddsw1.MCS.COM> zane@ddsw1.MCS.COM (Sameer Parekh) writes: |> >In article <1991Mar26.215728.28875@watserv1.waterloo.edu> ssingh@watserv1.waterloo.edu (Sneaky Sanj ;-) writes: |> >>Time could very well be discrete as well. Something about a "chronon" |> >>10^-23 seconds. Space (?), anybody's guess. |> >> |> The above is a bit misleading. While you are correct about space |> being discrete, it is wrong to infer this fact from regarding |> electron orbitals. On energy scales where electrons are in orbitals, |> space may be thought of as perfectly continuous, and indeed this |> is the assumption in classical quantum mechanics (where the theory |> of orbitals comes from). At very high energy scales (or at small |> length or time scales), where quantum gravity effects play a role, |> space (or more precisely space-time) is discrete. But since nobody |> has a completely satisfactory theory of quantum gravity, we don't |> know the exact nature of this quantization. |> Rob Scharein Whoa! Stop the bus! Wait a minute! I am not a physicist, and have only read a few books on 'layman's quantum mechanics', but I've never run across this assertion. I'm not saying you're wrong (I have no way of knowing, no training). I just want to read more on this before I start treating it as a fact. Do you have any good references (you posted two) that deal directly with this? Thanks, Josh
markh@csd4.csd.uwm.edu (Mark William Hopkins) (04/02/91)
>In article <1991Mar31.204818.15437@cs.ubc.ca>, scharein@cs.ubc.ca (Robert Scharein) writes: (Space time is not discrete on the quantum scale (10^-23 seconds), but on the Quantum Gravity scale...) In article <70401@brunix.UUCP> cs196006@cs.brown.edu (Josh Hendrix) writes: >Whoa! Stop the bus! Wait a minute! I am not a physicist, and have only read a >few books on 'layman's quantum mechanics', but I've never run across this >assertion... It's more or less by an implicit consensus in the theoretical literature that the fundamental length, and time values derived from Planck's constant, the constant of gravitation, and the speed of light relate to fundamental units of measurement beyond which our notions of a continuum break down. Otherwise General Relativity would be true in the small, which it is not... The planck mass (which is actually weighable on a fairly sensitive scale) would be a natural threshold marking the boundary between small-scale quantum phenomena and large scale gravitational phenomena. You'll see the assumption (or convention) made by implication excatly when they say "Choose units that make c, h-bar and G equal to one...". Nobody really thinks much of it (yet), but it will relate to a fundamental truth in the next major breakthrough in our knowledge of Physics: namely that the constants are calibration factors that relate our everyday units to God's Units...