muttiah@welsh.ecn.purdue.edu (Ranjan S Muttiah) (12/11/90)
What do researches think about why there are so many _individual_ neurons in the human brain (something like 10^19) ? Is there some critical number needed for "intelligent behaviour" ? [It's much like asking why there are so many epithelial cells, white blood cells and the like but I can think of a few reasons for these ...in the case of neurons I have difficulties].
minsky@media-lab.MEDIA.MIT.EDU (Marvin Minsky) (12/11/90)
In article <1990Dec11.040646.20760@noose.ecn.purdue.edu> muttiah@welsh.ecn.purdue.edu (Ranjan S Muttiah) writes: > >What do researches think about why there are so many _individual_ neurons >in the human brain (something like 10^19) ? No one thinks that. More like 10^11. (Many of them are little "granule cells" with few conections.) William Calvin mentioned neurologist's joke: the brain contains 10 billion beurons, of which 100 billion are in the cerebellum. It would seem likely, at least to me, that most single cells don't do useful operations alone, so that, for example, it might need a whole cluster or column of cells to do some little useful job. So for many functions, you'd be left with "only" a few hundred million" "units". And it would seem that an ordinary "commonsense database" semantic network might well require a few dozen million nodes and connections. No one knows that yet, of course. At the other end of the opinion spectrum are theorists who suggest that each single cell might store hundreds, or even millions of bits inside the cell. No one has made a good case for this, or for how they might be read-in or -out, but it certainly isn't inconceivable. But there simply aren't enough grams of brain for 10^19 cells, or anything of that magnitude. Each neuron is a few micrometers in diameter, so 10^18 of them would be meters in diameter.
esrmm@warwick.ac.uk (Denis Anthony) (12/11/90)
In article <1990Dec11.040646.20760@noose.ecn.purdue.edu> muttiah@welsh.ecn.purdue.edu (Ranjan S Muttiah) writes: > > >What do researches think about why there are so many _individual_ neurons >in the human brain (something like 10^19) ? Is there some critical number >needed for "intelligent behaviour" ? > >[It's much like asking why there are so many epithelial cells, white blood >cells and the like but I can think of a few reasons for these ...in the >case of neurons I have difficulties]. The speed of processing of the nervous system (milliseconds) and the amount of computation required for (e.g.) vision in real time, and the number of simultanious computation systems (vision, sound, proprioception, cognition etc.) require a large number of parallel processes. (all pretty obvious). Clearly there is a minimum number, but whether it must be of order 10^19 or not I do not know. There is some redundancy in that fairly large portions of the cerebrum may be lost before gross brain damage is manifest (though other parts of the brain are less robust). Perhaps some-one from the neuro-sciences can help out on this one. Denis
fahn@cs.columbia.edu (Paul N. Fahn) (12/12/90)
In article <1990Dec11.040646.20760@noose.ecn.purdue.edu> muttiah@welsh.ecn.purdue.edu (Ranjan S Muttiah) writes: > >What do researches think about why there are so many _individual_ neurons >in the human brain (something like 10^19) ? Is there some critical number >needed for "intelligent behaviour" ? > It seems to me that (computer-based) neural net researchers too often presume that biological organization is somehow optimal. It may be that the brain represents an enormous waste of resources, and that "intelligent behavior" can be achieved with far fewer neurons, if only they were organized more "intelligently". (or, equivalently, that higher intelligence could be achieved with the same number of neurons.) We should definitely look at biological nets for ideas, but in a very critical way, and be prepared to reject anything found in the biological realm. .Paul.
ian@ponder.csci.unt.edu (Ian Parberry) (12/12/90)
In article <1990Dec11.174921.26838@cs.columbia.edu> fahn@hudson.columbia.edu (Paul Fahn) writes: >It seems to me that (computer-based) neural net researchers too often >presume that biological organization is somehow optimal. It may be that >the brain represents an enormous waste of resources, and that >"intelligent behavior" can be achieved with far fewer neurons, if only >they were organized more "intelligently". (or, equivalently, that higher >intelligence could be achieved with the same number of neurons.) I used to take it for granted that we could hope to beat biology (and I was dumb enough to say it in print, see the Intro to my ``Primer on the Complexity Theory of Neural Networks'', in ``Formal Techniques in Artificial Intelligence: A Sourcebook'', R. B. Banerji (Ed.), in series Studies in Computer Science and Artificial Intelligence, Vol. 6, pp. 217-268, Elsevier, 1990). However, I met a biologist at the NIPS workshop who argued the opposite: that perhaps brains are the best way of doing things within reasonable constraints (speed, amount of hardware, volume, power, robustness, energy supply, fault-tolerance, etc.) Now I'm not so sure. _____ Ian Parberry ian@dept.csci.unt.edu Dept. of Computer Science, Univ. of North Texas, P.O. Box 13886, Denton, TX 76203-3886 A man with 1 watch knows what time it is. A man with 2 watches is never sure.
bellido@boromir.world (Ignacio Bellido) (12/12/90)
In article <1990Dec11.174921.26838@cs.columbia.edu>, fahn@hudson.columbia.edu (Paul Fahn) writes: >It seems to me that (computer-based) neural net researchers too often >presume that biological organization is somehow optimal. It may be that >the brain represents an enormous waste of resources, and that >"intelligent behavior" can be achieved with far fewer neurons, if only >they were organized more "intelligently". (or, equivalently, that higher >intelligence could be achieved with the same number of neurons.) > >We should definitely look at biological nets for ideas, but in a very >critical way, and be prepared to reject anything found in the biological >realm. Let's see, I believe that biological organization IS optimal taken acount of it's path, I mean, the selections made along the evolution, the advantage of biological optimization is a lot of million years over us (we just learned some years ago that the brain has cells and that they are independent elements), and a very big number of choices to select and eliminate on this time. May be another choice some millions years ago could have boosted another race to be intelligent with less amount of neurons better organized, but we will never know, and I don't think so. Are there any waste of resources in the human brain? I believe not, Nature trends always to the minimum energy, and it's not the way to waste. Also, in evolution is very important to have a big amount of variety, a really optimal organization probably means a cut of variety and this means less choices. It also means a bad tolerance to failures. There are also a lot of parameters to put into the acount, generality of the basic brain structure, fault tolerance (really important), information capacity, real time learnning, generalization,... These kind of things are very expensive in resources, but they are neccesary for an "intelligent" behavior, so it's not waste. Can we design a better organization? I don't think so, but let see how these field's evolution goes. Ignacio -- -------------------------------------------------------------------------- Ignacio Bellido Fernandez-Montes -1z Visiting Scholar at Stanford University e-mail: bellido@psych.stanford.edu Psychology Department Graduate Student Madrid University of Technology Department of Telematic Engineering e-mail: ibellido@dit.upm.es --------------------------------------------------------------------------
ld231782@longs.LANCE.ColoState.EDU (Lawrence Detweiler) (12/12/90)
muttiah@welsh.ecn.purdue.edu (Ranjan S Muttiah) in <1990Dec11.040646.20760@noose.ecn.purdue.edu : >What do researches think about why there are so many _individual_ neurons >in the human brain ... ? Is there some critical number >needed for "intelligent behaviour" ? esrmm@warwick.ac.uk (Denis Anthony) replies in <1990Dec11.152350.16930@warwick.ac.uk>: >The speed of processing of the nervous system (milliseconds) and the amount >of computation required for (e.g.) vision in real time, and the number of >simultanious computation systems (vision, sound, proprioception, cognition >etc.) require a large number of parallel processes. (all pretty obvious). Nearly any facet of human perception can be used as a plausible example for the necessity of the massive, unrivalled parallelism in the human brain, particularly vision. Clearly intelligence requires extraordinary computational power, but parallelism is only one way to satisfy this prerequisite (although the approach seems more and more a prerequisite itself!) Biology, in all its grandeur, is rarely a brilliant technological innovator in the eyes of one of its offspring, those sole thinking critics (after all, it hasn't even invented and applied the *wheel* in any of its abundant creations and variations.) Parallelism is an unmistakably intrinsic element in biology, however. It is the secret of life's success. Even though single-celled organisms (bacteria etc.) probably outnumber the multi-celled ones in our realm, I would wager the vast majority of cells are contained in multicellular organisms. So biology has had some bias for parallelism for some time, and we have something to learn from our patient and pragmatic engineer. I'm sure there are many opinions on what comprises the "critical mass" in human brain tissue that separates them so distinctly from the rest of the animal kingdom. One possible candidate is the huge amount of "interneurons" in the human brain. Neuroscience found a distinction of motor- vs. sensory- vs. inter- neuron useful in categorizing the nervous systems of invertebrates. Motor neurons connect to muscle fibers, or nearly so, and sensory neurons connect to receptors. Unfortunately, as we climb the vertebrate ladder, so to speak, this distinction becomes increasingly meaningless, whereupon in homo sapiens we reach a climax where only one in many thousands of neurons can be labelled a "sensory" or "motor" neuron, and the vast remaining majority take on that nebulous and mysterous term "interneurons". Therein lies thought, somewhere. Some might balk at the idea of some "critical mass" of neuron number whereupon we reach an explosion of intelligence. However, the idea is not so absurd to dismiss without thought...our brains have changed remarkably little in terms of organization in comparison with other invertebrates. The lobes and bulges tend to do the same thing in the same places across species (the hippocampus and motor cortex come to mind). We humans just have this incredibly large bulge in one of the usual bulges, namely the neocortex, that houses this vast majority of interneurons, and there is a lot of circumstantial evidence that this is precisely our differentiating advantage. The "phenomenon" of sign language in apes may be a good example of the role that the interneuron plays. It is very intuitive to think that the interneuron is what facilitates abstraction and association, arguably the foremost underlying qualities of speech. Here we may have a very vivid demonstration of interneuronal power--apes, which have a fraction of the interneurons that humans do, but still more than other animals, can communication in few-word sentences. Perhaps if we grafted additional associational cortex in their brains the complexity and abstraction of their "speech" would be enhanced! (When people criticize studies of sign language in apes as mimicry, I tend to consider this a manifestation of a superiority complex, or more precisely an "anthropomorphic" complex. Why discriminate against them just because we can do it better?! We don't scoff at our babies for their verbal mimicry! In terms of evolution, apes, just as babies, are the immature reflections of ourselves.) What about this "critical mass" of neurons? >Clearly there is a minimum number, [the order] I do not know. There is some >redundancy in that fairly large portions of the cerebrum may be lost before >gross brain damage is manifest (though other parts of the brain are less robust). There is a long history of misunderstanding the inherent quality of fault-tolerance in a massively parallel system. Reports of its nature have been greatly misinterpreted. Because we remove nine-tenths of a rat's brain and it still navigates a learned maze with minimal impairment, does that imply that nine-tenths of our mental capacity is dormant? We are simply applying a concept where it is not appropriate. In truth, this concept of "graceful degradation" is a somewhat novel concept to human engineers. There aren't too many engines that gradually slow down as we remove pistons, or buildings that gradually lean while we remove columns! Just how crucial is one neuron to a thousand? One one-thousandth! minsky@media-lab.MEDIA.MIT.EDU (Marvin Minsky) in <4311@media-lab.MEDIA.MIT.EDU> >It would seem likely, at least to me, that most single cells don't do >useful operations alone, so that, for example, it might need a whole >cluster or column of cells to do some little useful job. So for many >functions, you'd be left with "only" a few hundred million" "units". >And it would seem that an ordinary "commonsense database" semantic >network might well require a few dozen million nodes and connections. >No one knows that yet, of course. Every neuron is doing a useful job, its just that we have a difficult time measuring/quantifying/characterizing its diffuse and miniscule contribution. Perhaps we can duplicate some aspect of the effects of some clusters of neurons with some other device, but we will probably make compromises in the versatility and plasticity of the system. Perhaps the ultimate animal, "more than human", would be the synthesis of evolutionary biology and human technology. But there may be some features of the biological approach that are so ideal (for two, its versatility and plasticity) that they would be essential to the feasibility of the system. However, the idea of alternate forms of intelligence, perhaps to the point of improvement, is a very enticing idea! It is the motivation behind the field of AI! >At the other end of the opinion spectrum are theorists who suggest >that each single cell might store hundreds, or even millions of bits >inside the cell. No one has made a good case for this, or for how >they might be read-in or -out, but it certainly isn't inconceivable. At another end of the opinion spectrum are theorists who suggest that concepts such as "storing bits" in neurons is meaningless and embarassingly technocentric. fahn@cs.columbia.edu (Paul N. Fahn) in <1990Dec11.174921.26838@cs.columbia.edu> >It seems to me that (computer-based) neural net researchers too often >presume that biological organization is somehow optimal. It may be that >the brain represents an enormous waste of resources, and that >"intelligent behavior" can be achieved with far fewer neurons, if only >they were organized more "intelligently". (or, equivalently, that higher >intelligence could be achieved with the same number of neurons.) > >We should definitely look at biological nets for ideas, but in a very >critical way, and be prepared to reject anything found in the biological >realm. That's an interesting question that takes one step farther Marvin Minsky's favorite idea that neural networks can be condensed (ideally computationally)--not only can units be amalgamated, but some can be eliminated! Is there a neurological equivalent of an appendix or tonsils? Someday we may be surprised and aghast at the depth and vastness of the evolutionary baggage we carry... But only when we understand the design we will have the opportunity to transcend it. ld231782@longs.LANCE.ColoState.EDU
sci240s@monu6.cc.monash.edu.au (mr w.j. ho) (12/12/90)
bellido@boromir.world (Ignacio Bellido) writes: >In article <1990Dec11.174921.26838@cs.columbia.edu>, >fahn@hudson.columbia.edu (Paul Fahn) writes: >>It seems to me that (computer-based) neural net researchers too often >>presume that biological organization is somehow optimal. It may be that >>the brain represents an enormous waste of resources, and that >>"intelligent behavior" can be achieved with far fewer neurons, if only >>they were organized more "intelligently". (or, equivalently, that higher >>intelligence could be achieved with the same number of neurons.) >> >>We should definitely look at biological nets for ideas, but in a very >>critical way, and be prepared to reject anything found in the biological >>realm. > Let's see, I believe that biological organization IS optimal >taken acount of it's path, I mean, the selections made along the >evolution, the advantage of biological optimization is a lot of million >years over us (we just learned some years ago that the brain has cells >and that they are independent elements), and a very big number of >choices to select and eliminate on this time. May be another choice some >millions years ago could have boosted another race to be intelligent >with less amount of neurons better organized, but we will never know, ^^^^^^^^^^^^^^^^^^^^^^^ less robust too ( more susceptible to neuron degradation ) ? Does not offer "as graceful degradation as it is now" ? >and I don't think so. -- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^ Wey Jing Ho Tel: 61-3-5732567 E-mail : sci240s@monu6.cc.monash.edu.au ^ ^ Physics Dept., Monash University ( Caulfield Campus ), Melbourne, AUSTRALIA ^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
msellers@mentor.com (Mike Sellers) (12/13/90)
In article <11678@ccncsu.ColoState.EDU> ld231782@longs.LANCE.ColoState.EDU (Lawrence Detweiler) writes: > >muttiah@welsh.ecn.purdue.edu (Ranjan S Muttiah) in ><1990Dec11.040646.20760@noose.ecn.purdue.edu : > >>What do researches think about why there are so many _individual_ neurons >>in the human brain ... ? Is there some critical number >>needed for "intelligent behaviour" ? > >esrmm@warwick.ac.uk (Denis Anthony) replies in ><1990Dec11.152350.16930@warwick.ac.uk>: > >>The speed of processing of the nervous system (milliseconds) and the amount >>of computation required for (e.g.) vision in real time, and the number of >>simultanious computation systems (vision, sound, proprioception, cognition >>etc.) require a large number of parallel processes. (all pretty obvious). Feldman articulated this in his "100-step" rule. Since neurons are slow, operating in terms of milliseconds rathern than nano-seconds, and yet we are able to perform significant feats of cognition in a relatively short time frame (< 1 second), only about 100 sequential processing steps could fit into the time needed to perform the sensation, perception, and cognition of, say, recognizing a written word. Since 100 sequential steps are nowhere near what is necessary to do this sort of thing, massive parallelism must be used. Thus a large number of neurons, each with a large number of interconnections, is required for fast processing with slow units. (The number of neurons probably is something like 10^11, though this is obviously a _very_ gross estimate, and the total number of synapses --where the processing probably takes place-- is something like 4 to 6 orders of magnitude greater than the number of neurons.) > [...] Biology, in all its grandeur, is rarely a brilliant technological >innovator in the eyes of one of its offspring, those sole thinking critics I would argue this point, but that is probably best left for elsehwere. >(after all, it hasn't even invented and applied the *wheel* in any of its >abundant creations and variations.) Not so. At least some flagellate bacteria drive their flagella in a circular, not back-and-forth type motion via a small "wheel" at the base of the flagellum. At this scale having a proteinaceous wheel structure seems to make sense; at the macroscopic scale it apparently doesn't (I have an article around somewhere about why animals don't use wheels -- from SciAm or Discover or something, I think). > Parallelism is an unmistakably intrinsic >element in biology, however. It is the secret of life's success. Even though >single-celled organisms (bacteria etc.) probably outnumber the multi-celled >ones in our realm, I would wager the vast majority of cells are contained in >multicellular organisms. I realize that this is another side-comment, but considering the huge amount of the earth's biomass that is held in bacteria, algae, and other unicellular organisms, I think multicelled organisms are almost certainly in the minority. And anyway, the survival strategy of being multicellular is more an argument for object-oriented paradigms than parallel ones. :-) > So biology has had some bias for parallelism for >some time, and we have something to learn from our patient and pragmatic >engineer. > >I'm sure there are many opinions on what comprises the "critical mass" in >human brain tissue that separates them so distinctly from the rest of the >animal kingdom. I'm not so sure a distinct separation between human cognitive abilities and those of all other animals exists. The cognitive abilities of some of the great apes have been demonstrated, as have those of some cetaceans. We do not (yet?) have enough of a common basis upon which to judge their neural/cognitive abilities in comparison to ours, but I think it is safe to say that the traditional view of a chasm separating "man and beast" has been replaced with the realization that there are great but gradual changes in neural, perceptual, and cognitive abilities from planaria to humans. > One possible candidate is the huge amount of "interneurons" >in the human brain. Neuroscience found a distinction of motor- vs. sensory- >vs. inter- neuron useful in categorizing the nervous systems of invertebrates. >Motor neurons connect to muscle fibers, or nearly so, and sensory neurons >connect to receptors. Unfortunately, as we climb the vertebrate ladder, so to >speak, this distinction becomes increasingly meaningless, whereupon in homo >sapiens we reach a climax where only one in many thousands of neurons can be >labelled a "sensory" or "motor" neuron, and the vast remaining majority take >on that nebulous and mysterous term "interneurons". Therein lies thought, >somewhere. Some might balk at the idea of some "critical mass" of neuron >number whereupon we reach an explosion of intelligence. However, the idea is >not so absurd to dismiss without thought...our brains have changed remarkably >little in terms of organization in comparison with other invertebrates. I agree and disagree. :-) I agree that "thought" is embodied (if that terms makes any sense in this case) in the number and configuration of interneurons, particularly those in the cerebral cortex. However, I do not think it is correct to say that our brains have changed little from our invertebrate ancestors (or contemporaries). Even among vertebrates there are major differences in large-scale neural structures; it isn't the case that human brains just have an extra lobe tacked onto them. >What about this "critical mass" of neurons? > >>Clearly there is a minimum number, [the order] I do not know. There is some >>redundancy in that fairly large portions of the cerebrum may be lost before >>gross brain damage is manifest (though other parts of the brain are >less robust). > >There is a long history of misunderstanding the inherent quality of >fault-tolerance in a massively parallel system. Reports of its nature have >been greatly misinterpreted. Because we remove nine-tenths of a rat's brain >and it still navigates a learned maze with minimal impairment, does that imply >that nine-tenths of our mental capacity is dormant? We are simply applying a >concept where it is not appropriate. In truth, this concept of "graceful >degradation" is a somewhat novel concept to human engineers. There aren't too >many engines that gradually slow down as we remove pistons, or buildings that >gradually lean while we remove columns! Just how crucial is one neuron to a >thousand? One one-thousandth! I agree that the case for graceful degradation in neural systems is often overstated. However, it is true that many people who have massive cortical damage rarely or never exhibit changed behavior. There are other cases where less severe damage has resulted in spectacular cognitive or behavioral deficits, so the situation is not a clear-cut one. It is not the case, however, that if you remove some fraction of the neurons from a neural system that that system will necessarily be impaired to a similar degree. >minsky@media-lab.MEDIA.MIT.EDU (Marvin Minsky) in ><4311@media-lab.MEDIA.MIT.EDU> > >>It would seem likely, at least to me, that most single cells don't do >>useful operations alone, so that, for example, it might need a whole >>cluster or column of cells to do some little useful job. So for many >>functions, you'd be left with "only" a few hundred million" "units". >>And it would seem that an ordinary "commonsense database" semantic >>network might well require a few dozen million nodes and connections. >>No one knows that yet, of course. > >Every neuron is doing a useful job, its just that we have a difficult time >measuring/quantifying/characterizing its diffuse and miniscule contribution. I think you're using the term "useful job" in a different sense, or at a different scale. If you define "recognizing a vertical bar" or "recognizing my grandmother" as useful jobs or operations, then almost certainly single neurons do not perform useful operations. If you define reporting the existance of light in a particular part of the visual field as a useful job, then at least in some cases single neurons are doing useful things. The point of arguments like Minsky's is that it may be that neither the neuron nor the synapse is the correct unit of perception and cognition, but that ensembles of neurons provide more information for explanation and prediction of how thought is accomplished. >fahn@cs.columbia.edu (Paul N. Fahn) in ><1990Dec11.174921.26838@cs.columbia.edu> > >>It seems to me that (computer-based) neural net researchers too often >>presume that biological organization is somehow optimal. It may be that >>the brain represents an enormous waste of resources, and that >>"intelligent behavior" can be achieved with far fewer neurons, if only >>they were organized more "intelligently". (or, equivalently, that higher >>intelligence could be achieved with the same number of neurons.) >> >>We should definitely look at biological nets for ideas, but in a very >>critical way, and be prepared to reject anything found in the biological >>realm. > >That's an interesting question that takes one step farther Marvin Minsky's >favorite idea that neural networks can be condensed (ideally >computationally)--not only can units be amalgamated, but some can be >eliminated! Is there a neurological equivalent of an appendix or tonsils? >Someday we may be surprised and aghast at the depth and vastness of the >evolutionary baggage we carry... But only when we understand the design we >will have the opportunity to transcend it. > >ld231782@longs.LANCE.ColoState.EDU I think we can look at evolution as a process of finding minima. Our brain structure representss a local minimum in terms of using space, energy, speed, etc.. There may well be a global minimum just over the next (silicon) hill, but we shouldn't be too quick to discard what a much less innovative but far more tenacious process has come up with over a period of several million years. -- Mike Sellers msellers@mentor.com Mentor Graphics Corp. "I used to think that the brain was the most wonderful organ in my body. Then I realized who was telling me this." -- Emo Phillips
dominic@debussy.cs.colostate.edu (Zpwdrhoond) (12/13/90)
In article <1990Dec11.040646.20760@noose.ecn.purdue.edu> muttiah@welsh.ecn.purdue.edu (Ranjan S Muttiah) writes: > > >What do researches think about why there are so many _individual_ neurons >in the human brain (something like 10^19) ? Is there some critical number >needed for "intelligent behaviour" ? I have seen any estimates quite this high for the number of neurons in the human brain. Wasserman states that there are approximately 10^11 neurons comprising something like 10^15 interconnections. More neurons than stars in the Milky Way I think. Intelligent behavior is something very elusive to discuss because it is quite difficult to quantify. Personally, I think the coastal snail shows quite a bit of intelligence but I'm not sure how many neurons it is estimated to have. dominic@debussy.cs.colostate.edu