psotka@ARI-HQ1.ARPA ("PSOTKA, JOSEPH") (12/21/86)
Landauer (Cognitive Science, 10, 1986, pp. 477 - 493) argues that there is a calculable limit to the amount people remember. Estimates based on input and forgetting rates ranged around (EXPT 10 9) OR 1,000,000,000 or one billion bits. This is vastly less than the figure (EXPT 10 20) quoted from Von Neumann. On the basis of this he argues that possibly " we should not be looking for models and mechanisms that produce storage economies, but rather ones in which marvels are produced by profligate use of capacity." The key estimates for this figure are shown in the following table: Task: Input rate total: bits/sec bits Reading 1.2 1.8E9 Picture Recog. 2.3 3.4E9 To me these figures are unbelievably low. A gigabyte of facts on a CD-ROM cannot possibly represent my memory system. For the moment let us look at the reasoning for reading and then a brief look at pictorial memory. READING: Landauer uses a relatively straightforward set of assumptions (not without their perils!) to infer rate of storage into longterm memory. He has some experiments that back them up. Basically he says that given any text (in the average) with words deleted at random people are able to predict about half the words (.48) from context and previous knowledge, and this increases only slightly (e.g. to .63) after reading the text quickly. The net gain is log2(.63/.48) = .42. From this he argues that the new information available in the text is .42 BITS per word. Over a lifetime of reading 3 words per second, storage in memory would be roughly 1.8 X 10E9 bits. It seems reasonable enough, but it is not very convincing. For one thing, surely people are reading the "context" too, and not just getting information from the individual words: there are higher order chunks called sentences that are very meaningful. To eliminate the information value of the context so abruptly is a disservice to our information gathering abilities. Surely we are processing this "context" too! Another point is that there are aspects entering memory not just connected to the words: the episode itself provides information; that fact that this particular word is seen at this particular time is important; the auditory and somesthetic context comes along too ( e.g. the room was quiet; the chair was soft, etc.). BRAIN CYCLE TIME: Finally the natural cycle of information entry seems much too long; one to five seconds. There is much perceptual and cognitive information that suggests a basic cycle of 1/10th of a second ( e.g., perceptual integration, apparent movement, backward masking, sensory stores, etc.). BASIC BRAIN BYTE: As a counterexample for this low estimate, consider the following simple example: Two words are flashed on the screen for one-tenth of a second. Any person with eyes open reads and remembers them. If the words were chosen at random, the guessing rate would be very low (given approximately 1, 000, 000 words to choose from, the likelihood of getting both words right is roughly 10E-12) but the hit rate would surely be in the 90s for percent correct. Even after a few weeks it would be substantial. The storage transfer rate is now 17 bits in .1 sec. Over a lifetime, this comes out to 1.7 E 11, a factor of one hundred greater, without becoming too unreasonable in our assumptions. But there is yet another perspective on the same phenomenon. Much of the time, when I read a text my most prominent reaction is "Hohum. Nothing new here!" Has no information been transferred? Well, my text - prediction (Cloze) performance would probably be as only good as Landauer's claim, and even if it were much better, the baseline of .5 mitigates any drastic change in the total figures. Clearly, a lot of information has been transferred, not measured by this technique: I know the author of the text has wasted my time; I probably judged something about his writing and thinking abilities, his vocabulary, and other characteristics; I may have changed my desire to see him and any plans that went along with it; etc. etc. Surely a very large number of consequences arose from this interaction; consequences whose information content is surely constrained by the set size of potential reactions and current memories. In a sense this is the meaning and context of the reading task. Let me suggest a recursive procedure on the estimation of our lifetime memory. Given Landauer's basic lifetime estimate of information extracted from text of 1.0E9 bits, let us take an individual who lives 70 years and hypothesize a memory of 9.0E8 bits. Let us then suggest that any word he reads must be coded to be able to make contact with one of these (potential) memories and is stored in (some abstract) connection with that memory. The information content of that word is then 30 bits instead of .4 and total lifetime information (at 3 words per second) is 1.3E11 (given 1.5E9 sec. in a lifetime). Given this new measure of information we can redo the cycle. The next round is 1.7E11 bits. This is roughly stable, and it is about the same as our previous measure. Here is the function: (SETQ BitRate (QUOTIENT (LOG TotalBits )) (LOG 2))) (SETQ TotalBits (TIMES BitRate 4.5E9)) Both these procedures yield measures roughly 100 times higher than Landauer's. But there is a suggestion that the true measure is still much higher: that in fact we don't know how the brain codes information in all its many relationships. Really, we have very little information about the relative size of pictorial and other abstract knowledge structures. PICTORIAL REPRESENTATION; A series of experiments by careful and reputable researchers (Nickerson, Standing, and Shepard ) found very high recognition rates for pictures shown very briefly (4 to 6 seconds) even after hours, days, or weeks before testing. The relation between size of the set of pictures and accuracy is surprisingly flat: Number of Pictures Shown Percent Correct: 20 99 40 96 100 95 200 92 400 86 1000 88 4,000 81 10,000 83 One wonders when this function would break down so that showing a picture would result in no memory. Of course, that seems clearly impossible. At one second per picture over 70 years, one could only look at 2.268E9 pictures (WITHOUT SLEEPING) and these data show that at the very least one would remember 8, 300 of them and probably a lot more. Given the limited accuracy of these data it seems unwarranted to fit a curve to the numbers, but a rough estimate would say that recognition percentage becomes very small at about 1.0E9 pictures. At this point Landauer might say that the basic brain byte can no longer encode a new picture. The question then becomes "What is stored?" Landauer makes the parsimonious suggestion that all that is stored is the minimal code that would separate one picture from any other. Without any special coding procedures that make use of internal redundancies, it would take a 36 bit code to store all the pictures. This is about twice the estimate Landauer makes on other grounds: certainly within reasonable agreement for such rough estimates. However, it seems most improbable that only some abstract code is stored. Our computers need to store much more to do anything with these pictures: a 50,000 Bits bitmap is still a very rough representation of the real thing. A 35 Bits BITMAP would not represent very much at all. To say that stereoscopic vision adds one bit to the representation is to misrepresent the obvious. Naturally, the existence of veridical representations (e.g. eidetic images ) is difficult to verify; but fragmentary report suggests the decomposability of the memory code into useable fragments and features that are realistically detailed, with very fine grain size. Again, the estimate that Landauer suggests has to be considered as an absolute lower bound, with more realistic estimates surely orders of magnitude larger. The key to understanding memory size is the understanding of the transformations and codes the mind applies. Given this simple perspective, the conclusions that Landauer draws need to be modified. Given the many visual, auditory, and sensory storage systems that are possible, and the existence of abstract representation (ideas) in other forms, the used memory does indeed begin to approach the 10E12 figure that is a rough estimate for number of synapses. Profligacy of control structures is not quite in order: in fact there may be no room for control structures; everything may be in the code. None of this, to emphasize, disagrees with Landauer's basic conclusions that there is no one to one correspondence between functional memory and the component capacity needed for its support; this could always be much, much larger. What is so intriguing is that current computers are indeed beginning to approach these estimates of physical capacities. The brain's byte size and component stores are beginning to be realizable in silicon form. It is an audacious person who is no longer willing to admit the possibility of silicon intelligence. ------