[mod.ai] BASIC BRAIN BYTES AND LIFELONG MEMORY

psotka@ARI-HQ1.ARPA ("PSOTKA, JOSEPH") (12/21/86)

 Landauer (Cognitive Science, 10, 1986, pp. 477 -  493) 
 argues that there is a calculable limit to the
 amount people remember.  Estimates based on input and
 forgetting rates ranged around (EXPT 10 9) OR 1,000,000,000
 or one billion bits.  This is vastly less than
 the figure (EXPT 10 20) quoted from Von Neumann.  On
 the basis of this he argues that possibly " we should
 not be looking for models  and mechanisms that produce
 storage economies, but rather ones in which marvels
 are produced by profligate use of capacity."

The key estimates for this figure are shown in the following table:


Task:		Input rate	total:
			bits/sec			bits

Reading		1.2			1.8E9
Picture Recog. 	2.3			3.4E9

To me these figures are unbelievably low.
   A gigabyte of facts on a CD-ROM  cannot possibly represent
 my memory system. 

For the moment  let us look
 at the reasoning for reading 
 and then a brief look
 at  pictorial memory.

READING:


Landauer uses a relatively
 straightforward set of assumptions (not without
 their perils!) to infer  rate of storage into longterm
 memory.   He has some experiments that back them up.
  Basically he says that given any text (in the average)
 with words deleted at random people are able to predict
 about half the words (.48) from context and previous
 knowledge, and this increases only slightly (e.g.
 to .63) after reading the text quickly.  The net gain
 is log2(.63/.48) = .42.

From this he argues that the
 new information available in the text is .42 BITS per
 word.  Over a lifetime of reading 3 words per second,
 storage in memory would be roughly 1.8 X 10E9 bits.


It seems reasonable enough, but it is not very convincing.
  For one thing, surely people are reading the "context"
 too, and not just getting information from the
 individual words:  there are higher order chunks called
 sentences that are very meaningful.  To eliminate the
 information value of the context so abruptly is a disservice
 to our information gathering abilities.  Surely
 we are processing this "context" too!

Another point
 is that there are aspects entering memory not just connected
 to the words:  the episode itself provides information;
 that fact that this particular word is seen
 at this particular time is important; the auditory and
 somesthetic context comes along too ( e.g. the room
 was quiet; the chair was soft, etc.).


BRAIN CYCLE TIME:

Finally
 the natural cycle of information entry seems
 much too long; one to five seconds.  There is much
 perceptual and cognitive information that suggests a basic
 cycle of 1/10th of a second ( e.g., perceptual integration,
 apparent movement, backward masking, sensory
 stores, etc.).

BASIC BRAIN BYTE:

As a counterexample
 for this low estimate, consider the following simple
 example:  Two words are flashed on the screen for one-tenth
 of a second.  Any person with eyes open reads
 and remembers them.   If the words were chosen at random,
 the guessing rate would be very low (given approximately
 1, 000, 000 words to choose from, the likelihood
 of getting both words right is roughly 10E-12) but the
 hit rate  would surely be in the 90s for percent correct.
 Even after a few weeks it would be substantial.
  The storage transfer rate is now 17 bits in .1 sec.
  Over a lifetime, this comes out to 1.7  E 11, a factor
 of one hundred greater, without becoming too unreasonable
 in our assumptions.


But there is yet another perspective
 on the same phenomenon.  Much of the time, when
 I read a text my most prominent reaction is "Hohum.
  Nothing new here!"  Has no information been transferred?
  Well, my text - prediction (Cloze) performance would 
probably be as only good as Landauer's claim, and even
 if it were much better,  the baseline of .5 mitigates
 any drastic change in the total figures.  Clearly, a
 lot of information has been transferred, not measured
 by this technique: I know the author of the text has
 wasted my time; I probably judged something about his
 writing and thinking abilities, his vocabulary, and other
 characteristics; I may have changed my desire to see
 him and any plans that went along with it; etc. etc.
  Surely a very large number of consequences arose from
 this interaction; consequences whose information content
 is surely constrained by the set size of potential
 reactions and current memories.  In a sense this is
 the meaning and context of the reading task.

Let me suggest a recursive procedure  on the estimation of our
 lifetime memory.  Given Landauer's basic  lifetime estimate
 of information extracted from text of 1.0E9 bits,
 let us take an individual who lives 70 years  and
 hypothesize a memory of 9.0E8 bits.  Let us then suggest
 that any word he reads must be coded to be able to
  make  contact with one of these  (potential) memories
 and is stored in (some abstract) connection with that
 memory.  The information content of that word is then
 30 bits instead of .4 and total lifetime information
 (at 3 words per second) is 1.3E11  (given 1.5E9 sec. in
 a lifetime).  Given this new measure of information
 we can redo the cycle. The next round is   1.7E11 bits.
 This is roughly stable, and it is about the same as
 our previous measure.  


Here is the function:

(SETQ
 BitRate (QUOTIENT (LOG   TotalBits  )) (LOG 2)))
(SETQ
 TotalBits (TIMES   BitRate   4.5E9))


Both these procedures
 yield measures roughly 100 times higher than Landauer's.
  But there is a suggestion that the true measure
 is still much higher: that in fact we don't know
 how the brain codes information in all its many relationships.


 Really, we have very little information about
 the relative size of pictorial and other abstract
 knowledge structures.


PICTORIAL REPRESENTATION;



A series of experiments by careful and reputable researchers
 (Nickerson, Standing, and Shepard ) found very high
 recognition rates for pictures shown very briefly
 (4 to 6 seconds) even after hours, days, or weeks before
 testing.  The relation between size of the set of pictures
 and accuracy is surprisingly flat: 


Number of
 Pictures Shown		Percent Correct:
20							99
40 							96
100							95
200							92
400							86
1000							88
4,000						81
10,000						83


One wonders when  this
 function would break down so that showing a picture
 would result in no memory.  Of course, that seems clearly
 impossible.  At one second per picture over 70 years,
 one could only look at 2.268E9 pictures (WITHOUT
 SLEEPING) and these data show that at the very least
 one would remember  8, 300 of them and probably a lot
 more.  Given the limited accuracy of these data it seems
 unwarranted to fit a curve to the numbers, but a rough
 estimate would say that recognition percentage becomes
 very small at about 1.0E9 pictures.  At this point
 Landauer might say that the basic brain byte  can no
 longer encode a new picture.

The question then becomes
 "What is stored?"  Landauer makes the parsimonious
  suggestion that all that is stored is the minimal code
 that would separate one picture from any other.  Without
 any special coding procedures that make use of internal
 redundancies, it would take a 36 bit code to store
 all the pictures. This is about twice the estimate
 Landauer makes on other grounds: certainly within reasonable
 agreement for such rough estimates. 

However,
 it seems most improbable that only some abstract code
 is stored.  Our computers need to store much more to
 do anything with these pictures:  a 50,000 Bits bitmap
 is still a very rough representation of the real thing.
  A 35 Bits BITMAP would not represent very much at
 all.  To say that stereoscopic vision adds one bit to
 the representation is to misrepresent the obvious.  Naturally,
 the existence of veridical representations (e.g.
 eidetic images ) is difficult to verify; but fragmentary
 report suggests the decomposability of the memory
 code into useable fragments and features that are realistically
 detailed, with very fine grain size.    Again,
 the estimate that Landauer suggests has to be considered
 as an absolute lower bound, with more realistic
 estimates surely orders of magnitude larger.  The key
 to understanding memory size is the understanding of
 the transformations and codes the mind applies. 

Given this simple perspective, the conclusions that Landauer
 draws need to be modified.  Given the many visual,
 auditory, and sensory storage systems that are possible,
 and the existence of abstract representation (ideas)
 in other forms, the used memory does indeed begin
 to approach the 10E12 figure  that is a rough estimate
 for number of synapses.  Profligacy of control structures
 is not quite in order: in fact there may be no room
 for control structures; everything may be in the code.

None of this, to emphasize, disagrees with Landauer's
 basic conclusions that there is no one to one correspondence
 between functional memory and the component
 capacity needed for its support; this could always be
 much, much larger.


What is so intriguing is that current
 computers are indeed beginning to approach these
 estimates of physical capacities.  The brain's byte size
 and component stores are beginning to be realizable in 
silicon form.  It is an audacious person who is no longer 
willing to admit the possibility of silicon intelligence.


------