[comp.ai] Chinese Room Experiment: empirical tests

markh@csd4.csd.uwm.edu (Mark William Hopkins) (11/25/90)

I offer you a sinple empirical test that will verify or refute the assertion
Searle makes with this thought experiment.  This is, however, a minor twist
on the original idea, since it is YOU who will be the learning agent.

The concept is real simple.  Try to learn a new language by imitation.

So here's what you should do: take out about 10 books written in a language you
don't understand (like 10 books written in Spanish).  The language can be one
that uses the Latin alphabet, or not.  It doesn't matter.

Rewrite the exact contents of each book.  That means, WRITE the contents, don't
analyse them.  Don't even think about what it all might mean, just write it.
And write it all.

This is what will happen.  Within about 60 minutes your brain WILL begin to
adapt itself to the regularities of the language.  In one day (supposing you
work about 8 hours that day), you will already have a good feel for the
syntax of the language.  In about 3 to 5 days, you'll gradually begin to
recognize stylistic regularities.

And that means you work constantly 8 hours each day, no cheating, no breaks.

After a week you will be able to reproduce very complex novel (and valid)
written segments in that language, though you may not have a glimmer of what
it all means.

At this point, you'll find yourself in a very unusual position of being able
to recognize a new written language, and even being able to write in that
language, without having the faintest idea of what you're saying.

Now to address Searle's question about whether or not you *understand* the
language or not (remember, you're working off of 10 separate sources, so
you are getting a wide perspective on the language that even foreign language
students don't get), all you need to do is look inward.  Just ask youself:
what is your subjective impression?

Trust me, it's a very weird feeling, I've tried something very similar to
this experiment a countless number of times.

I suspect that even without the aid of resources to tell you what means what,
you will converge onto a sudden understanding of the language, almost as if
you were hit by a bolt out of the blue.

wido@isgtec.uucp (Wido Menhardt) (11/25/90)

In article <7852@uwm.edu> markh@csd4.csd.uwm.edu (Mark William Hopkins) writes:
> I offer you a sinple empirical test 
[ text deleted ] 
> Trust me, it's a very weird feeling, I've tried something very similar to
                                                  ^^^^^^^^^^^^^^^^^^^^^^
just what is it YOU tried ?

> this experiment a countless number of times.
                    ^^^^^^^^^^^^^^^^
so not even an INTEGER number ? this does not sound like experiments,
more like speculation ...

> I suspect that even without the aid of resources to tell you what means what,
    ^^^^^^^
right.

> you will converge onto a sudden understanding of the language, almost as if
> you were hit by a bolt out of the blue.

whow.

If this whole story is true, some other people will feel like hit by a bolt ..
myself, for instance.

I am looking forward to a reference to a (somewhat scientific) report/paper 
describing the experiments (including alphabet, language and ten book titles)
in detail.

If this confirms what you are saying, I am going to take the next three weeks
off;  buy ten titles of contemporary japanese literature plus paper
and pens (which is probably a wrong start already); and start writing
them; I always thought that language might be an asset, but never
thought to have the time for it.

-wido.

jwl@garnet.berkeley.edu (James Wilbur Lewis) (11/26/90)

In article <7852@uwm.edu> markh@csd4.csd.uwm.edu (Mark William Hopkins) writes:
-I offer you a sinple empirical test that will verify or refute the assertion
-Searle makes with this thought experiment.  This is, however, a minor twist
-on the original idea, since it is YOU who will be the learning agent.
-
-The concept is real simple.  Try to learn a new language by imitation.
-
-So here's what you should do: take out about 10 books written in a language you
-don't understand (like 10 books written in Spanish).  The language can be one
-that uses the Latin alphabet, or not.  It doesn't matter.
-
-Rewrite the exact contents of each book.  That means, WRITE the contents, don't
-analyse them.  Don't even think about what it all might mean, just write it.
-And write it all.
-
-This is what will happen.  Within about 60 minutes your brain WILL begin to
-adapt itself to the regularities of the language.  In one day (supposing you
-work about 8 hours that day), you will already have a good feel for the
-syntax of the language.  In about 3 to 5 days, you'll gradually begin to
-recognize stylistic regularities.
-
-And that means you work constantly 8 hours each day, no cheating, no breaks.
-
-After a week you will be able to reproduce very complex novel (and valid)
-written segments in that language, though you may not have a glimmer of what
-it all means.
-
-At this point, you'll find yourself in a very unusual position of being able
-to recognize a new written language, and even being able to write in that
-language, without having the faintest idea of what you're saying.
-
-Now to address Searle's question about whether or not you *understand* the
-language or not (remember, you're working off of 10 separate sources, so
-you are getting a wide perspective on the language that even foreign language
-students don't get), all you need to do is look inward.  Just ask youself:
-what is your subjective impression?

But this has *nothing* to do with Searle's argument, which is based on the
premise that the symbol manipulation algorithms are powerful enough to
enable the man in the room to *answer* questions that are posed in Chinese.

So far, you've only asserted that someone who follows your program will
be able to *recognize* the syntax of the foreign language, and perhaps
*generate* some syntactically valid strings.  But that is an entirely
different set of skills than the ones required to *respond* to input 
with the competence of a native speaker.

-I suspect that even without the aid of resources to tell you what means what,
-you will converge onto a sudden understanding of the language, almost as if
-you were hit by a bolt out of the blue.

I'll buy the idea that you might have a good feel for the syntax of a language
after such an intensive program of imitation, but *understanding*, in the sense
that Searle uses the term?  No way!

-- Jim Lewis

smoliar@vaxa.isi.edu (Stephen Smoliar) (11/27/90)

(Note:  I am cross-posting this to comp.ai.philosophy, since that is where
these discussions belong;  and I hope future debate will be conducted there.)

In article <7852@uwm.edu> markh@csd4.csd.uwm.edu (Mark William Hopkins) writes:
>
>The concept is real simple.  Try to learn a new language by imitation.
>
>So here's what you should do: take out about 10 books written in a language
>you
>don't understand (like 10 books written in Spanish).  The language can be one
>that uses the Latin alphabet, or not.  It doesn't matter.
>
>Rewrite the exact contents of each book.  That means, WRITE the contents,
>don't
>analyse them.  Don't even think about what it all might mean, just write it.
>And write it all.
>
>This is what will happen.  Within about 60 minutes your brain WILL begin to
>adapt itself to the regularities of the language.  In one day (supposing you
>work about 8 hours that day), you will already have a good feel for the
>syntax of the language.  In about 3 to 5 days, you'll gradually begin to
>recognize stylistic regularities.
>
I noticed that the first reaction to this proposal was one of self-righteous
skepticism.  Unfortunately, it is very hard to deal with this situation with
anything other than anecdotal evidence.  Having said that, let me throw out
two such anecdotes.

One is personal.  When I was a graduate student, we used to eat at a Chinese
restaurant whose Chinese menu differed from the English menu.  One of our group
made up a pony, but the waiters refused to let us order by pointing.  The
eventual compromise which was established with the management was that we
would be allowed to WRITE our orders.  It was amazing how little time it
took to begin to assimilate knowledge of how Chinese characters were put
together and even to "parse" those symbols in terms of components which
referred to specific food items.  (Note that James McCawley has now taken
some of the fun out of this exercise with his EATER'S GUIDE TO CHINESE
CHARACTERS, but I suspect that variations on this anecdote will continue
to unfold among Chinese food fanatics.)

The other anecdote is not personal but IS pedagogical.  Many decades ago, a
fundamental element in the instruction of counterpoint at the Curtis Institute
of Music involved copying out entire works by Palestrina.  (This approach is,
to the best of my knowledge, no longer in practice.)  The point here is that
the study of counterpoint involves learning a vast complex of constraint rules
with little guidance about what to do with them.  Copying "real" music gave the
student an opportunity to observe the rules in action without having everything
pointed out to him explicitly.  The assumption was that there was something to
be gained by picking up the habit of "writing Palestrina;"  and for quite some
time this was recognized as a valid pedagogical approach.  (Perhaps we are
encouraged to write quotations down on 3 X 5 cards not only as a source of
reference material but also as an exercise in writing someone else's text.)

=========================================================================

USPS:	Stephen Smoliar
	5000 Centinela Avenue  #129
	Los Angeles, California  90066

Internet:  smoliar@vaxa.isi.edu

"It's only words . . . unless they're true."--David Mamet

hugh@fivegl.co.nz (Hugh Grierson) (11/28/90)

In article <7852@uwm.edu> markh@csd4.csd.uwm.edu (Mark William Hopkins) writes:
>So here's what you should do: take out about 10 books written in a language you
>don't understand (like 10 books written in Spanish).  The language can be one
>that uses the Latin alphabet, or not.  It doesn't matter.
>
>Rewrite the exact contents of each book.  That means, WRITE the contents, don't
>analyse them.  Don't even think about what it all might mean, just write it.
>And write it all.
>
>This is what will happen.  Within about 60 minutes your brain WILL begin to
>adapt itself to the regularities of the language.  In one day (supposing you
>work about 8 hours that day), you will already have a good feel for the
>syntax of the language.  In about 3 to 5 days, you'll gradually begin to
>recognize stylistic regularities.
>...
>After a week you will be able to reproduce very complex novel (and valid)
>written segments in that language, though you may not have a glimmer of what
>it all means.
>
>At this point, you'll find yourself in a very unusual position of being able
>to recognize a new written language, and even being able to write in that
>language, without having the faintest idea of what you're saying.
>

All the way through I was looking for the smilies...

>
>Trust me, it's a very weird feeling, I've tried something very similar to
>this experiment a countless number of times.

...but now you have me interested.  

Don't leave us hanging here - what actually were the experiments you tried,
and what were the results?  What other history is there of such absorbtion 
techniques: for oral/written language, or for non-linguistic learning?

-- 
                               | Hugh Grierson         | hugh@fivegl.co.nz 
                               | 5GL International Ltd | PH:  +64.9.3021621
                               | Auckland, New Zealand | FAX: +64.9.3021617

longc@cs.fau.edu (Courtney Long) (11/29/90)

In article <1990Nov28.003642.22319@fivegl.co.nz> hugh@fivegl.co.nz (Hugh Grierson) writes:
>In article <7852@uwm.edu> markh@csd4.csd.uwm.edu (Mark William Hopkins) writes:
>>So here's what you should do: take out about 10 books written in a language you
>>don't understand (like 10 books written in Spanish).  The language can be one
>>that uses the Latin alphabet, or not.  It doesn't matter.
>>
>>Rewrite the exact contents of each book.  That means, WRITE the contents, don't
>>analyse them.  Don't even think about what it all might mean, just write it.
>>And write it all.
>>
>>This is what will happen.  Within about 60 minutes your brain WILL begin to
>>adapt itself to the regularities of the language.  In one day (supposing you
>>work about 8 hours that day), you will already have a good feel for the
>>syntax of the language.  In about 3 to 5 days, you'll gradually begin to
>>recognize stylistic regularities.
>>...
>>After a week you will be able to reproduce very complex novel (and valid)
>>written segments in that language, though you may not have a glimmer of what
>>it all means.
>>
>>At this point, you'll find yourself in a very unusual position of being able
>>to recognize a new written language, and even being able to write in that
>>language, without having the faintest idea of what you're saying.
>>
>
>All the way through I was looking for the smilies...
>
>>
>>Trust me, it's a very weird feeling, I've tried something very similar to
>>this experiment a countless number of times.
>techniques: for oral/written language, or for non-linguistic learning?

It's true.  When I studied jazz, I would often transcribe solos of various
artists.  As a guitar player I studied Wes Montgomery, and transcribed many
of his solos.  What made this different, however, is that I would listen to
the solo, and then write down the notes.  Maybe this is slightly different
that just copying the notes of his solo to another page?
>
>-- 
>                               | Hugh Grierson         | hugh@fivegl.co.nz 
>                               | 5GL International Ltd | PH:  +64.9.3021621
>                               | Auckland, New Zealand | FAX: +64.9.3021617

markh@csd4.csd.uwm.edu (Mark William Hopkins) (11/29/90)

In article <1990Nov28.003642.22319@fivegl.co.nz> hugh@fivegl.co.nz (Hugh Grierson) writes:
>>Trust me, it's a very weird feeling, I've tried something very similar to
>>this experiment a countless number of times.
>
>...but now you have me interested.  
>
>Don't leave us hanging here - what actually were the experiments you tried,
>and what were the results?

One of the more interesting cases that comes to mind is VAX binary: i.e.
reading executable files without any translation references.  I didn't spend
more than a couple days on that but got as far as being able to recognize what
happen to correspond to addressing modes and instruction opcodes.

If it isn't too late for you (meaning if you already have experience with
and knowledge of assembly languages), try it.  The analogy of this experience
will give you an appreciation of what a machine has to go through in 'learning'
human language.

>What other history is there of such absorbtion techniques: for oral/written
>language, or for non-linguistic learning?

As a matter of course, (last I heard), Hungarians teach their foreign-language
students from texts written completely in Hungarian.  It's very effective.

This is slightly different though, in that you have a translation reference
(initially: pictures, later: subsets of Hungarian itself).

markh@csd4.csd.uwm.edu (Mark William Hopkins) (12/01/90)

On the question:
If you undergo intensive immersion training of a language using no translation
references is it possible to eventually converge onto an understanding of that
language?

In article <1990Nov26.055429.8883@agate.berkeley.edu> jwl@garnet.berkeley.edu (James Wilbur Lewis) writes:
>I'll buy the idea that you might have a good feel for the syntax of a language
>after such an intensive program of imitation, but *understanding*, in the sense
>that Searle uses the term?  No way!
>
>-- Jim Lewis

A very similar situation is going on collectively with researchers who are
progressively deciphering Mayan Hieroglyphic text ... and recently there has
been some success at partially undercovering the meaning of the symbols.

Even the little understanding accomplished to date is enough to prove that
understanding may be possible without any substantial external references...

jwl@garnet.berkeley.edu (James Wilbur Lewis) (12/01/90)

In article <7989@uwm.edu> markh@csd4.csd.uwm.edu (Mark William Hopkins) writes:
-On the question:
-If you undergo intensive immersion training of a language using no translation
-references is it possible to eventually converge onto an understanding of that
-language?
-
-In article <1990Nov26.055429.8883@agate.berkeley.edu> jwl@garnet.berkeley.edu (James Wilbur Lewis) writes:
->I'll buy the idea that you might have a good feel for the syntax of a language
->after such an intensive program of imitation, but *understanding*, in the sense
->that Searle uses the term?  No way!
-
-A very similar situation is going on collectively with researchers who are
-progressively deciphering Mayan Hieroglyphic text ... and recently there has
-been some success at partially undercovering the meaning of the symbols.

Ok, but remember, you stipulated "no analysis, just copying" in your original
post -- these folks are doing something (intensve analysis) quite different 
from what you proposed!

-Even the little understanding accomplished to date is enough to prove that
-understanding may be possible without any substantial external references...

I guess the interesting question is "how much of a reference do you need?"
What is the "critical mass" of semantic content you need to infer from
other sources before you can start bootstrapping your degree of understanding?
How much of a reference do we have with respect to an unknown human culture,
just by virtue of the fact that, as humans, a large kernel of knowledge and
experiences are going to be represented in both our "knowledge bases"?

Is SETI doomed from the start, or would we be able to communicate with a
hypothetical extra-terrestrial culture given only that we both possess the
ability to (say) send and receive radio signals?

-- Jim Lewis

wachtel@canon.co.uk (Tom Wachtel) (12/03/90)

In article <1990Nov26.055429.8883@agate.berkeley.edu> jwl@garnet.berkeley.edu (James Wilbur Lewis) writes:
>I'll buy the idea that you might have a good feel for the syntax of a language
>after such an intensive program of imitation, but *understanding*, in the sense
>that Searle uses the term?  No way!
>
>-- Jim Lewis

The technique of teaching a foreign language using the target language
as the sole medium of communication has been around for several decades
(literally) and is very successful, in general. In fact, using
translation to convey the meaning of a foreign language is the thing
that is out of date and known to be self-defeating. This may be hard to
believe if you have never experienced it (from either side).  The thing
is that it's not just imitation, but communication. It's really not
even a radical issue any more.

-- Tom Wachtel

fleck@cressida.ecs.ox.ac.uk (Margaret Fleck) (12/04/90)

>A very similar situation is going on collectively with researchers who are
>progressively deciphering Mayan Hieroglyphic text ... and recently there has
>been some success at partially undercovering the meaning of the symbols.

It's been a few years since I was last up-to-date with research in
that area.  However, external knowledge was definitely used in early
stages of the decipherment, including (at least):

  (1) A pseudo-alphabet surviving from the Spanish conquest, critical
      in early stages of breaking the phonetic parts of the sign system.
  (2) Knowledge of living Mayan languages, including meanings of words
      in them (since some of the bits of signs are phonetic).  Information
      from related languages was also used in deciphering many of the
      ancient Near Eastern languages.  A big exception was Sumerian, 
      not closely related to any known language, for which explicit
      dictionaries (from Akkadian) were available.
  (3) Surviving knowledge of the calendar system, some of which is *still*
      in use by certain Indian groups.   The modern languages also preserve
      the base 20 number system.
  (4) Semantic cues in the form of signs, e.g. one of the two systems of
      numbers has a really obvious structure of bars and dots.
  (5) Modern astonomical knowledge, used to pin down dates for events
      reported in the texts and check hypotheses that certain texts
      referred to astronomical events.  
  (6) Knowledge about what humans write on inscriptions:  a big breakthrough
      came when someone noticed that certain sequences of dates fit the
      pattern of birth, death, and accession dates for a sequence of
      rulers.  If all the surviving texts had been abstract poetry, they
      would have been much harder to decipher.

A lot of information about the pattern of signs was extracted by
simply examining lots of text.  This is a good method, for example, of
getting preliminary ideas about which patterns of lines represent the
same character.  (Not obvious at first glance for e.g. the head-shaped
numbers in Mayan hieroglyphics.)  But the decipherment proper--the
matching of symbols to meanings--used external information.

Margaret Fleck (Oxford)

jwl@garnet.berkeley.edu (James Wilbur Lewis) (12/05/90)

In article <1990Dec3.134710.25467@canon.co.uk> wachtel@canon.co.uk (Tom Wachtel) writes:
>In article <1990Nov26.055429.8883@agate.berkeley.edu> jwl@garnet.berkeley.edu (James Wilbur Lewis) writes:
>>I'll buy the idea that you might have a good feel for the syntax of a language
>>after such an intensive program of imitation, but *understanding*, in the sense
>>that Searle uses the term?  No way!
>
>The technique of teaching a foreign language using the target language
>as the sole medium of communication has been around for several decades
>(literally) and is very successful, in general.

In learning by immersion, the foreign text is not the only channel of
communication.  The student can often infer the meaning of a new phrase
from social convention (e.g. the first foreign phrase taught will almost
certainly be some sort of greeting), by gestures (pointing to objects
when teaching their names), and by trial-and-error (taking advantage
of the positive and negative feedback provided by the teacher).

In learning by *imitation* (as discussed here), as opposed to immersion, 
none of these cues are present.  Under those circumstances, I don't see 
any way to assign meaning to the symbols being manipulated, even if we 
assume that the syntax is learnable.  (I'll concede the theoretical
possibility, given a sufficiently large body of foreign text, but
the original proposal stipulated "no analysis, just copying", which
is a pretty tough constraint.)

-- Jim Lewis

osborn@ultima.socs.uts.edu.au (An Approxamaton) (12/17/90)

miller@b17a.INGR.COM (David S. Miller) writes:

> Symbols that don't stand for anything aren't symbols at all!

OK... ...what do you mean by anything? And how does the
something (proto-symbol) come to stand for it? Maybe the
something also stands for "something" when it doesn't 
standing for anything else? I wonder what!

Tomasso.
-- 
Tom Osborn,                        "Integrity is not enough...
School of Computing Sciences,          ...you need pepper and salt, too."
University of Technology, Sydney,
PO Box 123 Broadway 2007,  AUSTRALIA.                      Gregory Peck.