smoliar@vaxa.isi.edu (Stephen Smoliar) (11/27/90)
(Note: I am cross-posting this to comp.ai.philosophy, since that is where these discussions belong; and I hope future debate will be conducted there.) In article <7852@uwm.edu> markh@csd4.csd.uwm.edu (Mark William Hopkins) writes: > >The concept is real simple. Try to learn a new language by imitation. > >So here's what you should do: take out about 10 books written in a language >you >don't understand (like 10 books written in Spanish). The language can be one >that uses the Latin alphabet, or not. It doesn't matter. > >Rewrite the exact contents of each book. That means, WRITE the contents, >don't >analyse them. Don't even think about what it all might mean, just write it. >And write it all. > >This is what will happen. Within about 60 minutes your brain WILL begin to >adapt itself to the regularities of the language. In one day (supposing you >work about 8 hours that day), you will already have a good feel for the >syntax of the language. In about 3 to 5 days, you'll gradually begin to >recognize stylistic regularities. > I noticed that the first reaction to this proposal was one of self-righteous skepticism. Unfortunately, it is very hard to deal with this situation with anything other than anecdotal evidence. Having said that, let me throw out two such anecdotes. One is personal. When I was a graduate student, we used to eat at a Chinese restaurant whose Chinese menu differed from the English menu. One of our group made up a pony, but the waiters refused to let us order by pointing. The eventual compromise which was established with the management was that we would be allowed to WRITE our orders. It was amazing how little time it took to begin to assimilate knowledge of how Chinese characters were put together and even to "parse" those symbols in terms of components which referred to specific food items. (Note that James McCawley has now taken some of the fun out of this exercise with his EATER'S GUIDE TO CHINESE CHARACTERS, but I suspect that variations on this anecdote will continue to unfold among Chinese food fanatics.) The other anecdote is not personal but IS pedagogical. Many decades ago, a fundamental element in the instruction of counterpoint at the Curtis Institute of Music involved copying out entire works by Palestrina. (This approach is, to the best of my knowledge, no longer in practice.) The point here is that the study of counterpoint involves learning a vast complex of constraint rules with little guidance about what to do with them. Copying "real" music gave the student an opportunity to observe the rules in action without having everything pointed out to him explicitly. The assumption was that there was something to be gained by picking up the habit of "writing Palestrina;" and for quite some time this was recognized as a valid pedagogical approach. (Perhaps we are encouraged to write quotations down on 3 X 5 cards not only as a source of reference material but also as an exercise in writing someone else's text.) ========================================================================= USPS: Stephen Smoliar 5000 Centinela Avenue #129 Los Angeles, California 90066 Internet: smoliar@vaxa.isi.edu "It's only words . . . unless they're true."--David Mamet
markh@csd4.csd.uwm.edu (Mark William Hopkins) (11/27/90)
In article <15799@venera.isi.edu> smoliar@vaxa.isi.edu (Stephen Smoliar) writes: >>The concept is real simple. Try to learn a new language by imitation. ... >The other anecdote is not personal but IS pedagogical. Many decades ago, a >fundamental element in the instruction of counterpoint at the Curtis Institute >of Music involved copying out entire works by Palestrina. (This approach is, >to the best of my knowledge, no longer in practice.) The point here is that >the study of counterpoint involves learning a vast complex of constraint rules >with little guidance about what to do with them. Copying "real" music gave the >student an opportunity to observe the rules in action without having everything >pointed out to him explicitly... This anecdote is a real interesting one, in that it's more or less where I derived the idea from. I learned some digital circuit design, a host of programming languages, assembly languages and even binaries this way, with a great deal of success. And that's not counting the non-computer-related tasks I applied the idea to :) It just occurred to me a couple days ago to try it on natural language. Forget formal training, just go pick up books written in the languages and simply copy them. That was the idea. But the question, having to do with the Chinese Room experiment, is whether you can actually converge onto an understanding of the language in this way WITHOUT ever 'cheating' by looking up translation references. In all the cases above I 'cheated' by getting a hold of electronics texts (for TTL circuits), and language references, and by using my own high-level language programs as compiler inputs (source code written in C is a 'translation' of assembly code written, say, for a VAX). Of course, if you really want to try this on a task where there's no opportunity for cheating, you'd get a hold of all the Mayan Hieroglyphic sources and copy them all by tracing. I almost did that a few years ago... :)
markh@csd4.csd.uwm.edu (Mark William Hopkins) (11/27/90)
In article <15799@venera.isi.edu> smoliar@vaxa.isi.edu (Stephen Smoliar) writes: >(Note: I am cross-posting this to comp.ai.philosophy, since that is where >these discussions belong; and I hope future debate will be conducted there.) Actually, I was going to use the experiment (which is more than just a thought experiment) as the basis for an argument simultaneously against Searle and advocates of strong AI showing why neurocomputers provide a better means for realising true AI, compared to purely symbolic architectures ... and why a machine (with a connectionist architecture) might be able to 'understand' language, rather than just imitate it. (I'm assuming here that Searle's position entails an argument against *any* machine ever 'understanding' language). (The experiment was): >>The concept is real simple. Try to learn a new language by imitation. ... >>So here's what you should do: take out about 10 books written in a language >>you don't understand (like 10 books written in Spanish)... Rewrite the exact >>contents of each book. >I noticed that the first reaction to this proposal was one of self-righteous >skepticism. Unfortunately, it is very hard to deal with this situation with >anything other than anecdotal evidence. Having said that, let me throw out >two such anecdotes. Both of your anecdotes actually relate to a general situation that you might call learning-by-imitation, which bears an obvious relation to the ancient method of learning-by-apprenticeship. It seems to me that this is by far the most natural and most efficient way for a human being to learn new information and to accquire new skills and new expertise. It's also obviously the oldest method, since it's the only one that doesn't require formal schooling or reading and writing. Symbolic architectures can only incorporate this desireable feature in a roundabout way ... because they're not explicitly designed to adapt over time. On the other hand, the ability for the system to adapt itself to the regularities of its environment is something that almost characterises connectionist learning architectures (like backpropagation). Another part of the experiment relates to a situation (which will eventually occur) where you can actually become fluent or knowledgeable in an area (particularily in a new language) without actually having the slightest idea of what it is you're saying or doing (or why it works). Though I expressed an opinion that an understanding of the language probably will suddenly emerge like a bolt out of the blue, even without the aid of translation resources, I'd have to also say that a symbolic architecture would never be able to do this unless it's design were eventually modified in a way that it came to more and more resemble a neural net. There's a phenomena of 'convergence' pervading this entire experiment that in and of itself seems to entail the existence of an underlying connnectionist architecture.
smoliar@vaxa.isi.edu (Stephen Smoliar) (11/28/90)
In article <7890@uwm.edu> markh@csd4.csd.uwm.edu (Mark William Hopkins) writes: >In article <15799@venera.isi.edu> smoliar@vaxa.isi.edu (Stephen Smoliar) >writes: > >(The experiment was): >>>The concept is real simple. Try to learn a new language by imitation. >... >>>So here's what you should do: take out about 10 books written in a language >>>you don't understand (like 10 books written in Spanish)... Rewrite the exact >>>contents of each book. > >>I noticed that the first reaction to this proposal was one of self-righteous >>skepticism. Unfortunately, it is very hard to deal with this situation with >>anything other than anecdotal evidence. Having said that, let me throw out >>two such anecdotes. > >Both of your anecdotes actually relate to a general situation that you might >call learning-by-imitation, which bears an obvious relation to the ancient >method of learning-by-apprenticeship. > >It seems to me that this is by far the most natural and most efficient way for >a human being to learn new information and to accquire new skills and new >expertise. It's also obviously the oldest method, since it's the only one >that >doesn't require formal schooling or reading and writing. > >Symbolic architectures can only incorporate this desireable feature in a >roundabout way ... because they're not explicitly designed to adapt over >time. On the other hand, the ability for the system to adapt itself to the >regularities of its environment is something that almost characterises >connectionist learning architectures (like backpropagation). > Now that the propaganda ploy has been revealed, perhaps we can try and sort out just what issues are at stake. Of the above three paragraphs, I agree entirely with the first two. The third, however, gives me pause. My own feeling is that BOTH symbolic and connectionist architectures may be laboring under the illusion that one may implement some sort of direct path to learning-by-apprenticeship. By using the word "illusion," I hope to have conveyed the opinion that such a direct path may not be particularly feasible. My current feeling is that Gerald Edelman's recent work on a "biological theory of consciousness" (THE REMEMBERED PRESENT) might provide a better approach to how such a goal may eventually be attained. Let me try to outline my interpretation of his story. Edelman's basic approach is that the ability to form perceptual categories lies at the heart of all our cognitive behavior. Furthermore, it is important that such an ability develop in the absence of any A PRIORI terminology which would basically serve as "hints" about what categories to look for. This is his fundamental beef against advocates of symbolic architectures: You're garden variety scene analysis system on a symbolic architecture is helpless without a knowledge base which endows it with some sort of "world model." Connectionist architectures, on the other hand, do not require such A PRIORI terminology. (See "Feature Discovery by Competitive Learning" by Rumelhart and Zipser--Chapter 5 of the "PDP Bible"-- for a nice demonstration of this claim.) However, there is much more to Edelman's story than perceptual categorization. Most important is that his whole story about memory is one of REcategorization. Categories are not static. They are constantly being revised with every new stimulus that comes along. As Oliver Sacks recently pointed out in his NEW YORK REVIEW article on this subject, a fundamental difference between computers and people is that computer memories retrieve the same thing each time they are probed, while people are far more fallible (which, as Sacks points out, is less of a bug and more of a feature when you start getting into more advanced forms of behavior). This is where Edelman begins to depart from the connectionists, since convergence is not one of his objectives. His argument is that if one wishes to build a "conscious machine" (and let us assume that such consciousness is necessary for such behavior as learning-by-apprenticeship), that machine had better be VERY dynamic, even at the level of something such as memory which we tend to think must be relatively static. If you are going to deal with a system which is always in flux, the next thing you are going to need is a way to deal with time. In other words this dynamic system must deal with the fact that it resides in a dynamic world. Another way of putting it is that you are going to need some kind of machinery which can sort out those system dynamics concerned with stimuli of the present from those based on stimuli of the past, as well as motor control hardware which can plan actions for the future. Edelman argues that the brain has such hardware in the hippocampus, the cerebellum, and the basal ganglia. At this point you can gingerly start to approach questions of learning. With all those dynamics, "concepts" cannot be expected to be implemented in the clean logical forms of symbolic knowledge bases. Rather, they may be regarded as a recursive layer of categorization--categories of the patterns of behavior arising from the dynamics of the system described thus far. Actually, there will be categories for both what our intuition would call concepts and RELATIONS between those concepts. Such categories ultimately will entail associations between perceived stimuli and recollections of past stimuli, to the point where we have some level of "imaging" power by which a memory induces neural activity similar to that triggered by "real" stimuli. This brings us to a capability which Edelman calls "primary consciousness." Note that no symbols have yet entered the picture. They come in through the acquisition of linguistic capabilities, which Edelman views as yet another recursive layer of category formation. At this stage we begin to associate LABELS with out memories. The labels, themselves, then assume the role of new categories; and relations among those categories bring us to the structure of language. Only after all these layers are in place are we in a position to "reason" with them, i.e. use them to expand our view of the world and our associated "mental state." This is the point at which we are doing learning, and we are using all the machinery outlined in the preceding paragraphs to do it. The bottom line, then, is that symbolic architectures definitely have critical shortcomings; but connectionist learning architectures which throw so much weight on convergence are probably no better off. We are too interested in building some sort of transducer with predictable input-output behavior than a dynamic system constantly reconfiguring itself as a result of its exposure to a changing world. Needless to say, building such a system is intimidating, to say the least. Therefore, we should take some comfort in the fact that Edelman and his colleagues have already begun to build working simulations of the first stages of his model. Such simulations may point the way to systems capable of such skills as learning-by-apprenticeship. >Another part of the experiment relates to a situation (which will eventually >occur) where you can actually become fluent or knowledgeable in an area >(particularily in a new language) without actually having the slightest idea >of what it is you're saying or doing (or why it works). > This can happen quite frequently when you learn by imitation. I would also argue that one of the reasons it happens in that imitation requires that the BODY learn. In other words we cannot take a dualist view which divorces mind from body. The WHOLE SYSTEM is what is doing the learning; and often the body gets involved in the "rhythm" or "ritual" of a situation, guiding behavior to those actions which are appropriate to solving a given problem or making an appropriate decision. ========================================================================= USPS: Stephen Smoliar 5000 Centinela Avenue #129 Los Angeles, California 90066 Internet: smoliar@vaxa.isi.edu "It's only words . . . unless they're true."--David Mamet
smoliar@vaxa.isi.edu (Stephen Smoliar) (11/28/90)
In article <7888@uwm.edu> markh@csd4.csd.uwm.edu (Mark William Hopkins) writes: > >But the question, having to do with the Chinese Room experiment, is whether >you can actually converge onto an understanding of the language in this way >WITHOUT ever 'cheating' by looking up translation references. Having gone on at some length about the general issue of learning, I would only like to make a few minor comments about Mark's other article. First of all, I STILL think it is important that those of us who are really interested in BUILDING systems steer clear of words like "understanding." As I have pointed out in reviewing Turing's original paper, Turing was well aware of this danger and knew how to think like an engineer when the situation demanded. This bulletin board may be devoted to questions of philosophy, but that does not mean we should fall into all the traps inherent in philosophical digressions. The second point is that, even if we substitute some more mundane goal for "understanding"--such as some variation on Turing's "imitation game" which would provide a demonstration of learning-by-apprenticeship--we should not assume that we can attain any sort of "convergence" on that goal. We are not interested in systems that converge; rather, we should attend to systems that manage in the world. Systems that converge tend to lack the flexibility which such management demands. Having said all that, I want to raise one final point on this issue of "cheating." The fact is that what Mark calls "cheating" is part of what we accept as intelligent behavior among humans. After all, human memory is not so perfect that we never have to use a dictionary. (Indeed, we probably can all mention entries which we have consulted not just once but multiple times.) Indeed, what makes the GEDANKENEXPERIMENT interesting is that there is some point in the accumulation of experience after which such reference materials become USEFUL--rather than some obscure resource of symbols. Learning-by-apprenticeship is as much a matter of learning to use reference materials as it is learning to work with "primary objects." > >Of course, if you really want to try this on a task where there's no >opportunity for cheating, you'd get a hold of all the Mayan Hieroglyphic >sources and copy them all by tracing. I almost did that a few years ago... :) My conjecture is that you would certainly get as far as forming perceptual categories. You might even get a bit farther. However, you would never get to a point of associating what you were transcribing with any observations of the real world. That would severely inhibit your ability to "learn" very much. ========================================================================= USPS: Stephen Smoliar 5000 Centinela Avenue #129 Los Angeles, California 90066 Internet: smoliar@vaxa.isi.edu "It's only words . . . unless they're true."--David Mamet