jones@amarna.gsfc.nasa.gov (JONES, THOMAS) (12/11/90)
Dear comp.ai.philosophy,
The question has been raised as to whether or not we could put a learning or
"reinforcement" algorithm, perhaps along the lines of Skinner's concepts, and
make the machine *learn* all sorts of neat things without their having to
be programmed in by humans. This is the oldest, worst idea in AI. Dozens
of attempts have been made to carry this effort through (I myself have made
a dozen or so.), essentially without success. The problem is that *all*
theories of learning in psychology are *unsound.* For example, Skinner
would have us believe that, if an organism is rewarded for doing a certain
action in a certain situation, then he/she/it will become more likely to perform
the action in question in "similar" situations. Horsefeathers! What is a
similar action? What is a similar situation? All sorts of heavy machinery
have been swept under the rug and labeled "similarity."
From the above it might be concluded that I am opposed to machine learning
in general. On the contrary, I consider it one of the most important areas
of AI. One bad habit which afflicts learning research is the failure to
distinguish between that which the machine can legitimately learn for itself,
and that which the human programmers jolly well better program in by hand.
For example, I doubt very much if a machine could do more than a few
rudimentary things without the concept of a *subroutine hierarchy* (or the
related GPS goal tree). Hence I believe that the machine should have
software for building up, testing, and using subroutine hierarchies on
its own. But can the machine *invent* subroutine hierarchies? Doubt.
Much of the *nerve net* research is marred by lack of making this distinction.
My experience with learning codes is that you start by working out just how
the performance program is to look and to operate.
References:
Jones, Thomas L., "A Computer Model of Simple Forms of Learning," MIT Ph.D.
thesis, September, 1970.
Jones, Thomas L., "A Computer Model of Simple Forms of Learning in Infants,"
Proc. AFIPS 1972 Spring Joint Computer Conference.
Tom Jones
All opinions are my own.greenba@gambia.crd.ge.com (ben a green) (12/12/90)
In article <4158@dftsrv.gsfc.nasa.gov> jones@amarna.gsfc.nasa.gov (JONES, THOMAS) writes:
The question has been raised as to whether or not we could put a learning or
"reinforcement" algorithm, perhaps along the lines of Skinner's concepts, and
make the machine *learn* all sorts of neat things without their having to
be programmed in by humans. This is the oldest, worst idea in AI. Dozens
of attempts have been made to carry this effort through (I myself have made
a dozen or so.), essentially without success. The problem is that *all*
theories of learning in psychology are *unsound.* For example, Skinner
would have us believe that, if an organism is rewarded for doing a certain
action in a certain situation, then he/she/it will become more likely to perform
the action in question in "similar" situations. Horsefeathers! What is a
similar action? What is a similar situation? All sorts of heavy machinery
have been swept under the rug and labeled "similarity."
An excellent point, but I haven't given up trying yet.
First, we have to choose the right level of description. Subroutines (mentioned
by Tom later) are too low. My choice is based on a robot with, say, 0.1 sec
clock speed that emits behavior (motor control signals) at that rate.
The problem of similarity is, indeed, swept under the rug by Skinner and all
exponents of his ideas I have read. My robot will deal with it in the following
way. First, invert the concept to that of _dissimilarity_. We need a map
from a pair of environmental vectors to a scalar dissimilarity, which can
be thought of as a metric in environmental space.
It will have to be a plastic map, since discrimination training increases
dissimilarity between previously similar environments. Dissimilarity
increases with differential reinforcement. (Think of a spanish speaker
learning to distinguish "b" and "v".)
My suggestion: Initialize the mapping to near zero and stretch the space
as a result of reinforcement. The volume of discriminable dissimilarity
starts out as near zero and expands rapidly in the first hours of the
robot's life.
It's the big bang theory of perception.
--
Ben A. Green, Jr.
greenba@crd.ge.com
Speaking only for myself, of course.smoliar@vaxa.isi.edu (Stephen Smoliar) (12/12/90)
In article <4158@dftsrv.gsfc.nasa.gov> jones@amarna.gsfc.nasa.gov writes: > One bad habit which afflicts learning research is the failure to >distinguish between that which the machine can legitimately learn for itself, >and that which the human programmers jolly well better program in by hand. There is an even worse habit which Minsky discusses in THE SOCIETY OF MIND: The problem is that we use the single word "learning" to cover too diverse a society of ideas. Such a word can be useful in the title of a book, or in the name of an institution. But when it comes to studying the subject itself, we need more distinctive terms for important, different ways to learn. Minsky then goes on to propose some of these terms, not all of which I am sure I agree with; and I suspect I could think up some more given the time. The point is that, like intelligence itself, we assume that anything that can be captured in a single word can, somehow or another, be implemented in code. Anything which counts as a result in machine learning has involved results in a very narrow, highly specific scope. Unfortunately, rather than trying to explore the nature of that scope (let alone consider how it might interact with other, equally narrow scopes), researchers are forever tempted to advertise their results as advances in "machine learning," a claim which lends little to our understanding of just what they have achieved. If we had less inflation of accomplishment, we might discover that our achievements are not as weak as they tend to appear. ========================================================================= USPS: Stephen Smoliar 5000 Centinela Avenue #129 Los Angeles, California 90066 Internet: smoliar@vaxa.isi.edu "It's only words . . . unless they're true."--David Mamet
powers@uklirb.informatik.uni-kl.de (David Powers ) (12/13/90)
Wow! I don't know where to start and I haven't got the time for a full analysis of Machine Learning vis-a-vis Human Learning vis-a-vis Hand Coding. But there's more to learning than meets the eye - that much is agreed. >In article <4158@dftsrv.gsfc.nasa.gov> jones@amarna.gsfc.nasa.gov writes: >The question has been raised as to whether or not we could put a learning or >"reinforcement" algorithm, perhaps along the lines of Skinner's concepts, and >make the machine *learn* all sorts of neat things without their having to >be programmed in by humans. This is the oldest, worst idea in AI. Dozens >of attempts have been made to carry this effort through (I myself have made >a dozen or so.), essentially without success. The problem is that *all* >theories of learning in psychology are *unsound.* For example, Skinner >would have us believe that, if an organism is rewarded for doing a certain >action in a certain situation then he/she/it will become more likely to perform >the action in question in "similar" situations. Horsefeathers! What is a >similar action? What is a similar situation? All sorts of heavy machinery >have been swept under the rug and labeled "similarity." Similarity (or Metaphor) is one of the most important concepts in Learning, and in Science for that matter. Nothing is ever the same as anything else. Even perceptions of the same object at different times are different. So classification of similar things is the first and major step in much of Machine Learning and metaphor is actually the outworking of the same ubiquitous phenomenon in our use of language. Theories of learning which are precomputational are not intended to be complete in the sense that they dot every i and cross every t necessary to code them into a learning program. But they can be used, and the empirical work laying behind them can be reinterpreted and used, to guide and inspire computational theories of learning. I personally agree that some aspects of the dogmatics of certain of the greats of psychology, linguistics and psycholinguistics are misdirected. But who's perfect. The real problem lies with the blind followers who recognize the fundamental truths their mentors exposed, but swallow blindly the inessential baggage as well. No wonder we get indigestion when we try to do some useful work! > From the above it might be concluded that I am opposed to machine learning >in general. On the contrary, I consider it one of the most important areas >of AI. One bad habit which afflicts learning research is the failure to >distinguish between that which the machine can legitimately learn for itself, >and that which the human programmers jolly well better program in by hand. >For example, I doubt very much if a machine could do more than a few >rudimentary things without the concept of a *subroutine hierarchy* (or the >related GPS goal tree). Hence I believe that the machine should have >software for building up, testing, and using subroutine hierarchies on >its own. But can the machine *invent* subroutine hierarchies? Doubt. >Much of the *nerve net* research is marred by lack of making this distinction. >My experience with learning codes is that you start by working out just how >the performance program is to look and to operate. "the failure to distinguish" is actually a consequence of "a failure to examine". The old maxim "you can only learn what you almost already know" is really fundamental. And as your system bootstraps itself from one level to the next - which may not be very far away - A. you need to look for the right techniques and the correct characterization of the prerequisites for this learning (including teacher - what sort of examples, critic - what sort of feedback,input - what sort of features, ...), and B. you must expect that you will only achieve bootstrapping to a level which is not that far removed from where you started. After all, "you can only learn..." smoliar@vaxa.isi.edu (Stephen Smoliar) writes: >There is an even worse habit which Minsky discusses in THE SOCIETY OF MIND: > The problem is that we use the single word "learning" > to cover too diverse a society of ideas. Such a word can be > useful in the title of a book, or in the name of an institution. > But when it comes to studying the subject itself, we need more > distinctive terms for important, different ways to learn. >Minsky then goes on to propose some of these terms, not all of which I am sure >I agree with; and I suspect I could think up some more given the time. The >point is that, like intelligence itself, we assume that anything that can be >captured in a single word can, somehow or another, be implemented in code. >Anything which counts as a result in machine learning has involved results >in a very narrow, highly specific scope. Unfortunately, rather than trying >to explore the nature of that scope (let alone consider how it might interact >with other, equally narrow scopes), researchers are forever tempted to >advertise their results as advances in "machine learning," a claim which >lends little to our understanding of just what they have achieved. If we >had less inflation of accomplishment, we might discover that our achievements >are not as weak as they tend to appear. Here I tend to agree more with the spirit of the comment. And not only may our "weak" achievements be more significant than they appear, in recent times people have tended to apply the "strongest" techniques they can to the learning in an attempt to make the "strongest" achievement, or "biggest" jump in the level of complexity. In fact, it is helpful to consider the inherent structure of what we are learning and what is the weakest form of learning we can use. Some of the classical results about what cannot be learnt are applicable to classes far more general and less restricted than those where we actually want to learn. We need to understand the restrictions, and relate them to the observed phenomena, classes, learning paradigms, etc. We do, of course, have names for some different types of learning. We need no doubt to develop and characterize more. And I have had some success in machine learning in several domains, and in relating the restrictions of the "language" to be learned, the appropriate learning algorithm, the "psychological" correlates, and the "linguistic" classes and rules. I won't list my book and other references here, but feel free to write for a bibliography and/or a paper. David Powers ------------------------------------------------------------------------ David Powers +49-631/205-3449 (Uni); +49-631/205-3200 (Fax) FB Informatik powers@informatik.uni-kl.de; +49-631/13786 (Prv) Univ Kaiserslautern * COMPULOG - Language and Logic 6750 KAISERSLAUTERN * MARPIA - Concurrent Logic Programming WEST GERMANY * STANLIE - Natural Language Learning Riddle: What is the difference between the university and me. Disclaimer: My opinion.