neuron-request@HPLABS.HP.COM ("Neuron-Digest Moderator Peter Marvit") (01/11/90)
Neuron Digest Wednesday, 10 Jan 1990
Volume 6 : Issue 2
Today's Topics:
What is a Symbol System?
Re: What is a Symbol System?
Re: What is a Symbol System?
Re: What is a Symbol System?
Re: What is a Symbol System?
Re: What is a Symbol System?
Re: What is a Symbol System?
Re: What is a Symbol System?
Re: What is a Symbol System?
What is a symbol system?
Re: What is a Symbol System?
Send submissions, questions, address maintenance and requests for old issues to
"neuron-request@hplabs.hp.com" or "{any backbone,uunet}!hplabs!neuron-request"
Use "ftp" to get old issues from hplpm.hpl.hp.com (15.255.176.205).
------------------------------------------------------------
Subject: What is a Symbol System?
From: harnad@phoenix.Princeton.EDU (S. R. Harnad)
Organization: Princeton University, NJ
Date: 20 Nov 89 04:39:12 +0000
What is a symbol system? From Newell (1980) Pylyshyn (1984), Fodor (1987)
and the classical work of Von Neumann, Turing, Goedel, Church, etc.(see
Kleene 1969) on the foundations of computation, we can reconstruct the
following definition:
A symbol system is:
(1) a set of arbitrary PHYSICAL TOKENS (scratches on paper, holes on a
tape, events in a digital computer, etc.) that are
(2) manipulated on the basis of EXPLICIT RULES that are
(3) likewise physical tokens and STRINGS of tokens. The rule-governed
symbol-token manipulation is based
(4) purely on the SHAPE of the symbol tokens (not their "meaning"), i.e.,
it is purely SYNTACTIC, and consists of
(5) RULEFULLY COMBINING and recombining symbol tokens. There are
(6) primitive ATOMIC symbol tokens and
(7) COMPOSITE symbol-token strings. The entire system and all its parts --
the atomic tokens, the composite tokens, the syntactic manipulations (both
actual and possible) and the rules -- are all
(8) SEMANTICALLY INTERPRETABLE: The syntax can be SYSTEMATICALLY assigned a
meaning (e.g., as standing for objects, as describing states of affairs).
According to proponents of the symbolic model of mind such as Fodor (1980)
and Pylyshyn (1980, 1984), symbol-strings of this sort capture what mental
phenomena such as thoughts and beliefs are. Symbolists emphasize that the
symbolic level (for them, the mental level) is a natural functional level
of its own, with ruleful regularities that are independent of their
specific physical realizations. For symbolists, this
implementation-independence is the critical difference between cognitive
phenomena and ordinary physical phenomena and their respective
explanations. This concept of an autonomous symbolic level also conforms to
general foundational principles in the theory of computation and applies to
all the work being done in symbolic AI, the branch of science that has so
far been the most successful in generating (hence explaining) intelligent
behavior.
All eight of the properties listed above seem to be critical to this
definition of symbolic. Many phenomena have some of the properties, but
that does not entail that they are symbolic in this explicit, technical
sense. It is not enough, for example, for a phenomenon to be INTERPRETABLE
as rule-governed, for just about anything can be interpreted as
rule-governed. A thermostat may be interpreted as following the rule: Turn
on the furnace if the temperature goes below 70 degrees and turn it off if
it goes above 70 degrees, yet nowhere in the thermostat is that rule
explicitly represented.
Wittgenstein (1953) emphasized the difference between EXPLICIT and IMPLICIT
rules: It is not the same thing to "follow" a rule (explicitly) and merely
to behave "in accordance with" a rule (implicitly). The critical difference
is in the compositeness (7) and systematicity (8) criteria. The explicitly
represented symbolic rule is part of a formal system, it is decomposable
(unless primitive), its application and manipulation is purely formal
(syntactic, shape-dependent), and the entire system must be semantically
interpretable, not just the chunk in question. An isolated ("modular")
chunk cannot be symbolic; being symbolic is a combinatory, systematic
property.
So the mere fact that a behavior is "interpretable" as ruleful does not
mean that it is really governed by a symbolic rule. Semantic
interpretability must be coupled with explicit representation (2),
syntactic manipulability (4), and systematicity (8) in order to be
symbolic. None of these criteria is arbitrary, and, as far as I can tell,
if you weaken them, you lose the grip on what looks like a natural category
and you sever the links with the formal theory of computation, leaving a
sense of "symbolic" that is merely unexplicated metaphor (and probably
differs from speaker to speaker).
Any rival definitions, counterexamples or amplifications?
Excerpted from:
Harnad, S. (1990) The Symbol Grounding Problem. Physica D (in press)
- -----------------------------------------------------
References:
Fodor, J. A. (1975) The language of thought. New York: Thomas Y. Crowell
Fodor, J. A. (1987) Psychosemantics. Cambridge MA: MIT/Bradford.
Fodor, J. A. & Pylyshyn, Z. W. (1988) Connectionism and cognitive
architecture: A critical appraisal. Cognition 28: 3 - 71.
Harnad, S. (1989) Minds, Machines and Searle. Journal of Theoretical
and Experimental Artificial Intelligence 1: 5-25.
Kleene, S. C. (1969) Formalized recursive functionals and formalized
realizability. Providence, R.I.: American Mathematical Society.
Newell, A. (1980) Physical Symbol Systems. Cognitive Science 4: 135-83.
Pylyshyn, Z. W. (1980) Computation and cognition: Issues in the
foundations of cognitive science. Behavioral and Brain Sciences
3: 111-169.
Pylyshyn, Z. W. (1984) Computation and cognition. Cambridge MA:
MIT/Bradford
Turing, A. M. (1964) Computing machinery and intelligence. In: Minds
and machines, A.R. Anderson (ed.), Engelwood Cliffs NJ: Prentice Hall.
Stevan Harnad INTERNET: harnad@confidence.princeton.edu
harnad@princeton.edu srh@flash.bellcore.com harnad@elbereth.rutgers.edu
harnad@princeton.uucp CSNET: harnad%confidence.princeton.edu@relay.cs.net
BITNET: harnad1@umass.bitnet harnad@pucc.bitnet (609)-921-7771
------------------------------
Subject: Re: What is a Symbol System?
From: fateman@renoir.Berkeley.EDU (Richard Fateman)
Organization: University of California, Berkeley
Date: 20 Nov 89 17:19:49 +0000
>A symbol system is:
>
>(1) a set of arbitrary PHYSICAL TOKENS (scratches on paper, holes on
>a tape, events in a digital computer, etc.) that are
......>
[manipulated based on a rule system that is]
>i.e., it is purely SYNTACTIC, ....
If you believe the syntactic rules (a^b)^c <--> a^(b*c) and a*b <--> b*a then
- -1 = (-1)^1 = (-1)^(2* (1/2)) = ((-1)^2)^1/2) = 1^(1/2) = 1.
from which anything else can be proven. So if you want to do mathematics
correctly, this approach or representation is wrong.
....
>All eight of the properties listed above seem to be critical to this
>definition of symbolic. Many phenomena have some of the properties, but
>that does not entail that they are symbolic in this explicit, technical
>sense.
In other words, with respect to symbolic mathematical symbol manipulation
systems, Harnad is making an observation that pertains to those "syntactic"
ones which (perhaps inevitably incorrectly) manipulate uninterpreted trees.
His observation is merely that anything else should not be called a "symbol
system". Since usage contradicts Harnad's contention, what's the point?
Should we change the name of our newsgroup? :)
Richard Fateman
fateman@renoir.berkeley.edu
------------------------------
Subject: Re: What is a Symbol System?
From: lee@uhccux.uhcc.hawaii.edu (Greg Lee)
Organization: University of Hawaii
Date: 20 Nov 89 18:32:22 +0000
" ...
" Wittgenstein (1953) emphasized the difference between EXPLICIT and
" IMPLICIT rules: [[...]]
`Explicit' is not the same thing as `explicitly represented'. I'm not sure
I see how all the rules could be formalized, but in any case, they needn't
be.
Greg, lee@uhccux.uhcc.hawaii.edu
------------------------------
Subject: Re: What is a Symbol System?
From: mcdermott-drew@CS.YALE.EDU (Drew McDermott)
Organization: Yale University Computer Science Dept, New Haven CT 06520-2158
Date: 20 Nov 89 18:44:01 +0000
[[Regarding Harnad's list of symbol systems]]
I have two quibbles with this list:
(a) Items 2&3: I agree that the rules have to be explicit, but they are
usually written in a different notation from the one they manipulate.
Example: A theorem prover written in Lisp. Another example: The weights in
a neural net.
(b) Item 8: Why is it necessary that a symbol system have a semantics in
order to be a symbol system? I mean, you can define it any way you like,
but then most AI programs wouldn't be symbol systems in your sense. I and
others have spent some time arguing that symbol systems *ought* to have a
semantics, and it's odd to be told that I was arguing in favor of a
tautology. (Or that, now that I've changed my mind, I believe a
contradiction.)
Perhaps you have in mind that a system couldn't really think, or couldn't
really refer to the outside world without all of its symbols being part of
some seamless Tarskian framework. (Of course, *you* don't think this, but
you feel that charity demands you impute this belief to your opponents.) I
think you have to buy several extra premises about the potency of knowledge
representation to believe that formal semantics is that crucial.
-- Drew McDermott
------------------------------
Subject: Re: What is a Symbol System?
From: geddis@Neon.Stanford.EDU (Donald F. Geddis)
Organization: Computer Science Department, Stanford University
Date: 20 Nov 89 20:38:23 +0000
In previous article fateman@renoir.Berkeley.EDU (Richard Fateman) writes:
>If you believe the syntactic rules (a^b)^c <--> a^(b*c) and a*b <--> b*a then
>-1 = (-1)^1 = (-1)^(2* (1/2)) = ((-1)^2)^1/2) = 1^(1/2) = 1.
1^(1/2) = (-1 or 1), not just 1. Your last step was not one of the
"syntactic rules" that I "believe".
If you reverse the order, so you start with 1=1 and then go to 1=1^(1/2),
that is false, in the strict sense. The left-hand side is equivalent to 1,
while the right-hand side is equivalent to (-1 or 1).
-- Don Geddis
Geddis@CS.Stanford.Edu
"There is no dark side of the moon, really. Matter of fact, it's all dark."
------------------------------
Subject: Re: What is a Symbol System?
From: hardt@linc.cis.upenn.edu (Dan Hardt)
Organization: University of Pennsylvania
Date: 20 Nov 89 22:06:40 +0000
I'm not sure how you can sharply distinguish between a system that is
interpretable as rule-governed and one that is explicitly rule governed.
Perhaps you have in mind a connectionist network on the one hand, where
what is syntactically represented might be things like weights of
connections, and the rules only emerge from the overall behavior of the
system; on the other hand, an expert system, where the rules are all
explicitly written in some logical notation. Would you characterize the
connectionist network as only interpretable as being rule-governed, and the
expert system as being explicitly rule governed? If it is that sort of
distinction you have in mind, I'm not sure how the criteria given allow you
make it. If fact, I wonder how you can rule out any turing machine.
------------------------------
Subject: Re: What is a Symbol System?
From: harnad@phoenix.Princeton.EDU (S. R. Harnad)
Organization: Princeton University, NJ
Date: 20 Nov 89 22:07:02 +0000
mcdermott-drew@CS.YALE.EDU (Drew McDermott) of Yale University Computer
Science Dept asked:
> Why is it necessary that a symbol system have a semantics in order to
> be a symbol system? I mean, you can define it any way you like, but
> then most AI programs wouldn't be symbol systems in your sense.
>
> Perhaps you have in mind that a system couldn't really think, or
> couldn't really refer to the outside world without all of its symbols
> being part of some seamless Tarskian framework... I think you have to
> buy several extra premises about the potency of knowledge
> representation to believe that formal semantics is that crucial.
I'd rather not define it any way I like. I'd rather pin people down on a
definition that won't keep slipping away, reducing all disagrements about
what symbol systems can and can't do to mere matters of interpretation.
I gave semantic interpretability as a criterion, because it really seems to
be one of the properties people have in mind when they single out symbol
systems. However, semantic interpretability is not the same as having an
intrinsic semantics, in the sense that mental processes do. But I made no
reference to anything mental ("thinking," reference," "knowledge") in the
definition.
So the only thing at issue is whether a symbol system is required to be
semantically interpretable. Are you really saying that most AI programs are
not? I.e., that if asked what this or that piece of code means or does, the
programmer would reply: "Beats me! It's just crunching a bunch of
meaningless and uninterpretable symbols."
No, I still think an obvious sine qua non of both the formal symbol systems
of mathematics and the computer programs of computer science and AI is that
they ARE semantically interpretable.
Stevan Harnad Department of Psychology Princeton University
harnad@confidence.princeton.edu srh@flash.bellcore.com
harnad@elbereth.rutgers.edu harnad@pucc.bitnet (609)-921-7771
------------------------------
Subject: Re: What is a Symbol System?
From: harnad@phoenix.Princeton.EDU (S. R. Harnad)
Organization: Princeton University, NJ
Date: 21 Nov 89 01:06:33 +0000
[[ Referencing Dan Hardt's previous message ]]
I'm willing to let the chips fall where they may. All I'm trying to do is
settle on criteria for what does and does not count as symbol, symbol
system, symbol manipulation.
Here is an easy example. I think it contains all the essentials: We have
two Rube Goldberg devices, both beginning with a string you pull, and both
ending with a hammer that smashes a piece of china. Whenever you pull the
string, the china gets smashed by the hammer in both systems. The question
is: Given that they can both be described as conforming to the rule "If the
string is pulled, smash the china," is this rule explicitly represented in
both systems?
Let's look at them more closely: One turns out to be pure causal
throughput: The string is attached to the hammer, which is poised like a
lever. Pull the string and the hammer goes down. Bang!
In the other system the string actuates a transducer which sends a data
bit to a computer program capable of controlling a variety of devices all
over the country. Some of its input can come from strings at other
locations, some from airline reservations, some from missile control
systems. Someone has written a lot of flexible code. Among the primitives
of the system are symbol tokens such as STRING, ROPE, CABLE, PULL, HAMMER,
TICKET, BOMB, LOWER, LAUNCH, etc. In particular, one symbol string is "IF
PULL STRING(I), LOWER HAMMER(J)," and this sends out a data bit that
triggers and effector that brings the hammer down. Bang! The system also
represents "If PULL STRING(J), LOWER HAMMER(J)," "IF PULL STRING(J),
RELEASE MISSILE(K)," etc. etc. The elements can be recombined as you would
expected, based on a gloss of their meanings, and the overall
interpretation of what they stand for is systematically sustained. (Not all
possible symbol combinations are enabled, necessarily, but they all make
systematic sense.) The explicitness of rules and representations is based
on this combinatory semantics.
It is in the latter kind of symbol economy that the rule is said to be
explicitly represented. The criteria I listed do allow me to make this
distinction. And I'm certainly not interested in ruling out a Turing
Machine -- the symbol system par excellence. The extent to which
connectionist networks can and do represent rules explicitly is still
unsettled.
Stevan Harnad Department of Psychology Princeton University
harnad@confidence.princeton.edu srh@flash.bellcore.com
harnad@elbereth.rutgers.edu harnad@pucc.bitnet (609)-921-7771
------------------------------
Subject: Re: What is a Symbol System?
From: Mitsuharu Hadeishi <well!mitsu@apple.com>
Date: Mon, 20 Nov 89 19:31:20 -0800
This is an interesting question. First of all, I think it is clear that
since a recurrent neural network can emulate any finite-state automaton
that they are Turing equivalent goes almost without saying, so it is also
clear that recurrent NNs should be capable of the symbolic-level processing
of which you speak.
First of all, however, I'd like to address the symbolist point of view that
higher-level cognition is purely symbolic, irrespective of the
implementation scheme. I submit this is patently absurd. Symbolic
representations of thought are simply models of how we think, and quite
crude models at that. They happen to have several redeeming qualities
however, among them that they are simple, well-defined, and easy to
manipulate.
However, in truth, though it is clear that many operations (such as
syntactic analysis of language) operate within the structure, at least in
part, of symbolic processing, others go outside (such as understanding a
subtle poem). In addition, there are many other forms of higher-level
cognition, such as that which visual artists engage themselves in, which do
not easily lend themselves to symbolic decomposition. I submit that even
everyday actions and thoughts do not follow any strict symbolic
decomposition, though to some degree of approximation they can be modelled
*as though* they were following rules of some kind.
I think the comparison between rule-based and analog systems is apt;
however, in my opinion it is the analog systems which have the greater
flexibility, or one might say economy of expression. That is to say,
inasmuch as one can emulate one with the other they are equivalent, but
given limitations on complexity and size I think it is clear the complex
analog dynamical systems have the edge.
The fact is that as a model for the world or how we think rule-based
representations are sorely lacking. It is similar to trying to paint a
landscape using polygons; one can do it, but it is not particularly
well-suited for the task, except in very simple situations (or situations
where the landscape happens to be man-made.)
We should not confuse the map with the territory. Just because we happen
to have this crude model for thinking, i.e., the symbolic model, does not
mean that is *how* we think. We may even describe our decisions this way,
but the intractability of AI problems except for very limited-domain
applications indicates or suggests the weaknesses with our model. For
example, natural language systems only work with extremely limited context.
The fact that they do work at all is evidence that our symbolic models are
not completely inadequate, however, that they are limited in domain
suggests they are nonetheless mere approximations. Connectionist models, I
believe, have much greater chance at capturing the true complexity of
cognitive systems.
In addition, the recent introduction of fuzzy reasoning and nonmonotonic
logic are extensions of the symbolic model which certainly improve the
situation, but also point out the main weaknesses with symbolic models of
cognition. Symbolic models address only one aspect of the thinking
process, perhaps not even the most important part. For example, a master
chess player typically only considers about a hundred possible moves, yet
can beat a computer program that considers tens of thousands of moves. The
intractability of even more difficult problems than chess also points this
out. Before the symbolic engine can be put into action, a great deal of
pre-processing goes on which will likely not be best described in symbolic
terms.
Mitsu Hadeishi
Open Mind
16110 S. Western Avenue
Gardena, CA 90247
(213) 532-1654
(213) 327-4994
mitsu@well.sf.ca.us
------------------------------
Subject: What is a symbol system?
From: mclennan%MACLENNAN.CS.UTK.EDU@cs.utk.edu
Date: Tue, 21 Nov 89 14:29:24 -0400
Steve Harnad has invited rival definitions of the notion of a symbol
system. I formulated the following (tentative) definition as a basis for
discussion in a connectionism course I taught last year. After stating the
definition I'll discuss some of the ways it differs from Harnad's.
PROPERTIES OF DISCRETE SYMBOL SYSTEMS
A. Tokens and Types
1. TOKENS can be unerringly separated from the background.
2. Tokens can be unambiguously classified as to TYPE.
3. There are a finite number of types.
B. Formulas and Schemata
1. Tokens can be put into relationships with one another.
2. A FORMULA is an assemblage of interrelated tokens.
3. Formulas comprise a finite number of tokens.
4. Every formula results from a computation (see below)
starting from a given token.
5. A SCHEMA is a class of relationships among tokens that
depends only on the types of those tokens.
6. It can be unerringly determined whether a formula
belongs to a given schema.
C. Rules
1. Rules describe ANALYSIS and SYNTHESIS.
2. Analysis determines if a formula belongs to a given
schema.
3. Synthesis constructs a formula belonging to a given
schema.
4. It can be unerringly determined whether a rule applies
to a given formula, and what schema will result from
applying that rule to that formula.
5. A computational process is described by a finite set of
rules.
D. Computation
1. A COMPUTATION is the successive application of the
rules to a given initial formula.
2. A computation comprises a finite number of rule appli-
cations.
COMPARISON WITH HARNAD'S DEFINITION
1. Note that my terminology is a little different from Steve's:
his "atomic tokens" are my "tokens", his "composite tokens"
are my "formulas". He refers to the "shape" of tokens,
whereas I distinguish the "type" of an (atomic) token from
the "schema" of a formula (composite token).
2. So far as I can see, Steve's definition does not include
anything corresponding to my A.1, A.2, B.6 and C.4. There
are all "exactness" properties -- central, although rarely
stated, assumptions in the theory of formal systems. For
example, A.1 and A.2 say that we (or a Turing machine) can
tell when we're looking at a symbol, where it begins and
ends, and what it is. It is important to state these
assumptions, because they need not hold in real-life pattern
identification, which is imperfect and inherently fuzzy.
One reason connectionism is important is that by questioning
these assumptions it makes them salient.
3. Steve's (3) and (7), which require formulas to be LINEAR
arrangements of tokens, are too restrictive. There is noth-
ing about syntactic arrangement that requires it to be
linear (think of the schemata used in long division).
Indeed, the relationship between the constituent symbols
need not even be spatial (e.g., they could be "arranged" in
the frequency domain, e.g., a chord is a formula comprising
note tokens). This is the reason my B.5 specified only
"relationships" (perhaps I should have said "physical rela-
tionships").
4. Steve nowhere requires his systems to be finite (although it
could be argued that this is a consequence of their being
PHYSICAL systems). I think finiteness is essential. The
theory of computation grew out of Hilbert's finitary
approach to the foundations of mathematics, and you don't
get the standard theory of computation if infinite formulas,
rules, sets of rules, etc. are allowed. Hence my A.3, B.3,
C.5, D.2.
5. Steve requires symbol systems to be semantically interpret-
able (8), but I think this is an empty requirement. Every
symbol system is interpretable -- if only as itself (essen-
tially the Herbrand interpretation). Also, mathematicians
routinely manipulate formulas (e.g., involving differen-
tials) that have no interpretation (in standard mathematics,
and ignoring "trivial" Herbrand-like interpretations).
6. Steve's (1) specifies a SET of formulas (physical tokens),
but places no restrictions on that set. I'm concerned that
this may permit uncountable or highly irregular sets of for-
mulas (e.g., all the uncomputable real numbers). I tried to
avoid this problem by requiring the formulas to be generat-
able by a finite computational process. This seems to hold
for all the symbol systems discussed in the literature; in
fact the formation rules are usually just a context-free
grammar. My B.4 says, in effect, that there is a generative
grammar (not necessarily context free) for the formulas, in
fact, that the set of formulas is recursively enumerable.
7. My definition does not directly require a rule itself to be
expressible as a formula (nearly Steve's 3), but I believe I
can derive this from my C.1, C.2, C.3, although I wouldn't
want to swear to it. (Here's the idea: C.2 and C.3 imply
that analysis and synthesis can be unambiguously described
by formulas that are exemplars of those schemata. Hence, by
C.1, every rule can be described by two examplars, which are
formulas.)
Let me stress that the above definition is not final. Please
punch holes in it!
Bruce MacLennan
Department of Computer Science
107 Ayres Hall
The University of Tennessee
Knoxville, TN 37996-1301
(615)974-0994/5067
maclennan@cs.utk.edu
------------------------------
Subject: Re: What is a Symbol System?
From: jiii@visdc.UUCP (John E Van Deusen III)
Organization: VI Software Development, Boise, Idaho
Date: 21 Nov 89 20:38:57 +0000
> Here is an easy example. I think it contains all the essentials:
[[ text omitted ]]
I believe that artificial intelligence is only concerned with the problem
of if and when to pull one of the strings. Once the string is pulled or
not, the result is deterministic. The china may break or it may not, but
the result requires no intelligence. It seems kind of clear that if we
want to consider artificial intelligence distinct from the chaotic
determinism in which it is embedded, we have to resort to some sort of
contrived formalism.
Like others, I think of intelligence as a recognizer of a language taken
over an alphabet of symbols. Not only is this mathematical contraption
capable of doing anything, in the sense of "knowing" precisely when to to
pull the string, but it is brutishly mechanistic and free from subjective
magic, (although seldom possible to build). In such a model it is
fruitless to quibble about what is to be included in the set of symbols,
since the set of possible languages taken over an alphabet even as simple
as {a, b} is infinite.
John E Van Deusen III, PO Box 9283, Boise, ID 83707, (208) 343-1865
uunet!visdc!jiii
------------------------------
End of Neurons Digest
*********************