[comp.ai] Objective measurement of subjective variables

adam@mtund.UUCP (01/25/87)

John Cugini:
>>	                        .... to explain, or at least discuss, private
>>	subjective events.  If it be objected that the latter are outside the
>>	proper realm of science, so be it, call it schmience or philosophy or
>>	whatever you like. - but surely anything that is REAL, even if
>>	subjective, can be the proper object for some sort of rational
>>	study, no?
Stevan Harnad:
>   Some sort, no doubt. But not an objective sort, and that's the point.
>   Empirical psychology, neuroscience and artificial intelligence are
>   all, I presume, branches of objective inquiry.
>          ....   Let's leave the subjective discussion of private events
>   to lit-crit, where it belongs.

Stevan Harnad makes an unstated assumption here, namely, that subjective
variables are not amenable to objective measurement. But if by
"objective" Steve means, as I think he does, "observer-invariant", than
this assumption is demonstrably false. I shall proceed to demonstrate
this in two parts: (1) private events are amenable to parametric
measurement; and (2) relevant results of such measurement can be
observer-invariant.

(1) Whether or not a stimulus is experienced as belonging to some target
category is clearly a private event. Now data for the measurement of d',
the detection-theoretic measure of discriminability, are usually
gathered using overt behavior, such as pressing "target" and
"non-target" buttons. But in principle, d' can be measured without any
resort to externally observable behavior. Suppose I program a computer
to present a sequence of stimuli and, following enough time after
each stimulus to allow the observer to mentally classify the experience
as target or non-target, display the actual category of the preceding
stimulus. The observer would use this information to maintain a mental
count of hits and false alarms. The category feedback for the last
stimulus could be followed by a display of a table for the conversion of
hit and false alarm rates into d'. Thus, the observer would be able to
mentally compute d' without engaging in any externally observable
behavior whatever.

(2) In some well-defined contexts, the variation of d' with an
independent variable is as lawful as anything in the "known to be
objective" sciences such as physics (see Reed, Memory and Cognition
1976, 4(4), 453-458, equation 5 and bottom panel of figure 1, for an
example of this). The parameters of such lawful relationships will
differ from observer to observer, but their form is observer-invariant.
In principle, two investigators could perform the experiment as in (1)
above, and obtain objective (in the sense of observer-independent)
results as to the form of the resulting lawful relationships between,
for example, d' and memory retention time, *without engaging in any
externally observable behavior until it came time to compare results*.

The following analogy (proposed, if I remember correctly, by Robert
Efron) may illuminate what is happening here. Two physicists, A and B,
live in countries with closed borders, so that they may never visit each
other's laboratories and personally observe each other's experiments.
Relative to each other's personal perception, their experiments are
as private as the conscious experiences of different observers. But, by
replicating each other's experiments in their respective laboratories,
they are capable of arriving at objective knowledge. This is also true,
I submit, of the psychological study of private, "subjective"
experience.
										   Adam Reed
										   mtund!adam,attmail!adamreed

harnad@mind.UUCP (01/27/87)

adam@mtund.UUCP (Adam V. Reed), of AT&T ISL Middletown NJ USA, wrote:

>	Stevan Harnad makes an unstated assumption... that subjective
>	variables are not amenable to objective measurement. But if by
>	"objective" Steve means, as I think he does, "observer-invariant", then
>	this assumption is demonstrably false.

I do make the assumption (let me state it boldly) that subjective
variables are not objectively measurable (nor are they objectively
explainable) and that that's the mind/body problem. I don't know what
"observer-invariant" means, but if it means the same thing as in
physics -- which is that the very same physical phenomenon can
occur independently of any particular observation, and can in
principle be measured by any observer, then individuals' private events
certainly are not such, since the only eligible observer is the
subject of the experience himself (and without an observer there is no
experience -- I'll return to this below). I can't observe yours and you
can't observe mine. That's one of the definitive features of the
subjective/objective distinction itself, and it's intimately related to
the nature of experience, i.e., of subjectivity, of consciousness.

>	Whether or not a stimulus is experienced as belonging to some target
>	category is clearly a private event...[This is followed by an
>	interesting thought-experiment in which the signal detection parameter
>	d' could be calculated for himself by a subject after an appropriate
>	series of trials with feedback and no overt response.]... the observer
>	would be able to mentally compute d' without engaging in any externally
>	observable behavior whatever.

Unfortunately, this in no way refutes the claim that subjective experience
cannot be objectively measured or explained. Not only is there (1) no way
of objectively testing whether the subject's covert calculations on
that series of trials were correct, not only is there (2) no way of
getting any data AT ALL without his overt mega-response at the end
(unless, of course, the subject is the experimenter, which makes the
whole exercise solipsistic), but, worst of all, (3) the very same
performance data could be generated by presenting inputs to a
computer's transducer, and no matter how accurately it reported its
d', we presumably wouldn't want to conclude that it had experienced anything
at all. So what's OBJECTIVELY different about the human case?

At best, what's being objectively measured happens to correlate
reliably with subjective experience (as we can each confirm in our own
cases only -- privately and subjectively). What we are actually measuring
objectively is merely behavior (and, if we know what to look for, also
its neural substrate). By the usual objective techniques of scientific
inference on these data we can then go on to formulate (again objective)
hypotheses about underlying functional (causal) mechanisms. These should
be testable and may even be valid (all likewise objectively). But the
testability and validity of these hypotheses will always be objectively
independent of any experiential correlations (i.e., the presence or
absence of consciousness).

To put it my standard stark way: The psychophysics of a conscious
organism (or device) will always be objectively identical to that
of a turing-indistinguishable unconscious organism (or device) that
merely BEHAVES EXACTLY AS IF it were conscious. (It is irrelevant whether
there are or could be such organisms or devices; what's at issue here is
objectivity. Moreover, the "reliability" of the correlations is of
course objectively untestable.) This leaves subjective experience a
mere "nomological dangler" (as the old identity theorists used to call
it) in a lawful psychophysical account. We each (presumably) know it's
there from our respective subjective observations. But, objectively speaking,
psychophysics is only the study of, say, the detecting and discriminating
capacity (i.e., behavior) of our trandsucer systems, NOT the qualities of our
conscious experience, no matter how tight the subjective correlation.
That's no limit on psychophysics. We can do it as if it were the study
of our conscious experience, and the correlations may all be real,
even causal. But the mind/body problem and the problem of objective
measurement and explanation remain completely untouched by our findings,
both in practise and in principle.

So even in psychophysics, the appropriate research strategy seems to
be methodological epiphenomenalism. If you disagree, answer this: What
MORE is added to our empirical mission in doing psychophysics if we
insist that we are not "merely" trying to account for the underlying
regularities and causal mechanisms of detection, discrimination,
categorization (etc.) PERFORMANCE, but of the qualitative experience
accompanying and "mediating" it? How would someone who wanted to 
undertake the latter rather than merely the former go about things any
differently, and how would his methods and findings differ (apart from
being embellished with a subjective interpretation)? Would there be any
OBJECTIVE difference?

I have no lack of respect for psychophysics, and what it can tell us
about the functional basis of categorization. (I've just edited and
contributed to a book on it.) But I have no illusions about its being
in any better a position to make objective inroads on the mind/body
problem than neuroscience, cognitive psychology, artificial
intelligence or evolutionary biology -- and they're in no position at all.

>	In principle, two investigators could perform the [above] experiment
>	...and obtain objective (in the sense of observer-independent)
>	results as to the form of the resulting lawful relationships between,
>	for example, d' and memory retention time, *without engaging in any
>	externally observable behavior until it came time to compare results*.

I'd be interested in knowing how, if I were one of the experimenters
and Adam Reed were the other, he could get "objective
(observer-independent) results" on my experience and I on his. Of
course, if we make some (question-begging) assumptions about the fact
that the experience of our respective alter egos (a) exists, (b) is
similar to our own, and (c) is veridically reflected by the "form" of the
overt outcome of our respective covert calculations, then we'd have some
agreement, but I'd hardly dare to say we had objectivity.

(What, by the way, is the difference in principle between overt behavior
on every trial and overt behavior after a complex-series-of-trials?
Whether I'm detecting individual signals or calculating cumulating d's
or even more complex psychophysical functions, I'm just an
organism/device that's behaving in a certain way under certain
conditions. And you're just a theorist making inferences about the
regularities underlying my performance. Where does "experience" come
into it, objectively speaking? -- And you're surely not suggesting that
psychophyics be practiced as a solipsistic science, each experimenter
serving as his own sole subject: for from solipsistic methods you can
only arrive at solipsistic conclusions, trivially observer-invariant,
but hardly objective.)

>	The following analogy (proposed, if I remember correctly, by Robert
>	Efron) may illuminate what is happening here. Two physicists, A and B,
>	live in countries with closed borders, so that they may never visit each
>	other's laboratories and personally observe each other's experiments.
>	Relative to each other's personal perception, their experiments are
>	as private as the conscious experiences of different observers. But, by
>	replicating each other's experiments in their respective laboratories,
>	they are capable of arriving at objective knowledge. This is also true,
>	I submit, of the psychological study of private, "subjective"
>	experience.

As far as I can see, Efron's analogy casts no light at all.
It merely reminds us that even normal objectivity in science (intersubjective
repeatability) happens to be piggy-backing on the existence of
subjective experience. We are not, after all, unconscious automata. When we
perform an "observation," it is not ONLY objective, in the sense that
anyone in principle can perform the same observation and arrive at the
same result. There is also something it is "like" to observe
something -- observations are also conscious experiences.

But apart from some voodoo in certain quantum mechanical meta-theories,
the subjective aspect of objective observations in physics seems to be
nothing but an innocent fellow-traveller: The outcome of the
Michelson-Morley Experiment would presumably be the same if it were
performed by an unconscious automaton, or even if WE were unconscious automata.
This is decidely NOT true of the (untouched) subjective aspect of a
psychophysical experiment. Observer-independent "experience" is a
contradiction in terms.

(Most scientists, by the way, do not construe repeatability to require
travelling directly to one another's labs; rather, it's a matter of
recreating the same objective conditions. Unfortunately, this does not
generalize to the replication of anyone else's private events, or even
to the EXISTENCE of any private events other than one's own.)

Note that I am not denying that objective knowledge can be derived
from psychophysics; I'm only denying that this can amount to objective
knowledge about anything MORE than psychophysical performance and its
underlying causal substrate. The accompanying subjective phenomenology is
simply not part of the objective story science can tell, no matter how, and
how tightly, it happens to be coupled to it in reality. That's the
mind/body problem, and a fundamental limit on objective inquiry.
Methodological epiphenomenalism recommends we face it and live with
it, since not that much is lost. The "incompleteness" of an objective
account is, after all, just a subjective problem. But supposing away
the incompleteness -- by wishful thinking, hopeful over-interpretation,
hidden (subjective) premises or blurring of the objective/subjective
distinction -- is a logical problem.
-- 

Stevan Harnad                                  (609) - 921 7771
{allegra, bellcore, seismo, rutgers, packard}  !princeton!mind!harnad
harnad%mind@princeton.csnet           

adam@mtund.UUCP (02/01/87)

This is a reply to Stevan Harnad, who wrote:
> adam@mtund.UUCP (Adam V. Reed), of AT&T ISL Middletown NJ USA, wrote:
> 
> >	Stevan Harnad makes an unstated assumption... that subjective
> >	variables are not amenable to objective measurement. But if by
> >	"objective" Steve means, as I think he does, "observer-invariant", then
> >	this assumption is demonstrably false.
> 
> I do make the assumption (let me state it boldly) that subjective
> variables are not objectively measurable (nor are they objectively
> explainable) and that that's the mind/body problem. I don't know what
> "observer-invariant" means, but if it means the same thing as in
> physics -- which is that the very same physical phenomenon can
> occur independently of any particular observation, and can in
> principle be measured by any observer, then individuals' private events
> certainly are not such, since the only eligible observer is the
> subject of the experience himself (and without an observer there is no
> experience -- I'll return to this below). I can't observe yours and you
> can't observe mine.

Yes, and in Efron's analogy, A can't observe B's, and vice versa.
However, I don't buy the assumption that two must *observe the same
instance of a phenomenon* in order to perform an *observer-independent
measurement of the same (generic) phenomenon*. The two physicists can
agree that they are studying the same generic phenomenon because they
know they are doing similar things to similar equipment, and getting
similar results. But there is nothing to prevent two psychologists from
doing similar (mental) things to similar (mental) equipment and getting
similar results, even if neither engages in any overt behavior apart
from reporting the results of his measurements to the other. My point is
that this constitutes objective (observer-independent) measurement of
private (no behavior observable by others) mental processes.

> That's one of the definitive features of the
> subjective/objective distinction itself, and it's intimately related to
> the nature of experience, i.e., of subjectivity, of consciousness.
> 
> >	Whether or not a stimulus is experienced as belonging to some target
> >	category is clearly a private event...[This is followed by an
> >	interesting thought-experiment in which the signal detection parameter
> >	d' could be calculated for himself by a subject after an appropriate
> >	series of trials with feedback and no overt response.]... the observer
> >	would be able to mentally compute d' without engaging in any externally
> >	observable behavior whatever.
> 
> Unfortunately, this in no way refutes the claim that subjective experience
> cannot be objectively measured or explained. Not only is there (1) no way
> of objectively testing whether the subject's covert calculations on
> that series of trials were correct,

This objection applies with equal force to the observation, recording
and calculations of externally observable behavior. So what?

> not only is there (2) no way of
> getting any data AT ALL without his overt mega-response at the end

Yes, but *this is not what is being measured*. Or is the subject matter
of physics the communication behavior of physicists?

> (unless, of course, the subject is the experimenter, which makes the
> whole exercise solipsistic), but, worst of all, (3) the very same
> performance data could be generated by presenting inputs to a
> computer's transducer, and no matter how accurately it reported its
> d', we presumably wouldn't want to conclude that it had experienced anything
> at all. So what's OBJECTIVELY different about the human case?

What is objectively different about the human case is that not only is
the other human doing similar (mental) things, he or she is doing those
things to similar (human mind implemented on a human brain) equipment.
If we obtain similar results, Occam's razor suggests that we explain
them similarly: if my results come from measurement of subjectively
experienced events, it is reasonable for me to suppose that another
human's similar results come from the same source. But a computer's
"mental" equipment is (at this point in time) sufficiently dissimilar
from a human's that the above reasoning would break down at the point
of "doing similar things to similar equipment with similar results",
even if the procedures and results somehow did turn out to be identical.

> At best, what's being objectively measured happens to correlate
> reliably with subjective experience (as we can each confirm in our own
> cases only -- privately and subjectively). What we are actually measuring
> objectively is merely behavior

Not true. As I have shown in my original posting, d' can be measured
without there *being* any behavior prior to measurement. There is
nothing in Harnad's reply to refute this.

> (and, if we know what to look for, also
> its neural substrate). By the usual objective techniques of scientific
> inference on these data we can then go on to formulate (again objective)
> hypotheses about underlying functional (causal) mechanisms. These should
> be testable and may even be valid (all likewise objectively). But the
> testability and validity of these hypotheses will always be objectively
> independent of any experiential correlations (i.e., the presence or
> absence of consciousness).

Why? And how can this be true in cases when it is the conscious
experience that is being measured?

> To put it my standard stark way: The psychophysics of a conscious
> organism (or device) will always be objectively identical to that
> of a turing-indistinguishable unconscious organism (or device) that
> merely BEHAVES EXACTLY AS IF it were conscious. (It is irrelevant whether
> there are or could be such organisms or devices; what's at issue here is
> objectivity. Moreover, the "reliability" of the correlations is of
> course objectively untestable.) This leaves subjective experience a
> mere "nomological dangler" (as the old identity theorists used to call
> it) in a lawful psychophysical account. We each (presumably) know it's
> there from our respective subjective observations. But, objectively speaking,
> psychophysics is only the study of, say, the detecting and discriminating
> capacity (i.e., behavior) of our trandsucer systems, NOT the qualities of our
> conscious experience, no matter how tight the subjective correlation.
> That's no limit on psychophysics. We can do it as if it were the study
> of our conscious experience, and the correlations may all be real,
> even causal. But the mind/body problem and the problem of objective
> measurement and explanation remain completely untouched by our findings,
> both in practise and in principle.

The above re-states Steve's position, but fails deal with my objections
to it.

> So even in psychophysics, the appropriate research strategy seems to
> be methodological epiphenomenalism. If you disagree, answer this: What
> MORE is added to our empirical mission in doing psychophysics if we
> insist that we are not "merely" trying to account for the underlying
> regularities and causal mechanisms of detection, discrimination,
> categorization (etc.) PERFORMANCE, but of the qualitative experience
> accompanying and "mediating" it? How would someone who wanted to 
> undertake the latter rather than merely the former go about things any
> differently, and how would his methods and findings differ (apart from
> being embellished with a subjective interpretation)? Would there be any
> OBJECTIVE difference?

I think so - I would not accept as legitimate any psychological theory
which appeared to contradict my conscious experience, and failed to
account for the apparent contradiction. As far as I can tell, Steve's
position means that he would not disqualify a psychological theory just
because it happened to be contradicted by his own conscious experience.

> I have no lack of respect for psychophysics, and what it can tell us
> about the functional basis of categorization. (I've just edited and
> contributed to a book on it.) But I have no illusions about its being
> in any better a position to make objective inroads on the mind/body
> problem than neuroscience, cognitive psychology, artificial
> intelligence or evolutionary biology -- and they're in no position at all.

> >	In principle, two investigators could perform the [above] experiment
> >	...and obtain objective (in the sense of observer-independent)
> >	results as to the form of the resulting lawful relationships between,
> >	for example, d' and memory retention time, *without engaging in any
> >	externally observable behavior until it came time to compare results*.
> 
> I'd be interested in knowing how, if I were one of the experimenters
> and Adam Reed were the other, he could get "objective
> (observer-independent) results" on my experience and I on his. Of
> course, if we make some (question-begging) assumptions about the fact
> that the experience of our respective alter egos (a) exists, (b) is
> similar to our own, and (c) is veridically reflected by the "form" of the
> overt outcome of our respective covert calculations, then we'd have some
> agreement, but I'd hardly dare to say we had objectivity.

These assumptions are not "question-begging": they are logically
necessary consequences of applying Occam's razor to this situation (see
above). And yes, I would tend to regard the resulting agreement among
different subjective observers as evidence for the objectivity of their
measurements.

> (What, by the way, is the difference in principle between overt behavior
> on every trial and overt behavior after a complex-series-of-trials?
> Whether I'm detecting individual signals or calculating cumulating d's
> or even more complex psychophysical functions, I'm just an
> organism/device that's behaving in a certain way under certain
> conditions. And you're just a theorist making inferences about the
> regularities underlying my performance. Where does "experience" come
> into it, objectively speaking? -- And you're surely not suggesting that
> psychophyics be practiced as a solipsistic science, each experimenter
> serving as his own sole subject: for from solipsistic methods you can
> only arrive at solipsistic conclusions, trivially observer-invariant,
> but hardly objective.)

For measurement to be *measurement of behavior*, the behavior must be,
in the temporal sequence, prior to measurement. But if the only overt
behavior is the communication of the results of measurement, then the
behavior occurs only after measurement has already taken place. So the
measurement in question cannot be a measurement of behavior, and must be
a measurement of something else. And the only plausible candidate for
that "something else" is conscious experience.

> >	The following analogy (proposed, if I remember correctly, by Robert
> >	Efron) may illuminate what is happening here. Two physicists, A and B,
> >	live in countries with closed borders, so that they may never visit each
> >	other's laboratories and personally observe each other's experiments.
> >	Relative to each other's personal perception, their experiments are
> >	as private as the conscious experiences of different observers. But, by
> >	replicating each other's experiments in their respective laboratories,
> >	they are capable of arriving at objective knowledge. This is also true,
> >	I submit, of the psychological study of private, "subjective"
> >	experience.
> 
> As far as I can see, Efron's analogy casts no light at all.

See my comments at the beginning of this reply.

> It merely reminds us that even normal objectivity in science (intersubjective
> repeatability) happens to be piggy-backing on the existence of
> subjective experience. We are not, after all, unconscious automata. When we
> perform an "observation," it is not ONLY objective, in the sense that
> anyone in principle can perform the same observation and arrive at the
> same result. There is also something it is "like" to observe
> something -- observations are also conscious experiences.
> 
> But apart from some voodoo in certain quantum mechanical meta-theories,
> the subjective aspect of objective observations in physics seems to be
> nothing but an innocent fellow-traveller: The outcome of the
> Michelson-Morley Experiment would presumably be the same if it were
> performed by an unconscious automaton, or even if WE were unconscious automata.
> This is decidely NOT true of the (untouched) subjective aspect of a
> psychophysical experiment. Observer-independent "experience" is a
> contradiction in terms.

Yes, but observer-independent *measurement of* experience is not. See
above.

> (Most scientists, by the way, do not construe repeatability to require
> travelling directly to one another's labs; rather, it's a matter of
> recreating the same objective conditions. Unfortunately, this does not
> generalize to the replication of anyone else's private events, or even
> to the EXISTENCE of any private events other than one's own.)

Yes it does: see the argument from Occam's razor earlier in this
article.

> Note that I am not denying that objective knowledge can be derived
> from psychophysics; I'm only denying that this can amount to objective
> knowledge about anything MORE than psychophysical performance and its
> underlying causal substrate. The accompanying subjective phenomenology is
> simply not part of the objective story science can tell, no matter how, and
> how tightly, it happens to be coupled to it in reality. That's the
> mind/body problem, and a fundamental limit on objective inquiry.

Steve seems to be saying that the mind-body problem constitutes "a
fundamental limit on objective inquiry", i.e. that this problem is *in
principle* incapable of ever being solved. I happen to think that human
consciousness is a fact of reality and, like all facts of reality, will
prove amenable to scientific explanation. And I like to think that
this explanation will constitute, in some scientifically relevant sense,
a solution to the "mind-body problem". So I don't see this problem as a
"fundamental limit".

> Methodological epiphenomenalism recommends we face it and live with
> it, since not that much is lost. The "incompleteness" of an objective
> account is, after all, just a subjective problem. But supposing away
> the incompleteness -- by wishful thinking, hopeful over-interpretation,
> hidden (subjective) premises or blurring of the objective/subjective
> distinction -- is a logical problem.

Yes, but need it remain one forever?

		Adam Reed (mtund!adam, attmail!adamreed)