kirlik@chmsr (Alex Kirlik) (03/02/90)
A poster recently inquired about the possible existence of "optimal" ways
to go about determining the inputs to a network. As is well known, a
judicious selection of primitive inputs (or features) may allow for a very
simple categorization rule, whereas a poor selection can result in the
need for a very cumbersome rule to perform the same task. So much of
human expertise, it seems, is in "knowing what to look for," and this
ability can obviate the need for very complex processing operations (those
operations required by the novice who hasn't yet learned to attend to the
"right" features). In addition to this selective attentional advantage,
the skilled performer is sometimes also able to better differentiate the
input space, or in psychological terms, is able to develop a "perceptual
sensitivity to variables of stimulation not previously available." Consider
an expert wine taster, for example, or perhaps a musical/visual art critic.
The problem of determining an "optimal" or even good input representation
for a network, or any other model for that matter, is in my opinion one of
the must central and difficult tasks in modeling. My comments are geared
toward the problem of psychological modeling, but they also apply to the
issue of seeking an engineering solution to a given task. When one
selects a supposedly primitive representation of the input environment,
one is, by that very act, building a model of that environment that
indicates what information is thought to be relevant to the task. This
selection will by itself make certain inferences or categorization
operations impossible, and for the rest it induces a "difficulty metric"
that indicates a minimal level of processing complexity needed to make
those inferences.
Unless one starts with a full-blown representation of the environment
(the environment itself), one always runs the risk of excluding features
to which it would pay the network to be sensitive. This happens all the
time in psychological modeling under the name "context effects." In case
after case human behavior has been found to defeat computational models
by exhibiting behavioral variance to contextual features of the environment
that are equivalently classed, or even abstracted away entirely, in the
environmental representation used by the model. And sometimes humans use
this context to advantage.
Of course the "context effect" is not a single empirical phenomenon; it
is always defined relative to a model. What might be an essential input
feature for one model is just more context to another. Nevertheless, the
moral to be learned here is that intuition is not always the best guide
in selecting a primitive environmental representation. And, I believe
there is no other solution to this problem than a lot of hard work.
There cannot be a formal optimization method for attacking such a problem
that works on anything less than a full blown input representation. Any
abstractions of the environment made by the modeler always open up the
possibility that the "optimal" solution is gone before the game can start.
Alex
UUCP: kirlik@chmsr.UUCP
{backbones}!gatech!chmsr!kirlik
INTERNET: kirlik@chmsr.gatech.eduandrew@dtg.nsc.com (Lord Snooty @ The Giant Poisoned Electric Head ) (03/03/90)
In article <6603@hydra.gatech.EDU>, kirlik@chmsr (Alex Kirlik) writes: > > A poster recently inquired about the possible existence of "optimal" ways > to go about determining the inputs to a network. As is well known, a > judicious selection of primitive inputs (or features) may allow for a very > simple categorization rule, whereas a poor selection can result in the > need for a very cumbersome rule to perform the same task. So much of > human expertise, it seems, is in "knowing what to look for,".... Note that babies bootstrap, so that "knowing what to look for" becomes increasingly sophisticated. > The problem of determining an "optimal" or even good input representation > for a network, or any other model for that matter, is in my opinion one of > the must central and difficult tasks in modeling.... Note that networks are especially good at feature extraction. > There cannot be a formal optimization method for attacking such a problem > that works on anything less than a full blown input representation. Any > abstractions of the environment made by the modeler always open up the > possibility that the "optimal" solution is gone before the game can start. So, combining my two short comments, why not build a network that bootstraps on an increasingly complex environment? - in other words, build two networks. There are a couple of ways you might do this; one way could be hierarchical; one on top of another. The top network is the "knowing what to look for" unit, which would provide a parallel "vigilance" input to the lower unit which is actually performing the task in hand. Another way might be a serial approach, whereby the "knowing what to look for" unit acts as an input preprocessor. Or a combination of the two.... just a thought. -- ........................................................................... Andrew Palfreyman andrew@dtg.nsc.com Albania before April!