laser-lovers@uw-beaver (05/23/85)
From: Les Earnest <LES@SU-AI.ARPA>
During the heat of an earlier battle [my message of 19 May 85 2144 PDT],
I remarked that I had some reservations about the utility of consensus
judgements by panels of "experts" or "common persons." I will now expand
on that remark.
It seems obvious that blind tastings generally can provide better information
about font quality than binary debates that evolve into "It is so!"
-- "It is not!" Consensus judgements can be of some use to potential
consumers and certainly should be of interest to typographers and
marketeers. I hope that individual consumers will continue to use their
own eyes in making buying decisions, however, or will at least calibrate
their surrogate "tasters" carefully.
We often say about various issues that "It is a matter of taste," but
people sometimes foolishly choose to "Let the experts decide." One of my
hobbies has yielded some experience in evaluating "expert" advice in a
field where taste is paramount.
In 1973 I invented a new kind of restaurant guide called "Yumyum," which
is a compilation of reader-written views forwarded to me by electronic
mail. I have edited and published Yumyum every year or so since then,
often with help from others. (Sidenote: one of the contributors in the
past has been Brian Reid. As you would expect, Brian's views of
restaurants are generally binary, with very few shades of grey.)
From the beginning, some readers got upset over some reviews and would
grumble things like "Charlie Brown has no taste -- he is totally wrong
about the Shirttail Restaurant." A more common kind of remark was that
"This is the most useful restaurant guide that I have -- by reading
several short reviews of a given restaurant by different people I can
usually get much better idea of how good it will be than I can from a much
longer review in one of the standard restaurant guides."
Naturally, I read some of the competing guides to assess what was good and
bad about them. I discovered that nearly all restaurant guides are done
in one of two ways: some are written by a single person based on his own
dining experience while others are written by an anonymous staff, usually
pretending to be one person. Those that are written by a single person
provide a consistent, calibratable view but suffer from a serious defect:
one person can't get around to all the restaurants in a sizable community
on a timely basis. Since chefs move and ownership changes fairly often,
the reviews are often badly out of date.
Those written by an anonymous staff are more timely but suffer from a more
serious defect: the different reviewers have substantialy different tastes,
so even though you may find that the guide's opinions agree with yours on
a couple of restaurants, there is no assurance that the rave review about
a new restaurant in town was written by the same person.
It is clear that different things are important to different people.
Some are looking for minimum cost. Others are looking for some kind of
cost-effectiveness (e.g. most calories/dollar). Others are looking for
good service or atmosphere or an impressive wine list. Some are actually
looking for good food.
I concluded that a restaurant guide that lists multiple short reviews from
identified people is much more useful because:
(1) it is practical to keep it up-to-date;
(2) the main thing that you want to know from a given reviewer is simply
whether or not he liked it; the rest is mostly "frosting";
(3) having multiple reviews of a given restaurant gives you more chances
of getting a data point from someone whose tastes you know.
I learned that by "reviewing the reviewers" I could find people whose tastes
appeared to agree with mine and who could then guide me to new restaurants.
I learned the hard way not to use negative correlations -- when I found that
my taste always disagreed with one reviewer, I tried using his reviews as
inverse evidence for selecting new restaurants to try. After being misled a
couple of times I figured out that he disliked almost *all* restaurants.
It wasn't clear why he ate out so much.
Being a restaurant review editor involves other perils. Each time I went
to one local restaurant whose owner knew me, he would corner and harangue
me about the "inaccurate" reviews in Yumyum. In actuality, nearly all of
the reviews of his restaurant were very positive (I thought overly so for
the most part), but he couldn't stand the idea that someone would have a
negative thought about the place. He would then not-so-subtly try to
bribe me in some way, such as slipping me a large bag of fortune cookies
as I left. (They turned out to be stale.)
Another hazard has been reviews from "ringers." For example, I received a
rave review about a new restaurant from a person who I later discovered
was a part owner of the place. That problem was self-correcting, though
at some expense to the readers -- the multiple reviews in the next edition
made it clear that the first review was nonsense. Awhile later that
restaurant folded.
An analogous problem from the world of technical newsletters such as those
that cover the fields of printing and office automation is that it is
fairly easy to buy good reviews of products, though it is bad form to
discuss it in those terms. If you would like to calibrate the reliability
of a given newsletter, I suggest that you undertake the following
experiment.
Call up the editor and tell him that you are planning to introduce a new
product and are offering some paid consultation on how best to market it.
A subtler approach is to suggest that they might like to "try out" your
new equipment for a few years -- at no cost, of course. If they take you
up on it, you may find it advantageous to get your products before the
public through their corrupt services, but you should also be careful not
to believe their evaluations of other products.
Of course, if you apply the above standard rigorously you will not have
much to read.
The point of all this is that in matters of taste, the consumer will
generally find it more useful to have multiple readings from calibratable
sources rather than either a single expert opinion or a consensus from a
panel of experts. Computer networks are especially well suited to collecting
and disseminating reviews.
I believe that review journals of this type will be much more useful if
they are edited rather than being a free-for-all. To defend against possible
capriciousness or corruption on the part of the editor, there should be
an appeal process leading to review by a technical jury.
Some day, then, I hope that we can all subscribe to an electronic journal that
will include font reviews by Chuck Bigelow such as the following:
"This ductal typeface shows strong intercene correlations in its
adjunctive and arrogative elements. The incipient counterfacies
show the influence of its primary antecedent, the 16th Century Gnostic-
Cordillian typeface, which was created in Southern Sardinia by a
Benedictine monk named Fr. Maserotti on his deathbed. It first appeared
in 1527 in a book published by his illegitimate daughter Bettina.
The color is light and airy and it has a superb nose, but the afterspace
is much too granular."
Brian Reid's view of the same font might be:
"Below the 12 point size, this font is despicable eye-rot.
Above that it is sexually arousing.
If you don't believe that, it is because either you are a Communist or
you have shaken hands with a Communist at some time during the last
ten years."
Once you calibrate the tasters and learn how to interpret their remarks,
you can get valuable information out of reviews such as these.
It seems to me that it will be useful eventually to have such a
"Fontasters Forum."
Cheers,
Les Earnest