[fa.laser-lovers] impartiality in blind comparisons

laser-lovers@uw-beaver (05/20/85)

From: Brian Reid <reid@Glacier>

I've had a wine tasting at my house every Monday night for several
years. One of the things that I have noticed is that people can often
recognize particular wines. From time to time one of the winemakers
shows up, and they always dazzle us with their ability not only to
recognize their own wines, but to tell one year from another. One
person owns many hundreds of bottles of wine from a certain winery, and
he can almost always detect that wine in these tastings. Whether he
*prefers* it is a separate issue from whether he can *recognize* it.

Similarly, I suspect that I could recognize, by brand name, the output
of this or that laser printer, and this or that text formatter, for a
reasonably large number of printers and text formatters. I'm not as
scholarly as Chuck Bigelow on these matters--I could never manage to
tell who the 16th-century Italian designer of some font might have
been--but I can normally spot Xerox or Imagen or Apple/Adobe or
Talaris/QMS output, even if I don't know whether I am looking at Times
New Roman or Optima.

I would guess that the people in Les Earnest's group can do this too.
Some of them probably have pretty good eyes for this kind of detail.
And therein lies the problem. Given that most experts in this field can
*recognize* the output of their own systems, and thereby pretend to
*prefer* it if they so choose, and given the nature of the accusations
Les has been making, it is very important to have comparisons like this
made by disinterested third parties.

Even if the people doing this "blind comparison" have no financial
interest in any of the products being tested, there is a tremendous
amount of interaction between people at Stanford and the various laser
printer companies, and there are going to be plenty of folks who are
quite partial even without owning stock in Imagen or Adobe or Apple or
QMS or Xerox. It is my understanding (I'm sure that Les will correct me
if I am wrong) that Imagen was founded by a bunch of people who left
Stanford (primarily from the TEX group) to sell commercially a laser
printer whose initial prototypes were made in the basement of the
Stanford CS department by folks in that research group. If that is
true, then I claim that a "blind tasting" by members of that group is
not an impartial tasting, and that Les therefore did exactly the right
thing by keeping the results private. 
--
	Brian Reid	decwrl!glacier!reid
	Stanford	reid@SU-Glacier.ARPA

laser-lovers@uw-beaver (05/20/85)

From: Howard Trickey <trickey@diablo>

I took place in Les's font tasting, and I don't think impartiality was
much of an issue.  I don't use either of the printers involved, and
though I'd seen output from them, I didn't recognize their output.
I think the same is true for a number of the other participants;
most of those that do use one or other of the printers probably never
use the Times Roman font on them.  And remember, Les asked those who
recognized either printer to disqualify themselves.

The reason that several people voiced objections to publicizing the
results was that it would in effect be a commercial endorsement,
and one based on not enough information.  (By the way, this objection
was raised before the tasting, and while we didn't definitely decide
not to release results until afterwards, the decision wasn't made
because anyone didn't like the way the group preference came out.)
It may have been that at a different point size the decision would
have been reversed.  I feel that my own preference was due partly to
the poor character placement given by one formatter/printer combination,
and thus not completely due to a "font" preference.
Another issue is that some believe it is often not possible to say that one
font is "better" than another; it depends on what use is being made
of them.  One of the participants made the point that we were all
studying the individual character forms rather closely, not reading
a large body of text, so we weren't making normal use of the fonts.

		Howard Trickey
		  Stanford