[comp.arch] Benchmarking Considered Harmful

eugene@pioneer.UUCP (06/02/87)

Martin McKendry wrote an excellent article on the hazards of benchmarking.
I too have been warned about not publishing results without prior approval
(not from within my organization), and have not "violated" agreements.

The shortsightedness evident in Martin's old company disturbs me.
Will we always kill the bearer of bad news?
Perhaps it is because I have been reading some articles on engineering
ethics and Morita's biography were he points out (to my surprise) that the
people of one certain culture (Morita's) like to takes to make radios
and other electronics parts to see what makes them run, but the
American perspective was to treat it more like a black box.  This is
certainly a gross generalization, but I think it is begins to reflect
the larger aspects of our society interested in short term gain.
Will we always fear management pressure over doing the right thing?

I personally think it is better to find weaknesses and strive to improve
rather than ignore the fact and cover it with lawyers and advertising.
[Yes, I am aware, as many others on the net, about the need to balance
the short-term pay check with the longer term `pay off.']  Who controls
your company?  The MBAs or the engineers?  [Don't answer that, I know
already ;-)]  Anyway, this discussion doesn't really involve
architecture  [please excuse the flame Garth].

On a separate note.  I will post my clock test.  I have not had a chance
to get to it due to the mess on my desk, but now there are no papers
there (just ask rafael@ucbernie.berkeley.edu).  So I can complete a few
details.  I might use some simple m4, so the changes required will only
be about 3 lines of code per machine.

From the Rock of Ages Home for Retired Hackers:

--eugene miya
  NASA Ames Research Center
  eugene@ames-aurora.ARPA
  "You trust the `reply' command with all those different mailers out there?"
  "Send mail, avoid follow-ups.  If enough, I'll summarize."
  {hplabs,hao,ihnp4,decwrl,allegra,tektronix,menlo70}!ames!aurora!eugene