jgreen@alliant.alliant.com (John Green) (01/04/91)
Public display of results of meaningful benchmarks improves competition which
in turn improves products to the benefit of consumers.
In 1982 I worked for Plexus Computers (now defunct) which built its own
Multibus 'intelligent' disk controller and asychronous terminal multiplexor
(ICP Intelligent Communications Processor). It also thought it had a hot
processor (4 MHz Z8000). I constructed a benchmark to show off the Plexus P/40
performance.
First you created a 50000 byte text file of sorted ascii text:
dd if=/usr/dict/words of=/tmp/file bs=500 count=100
and used stty to set the baud rate of /dev/tty00 and /dev/tty01 to 9600.
Then you used a bourne shell to spawn off in the background:
dd if=/dev/dsk0 of=/dev/null bs=8k count=2560 &
cat > /dev/tty00 /tmp/file &
cat > /dev/tty01 /tmp/file &
sort -r /tmp/file > /tmp/file1 &
The shell script waited until all completed and reported all 4 times.
The intent of the dd was to simulate timesharing disk access.
The intent of the cat commands was to simulate full screen painting as in
RMCobol or Unify/Informix or vi or word processing.
The intent of the sort was to simulate a compute bound job.
A marketing manager at Plexus talked a Unix trade magazine into adopting my
benchmark as its standard benchmark, and it was soon run on most of the
available Unix boxes (Onyx, Zilog, Plexus using Z8000, ISC using PDP-11,
various vendors using Unisoft ports on 8 MHz 68000).
Although I did not know it the Unisoft sort program was terrible at sort -r.
Under strong pressure from its customers Unisoft promptly put out a rev that
improved sort -r by several hundred per cent.
It also turned out that the Plexus intelligent disk controller and ICP weren't
so intelligent after all. Over the next couple of years Plexus improved the
P/40's Unix performance lots and also switched to Xylogics 450D disk
controllers.
This benchmark died in '83 and should not be revived now. (That's why I did not
give the complete sh script.) However it did cause numerous vendors to improve
their commercial Unix boxes while it was alive.
I strongly support the public display of results of meaningful benchmarks. I
believe SPECmark and SPECthruput in the short term allow consumers to compare
systems on a fair playing field and in the long term cause vendors to make
meaningful improvements to their Unix systems to the benefit of all customers.
On the other hand a benchmark such as bc offered neither benefit and died
quickly. Public benchmarks will come and go. The good ones will spur
competition and have long lives.
John Green Alliant CSC, 1 Monarch Dr, Littleton MA 01460 (508)486-1302