caf@omen.UUCP (Chuck Forsberg WA7KGX) (12/01/86)
In article <1172@brl-adm.ARPA> G.MDP@score.stanford.edu (Mike Peeler) writes:
:It's worth noting that optimizing doesn't improve the benchmark.
:What I'm interested in is how fast TYPICAL CODE runs. Typical
:code isn't optimal. Typical C code doesn't put all variables in
:registers. I don't always have source and I don't have the time
:to hand-optimize every program I run.
:
:What I want from a benchmark is a basis for a price-performance
:decision. If an automatic optimizer is available and typical
:code can take advantage of it, it's ok if the benchmark uses it.
:But if typical C programs have sub-optimal data declarations, I
:want to make my comparison on that basis.
Something I want from a benchmark is the ability to run it on real life
machines. Tales of Intel 386 boards getting 4000 to 6000 on Dhrystone
don't mean much to me when the fastest I can get my Intel 386
motherboard to go is about a third of that (my 9 mHz IBM PC-AT actually
runs faster, and doesn't lock up the keyboard either). It is also
interesting which vendors won't allow you to run your own benchmark at a
show; for example, none of the vendors of 386 accelerator boards would
allow my siev benchmark to run at Comdex. The 386 motherboard machines
that I was allowed to run it on were less than half as fast as the fastest
68k system I tried siev on.
One advantage of the siev benchmark is that it is simple enough to
understand. When making comparisions between different types of
systems, the text size of the main function is useful information. It
is even possible to look at a disassembly of the resultant code and make
some sense out of it, to see if the compiler is using 16 or 32 bit
operations, for example. This last point is especially relevant on 386
and 68k systems, where the program may run in either a 16 or 32 bit
model.
The advantage of the 2^4096 benchmark is that it is easy to type in and
run on a Unix system, even if the compiler is not present. When a 386
Unix system runs it more slowly than a PC-AT, you'd better believe some
tuning needs to be done.