ilan343@violet.berkeley.edu (Geraldo Veiga) (01/17/91)
I am trying to assemble a set of publicly available benchmarks to run on UNIX/386 machines. I am starting small. I got copies of Dhrystone, Whetstones and Linpack from NETLIB. Here is the problem: Running dhrystone 2.1 (c version) on a 486 machine running ISC 2.2, I get all kinds of crazy results, with Dhrystone ratings ranging from 3000 to 16000 in different runs. The same machine under DOS, running the Drystones version in QAPLUS rates (consistently) at 15172. Question: Is this normal? Is the hardware or software flaky? If I time the run with Sysv's "time", user time also show the same wide variation. Suggestions appreciated.
rosenkra@convex.com (William Rosencranz) (01/18/91)
In article <1991Jan16.210617.6484@agate.berkeley.edu> ilan343@violet.berkeley.edu (Geraldo Veiga) writes: >Running dhrystone 2.1 (c version) on a 486 machine running ISC 2.2, I >get all kinds of crazy results, with Dhrystone ratings ranging from >3000 to 16000 in different runs. The same machine under DOS, running >the Drystones version in QAPLUS rates (consistently) at 15172. no suprise here... >Is this normal? Is the hardware or software flaky? If I time the run >with Sysv's "time", user time also show the same wide variation. dhrystone is (far) more test of your machine's compiler than the hardware itself. i have seen similar results on systems from atari ST to cray YMP. 5:1 seems a little high; i have seen 3:1 difs between compilers on (say) the 8 Mhz 68000 atari ST...with identical hardware. compiler opimization switches also influence results. it sounds, however, like u see variations with the same executable. there may be other processes running, which is why u should always try to make bm runs on dedicated machines, without any daemons or other processes which may steal cycles. this is especially true with parallel systems. if ISC supports the equiv of nice, u may try boosting the priority of the process. since MSDOS is non-multitasking, it should give identical results (1% err) with identical executables. however, different compilers under DOS will still give different results, even with identical h/w. it can be a useful bm if u are writing a compiler, or if u are shopping for compilers, but it is not the best bm for performance analysis of h/w itself, unless u can guarantee that each compiler will generate similar instructions across architectures. caveat user... -bill rosenkra@convex.com -- Bill Rosenkranz |UUCP: {uunet,texsun}!convex!c1yankee!rosenkra Convex Computer Corp. |ARPA: rosenkra%c1yankee@convex.com
dafuller@sequent.UUCP (David Fuller) (01/20/91)
In article <1991Jan16.210617.6484@agate.berkeley.edu> ilan343@violet.berkeley.edu (Geraldo Veiga) writes: >I am trying to assemble a set of publicly available benchmarks to run >on UNIX/386 machines. I am starting small. I got copies of Dhrystone, >Whetstones and Linpack from NETLIB. > >Here is the problem: > >Running dhrystone 2.1 (c version) on a 486 machine running ISC 2.2, I >get all kinds of crazy results, with Dhrystone ratings ranging from >3000 to 16000 in different runs. The same machine under DOS, running >the Drystones version in QAPLUS rates (consistently) at 15172. > >Question: > >Is this normal? Is the hardware or software flaky? If I time the run >with Sysv's "time", user time also show the same wide variation. You have given no indication that your initial conditions have been analyzed and corrected for possible external influence. While significant single-run differences can be attributed to sterling initial conditions and a bad cache line arrangements, your results are worth breaking down to the source and completely reconsidering. > > >Suggestions appreciated. Examine your initial conditions. Eliminate every variable you can and document those you can't mess with. -- Dave Fuller Sequent Computer Systems Think of this as the hyper-signature. (708) 318-0050 (humans) It means all things to all people. dafuller@sequent.com