spuhler@hpcuhc.cup.hp.com (Tom Spuhler) (12/18/90)
Concerned that your 'bc' benchmark results may be skewed by vendor
optimization of the trivial case? Looking for a longer running version
for your faster CPU's? Does management want a richer instruction mix to
be tested?
For YOU, we proudly present:
nbc - a better bc benchmark.
It's easy, it's fun. Just type the following line on a system of your
choice under the SAME (or as close as possible) conditions in which the
system will normaly be used.
echo '2^9999/3^6308' |/bin/time bc
<Output is> 2
Record and report the values returned by /bin/time along with the system
configuration. Elapsed time is most important. Make sure that the
time command on the system under test is working correctly.
What you will learn: How fast bc will solve 2^9999/3^6308 on the tested
machine under the conditions specified. Of course, this may be somewhat
indicative of the performance of that machine running your application,
under the specified conditions. Don't count on it, but hey, the other
mistake is the other extreme (don't ignore it just because it's
simple).
---
My philosophy on benchmarks is something along the lines of:
Any correctly run and reported benchamrk is a datapoint that can
be used to gain a better understanding of a system's
performance characteristics..
and,
All accurate performance data is interesting and useful, but not
necessarly indicative (of the performance of that machine
running your application).
Some performane data is more indicative then others. The most
indicative is that which most accurately matches your target
workload. Probably also, the most expensive to obtain.
and finally,
It is better to have some data, no matter how limited, as long
as you understand it, then no data at all.
Happy benchmarking
-Tom Spuhlercrispin@csd.uwo.ca (Crispin Cowan) (12/18/90)
In article <115440001@hpcuhc.cup.hp.com> spuhler@hpcuhc.cup.hp.com (Tom Spuhler) writes: > echo '2^9999/3^6308' |/bin/time bc Ok, here we go again. uVAX 3200, BSD Tahoe 75.9 real 71.5 user 0.5 sys Sequent S27, Dynix 94.1 real 93.6 user 0.1 sys Sequent S27, PTX (POSIX Beta) 88.0 real 87.6 user 0.0 sys Sun 3/60, 8 MB, SUN OS 4.0.3 104.8 real 104.0 user 0.2 sys (All machines reported a result of 2) This benchmark is still predicting the uVAX to be the fastest of the above three computers. However, I KNOW the Sequent is faster (even as a single processor) because it does real loads, like latex, 50% faster on CPU time, and 100% faster on real time. Furthermore, I KNOW that the Sun 3/60 is about 40% faster (CPU time and real time) than the uVAX for building gcc. Therefore, this bc is just as valid for predicting bc performance as the last one, but is just as USELESS for predicting anything else. These bc tests are a great, indirect measure of the clock frequency being fed to a processor, if you use the same software and the same processor, but they're VERY non-predictive for anything useful. Crispin ----- Crispin Cowan, CS grad student, University of Western Ontario Work: MC28-C, x3342 crispin@csd.uwo.ca 890 Elias St., London, Ontario, N5W 3P2, 432-7823 ---> Support the GST: Canada's first fair tax <---
ereiamjh@jhunix.HCF.JHU.EDU (Tom B. O'Toole) (12/19/90)
In article <115440001@hpcuhc.cup.hp.com> spuhler@hpcuhc.cup.hp.com (Tom Spuhler) writes: >It's easy, it's fun. Just type the following line on a system of your >choice under the SAME (or as close as possible) conditions in which the >system will normaly be used. > > echo '2^9999/3^6308' |/bin/time bc > ><Output is> 2 Okay, On the system of my choice it took about .01 CPU seconds. But I didn't get the same answer. I got: %DCL-W-IVVERB, unrecognized command verb - check validity and spelling \ECHO\ Is this due to round off error? fffffffffffffffffffffffffffsssssssssssttttttttttttttppppppppppPPPP -- Tom O'Toole - ecf_stbo@jhuvms.bitnet - JHUVMS system programmer Homewood Computing Facilities, Johns Hopkins University, Balto. Md. 21218 ease!Trim!eeeaaaassse!trimtrimtrimeeeeeeaaaaassetrimease!trim!ease!trimeaase
spuhler@hpcuhc.cup.hp.com (Tom Spuhler) (12/20/90)
# > echo '2^9999/3^6308' |/bin/time bc # > # ><Output is> 2 # Okay, # On the system of my choice it took about .01 CPU seconds. But I didn't get # the same answer. I got: # %DCL-W-IVVERB, unrecognized command verb - check validity and spelling # \ECHO\ # Is this due to round off error? Naw, it's mostly due to poor you being stuck with some hackneyed Op. Sys :-). I guess I forgot to mention that this command works best on UNIX like systems (and /usr/bin and /bin are on your path and you are using a bourne shell compatible command interpreter and you are not in vi at the time you type the command string and you don't have 'bc' aliased to /etc/backup and ... But, that's probably not overly restrictive in this day and age. And, of course, there are lots of situations that don't match the above where it will work. Most importantly, You didn't get the correct output. Any benchmark which doesn't return the expected output is invalid (or at least VERY suspect). Work on it until you get '2'. -Tom
pc@flash.bellcore.com (Peter A Clitherow) (12/21/90)
>From: spuhler@hpcuhc.cup.hp.com (Tom Spuhler) ># > echo '2^9999/3^6308' |/bin/time bc ># ><Output is> 2 >Naw, it's mostly due to poor you being stuck with some hackneyed Op. Sys :-). >Most importantly, You didn't get the correct output. Any benchmark >which doesn't return the expected output is invalid (or at least VERY >suspect). Work on it until you get '2'. perhaps you should work on UN*X then. elementary considerations show that the numerator must end in "8", and the denominator must end in "1". How can the answer possibly be "2"? My LISP machine gives an answer of: 9975..(omit ~ 6300 digits)..4688 / 4795..(omit ~ 6300 digits)..2561 i.e. no integral answer. it took 17.4 seconds by the way. pc
tif@doorstop.austin.ibm.com (Paul Chamberlain) (12/22/90)
In article <1941@ria.ccs.uwo.ca> crispin@csd.uwo.ca (Crispin Cowan) writes: >In article <115440001@hpcuhc.cup.hp.com> spuhler@hpcuhc.cup.hp.com (Tom Spuhler) writes: >> echo '2^9999/3^6308' |/bin/time bc >Ok, here we go again. Ok, I'm game: real 12.8 user 12.4 sys 0.0 And I think I finally got my model number right: It's an IBM Risc System/6000 Model 930, which is identical to the 530 only it's rack-mounted. Now, I want to know what that Coherent thingy does... Paul Chamberlain | I do NOT represent IBM. IBM VNET: sc30661 at ausvm6 512/838-9662 | This is rumored to work now --> tif@doorstop.austin.ibm.com
gsnow@clark.edu (Gary Snow) (12/23/90)
On a 25mHz 486 running ESIX System V Unix we got.... Real 32.5 sec User 32.3 sec Sys 0.0 sec Gary -- ----- Gary Snow uunet!clark!gsnow or gsnow@clark.edu