shawn@jdyx.UUCP (Shawn Hayes) (02/28/90)
Well, this want answer all the questions about how fast the new RS 6000
machines really are, but here is my attempt to help.
I recently got a copy of the Dhrystone program V2.0 and ran it on our
RS 320 machine. The results aren't quite what IBM stated (using V1.0), but
that shouldn't be a big suprise. :> I got a value of 11,881.2 dhrystones/sec
using a non-optimized version, and a value of 18,867.9 dhrystones/second
using the optimizer. Averages out to about 11 MIPS which isn't bad.
In addition I decided to run another test today and ran 11 copies of the
optimized version at the same time. I got about 18,700 dhrystones/sec on
10 of the copies and about 14,000 on the last copy. Not bad at all.
Speaking of benchmarks I have two requests.
First, does anyone know of any good system level benchmarks? I've heard that
BYTE has some Unix benchmarks, but I haven't seen the program anywhere on the
network. I'd like to do some real system level timings if there's anything
available.
Second, does anyone know how much time an RS 6000/320 will take to do a context
switch?????? We've talked to our SE, but she doesn't have any hard info
and can't seem to get any from Austin. It's critical to our application that
it not be worse than 2 milliseconds, and her guess was 10 milliseconds. I don'tthink that's accurate so if anyone knows please email me or post in this group.
Shawn Hayesjon@ireland.uucp (Jon Doran) (03/03/90)
In article <3131@kilp.UUCP> kilp@kilp.UUCP (/20000) writes: >shawn@jdyx.UUCP (Shawn Hayes) writes: > >The current development version of AIX 3.1 on a 320 yields 46.8K Dhrystone >2.1's and 53.2K Dhrystone 1.1's. There are several possible reasons for >the difference, viz. > I believe that having slow memory cards installed will also drop the dhrystone numbers. Lets make sure we are comparing the same configurations when we quote benchmarks... Jon
luner@werewolf.cs.wisc.edu (David L. Luner) (03/03/90)
In a previous article shawn@jdyx.UUCP (Shawn Hayes) writes: > Well, this want answer all the questions about how fast the new RS 6000 >machines really are, but here is my attempt to help. Thank you. All we're (and, I suppose, We're all) looking for is a fair evaluation. > I recently got a copy of the Dhrystone program V2.0 and ran it on our >RS 320 machine. The current field releases of AIX V3.1 are preliminary. Furthermore, they are changing daily. I believe the current release is "8943Q". What version we you running on? If you run benchmarks on early releases, be aware that things may change *in either direction*. > The results aren't quite what IBM stated (using V1.0), but > that shouldn't be a big suprise. :> For any number of reasons. First: they even ran the Dhrystone V1.1 on a different release. Second: Dhrystone V2.0 makes things (intentionally) more difficult for the compiler. Third: Read on for caveats w.r.t. "43Q". > I got a value of 11,881.2 dhrystones/sec >using a non-optimized version, and a value of 18,867.9 dhrystones/second >using the optimizer. Averages out to about 11 MIPS which isn't bad. In trying to duplicate the published Dhrystone V1.1 numbers that were published I discovered two things on. For the time being I only assume this is specific to "43Q". Number one, the clock runs at 100Hz. The V1.1 code I was using assumed a 60Hz clock so the numbers looked low. Secondly, there is a problem in the C libraries related to the string handling functions. A bypass (to convince you of the "reasonableness" of the published numbers) is to #include <strings.h> This will in-line the string handling functions. *** I know this is not kosher. Until the C library is fixed in a later release, it will have to do. > > In addition I decided to run another test today and ran 11 copies of the >optimized version at the same time. I got about 18,700 dhrystones/sec on >10 of the copies and about 14,000 on the last copy. Not bad at all. > Assuming that the Dhrystone V2.0 code is like the V1.1 code in this respect, it only looked at user process time, not elapsed clock time. If I am wrong in this, I am impressed because what you stated implies over 187K Dhrystones. > Speaking of benchmarks I have two requests. >First, does anyone know of any good system level benchmarks? IBM has stated that SPEC (for sure) and RAMP-C (I think) numbers will be provided in the forseeable future. Other things you may look at are the MUSBUS and X-benchmarks. > Second, does anyone know how much time an RS 6000/320 will take to do a > context switch?????? We've talked to our SE... Please refer your SE to me. -- David David Luner IBM Madison (608) 273-5243 Disclaimer: #include <sys/aix/rs6000/disclaimer>
rcd@ico.isc.com (Dick Dunn) (03/03/90)
shawn@jdyx.UUCP (Shawn Hayes) writes: > I recently got a copy of the Dhrystone program V2.0 and ran it on our > RS 320 machine. The results aren't quite what IBM stated (using V1.0), but > that shouldn't be a big suprise. :> I got a value of 11,881.2 dhrystones/sec > using a non-optimized version, and a value of 18,867.9 dhrystones/second > using the optimizer. Averages out to about 11 MIPS which isn't bad. Hold on...this is a machine which is advertised at 27 MIPS, right? And you say 11 doesn't sound bad??? Let's recheck things here. Are you sure you got the clock speed right? The Dhrystone benchmarks depend on having the value HZ set properly (in 1.x it's in the code; in 2.x it's in the Makefile). Please do a sanity check, and could someone else try to replicate the results? The best "sanity check" is to do "time dry2" to get the "time" command's idea of total CPU time; then see if the benchmark's reported "Microseconds for one run" times the number of iterations gives you about the right CPU time number. ALSO, time it with your watch on a quiet machine (as an extra sanity check on the time command:-). I find it hard to believe that a machine could run less than half as fast on a 2.x benchmark as on a 1.x benchmark. I can believe 10% or maybe as much as 20% degradation, but not 50%. There must be something else going on. Let's see...if the machine has a 100 Hz clock and you ran the benchmark with it set for 60, you'd be at about 31-32000 Dhrystones, which still isn't enough. > In addition I decided to run another test today and ran 11 copies of the > optimized version at the same time. I got about 18,700 dhrystones/sec on > 10 of the copies and about 14,000 on the last copy. Not bad at all. Hold it. What's good about that? Dhrystone is purely a CPU benchmark, and it measures its results in CPU time. It should give nearly the same results regardless of machine load, but your results show something like a 1/3 increase in time (and only on one of the jobs, which is really strange). -- Dick Dunn rcd@ico.isc.com uucp: {ncar,nbires}!ico!rcd (303)449-2870 ...Don't lend your hand to raise no flag atop no ship of fools.
clarke@acheron.uucp (Ed Clarke/10240000) (03/04/90)
From article <1990Mar2.204943.13285@ico.isc.com>, by rcd@ico.isc.com (Dick Dunn): > Let's recheck things here. Are you sure you got the clock speed right? > The Dhrystone benchmarks depend on having the value HZ set properly (in 1.x > it's in the code; in 2.x it's in the Makefile). Please do a sanity check, The current field release may have tons of debugging code in it. We're getting a new build weekly from Austin. The one we put up yesterday made gnu-emacs (X version) update the screen about five times faster than the build we were replacing. Where can I get Dhrystone 2.0 source? -- Ed Clarke | Happiness, n. An agreeable sensation arising from acheron!clarke | contemplating the misery of another. - Ambrose Bierce
emv@math.lsa.umich.edu (Edward Vielmetti) (03/06/90)
In article <3132@d75.UUCP> jon@ireland.uucp (Jon Doran) writes:
I believe that having slow memory cards installed will also drop
the dhrystone numbers. Lets make sure we are comparing the same
configurations when we quote benchmarks...
Jon, could you clarify what you mean by "slow" memory cards ?
What are the part numbers for the slow cards (and how slow
are they), ditto for fast cards. Is it just the speed of the
DRAMs or is there something else coming into play ?
--Ed
Edward Vielmetti, U of Michigan math dept.webb@bass.tcspa.ibm.com (Bill Webb) (03/07/90)
> RS 320 machine. The results aren't quite what IBM stated (using V1.0), but > that shouldn't be a big suprise. :> I got a value of 11,881.2 dhrystones/sec > using a non-optimized version, and a value of 18,867.9 dhrystones/second > using the optimizer. Averages out to about 11 MIPS which isn't bad. > Shawn Hayes I'm surprised that the numbers you are getting are so low. The number's I've been getting are somewhat higher (I don't think that I should post them since I don't speak for IBM). The command I used to compile cc -I/usr/include/sys -DLOOPS=500000 -DTIMES -DHZ=100 -Q -O dhry[12].c note that you need to specify -DHZ=100 because a clock tick on an RS/6000 is 1/100 of a second. If you use the default (60 ticks/second) then you will understate the results by about 40%. I think that specifying -Q causes some functions to be inlined, which also improves the results. In any case, when I compiled it without -Q and without -DHZ=100 I got numbers that were close (with 10%) of what you specified above. With both -DHZ=100 and -Q I numbers that are about twice what you got. I hope this helps explain some of the difference between the claimed values and what you got. Note also that this is pre-release software and that you may not have the same version of the compiler or system software as I (or other people) do. ---------------------------------------------------------------- The above views are my own, not necessarily those of my employer. Bill Webb (IBM AWD Palo Alto), (415) 855-4457. UUCP: ...!uunet!ibmsupt!webb
palowoda@fiver.UUCP (Bob Palowoda) (03/09/90)
From article <1990Mar3.172644.10962@acheron.uucp>, by clarke@acheron.uucp (Ed Clarke/10240000): > From article <1990Mar2.204943.13285@ico.isc.com>, by rcd@ico.isc.com (Dick Dunn): >> Let's recheck things here. Are you sure you got the clock speed right? >> The Dhrystone benchmarks depend on having the value HZ set properly (in 1.x >> it's in the code; in 2.x it's in the Makefile). Please do a sanity check, > > The current field release may have tons of debugging code in it. We're > getting a new build weekly from Austin. The one we put up yesterday made > gnu-emacs (X version) update the screen about five times faster than the > build we were replacing. > > Where can I get Dhrystone 2.0 source? I beleive it's in the new UNIX benchmarks. I have it on my bbs file: Bbench.tar.Z Located in the 'unix_bench' files area. Please don't ask me to mail it, it's 800K. (Should compile on any unix system) Below is an example of the benchmark run on a 486. Hey it's no SPEC but I would like to see the numbers compared to the 6000 anyways. Start Benchmark Run (BYTE Version 2.1) Sun Mar 4 17:12:22 PST 1990 (long iterations times) 3 interactive users. sysname=XENIX nodename=tcomeng release=2.3.2 version=SysV machine=i80386 origin=3 oem=3000 serial#=9575 ============================================================================= Dhrystone 2 without register variables ----------------------------------------------------------------------------- Arithmetric Geometric Variance Mean Mean (6 tests) Microseconds/loop: 54 54 0.87 Dhrystones/sec: 18466 18464 96855.37 ============================================================================= Dhrystone 2 using register variables ----------------------------------------------------------------------------- Arithmetric Geometric Variance Mean Mean (6 tests) Microseconds/loop: 53 53 0.43 Dhrystones/sec: 18851 18849 60963.84 ============================================================================= Sequential Memory Access Test: 1000000 Accesses ----------------------------------------------------------------------------- Array Size: 512 bytes Arithmetric Geometric Variance Mean Mean (6 tests) CPU Time (secs): 0.10 0.10 42.59 Real Time (secs): 5.83 5.70 1.77 Rate (access/sec): 175436 ----------------------------------------------------------------------------- Array Size: 1024 bytes Arithmetric Geometric Variance Mean Mean (6 tests) CPU Time (secs): 0.11 0.10 49.19 Real Time (secs): 6.33 6.26 1.07 Rate (access/sec): 159813 ----------------------------------------------------------------------------- Array Size: 2048 bytes Arithmetric Geometric Variance Mean Mean (6 tests) CPU Time (secs): 0.11 0.10 49.19 Real Time (secs): 6.33 6.26 1.07 Rate (access/sec): 159813 ----------------------------------------------------------------------------- Array Size: 8192 bytes Arithmetric Geometric Variance Mean Mean (6 tests) CPU Time (secs): 0.10 0.10 46.59 Real Time (secs): 6.17 6.10 0.97 Rate (access/sec): 163972 ----------------------------------------------------------------------------- Array Size: 16384 bytes Arithmetric Geometric Variance Mean Mean (6 tests) CPU Time (secs): 0.10 0.10 46.19 Real Time (secs): 6.17 6.13 0.57 Rate (access/sec): 163204 ============================================================================= Random Memory Access Test: 1000000 Accesses ----------------------------------------------------------------------------- Array Size: 512 bytes Arithmetric Geometric Variance Mean Mean (6 tests) CPU Time (secs): 0.18 0.18 141.36 Real Time (secs): 10.83 10.81 0.57 Rate (access/sec): 92493 ----------------------------------------------------------------------------- Array Size: 1024 bytes Arithmetric Geometric Variance Mean Mean (6 tests) CPU Time (secs): 0.17 0.17 128.76 Real Time (secs): 10.33 10.31 0.67 Rate (access/sec): 97035 ----------------------------------------------------------------------------- Array Size: 2048 bytes Arithmetric Geometric Variance Mean Mean (6 tests) CPU Time (secs): 0.16 0.16 116.57 Real Time (secs): 9.83 9.81 0.57 Rate (access/sec): 101942 ----------------------------------------------------------------------------- Array Size: 8192 bytes Arithmetric Geometric Variance Mean Mean (6 tests) CPU Time (secs): 0.17 0.17 120.37 Real Time (secs): 10.00 9.98 0.40 Rate (access/sec): 100168 ----------------------------------------------------------------------------- Array Size: 16384 bytes Arithmetric Geometric Variance Mean Mean (6 tests) CPU Time (secs): 0.19 0.19 150.16 Real Time (secs): 11.17 11.15 0.57 Rate (access/sec): 89724 ============================================================================= Arithmetic Test (type = arithoh): 10000 Iterations ----------------------------------------------------------------------------- Arithmetric Geometric Variance Mean Mean (6 tests) User Time (secs): 0.30 0.30 0.00 System Time (secs): 0.00 1.00 0.00 Real Time (secs): 0.43 0.43 0.01 ============================================================================= Arithmetic Test (type = register): 10000 Iterations ----------------------------------------------------------------------------- Arithmetric Geometric Variance Mean Mean (6 tests) User Time (secs): 3.10 3.10 0.00 System Time (secs): 0.00 1.00 0.00 Real Time (secs): 3.22 3.22 0.00 ============================================================================= Arithmetic Test (type = short): 10000 Iterations ----------------------------------------------------------------------------- Arithmetric Geometric Variance Mean Mean (6 tests) User Time (secs): 3.50 3.50 0.00 System Time (secs): 0.00 1.00 0.00 Real Time (secs): 3.60 3.60 0.01 ============================================================================= Arithmetic Test (type = int): 10000 Iterations ----------------------------------------------------------------------------- Arithmetric Geometric Variance Mean Mean (6 tests) User Time (secs): 3.30 3.30 0.00 System Time (secs): 0.00 1.00 0.00 Real Time (secs): 3.33 3.33 0.01 ============================================================================= Arithmetic Test (type = long): 10000 Iterations ----------------------------------------------------------------------------- Arithmetric Geometric Variance Mean Mean (6 tests) User Time (secs): 3.28 3.28 0.00 System Time (secs): 0.00 1.00 0.00 Real Time (secs): 3.33 3.33 0.01 ============================================================================= Arithmetic Test (type = float): 10000 Iterations ----------------------------------------------------------------------------- Arithmetric Geometric Variance Mean Mean (6 tests) User Time (secs): 6.50 6.50 0.00 System Time (secs): 0.00 1.00 0.00 Real Time (secs): 6.55 6.55 0.02 ============================================================================= Arithmetic Test (type = double): 10000 Iterations ----------------------------------------------------------------------------- Arithmetric Geometric Variance Mean Mean (6 tests) User Time (secs): 7.20 7.20 0.00 System Time (secs): 0.00 1.00 0.00 Real Time (secs): 7.25 7.25 0.01 ============================================================================= System Call Overhead Test: 5 x 4000 Calls ----------------------------------------------------------------------------- Arithmetric Geometric Variance Mean Mean (6 tests) User Time (secs): 0.17 0.16 0.00 System Time (secs): 0.53 0.53 0.00 Real Time (secs): 0.82 0.82 0.00 ============================================================================= Pipe Throughput Test: read & write 2048 x 512 byte blocks ----------------------------------------------------------------------------- Arithmetric Geometric Variance Mean Mean (6 tests) User Time (secs): 0.02 0.68 0.00 System Time (secs): 0.65 0.65 0.00 Real Time (secs): 0.72 0.72 0.00 ============================================================================= Pipe-based Context Switching Test: 2 x 500 Switches ----------------------------------------------------------------------------- Arithmetric Geometric Variance Mean Mean (6 tests) User Time (secs): 0.00 1.00 0.00 System Time (secs): 0.20 0.20 0.00 Real Time (secs): 0.55 0.55 0.01 ============================================================================= Process Creation Test: 100 forks ----------------------------------------------------------------------------- Arithmetric Geometric Variance Mean Mean (6 tests) User Time (secs): 0.00 1.00 0.00 System Time (secs): 0.70 0.70 0.00 Real Time (secs): 0.83 0.83 0.01 ============================================================================= Execl Throughput Test: 100 execs ----------------------------------------------------------------------------- Arithmetric Geometric Variance Mean Mean (6 tests) User Time (secs): 0.02 0.68 0.00 System Time (secs): 1.20 1.17 0.08 Real Time (secs): 3.43 3.43 0.03 ============================================================================= Filesystem Throughput Test: ----------------------------------------------------------------------------- File Size: 50 (1024 byte) blocks Arithmetric Geometric Variance Mean Mean (6 tests) Read (Kbytes/sec): 0 1 0.00 Write (Kbytes/sec): 2500 2500 0.00 Copy (Kbytes/sec): 2222 2082 462964.81 ----------------------------------------------------------------------------- File Size: 100 (1024 byte) blocks Arithmetric Geometric Variance Mean Mean (6 tests) Read (Kbytes/sec): 5000 5000 0.00 Write (Kbytes/sec): 2917 2806 1041666.67 Copy (Kbytes/sec): 1597 1589 28935.65 ----------------------------------------------------------------------------- File Size: 200 (1024 byte) blocks Arithmetric Geometric Variance Mean Mean (6 tests) Read (Kbytes/sec): 3333 3333 0.00 Write (Kbytes/sec): 2778 2752 185183.70 Copy (Kbytes/sec): 1438 1433 17597.64 ----------------------------------------------------------------------------- File Size: 400 (1024 byte) blocks Arithmetric Geometric Variance Mean Mean (6 tests) Read (Kbytes/sec): 3254 3249 37792.82 Write (Kbytes/sec): 2560 2556 21258.16 Copy (Kbytes/sec): 93 92 197.72 ----------------------------------------------------------------------------- File Size: 800 (1024 byte) blocks Arithmetric Geometric Variance Mean Mean (6 tests) Read (Kbytes/sec): 177 177 0.74 Write (Kbytes/sec): 112 112 35.46 Copy (Kbytes/sec): 52 52 7.85 ----------------------------------------------------------------------------- File Size: 1600 (1024 byte) blocks Arithmetric Geometric Variance Mean Mean (6 tests) Read (Kbytes/sec): 80 79 145.35 Write (Kbytes/sec): 106 105 188.09 Copy (Kbytes/sec): 41 41 7.59 ============================================================================= Client/Server Database Engine: ----------------------------------------------------------------------------- 1 client processes. Arithmetric Geometric Variance Mean Mean (3 tests) User Time (secs): 1.20 1.18 0.07 System Time (secs): 3.03 3.03 0.02 Real Time (secs): 11.03 11.01 0.70 ----------------------------------------------------------------------------- 2 client processes. Arithmetric Geometric Variance Mean Mean (3 tests) User Time (secs): 2.60 2.60 0.01 System Time (secs): 6.27 6.27 0.02 Real Time (secs): 22.33 22.33 0.22 ----------------------------------------------------------------------------- 4 client processes. Arithmetric Geometric Variance Mean Mean (3 tests) User Time (secs): 4.93 4.93 0.02 System Time (secs): 12.37 12.37 0.01 Real Time (secs): 41.17 41.16 0.74 ----------------------------------------------------------------------------- 8 client processes. Arithmetric Geometric Variance Mean Mean (3 tests) User Time (secs): 10.43 10.42 0.34 System Time (secs): 24.60 24.59 0.43 Real Time (secs): 80.23 80.23 0.17 2 interactive users. End Benchmark Run (Sun Mar 4 17:12:22 PST 1990) .... -- Bob Palowoda indetech!fiver!palowoda *Home of Fiver BBS* login: bbs Home {sun|daisy}!ys2!fiver!palowoda (415)-623-8809 1200/2400 Work {sun|pyramid|decwrl}!megatest!palowoda (415)-623-8806 2400/9600/19200 TB Voice: (415)-623-7495 palowoda@fiver Public access UNIX XBBS