mac@gold.kpc.com (Mike McNamara) (04/16/91)
The following is motivated by laments heard at ASPLOS
concerning the derth of useful OS benchmarks. This benchmarks suffers
from all the usual aliments:
It generates a single figure of merit;
It is suspectable to point optimizations of the target
machine;
It gives different results when run in single user mode versus
when run at 3 pm the day before tapeout;
It assumes you are running a UNIX on a machine with /bin/csh;
It probably does not model your application real well;
It probably isn't real useful ;-)
However, it tests interesting things: How often can the system
create, run, and exit a simple program, like /bin/date ?
#!/bin/csh
set i = 0
set count = 500
set host = `/bin/hostname`
echo " The Date Per Second Benchmark "
echo " ----------------------------- "
echo " Timing $count /bin/dates..."
onintr die
## if you have a modern date...
#set k = `date '+%H %M %S' `
#set hours = $k[1]
#set minutes = $k[2]
#set seconds = $k[3]
# else
set k = `date | sed -e 's/:/ /g'`
set hours = $k[4]
set minutes = $k[5]
set seconds = $k[6]
# endif
set start = 0
@ start = $hours * 3600 + $minutes * 60 + $seconds
echo "Start time is $start ..."
while ( $i < $count )
/bin/date > /dev/null
@ i = $i + 1
end
die:
## if you have a modern date...
# set k = `date '+%H %M %S' `
#set hours = $k[1]
#set minutes = $k[2]
#set seconds = $k[3]
# else
set k = `date | sed -e 's/:/ /g'`
set hours = $k[4]
set minutes = $k[5]
set seconds = $k[6]
# endif
set finish = 0
@ finish = $hours * 3600 + $minutes * 60 + $seconds
echo ...End time is $finish
set delta = 0
@ delta = $finish - $start
echo $i dates in $delta seconds:
set dps = `echo " $i / $delta " | bc `
echo
echo The machine \"${host}\" executes $dps dates per second
Just to start things off, the machine on my desk runs:
mac@mack 25 % ./dps
The Date Per Second Benchmark
-----------------------------
Timing 500 /bin/dates...
Start time is 57001 ...
...End time is 57020
500 dates in 19 seconds:
The machine "mack" executes 26 dates per second
mac@mack 26 %
My machine is a 33MHz R3000 with a 33MHz bus.
Machine OS DPS Load
Sun 3/50 4.0 3 Unloaded
Sun 3/280 4.1.1 6 Normal fileserver load
Stardent 30xx 4.0 12 Unloaded
My machine -- 26 Unloaded
How does your machine compare?
--
+-----------+-----------------------------------------------------------------+
|mac@kpc.com| Increasing Software complexity lets us sell Mainframes as |
| | personal computers. Carry on, X windows/Postscript/emacs/CASE!! |
+-----------+-----------------------------------------------------------------+de5@ornl.gov (Dave Sill) (04/16/91)
In article <MAC.91Apr15161150@gold.kpc.com>, mac@gold.kpc.com (Mike McNamara) writes: > >Machine OS DPS Load >Sun 3/50 4.0 3 Unloaded >Sun 3/280 4.1.1 6 Normal fileserver load >Stardent 30xx 4.0 12 Unloaded >My machine -- 26 Unloaded > >How does your machine compare? IBM RS 320 3.1 23 Unloaded DEC 5000/200 4.1 35 Unloaded DEC 5810 3.1 15 Light Cray X-MP 5.1.11 27 Light Sun SS1+ 4.1 9 Light DEC 2100 4.0 13 Light -- Dave Sill (de5@ornl.gov) It will be a great day when our schools have Martin Marietta Energy Systems all the money they need and the Air Force Workstation Support has to hold a bake sale to buy a new bomber.
dayhoff@ddnvx1.afwl.af.mil (dayhoff@ddnvx1.afwl.af.mil -- harv) (04/16/91)
In article <2308@spim.mips.COM>, hawkes@mips.com (John Hawkes) writes: > In <1991Apr16.120134.5264@cs.utk.edu> Dave Sill <de5@ornl.gov> writes: >><MAC.91Apr15161150@gold.kpc.com>, mac@gold.kpc.com (Mike McNamara) writes: >>> >>>Machine OS DPS Load >>>Sun 3/50 4.0 3 Unloaded >>>Sun 3/280 4.1.1 6 Normal fileserver load >>>Stardent 30xx 4.0 12 Unloaded >>>My machine -- 26 Unloaded >>> >>>How does your machine compare? >> >>IBM RS 320 3.1 23 Unloaded >>DEC 5000/200 4.1 35 Unloaded >>DEC 5810 3.1 15 Light >>Cray X-MP 5.1.11 27 Light >>Sun SS1+ 4.1 9 Light >>DEC 2100 4.0 13 Light > > MIPS M/2000 4.52 33 Light > MIPS RC6280 4.52+ 50 Unloaded > -- VAXstation3100 4.0 8 light, 8 MB mem VAXserver3xxx 4.0 9 light, except for net, 16 Meg -- RS (harvey) Dayhoff OL-NS/ENSB (505) 846-5392 The USAF, and ASD have no opinions; as for me, I am a scientest... the TRUTH is the collective best guess.
hawkes@mips.com (John Hawkes) (04/16/91)
In article <1991Apr16.120134.5264@cs.utk.edu> Dave Sill <de5@ornl.gov> writes: >In article <MAC.91Apr15161150@gold.kpc.com>, mac@gold.kpc.com (Mike McNamara) writes: >> >>Machine OS DPS Load >>Sun 3/50 4.0 3 Unloaded >>Sun 3/280 4.1.1 6 Normal fileserver load >>Stardent 30xx 4.0 12 Unloaded >>My machine -- 26 Unloaded >> >>How does your machine compare? > >IBM RS 320 3.1 23 Unloaded >DEC 5000/200 4.1 35 Unloaded >DEC 5810 3.1 15 Light >Cray X-MP 5.1.11 27 Light >Sun SS1+ 4.1 9 Light >DEC 2100 4.0 13 Light MIPS M/2000 4.52 33 Light MIPS RC6280 4.52+ 50 Unloaded -- John Hawkes {ames,decwrl}!mips!hawkes OR hawkes@mips.com
jwk@Scripps.EDU (John Kupec) (04/17/91)
Convex C220 9.0 23 Lightly loaded (2.85 load avg.) SGI 4D/210 3.3.1 35 Unloaded E&S Workstation 1.3 41 Unloaded -- John Kupec, Agouron Pharmaceuticals, Inc., La Jolla, CA jwk@scripps.edu or uunet!agouron!kupec
rwa@cs.athabascau.ca (Ross Alexander) (04/17/91)
hawkes@mips.com (John Hawkes) writes: >MIPS M/2000 4.52 33 Light >MIPS RC6280 4.52+ 50 Unloaded SGI 320/D - four CPUs, Irix 3.2.2 - 38 dps, unloaded -- Ross Alexander rwa@cs.athabascau.ca (403) 675 6311 ve6pdq "I'd like MY data-base JULIENNED and stir-fried!" -- Zippy
john@jwt.UUCP (John Temples) (04/17/91)
In article <MAC.91Apr15161150@gold.kpc.com> mac@kpc.com writes: >How does your machine compare? >Machine OS DPS Load 386/33 SVR3.2 21 Unloaded -- John W. Temples -- john@jwt.UUCP (uunet!jwt!john)
john@iastate.edu (Hascall John Paul) (04/17/91)
In article <1991Apr16.120134.5264@cs.utk.edu> Dave Sill <de5@ornl.gov> writes: }In article <MAC.91Apr15161150@gold.kpc.com>, mac@gold.kpc.com (Mike McNamara) writes: }> }>Machine OS DPS Load }DEC 2100 4.0 13 Light DEC 2100 3.1d 22 Very Light DEC 2100 3.1d 27 Loop unrolled (x5) :-) DEC 2100 3.1d 29 Loop unrolled (x10) John -- John Hascall An ill-chosen word is the fool's messenger. Project Vincent Iowa State University Computation Center john@iastate.edu Ames, IA 50011 (515) 294-9551
john@iastate.edu (Hascall John Paul) (04/17/91)
In article <1991Apr16.120134.5264@cs.utk.edu> Dave Sill <de5@ornl.gov> writes: }In article <MAC.91Apr15161150@gold.kpc.com>, mac@gold.kpc.com (Mike McNamara) writes: }> }>Machine OS DPS Load }DEC 5000/200 4.1 35 Unloaded DEC 5000/200 4.1 45 Very Light }DEC 2100 4.0 13 Light DEC 2100 3.1d 22 Very Light Hmmm, seems to be some amount of variability here. Any guesses? Disk speed? Amount of memory? (or just "Light" isn't all that light!). John -- John Hascall An ill-chosen word is the fool's messenger. Project Vincent Iowa State University Computation Center john@iastate.edu Ames, IA 50011 (515) 294-9551
de5@ornl.gov (Dave Sill) (04/17/91)
In article <1991Apr17.043333.28977@news.iastate.edu>, john@iastate.edu (Hascall John Paul) writes: >In article <1991Apr16.120134.5264@cs.utk.edu> Dave Sill <de5@ornl.gov> writes: >} >}DEC 5000/200 4.1 35 Unloaded > DEC 5000/200 4.1 45 Very Light I ran it again. I was the only person logged in, the load average was straight 0's. The machine has 24MB, and provides NFS service to a handful of systems, none of which are currently active. I got 35 dps again. >}DEC 2100 4.0 13 Light > DEC 2100 3.1d 22 Very Light I ran this again, too. This is my personal machine. It has 16MB, no other users. I'm running the DECwindows X server, xrn, a local emacs, a remote emacs, one local xterm, two remote xterms, a remote Synchronize, and twm. The load averages were 0.67, 0.38, 0.00. I got 14 dps. > Hmmm, seems to be some amount of variability here. Any guesses? I suppose the 3.1d vs. 4.0 could be a factor for the 2100. >Disk speed? Seems unlikely, since little I/O is being done. >Amount of memory? Possibly. What have you got? >(or just "Light" isn't all that light!). What kind of load averages were you running at? What kind of application profile? This is a perfect example of value of tiny/trivial benchmarks... -- Dave Sill (de5@ornl.gov) It will be a great day when our schools have Martin Marietta Energy Systems all the money they need and the Air Force Workstation Support has to hold a bake sale to buy a new bomber.
lusol@vax1.cc.lehigh.edu (04/17/91)
All machines essentially unloaded: Vendor OS DPS CPU ------ -- --- --- Control Data 4680 EP/IX 1.2.3 62 R6000 Control Data 4380 EP/IX 1.2.3 35 R3000 DECstation 5000 Ultrix 4.1 33 R3000 IBM RS/6000 320 AIX 23 RS6000 Silicon Graphics Irix 3.3.1 21 R3000 Sun SPARC 1+ SunOS 4.1.1 9 SPARC
mis@Seiden.com (Mark Seiden) (04/17/91)
de5@ornl.gov (Dave Sill) and mac@gold.kpc.com (Mike McNamara) and hawkes@mips.com (John Hawkes) write: (and i append a couple machines, reformat and sort) Machine OS DPS Load Sun 3/50 4.0 3 Unloaded Sun 3/60 4.0.3 6 Light Sun 3/280 4.1.1 6 Normal fileserver load Sun SS1+ 4.1 9 Light Stardent 30xx 4.0 12 Unloaded DEC 2100 4.0 13 Light DEC 5810 3.1 15 Light Sun SS2 4.1.1 16 Light IBM RS 320 3.1 23 Unloaded MIPS R/3000-33mhz bus ? 26 Unloaded Cray X-MP 5.1.11 27 Light MIPS M/2000 4.52 33 Light DEC 5000/200 4.1 35 Unloaded MIPS RC6280 4.52+ 50 Unloaded How does your machine compare?
lusol@vax1.cc.lehigh.edu (04/18/91)
Why does one DEC 5000 have a DPS speed of 35 and another of 45..??? The DPS rate has 1 second granularity, that's why. So, although the CDC 4680 usually gives 62 DPS because it runs in 8 seconds, sometimes I get 71 DPS for a 7 second run!
john@iastate.edu (Hascall John Paul) (04/18/91)
lusol@vax1.cc.lehigh.edu writes: }Why does one DEC 5000 have a DPS speed of 35 and another of 45..??? }The DPS rate has 1 second granularity, that's why. Only 35 vs 45 is a 3 second difference (14 vs 11). }So, although the CDC 4680 usually }gives 62 DPS because it runs in 8 seconds, sometimes I get 71 DPS for a 7 }second run! As for why, my DS2100/DS5000 numbers were higher than the other posters, I really don't know. My load averages were probably right near 0 (not while it was running!), and the machines have 12 and 24 MB respectively. Must be the DECwindows (we use Motif/X11R4) ;-) John -- John Hascall An ill-chosen word is the fool's messenger. Project Vincent Iowa State University Computation Center john@iastate.edu Ames, IA 50011 (515) 294-9551
pdg@cs.uow.edu.au (Peter Gray) (04/19/91)
de5@ornl.gov (Dave Sill) writes: >In article <MAC.91Apr15161150@gold.kpc.com>, mac@gold.kpc.com (Mike McNamara) writes: >> >>Machine OS DPS Load >>Sun 3/50 4.0 3 Unloaded >>Sun 3/280 4.1.1 6 Normal fileserver load >>Stardent 30xx 4.0 12 Unloaded >>My machine -- 26 Unloaded >> >>How does your machine compare? >IBM RS 320 3.1 23 Unloaded >DEC 5000/200 4.1 35 Unloaded >DEC 5810 3.1 15 Light >Cray X-MP 5.1.11 27 Light >Sun SS1+ 4.1 9 Light It appears the SS1 suffers here due to shared libraries, or maybe just a bad implementation. If you try the "benchmark" with a statically linked date you get 20 DPS just on a normal SS1 (not a +). >DEC 2100 4.0 13 Light pdg Peter Gray Internet: pdg@draci.cs.uow.EDU.AU Professional Officer UUCP: ...!munnari!draci.cs.uow.EDU.AU!pdg Dept of Computer Science MHSnet: pdg@draci.cs.uow.oz.au University of Wollongong Phone: +61 42 213770 N.S.W. 2500 Australia Fax : +61 42 213262
rhb@mstr.hgc.edu (Roger H. Brown) (04/20/91)
In article <3630@dagobah.UUCP> mis@Seiden.com (Mark Seiden) writes: >de5@ornl.gov (Dave Sill) and mac@gold.kpc.com (Mike McNamara) and hawkes@mips.com (John Hawkes) write: >(and i append a couple machines, reformat and sort) > >Machine OS DPS Load >Sun 3/50 4.0 3 Unloaded >Sun 3/60 4.0.3 6 Light >Sun 3/280 4.1.1 6 Normal fileserver load >Sun SS1+ 4.1 9 Light Sun IPC 4.1.1 11 Unloaded Solbourne S4000 4.1 12 Unloaded >Stardent 30xx 4.0 12 Unloaded >DEC 2100 4.0 13 Light >DEC 5810 3.1 15 Light >Sun SS2 4.1.1 16 Light >IBM RS 320 3.1 23 Unloaded >MIPS R/3000-33mhz bus ? 26 Unloaded >Cray X-MP 5.1.11 27 Light >MIPS M/2000 4.52 33 Light >DEC 5000/200 4.1 35 Unloaded >MIPS RC6280 4.52+ 50 Unloaded > >How does your machine compare? |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| | Roger H. Brown GEnie: R.BROWN64 AOL: Roger48 | | Sch. of Eng. & Science rhb@mstr.hgc.edu ...uupsi!mstr!rhb | |_____________________________________________________________________|
andreess@mrlaxa.mrl.uiuc.edu (Marc Andreessen) (04/20/91)
In article <1991Apr20.001049.6681@mstr.hgc.edu> rhb@mstr.hgc.edu (Roger H. Brown) writes: >In article <3630@dagobah.UUCP> mis@Seiden.com (Mark Seiden) writes: >>de5@ornl.gov (Dave Sill) and mac@gold.kpc.com (Mike McNamara) and hawkes@mips.com (John Hawkes) write: [... lots of meaningless numbers ...] Maybe it's just lack of sleep, but it seems to me that this newsgroup, which I've been reading since its formation, spends about 95% of its time spewing out a plethora of meaningless numbers on meaningless and trivial little pseudo-hacks which don't deserve to be called benchmarks by any stretch of the imagination. Remember 'bc'? And now this... These numbers are useless. Marc -- Marc Andreessen___________University of Illinois Materials Research Laboratory Internet: andreessen@uimrl7.mrl.uiuc.edu____________Bitnet: andreessen@uiucmrl
john@iastate.edu (Hascall John Paul) (04/21/91)
andreess@mrlaxa.mrl.uiuc.edu (Marc Andreessen) writes: }[... lots of meaningless numbers ...] } ... a plethora of meaningless numbers on meaningless and trivial }little pseudo-hacks which don't deserve to be called benchmarks by any }stretch of the imagination. }These numbers are useless. I disagree. While it could use some work (see below) it does do a fair job of measuring one small, but extremely common, aspect of system performance -- process/image startup. To improve it, I would suggest the following: 1) unroll the loop [50x seems adequate] to minimize the effect of the shell loop overhead. 2) use /bin/true rather than /bin/date -- John Hascall An ill-chosen word is the fool's messenger. Project Vincent Iowa State University Computation Center john@iastate.edu Ames, IA 50011 (515) 294-9551
AER7101@TECHNION.BITNET (Zvika Bar-Deroma) (04/22/91)
Here's my $0.02 worth of benchmarking - Machine OS DPS Load IBM RS/6000-320 AIX 3.1 26 unloaded (except net) CONVEX C220 CONVEXOS 9.0 21 loaded IBM 6150 (RT) AIX 2.2.1 5 unloaded SUN 4/490 SunOS 4.? 10 light-average SUN 3/350 SunOS 4.? 5 unloaded DEC station 2100 ULTRIX 3.1 17 unloaded /Zvika Zvika Bar-Deroma Phone: (+972)-4-292706 Faculty of Aerospace Engineering, Fax : (+972)-4-231848 Technion Haifa 32000 Israel BITNET : AER7101@TECHNION Internet : AER7101@TECHNION.TECHNION.AC.IL UUCP : ...!uunet!pucc.princeton.edu!technion!aer7101