[comp.sys.next] Benchmarks

sfrank@orion.oac.uci.edu (Steven Frank) (09/23/90)

In article <10548@pt.cs.cmu.edu> avie@wb1.cs.cmu.edu (Avadis Tevanian) writes:
>In article <15640002@hpdmd48.boi.hp.com> sritacco@hpdmd48.boi.hp.com (Steve Ritacco) writes:
>>Actually the slab cpu is not as high performance as the cube, it has
>>lower throughput and can only hold half as much RAM.
>
>The NeXTstation and NeXTcube have the same performance (conservatively 15
>MIPS).  The NeXTstation can be expanded to 32MB of RAM, the NeXTcube can
>be expanded to 64MB.
>-- 
>Avadis Tevanian, Jr.    (Avie)
>Manager, System Software
>NeXT, Inc.
>avie@NeXT.COM

Are Dhrystone and Linpack benchmarks available??  Ideally, the same
benchmark versions as used in Personal Workstation, to allow comparison
with other systems included in their tables of benchmarks.

kline@cs.arizona.edu (Nick Kline) (09/24/90)

In article <26FCE703.25276@orion.oac.uci.edu> sfrank@orion.oac.uci.edu (Steven Frank) writes:
>
>Are Dhrystone and Linpack benchmarks available??  Ideally, the same
>benchmark versions as used in Personal Workstation, to allow comparison
>with other systems included in their tables of benchmarks.

Or even better, just give us the SPECmarks.  That would really be useful.

I would actually trust those, but for Linpack and Dhrystone its always
what compiler did you use, what version of the benchmark (makes a big
diff with Dhrystone), what compiler tricks did you use, etc.

Give me specmarks.  I know what those are for the sparcs.

The mathematica benchmark is basically worthless, because it could
be contrived to make the NeXT fast.  I mean, does it have IO?  What
is it?  etc etc, but SPECmarks are agreed upon by lots of
different manufactures, so I trust it.

-nick "specmark" kline
---

	"This isn't a game; this is garbage collection!"
	      - heard during my Ph. D. Oral Qualifier

Nick Kline, Univ. of Az., Computer Science, Tucson, AZ 85721
(kline@cs.arizona.edu -or- {noao|allegra|cmcl2}!arizona!kline)

sritacco@hpdmd48.boi.hp.com (Steve Ritacco) (09/24/90)

The throughput of the slab is about 30M/sec. and the cube is about 40M/sec.
It says this in the specs.  The slab also has 8 DMA chanels vrs. 9 DMA chanels
on the cube.

shwake@raysnec.UUCP (Ray Shwake) (09/24/90)

sfrank@orion.oac.uci.edu (Steven Frank) writes:

>Are Dhrystone and Linpack benchmarks available??  Ideally, the same
>benchmark versions as used in Personal Workstation, to allow comparison
>with other systems included in their tables of benchmarks.

	Sigh! It seems the Magic Number Syndrome is alive and well in
the Workstation world. The state of benchmarking in the UNIX world, while
still woefully inadequate, has at least advanced beyond such primitive
measurements. [For a real-world example of how one benchmark suite can
challenge another, see UnixWorld's "Big Blue's RS/6000 Isn't White Lightning"
(Sept., 1990)].
							shwake@rsxtech

sritacco@hpdmd48.boi.hp.com (Steve Ritacco) (09/25/90)

Even if the slab couldn't have an expansion slot, couldn't the CPU board
have been the same as the cube so that you could by an empty cube and
a NeXTdimension.  It is sad that the low-end customers are locked out
of the NeXTdimension.

henry@zoo.toronto.edu (Henry Spencer) (09/26/90)

In article <15640004@hpdmd48.boi.hp.com> sritacco@hpdmd48.boi.hp.com (Steve Ritacco) writes:
>Even if the slab couldn't have an expansion slot, couldn't the CPU board
>have been the same as the cube so that you could by an empty cube and
>a NeXTdimension...

Boards with chips on them are cheap.  Boxes are expensive.  You probably
would not save very much.  It would also drive up the price of the slab.
The reason why people build closed-box low-end systems, with no bus, is
that it is easier and cheaper to build the hardware when you have complete
control and don't need to allow for a wide range of unknown add-in boards.
-- 
TCP/IP: handling tomorrow's loads today| Henry Spencer at U of Toronto Zoology
OSI: handling yesterday's loads someday|  henry@zoo.toronto.edu   utzoo!henry

garnett@cs.utexas.edu (John William Garnett) (09/26/90)

In article <83@raysnec.UUCP> shwake@raysnec.UUCP (Ray Shwake) writes:
>sfrank@orion.oac.uci.edu (Steven Frank) writes:
>
>>Are Dhrystone and Linpack benchmarks available??
>>[comments deleted]
>[other comments deleted]
>The state of benchmarking in the UNIX world, while
>still woefully inadequate, has at least advanced beyond such primitive
>measurements. [For a real-world example of how one benchmark suite can
>challenge another, see UnixWorld's "Big Blue's RS/6000 Isn't White Lightning"
>(Sept., 1990)].

Does anyone really consider the benchmark suite described in the above
mentioned UnixWorld article to be a valid indicator of performance? 
According to the article, the benchmark in question measured floating
point performance by running an awk program [this awk test appears to
have been the basis for saying that the RS/6000 is slower at floating
point than a 25Mhz 486 machine...].

Followups to comp.unix.aix or other appropriate place...
-- 
John Garnett
                              University of Texas at Austin
garnett@cs.utexas.edu         Department of Computer Science
                              Austin, Texas

cyliao@hardy.u.washington.edu (Chun-Yao Liao) (09/26/90)

In article <15640004@hpdmd48.boi.hp.com> sritacco@hpdmd48.boi.hp.com (Steve Ritacco) writes:
>Even if the slab couldn't have an expansion slot, couldn't the CPU board
>have been the same as the cube so that you could by an empty cube and
>a NeXTdimension.  It is sad that the low-end customers are locked out
>of the NeXTdimension.

I wonder why NeXT didn't put a bus connector on the slab. If so, adding a
NeXTdimension card will be something like adding a second pizza-box on the
top or bottom of NeXTstation... kind like modular...

cyliao@wam.umd.edu     		o NeXT :  I put main frame power on two chips.
      @epsl.umd.edu		o people: We put main flame power on two guys.
      @bagend.eng.umd.edu       o ::::::::::::::::::::::::::::::::::::::::::::
 xxxxx@xxxxx.xxx.xxx (reserved)	o RC + Apple // + Classic Music + NeXT = cyliao

daugher@cs.tamu.edu (Dr. Walter C. Daugherity) (09/26/90)

In article <8135@milton.u.washington.edu> cyliao@hardy.acs.washington.edu (Chun-Yao Liao) writes:
>In article <15640004@hpdmd48.boi.hp.com> sritacco@hpdmd48.boi.hp.com (Steve Ritacco) writes:
>>Even if the slab couldn't have an expansion slot, couldn't the CPU board
>>have been the same as the cube so that you could by an empty cube and
>>a NeXTdimension.  It is sad that the low-end customers are locked out
>>of the NeXTdimension.
>
>I wonder why NeXT didn't put a bus connector on the slab. If so, adding a
>NeXTdimension card will be something like adding a second pizza-box on the
>top or bottom of NeXTstation... kind like modular...

It's called money: expandable systems need a bigger chassis, a bigger power
supply, a bigger fan, a bus, and bus drivers on all the cards.  Unfortunately
it's true that poor folk can't afford expandable systems so easily, but be
thankful NeXT produced some options they (we) ***CAN*** afford!  Sun et al.
have already done so--anybody want to buy some of our Sun 3/50's?

Anyway, at the rate VLSI is shrinking, the 1992 NeXT Model 3 may very well
fit 32-bit color into a slab!


-------------------------------------------------------------------------------
Walter C. Daugherity			Internet, NeXTmail: daugher@cs.tamu.edu
Knowledge Systems Research Center	uucp: uunet!cs.tamu.edu!daugher
Texas A & M University			BITNET: DAUGHER@TAMVENUS
College Station, TX 77843-3112		CSNET: daugher%cs.tamu.edu@RELAY.CS.NET
	---Not an official document of Texas A&M---

zilla@nyit.UUCP (John Lewis) (11/20/90)

In benchmarking the sun 4 (/280? - one of the first ones) we found
that the times reported by the 'time' command were as little as
1/3 of the real elapsed time when running with a single active
benchmark process and no network (though normal timesharing unix
was up).  
The time command also gave fishy utilization percentages like 150%.
We decided to use the single-user real-time to measure this machine
(since if it can't do it in real time, the numbers dont mean much),
with results that this supposedly 10-Mips machine was between 3.5-5 mips
on some benchmarks designed for our applications.
Based on this experience, i don't trust most sun benchmarks...
h
john lewis @nyit computer graphics

nerd@percy.rain.com (Michael Galassi) (01/05/91)

In article <10016@pasteur.Berkeley.EDU> jclee@cory.Berkeley.EDU (James C. Lee) writes:

>> The NeXTStation was about 20-30% faster on this group of benchmarks
>> than the DECStation 3100.  This is surprising since the DECStation is
>> rated and 13.9 Mips and the NeXTstation benches at 15.0 Mips.  You
>> would expect from the Mips rating only about a 10% increase.

>or maybe it's because the 15.0 MIPS is a conservative estimate. I know
>that some manufacturers using 68040 have rated their computers at
>17.0 MIPS or higher....

MIPS = Milions of Instructions Per Second.  An 040 does more in each
average instruction than the MIPS R[23]000 (in the DECstations) thus
1 Motorola MIPS yeilds more preformace than 1 R[23]000 MIPS all else
being equal.  It folows that no usefull information can be gotten by
comparing MIPS figures for processors of different families.

Since most of us have no real interest in how many instructions the
cpu executes but rather are concerned with how long our computer takes
to complete a task why don't we compare something more relevant.  I
would suggest the SPEC benchmarks if they have been run on the NeXT
or some other equaly general set.
-- 
Michael Galassi				| nerd@percy.rain.com
MS-DOS:  The ultimate PC virus.		| ...!tektronix!percy!nerd

romero@parc.xerox.com (Antonio Romero) (01/09/91)

In article <1991Jan5.022126.19747@percy.rain.com> nerd@percy.rain.com (Michael Galassi) writes:
>In article <10016@pasteur.Berkeley.EDU> jclee@cory.Berkeley.EDU (James C. Lee) writes:
>>> The NeXTStation was about 20-30% faster on this group of benchmarks
>>> than the DECStation 3100.  This is surprising since the DECStation is
>>> rated and 13.9 Mips and the NeXTstation benches at 15.0 Mips.  You
>>> would expect from the Mips rating only about a 10% increase.

>MIPS = Milions of Instructions Per Second.  An 040 does more in each
>average instruction than the MIPS R[23]000 (in the DECstations) 
>...  It folows that no usefull information can be gotten by
>comparing MIPS figures for processors of different families.

Nope, 'fraid not...  MIPS these days have been normalized to make 1 MIPS=
some canonical VAX 11/780 somewhere.

This is not to suggest that useful information can be gotten from
hype-sheet MIPS figures, but they're inaccurate because people tweak
the tests that generate the numbers to take advantage of on-chip caches,
unusually fast instructions unique to that chip, etc., not because the
processors do different amounts of work per instruction.

MIPS ratings are notoriously bogus.  Does anyone know if the SPECMARK is
any better, or if it's susceptible to the same sort of cheating that
more traditional benchmarks suffer from?

-Antonio Romero         romero@arisia.xerox.com

neil@ms.uky.edu (Neil Greene) (04/01/91)

Comparison of performance

Article By: 

NeXTWorld - April 1991

Edward Jung is the program manager for the information at Your Fingertips
Projec
t at Microsoft Corporation and a pricipal of the Deep Thought Group L.P., a
neur
al network research and development group.

Bruce F. Webster is author of The NeXT Book and vice president of reserch and
de
velopment at Pages, a software company in San Diego, California.


Benchmark (count)       NeXT    Mac IIfx        NeXT    SPARCstation 2
                        ('030)                  ('040)

Sieve   (1000)          39.5    19.9            15.6    7.3
Sort    (2000)          24.5    11.7            9.5     4.9
Matrix  (50)            39.0    25.1            16.4    5.8
Float   (10,000)        9.4     12.2            6.4     3.4
Savage  (20,000)        29.7    12.3            24.7    5.6
Circle  (500)           28.9    7.5             9.3     --
Square  (1000)          28.8    14.9            14.9    --
Dhrystone2              5,451   9,011           22,727  34,884

Compilers
Macintosh IIfx: cc -mc68020 -mc6881 -elems881 -opt full -opt speed
SPARCstation 2: cc -O4 (Sun C)
NeXT systems: cc -O (Objective C 2.0, from FNU cc version 1.36)

-- 
Neil Greene ---	University of Kentucky Mathmatics and Sciences
		University of Kentucky Computing Center 

neil@graphlab.cc.uky.edu [NeXT Attachments]

melling@cs.psu.edu (Michael D Mellinger) (04/03/91)

Has anyone gotten around to SPECmarking the NeXT yet?  I think some
results from HP's 68040 put the 040 at around 12.  By the way, has
everyone seen the SPECS on HP's Snake machines?  Looks like they might
have a NeXTDimension Board built into their CPU.

-Mike

victor@tesla.math.yale.edu (Mladen Victor Wickerhauser) (04/04/91)

In article <neil.670518725@s.ms.uky.edu> neil@ms.uky.edu (Neil Greene) writes:
>Comparison of performance
  [comparison deleted]
>Compilers
>Macintosh IIfx: cc -mc68020 -mc6881 -elems881 -opt full -opt speed
>SPARCstation 2: cc -O4 (Sun C)
>NeXT systems: cc -O (Objective C 2.0, from FNU cc version 1.36)

	According to the GNU C-compiler info, it is more efficient to use
	cc -O -g ...., since the debugging information (-g) helps the
	optimizer make certain heuristic choices.
	
>-- 
>Neil Greene ---	University of Kentucky Mathmatics and Sciences
>		University of Kentucky Computing Center 
>
>neil@graphlab.cc.uky.edu [NeXT Attachments]

	From a satisfied customer of both SUN and NeXT....


-- 
------
Mladen Victor Wickerhauser, victor@math.yale.edu, (203)498-1011
Dept. of Mathematics, Yale University, New Haven, CT 06520 (USA)

melling@cs.psu.edu (Michael D Mellinger) (04/05/91)

In article <29847@cs.yale.edu> victor@tesla.math.yale.edu (Mladen Victor Wickerhauser) writes:


	   According to the GNU C-compiler info, it is more efficient to use
	   cc -O -g ...., since the debugging information (-g) helps the
	   optimizer make certain heuristic choices.

Where does it say this?

-Mike

m.tiernan@pro-angmar.UUCP (Michael Tiernan) (04/06/91)

In-Reply-To: message from neil@ms.uky.edu

Is there a source code set for these benchmarks available?  I'd like to run
them on my UNIX box and see what they give.


<< MCT >>

GEnie       : M.Tiernan
AppleLinkPE : M Tiernan or BCS Mike
Internet    : pro-angmar!m.tiernan@alfalfa.com
UUCP        : ...!uunet!alfalfa!pro-angmar!m.tiernan

"God isn't dead, he's only missing in action."
                                             - Phil Ochs