[comp.arch] SPEC Bench-a-Thon

rnovak@mips.com (Robert E. Novak) (06/21/89)

During the week of June 5-9, the SPEC Bench-a-thon was held at MIPS
computers.  We had 7 vendors participating with 10 different architectures.
It was a grueling week for many of us.  In a single week we had to analyze 
24 different benchmarks and ensure that they would run on all of the
machines.  What we did not do was post results from running the benchmarks.

The participating vendors included:

Apollo			2 architectures
DEC			2 architectures and 2 Operating Systems
Hewlett-Packard		2 architectures
IBM
MIPS
Motorola
SUN

As a result of this meeting, the SPEC consortium produced an Alpha tape of
11 programs that could be run by all vendors.  This tape has been
distributed to all of the participants by now.  We are awaiting comments
from the participants to clean up any small flaws in the Alpha tapes and we
will generate a Beta release on June 30.  The Beta tape will undergo a 60
day evaluation period.  

This tape will be available to all companies that are members of 
the SPEC consortium.  The tape will be made publicly available at a to be
determined nominal cost at the end of the Beta period.

The tape contains the following benchmarks which have run on the machines
of the participants:

GCC 1.35	Integer intensive C compiler for CASE
espresso	C, Integer-intensive benchmark for ECAD
spice2g6	FORTRAN, Floating point intensive benchmark for ECAD
			applications (analog circuit simulation)
doduc		FORTRAN, Floating point intensive benchmark for ECAD
			applications (high-energy physics)[SIC].
nasker		FORTRAN, floating point intensive benchmark for scientific
			applications.
li		C, integer intensive benchmark for CASE
eqntott		C, integer intensive benchmark for ECAD
intmc		FORTRAN, integer intensive benchmark for scientific
			applications.
matrix300	FORTRAN, floating point intensive benchmark for wsceintific
			applications (Monte Carlo code that performs various matrix
			multiplications, including transposed Linpack routines).
fpppp		???
tomcatv		FORTRAN, floating point intensive benchmark for scientific
			applications.

These benchmarks are roughly 1/2 Fortran, 1/2 C, 1/2 integer and 1/2
floating point intensive.  Unfortunately, all of these benchmarks are
heavily CPU intensive and do very little to exercise the rest of a system.

The tape also contains a number of programs that are "Works in Progress."
These programs are candidates for future SPEC benchmark tapes.  They are as
follows:

Timberwolf
gbench		C, integer intensive benchmark for measuring X-terminal
			performance.
isajet		FORTRAN, floating point intensive (single prescision) benchmark
			for scientific applications (high energy physics
			applications).
mdljdp		FORTRAN, floating point intensive benchmark for scientific
			applications (particle simulation program--molecular dynamics).
wave		Maxwell's Equations?
spartan		Computational Chemistry

SPEC is actively soliciting for additional benchmarks.  There are several
requirements for the benchmarks:

1) They must be public domain or publicly available (e.g. GCC).
2) They should be highly portable.
3) They should run in a UNIX(tm) environment.
4) They should not use time() to seed the random number generator.
5) They should contain their own random number generator if they need one.
6) All calls to time() times() getrusage() et.al. calls should be removed.

SPEC is especially interested in benchmarks that exercise an entire system.
Areas that are under consideration include:
	disk I/O bandwidth
	network bandwidth
	DBMS performance in non-transaction environments (leave the
		transactions to TPC)
	Workstation support (i.e. how many workstations can a file/client
		server support?)

In all cases, it is highly desirable that the benchmarks be REAL
applications, not synthetic workloads.  It is okay if it is a crippled
version of a live application, but it should be exercising a real problem.

If you want more information about SPEC, contact Kim Shanley at Waterside
Associates, 39510 Paseo Padre Pkwy, Fremont, CA  94538, 415/792-2901
shanley@portal.com

All of the usual disclaimers apply.  I speak only for myself... etc.
-- 
Robert E. Novak                                     MIPS Computer Systems, Inc.
{ames,decwrl,pyramid}!mips!rnovak      928 E. Arques Ave.  Sunnyvale, CA  94086
rnovak@abbott.mips.COM (rnovak%mips.COM@ames.arc.nasa.gov)       +1 408 991-0402
--
Send compilers articles to compilers@ima.isc.com or, perhaps, Levine@YALE.EDU
Plausible paths are { decvax | harvard | yale | bbn}!ima
Please send responses to the originator of the message -- I cannot forward
mail accidentally sent back to compilers.  Meta-mail to ima!compilers-request

rnovak@mips.COM (Robert E. Novak) (06/22/89)

This may be a repost, my last posting seemed to drop into the bit-bucket.

During the week of June 5-9, the SPEC Bench-a-thon was held at MIPS
computers.  We had 7 vendors participating with 10 different architectures.
It was a grueling week for many of us.  In a single week we had to analyze 
24 different benchmarks and ensure that they would run on all of the
machines.  What we did not do was post results from running the benchmarks.

The participating vendors included:

Apollo				2 architectures
DEC					2 architectures and 2 Operating Systems
Hewlett-Packard		2 architectures
IBM
MIPS
Motorola
SUN

As a result of this meeting, the SPEC consortium produced an Alpha tape of
11 programs that could be run by all vendors.  This tape has been
distributed to all of the participants by now.  We are awaiting comments
from the participants to clean up any small flaws in the Alpha tapes and we
will generate a Beta release on June 30.  The Beta tape will undergo a 60
day evaluation period.  

This tape will be available to all companies that are members of 
the SPEC consortium.  The tape will be made publicly available at a to be
determined nominal cost at the end of the Beta period.

The tape contains the following benchmarks which have run on the machines
of the participants:

GCC 1.35	Integer intensive C compiler for CASE
espresso	C, Integer-intensive benchmark for ECAD
spice2g6	FORTRAN, Floating point intensive benchmark for ECAD
			applications (analog circuit simulation)
doduc		FORTRAN, Floating point intensive benchmark for ECAD
			applications (high-energy physics)[SIC].
nasker		FORTRAN, floating point intensive benchmark for scientific
			applications.
li			C, integer intensive benchmark for CASE
eqntott		C, integer intensive benchmark for ECAD
intmc		FORTRAN, integer intensive benchmark for scientific
			applications.
matrix300	FORTRAN, floating point intensive benchmark for wsceintific
			applications (Monte Carlo code that performs various matrix
			multiplications, including transposed Linpack routines).
fpppp		???
tomcatv		FORTRAN, floating point intensive benchmark for scientific
			applications.

These benchmarks are roughly 1/2 Fortran, 1/2 C, 1/2 integer and 1/2
floating point intensive.  Unfortunately, all of these benchmarks are
heavily CPU intensive and do very little to exercise the rest of a system.

The tape also contains a number of programs that are "Works in Progress."
These programs are candidates for future SPEC benchmark tapes.  They are as
follows:

Timberwolf
gbench		C, integer intensive benchmark for measuring X-terminal
			performance.
isajet		FORTRAN, floating point intensive (single prescision) benchmark
			for scientific applications (high energy physics
			applications).
mdljdp		FORTRAN, floating point intensive benchmark for scientific
			applications (particle simulation program--molecular dynamics).
wave		Maxwell's Equations?
spartan		Computational Chemistry

SPEC is actively soliciting for additional benchmarks.  There are several
requirements for the benchmarks:

1) They must be public domain or publicly available (e.g. GCC).
2) They should be highly portable.
3) They should run in a UNIX(tm) environment.
4) They should not use time() to seed the random number generator.
5) They should contain their own random number generator if they need one.
6) All calls to time() times() getrusage() et.al. calls should be removed.

SPEC is especially interested in benchmarks that exercise an entire system.
Areas that are under consideration include:
	disk I/O bandwidth
	network bandwidth
	DBMS performance in non-transaction environments (leave the
		transactions to TPC)
	Workstation support (i.e. how many workstations can a file/client
		server support?)

In all cases, it is highly desirable that the benchmarks be REAL
applications, not synthetic workloads.  It is okay if it is a crippled
version of a live application, but it should be exercising a real problem.

If you want more information about SPEC, contact Kim Shanley at Waterside
Associates, 39510 Paseo Padre Pkwy, Fremont, CA  94538, 415/792-2901

All of the usual disclaimers apply.  I speak only for myself... etc.
-- 
Robert E. Novak                                     MIPS Computer Systems, Inc.
{ames,decwrl,pyramid}!mips!rnovak      928 E. Arques Ave.  Sunnyvale, CA  94086
rnovak@abbott.mips.COM (rnovak%mips.COM@ames.arc.nasa.gov)       +1 408 991-0402

rnovak@mips.COM (Robert E. Novak) (06/22/89)

>Date: Wed, 21 Jun 89 13:38:18 PDT
>From: James A. Woods <jaw@eos.arc.nasa.gov>
>Subject: Re: SPEC Bench-a-Thon
>
>as a benchmark, what's wrong with unix 'compress',
>other than it messes up mipsco internal benchmark numbers?
>
>it's a nice cache-busting memory system benchmark, which is neat
>because it's not the instruction set which necessary dictates
>system performance these days...
>
>ames!jaw
>
>p.s. 'gnuchess' on some famous chess position would also make
>a fun integer bit-pushing measure to watch over the years
>(the metric being nodes/sec over five minutes or some such).

Actually, compress is on the list of candidate benchmarks.  The difficulty
that we have encountered is putting the proper wrapper around compress.  We
don't want to ship a huge file that will properly exercise compress.  What
we need is a program that will generate an 'interesting' file to compress
that will be the same on overy machine on which compress is run.



A note about Publicly Available.  Publicly Available does not imply Public
Domain.  It simply means that the program can be distributed with
copyrights in place that do not necessarily restrict further distribution
of the program.  It will even be fine with SPEC if the copyright restricts
the usage to performance testing.
-- 
Robert E. Novak                                     MIPS Computer Systems, Inc.
{ames,decwrl,pyramid}!mips!rnovak      928 E. Arques Ave.  Sunnyvale, CA  94086
rnovak@abbott.mips.COM (rnovak%mips.COM@ames.arc.nasa.gov)       +1 408 991-0402

elgie@canisius.UUCP (Bill Elgie) (06/22/89)

>From: James A. Woods <jaw@eos.arc.nasa.gov>
>Subject: Re: SPEC Bench-a-Thon
>
>as a benchmark, what's wrong with unix 'compress',
>other than it messes up mipsco internal benchmark numbers?
>
 We used compress as one of many tests between a MIPS system and several
 others.  It didn't "mess up" the MIPS in any way.  

  greg pavlov (under borrowed account), fstrf, amherst, ny 
                            
                            
                            
                            

jaw@eos.UUCP (James A. Woods) (06/23/89)

>>
>>as a benchmark, what's wrong with unix 'compress',
>>other than it messes up mipsco internal benchmark numbers?
>>
>  We used compress as one of many tests between a MIPS system and several
>  others.  It didn't "mess up" the MIPS in any way.  
> 
was referring here to the bit about how this integer benchmark,
since it scribbles semi-randomly over a sparse array bigger
than the cache, runs at about nine-ten vax mips on the m3000 system,
which is billed at twenty.

a mipsco employee informed me that the problematic memory write latency
exercised by 'compress' "dragged down their numbers for the r3000 chip"
during simulations.

other risc implementations, of course, may yield similar results.

trebor@biar.UUCP (Robert J Woodhead) (06/23/89)

In article <22033@abbott.mips.COM> rnovak@mips.COM (Robert E. Novak) writes:
>Actually, compress is on the list of candidate benchmarks.  The difficulty
>that we have encountered is putting the proper wrapper around compress.  We
>don't want to ship a huge file that will properly exercise compress.

	Well, if you want to get elegant, have compress compress it's
	own source code, then compress the compressed file, and repeat
	the process N times...  ;^)

-- 
(^;-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-;^)
Robert J Woodhead, Biar Games, Inc.   !uunet!biar!trebor | trebor@biar.UUCP
  ``I can read your mind - right now, you're thinking I'm full of it...''

trebor@biar.UUCP (Robert J Woodhead) (06/23/89)

In article <670@biar.UUCP> trebor@biar.UUCP (Yours Truly) writes:
>	Well, if you want to get elegant, have compress compress it's
>	own source code, then compress the compressed file, and repeat
>	the process N times...  ;^)

	Idea #2 - make the benchmark not compression but uncompression.
	Send out the whole benchmark suite compressed N times.  The
	first benchmark is uncompressing the benchmark suite!
-- 
(^;-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-;^)
Robert J Woodhead, Biar Games, Inc.   !uunet!biar!trebor | trebor@biar.UUCP
  ``I can read your mind - right now, you're thinking I'm full of it...''

jps@cat.cmu.edu (James Salsman) (06/23/89)

In article <671@biar.UUCP> trebor@biar.UUCP (Robert J Woodhead) writes:
> In article <670@biar.UUCP> trebor@biar.UUCP (Yours Truly) writes:
> >	Well, if you want to get elegant, have compress compress it's
> >	own source code, then compress the compressed file, and repeat
> >	the process N times...  ;^)
> 
> 	Idea #2 - make the benchmark not compression but uncompression.
> 	Send out the whole benchmark suite compressed N times.  The
> 	first benchmark is uncompressing the benchmark suite!

This is silly.

The composition of any compression algorithm with its self
can not do better than a single application of the algorithm.

Otherwise, things would be a lot smaller.

If you want to use compress as a benchmark, use it on a
large file that is similar (if not the same) everywhere.
Take the first 10k of /etc/hosts, for instance.  The site
differences really won't matter -- time compress on
different disjoint equal-size sections of any large
homogeneous file and see for yourself.

:James
-- 

:James P. Salsman (jps@CAT.CMU.EDU)
-- 

peter@ficc.uu.net (Peter da Silva) (06/23/89)

I think one thing that would make a nice benchmark is running a few
generations of LIFE from a known interesting starting position, such as the
R pentomino. It would probably want to be a naive implementation of the
algorithm, rather than something that does bounds checking to limit the
computation or stores the cells in some exotic data structure.

Of course if you allowed moderately exotic implementations it'd make a
nice benchmark for array processors. A couple of years back I suggested
this and people mailed me some awesome results from things like the
Connection Machine...
-- 
Peter da Silva, Xenix Support, Ferranti International Controls Corporation.

Business: uunet.uu.net!ficc!peter, peter@ficc.uu.net, +1 713 274 5180.
Personal: ...!texbell!sugar!peter, peter@sugar.hackercorp.com.

les@unicads.UUCP (Les Milash) (06/23/89)

In article <5297@pt.cs.cmu.edu> jps@cat.cmu.edu (James Salsman) writes:
>The composition of any compression algorithm with its self
                    ^^^
>can not do better than a single application of the algorithm.

you mean "any good", right? :-)


how bout generating a file with pseudorandom numbers.  
you could take your chances with srand, or deliver your own.
that's so obvious it must be wrong, or unacceptably risky, huh?

trebor@biar.UUCP (Robert J Woodhead) (06/24/89)

In article <5297@pt.cs.cmu.edu> jps@cat.cmu.edu (James Salsman) writes:
>The composition of any compression algorithm with its self
>can not do better than a single application of the algorithm.

	Well, first of all, in certain circumstances this
	is incorrect.  Every so often, recompressing the
	compressed file gains you a couple of percent.

	This was, however, not the point.  I suggested doing
	the compression N times merely to increase the amount
	of computation required, so one could get a better
	timing.  I thought this was obvious, and the basic
	idea (of getting an uncompress program and a file
	you have to pipe through it 100 times in order to
	get the other benchmarks) appealed to me.

>Take the first 10k of /etc/hosts, for instance.  The site
>differences really won't matter -- time compress on
>different disjoint equal-size sections of any large
>homogeneous file and see for yourself.

	You wouldn't want to do this.  Some companies would
	spend thousands of man-hours reorganising their
	/etc/hosts files for maximum compression performance.

	;^)

-- 
(^;-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-;^)
Robert J Woodhead, Biar Games, Inc.   !uunet!biar!trebor | trebor@biar.UUCP
  ``I can read your mind - right now, you're thinking I'm full of it...''

mash@mips.COM (John Mashey) (06/25/89)

In article <4042@eos.UUCP> jaw@eos.UUCP (James A. Woods) writes:
>>>as a benchmark, what's wrong with unix 'compress',
>>>other than it messes up mipsco internal benchmark numbers?
>>>
>>  We used compress as one of many tests between a MIPS system and several
>>  others.  It didn't "mess up" the MIPS in any way.  
>> 
>was referring here to the bit about how this integer benchmark,
>since it scribbles semi-randomly over a sparse array bigger
>than the cache, runs at about nine-ten vax mips on the m3000 system,
>which is billed at twenty.
>
>a mipsco employee informed me that the problematic memory write latency
>exercised by 'compress' "dragged down their numbers for the r3000 chip"
>during simulations.
>
>other risc implementations, of course, may yield similar results.

1) MIPS uses compress as one of its internal benchmarks.
2) It does indeed drag down the average; that's why we included it,
because of its nasty effect on caches.
3) It's on the "proposed" list for SPEC [by me], but I just ran out
of time to get it packaged, and certain characteristics now make
it look a little more appropriate for evaluation in the second round.
These characteristics are:
	the first round of the SPEC benchmarks are essentially CPU-bound
	programs, that is, on any machine big enough not to have
	them cause noticable paging, the elapsed time approx= user-cpu+
	system-cpu, which means it's easy to agree on how to measure
	and report it. (i.e., elapsed time).

compress, to be used for SPEC, had to be made to run much longer than
the case MIPS uses internally.  In doing so, it now turns into a benchmark
that: a) either does serious I/O, or b) has elapsed time that might be
substantially longer than CPU time.   We're going to be working on
a coherent policy of handling such things, but we'd like to get
the first tape out the door first.
-- 
-john mashey	DISCLAIMER: <generic disclaimer, I speak for me only, etc>
UUCP: 	{ames,decwrl,prls,pyramid}!mips!mash  OR  mash@mips.com
DDD:  	408-991-0253 or 408-720-1700, x253
USPS: 	MIPS Computer Systems, 930 E. Arques, Sunnyvale, CA 94086

bander@cs.odu.edu (James Bander) (06/25/89)

In article <670@biar.UUCP> trebor@biar.UUCP (Robert J Woodhead) writes:
>
>	Well, if you want to get elegant, have compress compress it's
>	own source code, then compress the compressed file, and repeat
>	the process N times...  ;^)

That does sound elegant; unfortunately it would not be a fair test.  File
compression is based on information theory; files are compressed by reducing
redundancy.  The reason that file compression programs work is that English 
text, programs, data files, and so forth, contain plenty of redundant
information.  Languages invented by human beings work that way.

The second time a file is compressed, there is less redundancy, so
the compression program does less work.

In fact, the Unix compress program uses an adaptive Lempel-Ziv file compression
algorithm, based on a sliding dictionary.  Lempel and Ziv proved that for very
large files their algorithm is optimal--it eliminates all redundancy.  Even
for smaller files the Lempel-Ziv algorithm does no compression the second time. 

Try it--take a file, run it through compress, rename the file so it doesn't 
have the .Z extension, and run it through compress again.  Unix's compress
utility refuses to generate output on the second pass.  (Even if it did 
generate output that was the same size as the input, the program would have
done less computation on the compressed file than on a random- or human-
generated file.)


References:  James A. Storer, Data Compression: Methods and Theory.
                   Computer Science Press 1988
             J. Ziv & A. Lempel, "A Universal Algorithm for Sequential Data
                   Compression", IEEE Transactions Info. Theory 23:3
                   (1977), 337-343.
             J. Ziv & A. Lempel, "Compression of Individual Sequences Via
                   Variable-Rate Coding", IEEE Trans. Info. Theory 24:5
                   (1978), 530-536.


-----------------------------------------------------------------------------
Jim Bander, Tidewater Consultants, Inc., Virginia Beach, VA
    
                                          Internet:  bander@cs.odu.edu
                                          Ma Bell:   (804) 497-8951
                                          USnail:    160 Newtown Rd. #415
                                                     Va. Beach, VA 23462

"The presence of humans, in a system containing high-speed electronic
computers and high-speed, accurate communications, is quite inhibiting." 
                                                 --Stuart L. Seaton, 1958
----------------------------------------------------------------------------
Disclaimer:  All my mistaken opinions are my own.

bukys@cs.rochester.edu (Liudvikas Bukys) (06/26/89)

In article <22197@winchester.mips.COM> mash@mips.COM (John Mashey) writes:
>compress, to be used for SPEC, had to be made to run much longer than
>the case MIPS uses internally.  In doing so, it now turns into a benchmark
>that: a) either does serious I/O, or b) has elapsed time that might be
>substantially longer than CPU time.   We're going to be working on
>a coherent policy of handling such things, but we'd like to get
>the first tape out the door first.

Perhaps it would be acceptable to replace all the input with a pseudo-random
stream (it's easy to find all the occurences of "stdin" and "getchar").

Output checking is harder; does the SPEC suite check the answers from
the other benchmarks?  If not, then the new benchmark-compress could
just throw away the output; otherwise, maybe just checksumming the
output would do as a first-order correctness check.

les@unicads.UUCP (Les Milash) (06/26/89)

In article <4714@ficc.uu.net> peter@ficc.uu.net (Peter da Silva) writes:
>[awesome "life" performance on the Connection Machine...
i'd love to see those numbers...

running cellular automata on a machine designed by a guy who asks
"what would computer science be if you had a mole of processors"
is starting to be like playing God, i think.

johng@cavs.syd.dwt.oz (John Gardner) (06/28/89)

In article <5297@pt.cs.cmu.edu> jps@cat.cmu.edu (James Salsman) writes:
>
>The composition of any compression algorithm with its self
>can not do better than a single application of the algorithm.
>
Not so.  I have seen repeated compression succeed in further compressing files
Although any algorithim that this is true for should of course be expanded to
say that it repeatably applys itself until it gets no further compression.


/*****************************************************************************/
Division Of Wool Technology - Where the programmers have something better to do
than think up funny lines.
PHONE          : (02) 436 3438
ACSnet         : johng@cavs.dwt.oz
JANET          : johng%cavs.dwt.oz@uk.ac.ukc
ARPA           : johng%cavs.dwt.oz@uunet.UU.NET
UUCP           : ..!uunet!munnari!cavs.dwt.oz.au!johng              
What's a disclaimer ?
/*****************************************************************************/

rnovak@mips.COM (Robert E. Novak) (06/28/89)

In article <1989Jun26.130517.29616@cs.rochester.edu> bukys@cs.rochester.edu (Liudvikas Bukys) writes:
>In article <22197@winchester.mips.COM> mash@mips.COM (John Mashey) writes:
>>compress, to be used for SPEC, had to be made to run much longer than
>>the case MIPS uses internally.  In doing so, it now turns into a benchmark
>>that: a) either does serious I/O, or b) has elapsed time that might be
>>substantially longer than CPU time.   We're going to be working on
>>a coherent policy of handling such things, but we'd like to get
>>the first tape out the door first.
>
>Perhaps it would be acceptable to replace all the input with a pseudo-random
>stream (it's easy to find all the occurences of "stdin" and "getchar").
>
>Output checking is harder; does the SPEC suite check the answers from
>the other benchmarks?  If not, then the new benchmark-compress could
>just throw away the output; otherwise, maybe just checksumming the
>output would do as a first-order correctness check.

The SPEC benchmarks check their results against known good results.  This
was mandatory to guarantee that every machine was computing the same
algorithm.
-- 
Robert E. Novak                                     MIPS Computer Systems, Inc.
{ames,decwrl,pyramid}!mips!rnovak      928 E. Arques Ave.  Sunnyvale, CA  94086
rnovak@abbott.mips.COM (rnovak%mips.COM@ames.arc.nasa.gov)       +1 408 991-0402

khb%chiba@Sun.COM (Keith Bierman - SPD Languages Marketing -- MTS) (06/28/89)

In article <4714@ficc.uu.net> peter@ficc.uu.net (Peter da Silva) writes:
>I think one thing that would make a nice benchmark is running a few
>generations of LIFE from a known interesting starting position, such as the

Why would this be an interesting benchmark ? The goal of any sensible
benchmark suite is to select codes which closely resemble real work
... SPEC has taken the approach that real applications come as close
as one can get... (which is sensible. I do think that the criteria for
accepting a benchmark should include a "performance functional" based
on what the new benchmark adds to the suite (it is currently FP heavy
.. so new FP codes should be discouraged ... otherwise any metric
based on a simple amalgam (mean, harmonic, geometric, total time, etc.)
will not reflect "common" usage ... if we can agree on what a "proper
workload is". In addition, the current rules ensure that identical
code is run on all machines. As a application programmer, I'd rather
call the local sort than expecting my portable version to be optimal
(and if I do have a portable one, do I use a recursive solution ? This
may adversely impact some otherwise fine machines). But the SPEC
effort is a good one, and like all right thinking people (:>) I support it.

If you really run life on a daily basis (for anything other than a
screen saver) then perhaps it does belong in. Please send it, along
with a description of what use the output is, and perhaps we will
propose it to SPEC).

Cheers.
Keith H. Bierman      |*My thoughts are my own. Only my work belongs to Sun*
It's Not My Fault     |	Marketing Technical Specialist    ! kbierman@sun.com
I Voted for Bill &    |   Languages and Performance Tools. 
Opus  (* strange as it may seem, I do more engineering now     *)

eugene@eos.UUCP (Eugene Miya) (06/28/89)

Point of information:
The code nasker was developed here at NASA Ames.  There are very strict
rules for the execution of this code among which 64-bit precision
is required [it was developed for 64-bit vector processors,
I wonder which.... 8)].

Anyway, simplying executing the code is insufficient to obtain a
measurement.  I too was at the SPEC announcement, and the emphasis on
32-bit was announced.  32-bit does not cut it for running this code,
the error bounds make it way too poor.  Modification of the just to
get it running is also limited by bounds on the numbers of modifiable
lines of code [typically for performance, not getting it to accurately].

In other words IMPLICIT REAL*8 won't cut it.  There is also 64-bit
precision COMPLEX as well.  The "rules" are in the text of the TR (TeX)
by Barton and Bailey which can also be obtained with the code
at either nbslib or netlib.

You can tell a person's perspective by asking the number of bits in a
"single-precision" word.

Another gross generalization from

--eugene miya, NASA Ames Research Center, eugene@aurora.arc.nasa.gov
  resident cynic at the Rock of Ages Home for Retired Hackers:
  "You trust the `reply' command with all those different mailers out there?"
  "If my mail does not reach you, please accept my apology."
  {ncar,decwrl,hplabs,uunet}!ames!eugene
  				Live free or die.

ram@shukra.Sun.COM (Renu Raman) (06/29/89)

In article <112809@sun.Eng.Sun.COM> khb@sun.UUCP (Keith Bierman - SPD Languages Marketing -- MTS) writes:
>If you really run life on a daily basis (for anything other than a
>screen saver) then perhaps it does belong in. Please send it, along
>with a description of what use the output is, and perhaps we will
>propose it to SPEC).
>
>Cheers.
>Keith H. Bierman      |*My thoughts are my own. Only my work belongs to Sun*

   How about perl?  It is Public Domain and runs on Vaxens, Suns (all kinds),
   386 machines and a few other assorted variety.  If you haven't used
   perl, I would be happy to send a brief description.

   Since it is an interpretive language, with much resemblance to C, sed, awk
   and sh, it would be a good test of success/failure of compiler optimzations.

   renu raman

ram@shukra.Sun.COM (Renu Raman) (06/29/89)

In article <113125@sun.Eng.Sun.COM> ram@sun.UUCP (Renu Raman) writes:

  Oooopsss! Sorry, hit the 'F' key instead of 'R'.

  renu raman

peter@ficc.uu.net (Peter da Silva) (06/29/89)

In article <112809@sun.Eng.Sun.COM>, khb%chiba@Sun.COM (Keith Bierman - SPD Languages Marketing -- MTS) writes:
> In article <4714@ficc.uu.net> peter@ficc.uu.net (Peter da Silva) writes:
> >I think one thing that would make a nice benchmark is running a few
> >generations of LIFE from a known interesting starting position, such as the

> Why would this be an interesting benchmark ?

It's simple to implement in different architectures and languages, and it
is a reasonably good test of massively parallel processing. For example,
with a connection machine running 64k processors, you could keep all
of them busy with a 256 by 256 LIFE matrix. I suppose you could get the
same result from a number of runs of some convolution on an image, but it's
not as much fun...

> The goal of any sensible
> benchmark suite is to select codes which closely resemble real work

For certain environments, that resembles real work.
-- 
Peter da Silva, Xenix Support, Ferranti International Controls Corporation.

Business: uunet.uu.net!ficc!peter, peter@ficc.uu.net, +1 713 274 5180.
Personal: ...!texbell!sugar!peter, peter@sugar.hackercorp.com.

khb%chiba@Sun.COM (chiba) (06/30/89)

In article <4828@ficc.uu.net> peter@ficc.uu.net (Peter da Silva) writes:
...life as a benchmark ...>
>> Why would this be an interesting benchmark ?
>
>It's simple to implement in different architectures and languages, and it
>is a reasonably good test of massively parallel processing. For example,
>with a connection machine running 64k processors, you could keep all
>of them busy with a 256 by 256 LIFE matrix.

Which seems very artifical to me. I have played with hypercubes and
such, and while they hold great promise, very few application codes START
OUT being massively parallel (Geof. Fox and friends to the contrary).
If someone wants to submit a lattice gague code as a benchmark that
would represent sensible work. Life is tiny bit twiddeling in
parallel, and that is very rare ... so it should not IMHO be part of SPEC.

>> The goal of any sensible
>> benchmark suite is to select codes which closely resemble real work
>
>For certain environments, that resembles real work.

Could you be more specific ?



Keith H. Bierman      |*My thoughts are my own. Only my work belongs to Sun*
It's Not My Fault     |	Marketing Technical Specialist    ! kbierman@sun.com
I Voted for Bill &    |   Languages and Performance Tools. 
Opus  (* strange as it may seem, I do more engineering now     *)