eugene@wilbur.nas.nasa.gov (Eugene N. Miya) (07/04/90)
> David Tayler notes and promotes SPEC from gcc discussion.
I video taped the ISCA'90 workshop on benchmarking in May
(a little over 6 hours of workshop). I specifically asked that
representatives from various efforts get a chance to speak, in
particular SPEC. The nearly unanimious opinion (less the SPEC
rep I was involved in inviting) was that SPEC was less than adequate
for supercomputers.
I only note this as a set of opinions which differs from Dave.
The workshop was organized by John Roberts (NIST), David Bailey, and
myself and included Frank McMahon, Ann Hayes, Dan Reed, and a few other
benchmarking people. I only wish the group as a whole had not come
down so hard on SPEC. Also PERFECT was not brought out and "bashed"
more.
P.S. attendees reading this expecting tapes (yes it takes a while
to get copies made of clusters of 4 (2 HR) tapes.
--e. nobuo miya, NASA Ames Research Center, eugene@orville.nas.nasa.gov
{uunet,mailrus,other gateways}!ames!eugeneddt@convex.COM (David Taylor) (07/06/90)
Did that event cover the SPECthruput or just the SPECmark? The SPECthruput is
the latest addition to the SPEC benchmark suite, and is tailored specifically
for multi-headed, larger machines. The test is as follows:
If N is the number of CPUS you have in your system, then you run 2 copies
of the same benchmark on each head, i.e. if N=4, you run 8 copies of a single
benchmark simultaneously on the system. You record the time it takes from
the beginning of the first program to the end of the last program.
You repeat this process for each of the 10 benchmarks, take the geometric
mean of the runtimes, and generate the SPECthruput by taking the ratio of
the VAX 11/780's geometric mean over the system under test's (SUT) geometric
mean. However, looking at the results of the individual benchmarks really
says much more about the system because the benchmarks do cover a fairly
large range of applications.
So far, they've published results for the SPECthruput mainly on faster
minis and workstations, not many mainframes or (any?) supers. But, I don't
see why it couldn't work out.
=-ddt->
--
David D. Taylor, Esq, Performance Measurement Intern, Convex. (whew!)
(214) 497-4860, ddt@convex.com or ddt@vondrake.cc.utexas.edu
Remember, flatulation is only natural.