dipto@umbc3.UMD.EDU (Dipto Chakravarty) (03/12/88)
Benchmarking programs needed
We are in need of benchmarking programs for various types of workstations
that are currently under evaluation + study. I need to get some enlightenment
in this topic of benchmarking. Of whatever little I know of benchmarks, it
appears that there are a variety of public domain benchmarks that are
available.
I will try to give some outline of what we need, hereunder.
(1) A floating-point based bench mark is required for a Microvax II running
VMS 4.7. We want to test it against a 11/785 and a 8600 machine. It will
be nice if I could get the src code or the .EXE file(s) for Application
oriented, Disc access oriented and CPU intensive benchmarks.
I have heard that in addition to public domain benchmarking programs,
there are source codes available, such as 'Wet-stone' (spelling?), for
evaluating floating point based applications. Can someone kindly e-mail
me the source code(s) for such standard routines?
(2) A floating point based and disc access based benchmark for a Sun station
running 4.2 BSD Unix and 2.1 DEC Ultrix will also be needed.
Readers, I do understand that my specifications mentioned above, could be
little off or vague. But, bare it with me; this is my first experience with
benchmarks. Thanks to all for taking the time to read this article.
Kindly send me e-mail directly, as I am not a regular reader of all the groups
where this is getting posted.
--
BITNET : dipto@umbc2 ------\
ARPANET: dipto@umbc3.UMD.EDU -------> In-real-life: Dipto Chakravarty
USMAIL : CMSC, UMBC,Md 21228 ------/carl@CITHEX.CALTECH.EDU (Carl J Lydick) (03/14/88)
> Readers, I do understand that my specifications mentioned above, could be > little off or vague. But, bare it with me; this is my first experience with > benchmarks. Thanks to all for taking the time to read this article. I'm afraid I can't help you by sending benchmark programs or references to them, but I can, perhaps, give you some useful advice regarding benchmarking. Any benchmarking program makes assumptions about the mix of instructions to be used on a machine, about the level of optimization users of that machine are likely to make, etc. It's my experience that these assumptions are ALWAYS wrong, at least to some extent, and generally to a major extent. For example, in benchmark floating-point tests, a 780 with 4 MB of memory will generally beat a 750 with 8 MB of memory. Now, put 4 physicists doing monte-carlo simulations on each of the machines. Since it's against the principles of a physicist to know anything about using a computer efficiently (I've been managing VAXen used by physicists for several years, so I speak from experience), each of these users will be doing calculations involving data scattered over at least 500 pages of memory, with the code scattered over an additional 500 pages (even if the total length of the code involved is about 1Kbyte and all of the data being used within a 5-minute period is only about 20Kbytes). This makes 1MB of memory being used by each of the 4 users. VMS takes up another 2 MB. On the 780, this means we're exceeding physical memory by 50%, and the computer spends up to 40% of its time paging. On the 750, we've still got 2 MB left over, so there's no page faulting going on. The result is that the 750 runs significantly faster than the 780. The moral of this story is, DON'T TRUST STANDARD BENCHMARKS. Write a suite of programs that simulates your actual load, and use them for benchmarking.
gkn@SDS.SDSC.EDU (Gerard K. Newman) (03/16/88)
From: carl@CitHex.Caltech.Edu (Carl J Lydick) Subject: Re: Need benchmarking wisdom Date: Mon, 14 Mar 88 03:28:27 PST ... Now, put 4 physicists doing monte-carlo simulations on each of the machines. Since it's against the principles of a physicist to know anything about using a computer efficiently (I've been managing VAXen used by physicists for several years, so I speak from experience) ... They don't do much better at designing networks, either. gkn ---------------------------------------- Internet: GKN@SDS.SDSC.EDU Bitnet: GKN@SDSC Span: SDSC::GKN (27.1) USPS: Gerard K. Newman San Diego Supercomputer Center P.O. Box 85608 San Diego, CA 92138-5608 AT&T: 619.534.5076