aglew@ccvaxa.UUCP (12/20/87)
John Mashey mentioned "understanding DEC's performance methodology" in one of his recent postings. Can anyone explain DEC's performance methodology to me, or give me references? I'm still new at this sort of thing. Andy "Krazy" Glew. Gould CSD-Urbana. 1101 E. University, Urbana, IL 61801 aglew@mycroft.gould.com ihnp4!uiucdcs!ccvaxa!aglew aglew@gswd-vms.arpa My opinions are my own, and are not the opinions of my employer, or any other organisation. I indicate my company only so that the reader may account for any possible bias I may have towards our products.
aglew@ccvaxa.UUCP (12/24/87)
>If you are testing the performance of a specific system configuration (CPU, >memory, disk, OS, etc, etc) fine, do so. If you are addressing the >performance of the CPU/FPU/cache (which seems to be what is discussed in this >group), don't use those benchmarks or at least try to factor out the >peripheral/OS/whatever differences. > > Randell Jesup Lunge Software Development I very much disagree - I/O system architecture should be just as much the topic of the comp.arch newsgroup as CPU and memory system architecture. The problem, of course, is that there are more varieties of I/O systems. Which is one of the best reasons for discussing them. I do not mean to compare things like the Berkeley Fast File System with a CP/M disk layout. But, incidents like the report that BSD filesystem performance can actually go down with a disk cache are appropriate (smart SW/smart HW don't always mix). As is anything that talks about faster I/O, since I/O is the bottleneck on too many systems. Andy "Krazy" Glew. Gould CSD-Urbana. 1101 E. University, Urbana, IL 61801 aglew@mycroft.gould.com ihnp4!uiucdcs!ccvaxa!aglew aglew@gswd-vms.arpa My opinions are my own, and are not the opinions of my employer, or any other organisation. I indicate my company only so that the reader may account for any possible bias I may have towards our products.
hansen@mips.UUCP (Craig Hansen) (01/06/88)
In article <28200078@ccvaxa>, aglew@ccvaxa.UUCP writes: > John Mashey mentioned "understanding DEC's performance methodology" > in one of his recent postings. Can anyone explain DEC's performance > methodology to me, or give me references? I'm still new at this > sort of thing. A good reference is "VAXstation 3200 Performance Analysis" from DEC "Worksystems Marketing", Dated Sept. 9, 1987. I think that what John meant by their performance methodology is their emphasis on not stating a single performance number. For example, they quote MicroVAX-II relative performance for the Sun-4/260 and the VS3200 as: PRODUCT VENDOR PERFORMANCE vs VSII BENCHMARK MIPS CLAIM MEAN RANGE SAMPLE SIZE Sun-4/260 10 4.56 1.8-13.3 38 Digital VS3200 - 3.11 2.2-4.3 61 DEC's methodology is not foolproof, but it's harder to lie with their statistics. If I were to criticize their methodology, it would be that they rely too much on small benchmarks, and not enough on "real" programs, but that's an industry-wide problem, and DEC is not the worst offender here. -- Craig Hansen Manager, Architecture Development MIPS Computer Systems, Inc. ...{ames,decwrl,prls}!mips!hansen or hansen@mips.com