grenley@nsc.UUCP (06/11/87)
Oh, boy, it looks like my benchmarking proposal has generated quite a bit of heat, and more importantly, light as well. My comments on the ongoing discussion: In article <20189@sun.uucp> klein@sun.UUCP (Mike Klein) writes: >In article <4667@fritz.UUCP> martin@felix.UUCP (Martin McKendry) writes: >>Gentlemen: >>Any bake-off will have one winner and many losers. As an engineer ^^^^^^^^^^^^^^^. Not true! benchmarks can (and should) be multi-dimensional. >>working for a particular vendor, you may have run tests that >>convince you that you will be a winner. But be assured that in >>some dimension you will be a loser. And if you proposed that >>your company enter, you will be the one "responsible" for the >>exposure of "weakness". I agree with you that some people, particularly bad marketing people, will see it this way. On the other hand, computer systems are sold on many points besides raw (or 'cooked' :-)) CPU power. Service, compatibility, and not taking risks, for instance. Look at IBM. My purpose in proposing the benchmark is simply to insure, by having each vendor monitor the other's work, that the benchmark was run strictly according to the rules of the benchmark; in other words, so the results will be truly comparable. I do not claim that the results will be meaningful - personally I think MIP ratings are among the least important parameters in selecting a computer. To a certain extent the vendors are already self-policing. Witness Motorola calling Intel to task for running crooked whetstones. (much good analysis deleted in the interests of brevity) > Submitting a product to an open benchmarking >when it is known in advance that the product will not be the "best" is >poor marketing. Submitting it to an open benchmarking when it is not known >in advance how the product will perform is also poor marketing. I disagree. If you know your product and your customers/market, you are aware of ALL of their purchasing criteria, not the least of which is cost. Cost/Performance is usually more important than performance alone, and software compatibility is the most important of all. > But I don't expect to >see the AMD booth or the National booth running the same benchmarks, and >I understand the reasons why. Then again Mike you just might... Regards, George Grenley