mash@mips.UUCP (John Mashey) (05/21/87)
In article <6024@steinmetz.steinmetz.UUCP> davidsen@kbsvax.steinmetz.UUCP (William E. Davidsen Jr) writes: > >As the developer of a benchmark suite of my own I would love to cast >bricks at the Nelson suite. In truth it's a pretty good set of benchmarks, >and has been run on hundreds of configurations. I agree that it would be >a suitable measure of machines. Is there anybody out there who: a) Knows enough of what's going on inside the Nelson suite that can explain it, and b) Is legally allowed to? I have several reasons for asking: a) Many people have referenced it, but I haven't seen a publicly-available description of how it works. (If I've just missed it, please point me at it). b) A while back, an (unnamed) prospect wished to get Nelson numbers on our machine, which I assisted in. There were several oddities: 1) From the outside, the test mostly seemed to measure raw disk performance, and the general performance statistics (as seen by vmstat) didn't look very much like what I've seen real systems run like. [I can't remember why; I just remember they looked odd. Can anybody else comment? This was on a very early version of the OS, so it may have been strange.] In particular, it seemed like there was a higher percentage of idle time, even with many scripts running, than one would expect. 2) Although I don't still have a copy of the benchmark source around, I do remember in the process of getting it working to have seen some "arithmetic" test that looked like: a = b + c; d = e - f; g = h * i; j = k / l; that purported to measure arithmetic speed of the processor. I have no idea how that counted in the tests, or how it was actually used, but seeing something like that instantly raises my skepticism level rather high, since it has no attempt whatsoever to act like real code. Anyway, this is not an attempt to throw rocks at that benchmark, but just a wish that somebody would tell us what it's supposed to measure, and generally, how it does it, so we can assess whether or not it means anything. -- -john mashey DISCLAIMER: <generic disclaimer, I speak for me only, etc> UUCP: {decvax,ucbvax,ihnp4}!decwrl!mips!mash, DDD: 408-720-1700, x253 USPS: MIPS Computer Systems, 930 E. Arques, Sunnyvale, CA 94086