pb@idca.tds.PHILIPS.nl (Peter Brouwer) (09/18/89)
In the discussion on which benchmarks to use as a reaction on a comparison done on machines by a unix magazin , neal nelson was mentioned as a benchmark to use. I have two points of comments on this bench mark. Some month back we used neal nelson benchmarks to compare machines. When i looked at the sources I saw that the time of the tests are measured with the system call time, so resolution is in seconds. This makes the fast tests on fast machines inacurate. I test that runs in 10 sec has an accuracy of 10%. This has to be thought of when looking at machine comparisons. An other remark I have is that all runs ( for n copies of the test ) run paralel. ( For those who are not familiar with neal neson bench mark, a number of runs can be executed paralel, each run consists of a series of tests that are executed in a fixed sequence. ) This means that the code of that test will fit in the system cache. The cache hit ratio will be high. This is not an realistic situation on a multi user machine. It would be better to make the order of the tests random for each or a number of copies of a run. The musbus benchmark does something like this. It created a number of execution scripts from a master script. Each script executes the test from the master script in a different order. -- Peter Brouwer, # Philips Telecommunications and Data Systems, NET : pb@idca.tds.philips.nl # Department SSP-P9000 Building V2, UUCP : ....!mcvax!philapd!pb # P.O.Box 245, 7300AE Apeldoorn, The Netherlands. PHONE:ext [+31] [0]55 432523, # Never underestimate the power of human stupidity