rws@EXPO.LCS.MIT.EDU (Bob Scheifler) (06/14/90)
My people have now had a good look at the difference between the standard
xstone calculation and the new Scheifler one.
His script uses a weighted arithmetic mean to do the calculations.
The original uses a weighted HARMONIC mean. Since (if my statisticians
are correct) harmonic means are better for figures which have "funny"
spreads, I would contend that the original is a better way of calculating
things (having said that, the script has lots of redundant code in it
and evidently evolved).
Having looked this over again, people should ignore the awk script that I
posted. It did indeed use an arithmetic mean, and a harmonic mean is probably
more useful in this context.
Having said that, I still believe the xstone number is relatively useless.
Your xstone rating is limited by the smallest weighting used. For example,
if arc speed were weighted at 10%, and you made everything else infinitely
fast, you would only improve the xstone rating by a factor of 10. This is
a meaningful result if your application really does use arcs 10% of the
time, but it is rather uninteresting if your application never draws arcs.
(For example, if your application only draws text, your performance improves
linearly with text performance.) It isn't obvious to me that there are
many real applications matching the particular set of weights used to compute
the xstone rating, so drawing conclusions about real applications from this
rating is suspect. As I said before, I think the raw data produced by xbench
is much more interesting, and can be applied to specific applications to
estimate performance (although I think x11perf produces a more comprehensive
set of raw data that can be used for even better estimates).