georgeb@fai.UUCP (George Bosworth) (06/25/91)
In an article in the ASPLOS-IV Proceedings (full citation later),
researchers from U. of Washington published a side-by-side
performance comparison of Mach 2.5 and Mach 3.0.
They averaged three runs of seven diverse applications running on a
MIPS R3000-based DECstation 5000/200 with 24 MB of memory. I condense
their tabulation of results by adding the applications together for
a few of their measured items:
Measured Item Mach 2.5 Mach 3.0
========================== ======== ========
Elapsed time (seconds) 307 416
Address space context switches 10,740 215,811
Kernel-level thread context 18,255 248,274
switches
System calls 90,515 278,771
Kernel mode TLB misses 439,067 3,867,602
Here are two questions:
1. The authors argue that RISC hardware and microkernel OS designs
are out of kilter, and deliberately chose a RISC machine for
the Mach comparisons. Are there similar timing runs for Mach 2.5
and Mach 3.0 on CISC machines?
2. Are there any reasons to believe that more current versions of
Mach 3.0 have improved in performance? If so, are there published
performance runs?
Full citation: Thomas E. Anderson, Henry M. Levy, Brian N. Bershad,
and Edward D Lazowska, "The Interaction of Architecture and Operating
System Design", in Proceedings of the Fourth International Conference
on Architectural Support for Programming Languages and Operating
Systems, April 1991, pages 108-120. The proceedings are available
from ACM by order number 556910.
George
George H. Bosworth georgeb@fai.com or
Fujitsu America B-2-7 uunet!fai.com!georgeb
3055 Orchard Drive
San Jose CA 95134 1-408/432-1300 x5033Rick.Rashid@cs.cmu.edu (06/26/91)
In looking at the numbers in Brian's paper I noted a couple of things:
1) The single application which is listed as having the
greatest disparity in time (150 vs 92 seconds) is the
remote Andrew benchmark. This is very deceptive because
Mach 2.5 incorporates AFS 3.0 directly in the kernel
(AFS is the remote file system being used) while there
is no direct support for AFS in Mach 3.0. In Mach 3.0
we run a server program called Venus which implements
an older version of the AFS interface. This older
implementation still talks to current AFS servers, but
it does so using the old protocol through a protocol
converter. Performance of AFS access through Venus is
thus particularly poor and not really indicative of any
specific feature of Mach 3.0. (The reason Mach 3.0 uses
Venus has to do with NFS/AFS licensing issues.)
2) The other tests indicated Mach 3.0 performance in the range of
.73 to 1.64 of Mach 2.5 with the latex and linking applications,
for example, running at .85 and the spelling checker at 1.64.
I don't routinely get number for the DecStation, but I do have some
older numbers for the Decstation Mach 3.0 vs. Ultrix (collected
last year for USENIX). The numbers are presented as ratios
of Ultrix to Mach 3.0 times (i.e. numbers below 1 are
bad for 3.0, above 1 are good):
Compilation benchmark: 1.23
Andrew benchmark: 1.01
Cached file reads: 1.60
Uncached reads: .90
Uncached writes: .88
For the Sun 3/60 I measured Mach 2.5 and Mach 3.0:
Compilation: 1.04
Andrew: .98
(additional numbers are in the paper).
I am currently compiling a set of performance number for the
i386 which I will pass on in a message in the next few days.
-Rick