partha@sbcs.UUCP (Partha Dasgupta) (08/06/84)
The following are some performance figures for a VAX780, a VAX750, the new Pyramid and a SUN-1. They are not standardized benchmarks, but do provide some comparisons.... The 780 has Berkeley 4.1, while all the others ran 4.2bsd. 1) C-CARD : A CPU intensive, recursive card arranging routine, written in C. 2) P-CARD : The exact same program, written in Pascal. 3) GREP : The command "grep 11 /usr/dict/words". Has considerable CPU and i/o activity. 4) SWAP : A tough one. Initializes a 1 Meg integer array twice. Uses 4 Meg bytes of core (2 Meg on SUN), and causes a lot of swapping, possibly thrashing. It favors the SUN, but the SUN has a slower swapping disk, and less core. Part I: Low Load tests This part was done at low load. I was the only person logged in. The data shows the CPU time used by the user process and the system, and the elapsed time. The elapsed time for VAX780 is omitted as there were a lot of background stuff running. 1) C-CARD User-CPU System-CPU Elapsed time VAX-780 6.8s 0.1s -- VAX-750 13.3s 0.1s 14s SUN 13.9s 0.1s 14s Pyramid 3.4s 0.1s 4s 2) P-CARD User-CPU System-CPU Elapsed time VAX780 23.3s 1.0s -- VAX750 32.5s 0.4s 33s SUN 41.0s 0.4s 42s Pyramid 3.4s 0.1s 4s 3) GREP User-CPU System-CPU Elapsed time VAX780 3.4s 0.8s -- VAX750 6.8s 0.9s 8s SUN 6.8s 0.9s 8s Pyramid 3.3s 0.5s 4s 4) SWAP! (Note the dramatic rise in system CPU times. Caused by swapping too much?) User-CPU System-CPU Elapsed time VAX780 13.6s 11.5s --- VAX750 22.7s 23.1s 1:30s SUN 33.1s 158.0s 5:56s Pyramid 5.4s 5.9s 0:58s Part II : High load testing. This was done by loading the system by up to 20 processes of the same command. 1) to 3) was run as 20 simultaneous processes. Naturally load average reached nearly 20. The figures are averages, per process. The SWAP! program could not be handled in large numbers. The Pyramid allowed 6 processes before saying "No more core". The 780, 750 and SUN gave up after 3 processes. The figures are for 5 processes on Pyramid and 3 on 780/750/SUN. 1) C-CARD (20 processes) User-CPU System-CPU Elapsed time VAX780 6.8s 0.2s 2:15s VAX750 13.4s 0.4s 4:37s SUN 16.0s 0.6s 5:28s Pyramid 3.4s 0.1s 1:04s 2)P-CARD (20 processes) User-CPU System-CPU Elapsed time VAX780 (no data) VAX750 33.5s 0.6s 11:10s SUN 47.8s 3.9 16:00s Pyramid 3.4s 0.1s 1:06s (!!) 3) GREP (20 processes) User-CPU System-CPU Elapsed time VAX780 3.5s 0.9s 1:20s VAX750 7.0s 0.8s 2:45s SUN 7.5s 1.4s 2:50s Pyramid 4.0s 0.5s 1:20s 4) SWAP! : This program brought all the systems to their knees. They probably thrashed badly. Even echoing characters on the terminal seemed to be a problem! The sun especially has a small slow swapping disk and was the hardest hit. User-CPU System-CPU Elapsed time VAX780 (3 processes) 14.0s 14.0s 3:40s VAX750 (3 processes) 23.5s 23.1s 4:26s SUN (3 processes) 35.6s 104.0s 13:00s Pyramid (3 processes) 5.9s(!) 6.0s 2:30s Pyramid (5 processes) 5.9s 6.0s 4:17s Thus the Pyramid-90 seems to be a winner. However the tiny SUN's performance was astounding. It matched the 750 all the time, except for the SWAP program, but then its disk is mainly to blame. Makes one wonder whether a 750 is really worth it, when a el-cheapo M68000 can get this sort of performance. (There has been a lot of mud slinging at the Sun on the net. I agree its not very impressive when run on its bitmap screen in single user mode, but on the ethernet, with a couple of users logged in it really looks as good (or bad?) as a 750. Wonder if you would notice it if someone yanked out your 750, and put in a Sun....) Partha Dasgupta SUNY Stony Brook ...allegra!sbcs!partha
smh@SRI-PRISM.ARPA (08/13/84)
From: "Scott M. Hinnrichs" <smh@SRI-PRISM.ARPA>
I was pleased at the results these comparison benchmarks came
up with so I requested a copy from Partha to run on our Pyramid 90x.
Needless to say, we came up with similar results.
Partha (at SUNY) did not say what the configuration of the
Pyramid (or VAX, or SUN) were, but I was anxious to see how our 90X
faired. We have 8 meg of memory, a Data-cache, 2 swap devices, and
the O/Sx 2.3 release software (which can all make a difference in
performance measurements).
Here are the results of running Partha's benchmarks on our
Pyramid 90x vs. his results. We were running an incremental dump at
the time. We also have a program (nice 20) running all the time
computing pi to 10,000,000 places soaking up spare cpu. :-)
1) C-CARD
User-CPU System-CPU Elapsed Time
Pyramid (SRI) 2.6s 0.0s 2s
Pyramid (SUNY) 3.4s 0.1s 4s
VAX-11/780 6.8s 0.1s --
VAX-11/750 13.3s 0.1s 14s
SUN 13.9s 0.1s 14s
2) P-CARD
User-CPU System-CPU Elapsed Time
Pyramid (SRI) 2.6s 0.0s 2s
Pyramid (SUNY) 3.4s 0.1s 4s
VAX-11/780 23.2s 1.0s --
VAX-11/750 32.5s 0.4s 33s
SUN 41.0s 0.4s 42s
3) GREP
User-CPU System-CPU Elapsed Time
Pyramid (SRI) 2.6s 0.4s 3s
Pyramid (SUNY) 3.3s 0.5s 4s
VAX-11/780 3.4s 0.8s --
VAX-11/750 6.8s 0.9s 8s
SUN 6.8s 0.9s 8s
4) SWAP
User-CPU System-CPU Elapsed Time
Pyramid (SRI) 4.8s 2.8s 0:08s (!)
Pyramid (SUNY) 5.4s 5.9s 0:58s
VAX-11/780 13.6s 11.5s -----
VAX-11/750 22.7s 23.1s 1:30s
SUN 33.1s 158.0s 5:56s
High-Load Tests
The high-load tests were fun. While the 5 process SWAP test
was going I attempted to run emacs. It took 16 seconds for emacs to
load my .emacs_pro.mo and read in /etc/termcap (usually takes 3
seconds). After emacs came up it was quite responsive and there did
not seem to be any delay in screen updating. Even though the 90X
shines on individual benchmarks, I think the real benefit we have
found is responsiveness under (simulated) heavy loads.
1) C-CARD (20 processes)
User-CPU System-CPU Elapsed Time
Pyramid (SRI) 2.6s 0.0s 0:47s (!)
Pyramid (SUNY) 3.4s 0.1s 1:04s
VAX-11/780 6.8s 0.2s 2:15s
VAX-11/750 13.4s 0.4s 4:37s
SUN 16.0s 0.6s 5:28s
2) P-CARD (20 processes)
User-CPU System-CPU Elapsed Time
Pyramid (SRI) 2.6s 0.0s 46s (!)
Pyramid (SUNY) 3.4s 0.1s 1:06s
VAX-11/780 -- No Data --
VAX-11/750 33.5s 0.6s 11:10s
SUN 47.8s 3.9s 16:00s
3) GREP (20 processes)
User-CPU System-CPU Elapsed Time
Pyramid (SRI) 2.6s 0.4s 48s (!)
Pyramid (SUNY) 4.0s 0.5s 1:20s
VAX-11/780 3.5s 0.9s 1:20s
VAX-11/750 7.0s 0.8s 2:45s
SUN 7.5s 1.4s 2:50s
4) SWAP (N processes)
User-CPU System-CPU Elapsed Time
SRI (3 procs) 4.9s 6.3s 2:25s
SUNY(3 procs) 5.9s 6.0s 2:30s
VAX-11/780(") 14.0s 14.0s 3:40s
VAX-11/750(") 23.5s 23.1s 4:26s
SUN 35.6s 104.0s 13:00s
SRI (5 procs) 4.9s 6.7s 4:17s (Max 10 procs)
SUNY(5 procs) 5.9s 6.0s 4:17s (Max 5 procs)
VAX-11/780(") -- Pooped out -- (Max 3 procs)
From these tests it is obvious that there is an advantage to
the Data-Cache, an additional swap device, and 8 Meg of memory.
The final results of this test gave similar results to 2 other
benchmark suites we have run here. The overall average performance of
the Pyramid 90x approaches 2.6 x VAX-11/780. Worst case I have found
since installing the Data-Cache was 1.26 x VAX-11/780, and the best is
4.8 x VAX-11/780 (former was a Prolog Interpreter, the latter was
passing a single int parameter on a 10e+06 iteration).
Until Pyramid comes out with their GPIO hardware for speeding
up (disk) I/O the machine will not realize it's full potential.
One question for Partha, what happened to the data for the
VAX-11/780 on many of the tests?
We will soon be unleashing our entire user community on the
90X; I will try to give some reasonable impressions of the results.
Scott M. Hinnrichs
SRI International
smh@sri-prism, sri-unix!sri-prism!smh