hedrick@topaz.ARPA (Chuck Hedrick) (06/29/85)
As I indicated in a previous message, we have become concerned about
compilation times. For many purposes these are more important than
execution times. Since I found little in the way of compilation
benchmarks, I have started doing my own. Here are some preliminary
results:
Pascal
------
The program is one of our Unix billing programs (which is a port of
our TOPS-20 project-based billing system). It is 26755 bytes, 899
lines. Here are the results on various systems. Times are CPU for
the timesharing systems (DEC-20, Pyramid, VMS), both CPU and real for
Sun and Apollo. The DEC-20, Pyramid, and VMS configurations were
fairly normal: a multi-pack RP06 for the DEC-20, an Eagle for the
Pyramid, and an RA81 for the 785. The Apollo we used was a diskless
node. Tests were done under their 4.2 port. The Sun we used had
their slow (SCSI) 42 Mbyte disk. SVS is Silicon Valley Software.
Their compilers are distributed by Unipress, in Highland Park, NJ.
Times include loading, except in one case.
DEC-20: 6.6
Pyramid: 66.2
Sun: 9:25, 10:21 real
Sun with SVS Pascal: 19.5, 24 real
Apollo: 41.7, 46 real [does not include the time to load, as
I forgot to write that down]
One amusing thing about the Apollo: The fastest compilation time is
with the option -dba, which is the equivalent of Unix -g. It
produces debugging information. We assumed the difference was that
it turned off optimization, which is on by default. But we tried
-nopt, which does just that, and it was just a tad slower. We are
curious why -dba should be faster than -nopt.
Fortran
-------
This program was Paranoia. This is a test of various features of
arithmetic, originally written by W.M.Kahan at U.C. Berkeley. It is
118922 bytes, or 3437 lines. Machine configurations are as above.
DEC-20: 18.2
785, VMS: 42.5
Pyramid: 6:18
Sun: 9:25, 10:21 real
Sun, Silicon Valley compiler: 1:06, 1:17 real
Apollo: 2:16, 2:20 real
The moral of the story (in case it wasn't obvious anyway) is that the
compilers distributed with 4.2 are incredibly slow. In all fairness,
I should say that both Sun and Pyramid are doing work on compiler
performance. I believe that they will both make significant
improvements. However I haven't seen any evidence that they are
going to get the order of magnitude performance that SVS and Apollo
have gotten.
Lest this be taken as an endorsement of the SVS compilers, let me say
that my only experience with them was in doing these benchmarks. I
have no idea whether they are reliable, or have the various other
qualities one expects in a good compiler.