maclab@reed.UUCP (S.Gillespie/Mac Dev. Lab) (11/06/85)
As a follow-up to the recent Byte review of 5 Macintosh C compilers, I have
compared Rascal's performance on the same benchmarks. Included are
time comparisons, notes, and Rascal sources. This is by no means a
stand-alone article -- I strongly recommend that you use this information
*in conjunction* with the Byte article, should you wish to draw
specific conclusions about these benchmarks.
Rascal is certainly not the fastest of the development systems, in terms
of these benchmarks, but I think it compares rather favorably with the C's.
Scott Gillespie
Reed College
Claimer: As a member of the Rascal development team, I am biased
beyond a doubt, but I hope you will find this to be a fair
and honest evaluation.
{decvax, ucbvax, pur-ee, uw-beaver, masscomp, cbosg, aat, mit-ems,
psu-cs, uoregon, orstcs, ihnp4, uf-cgrl, ssc-vax}!tektronix
\
+--!reed!maclab
{teneron, ogcvax, muddcs, cadic, oresoft, grpwre, /
harvard, psu-cs, omen, isonvax, nsc-pdc}----------+
---------------------------------------------------------------------------
Rascal Benchmarks
In a recent issue of Byte magazine (November 1985 Vol. 10, No. 12), Tim Field
reviewed 5 Macintosh development systems ("Five C Compilers for the
Macintosh," page 275), and included a number of benchmarks for timing
comparison. Because C is part of Rascal's heritage, we found it extremely
easy to translate the C benchmark sources into Rascal. What follows are
benchmark times, sources and notes which put Rascal's performance in
perspective with the C compilers. Complete C sources and useful descriptions
of the various benchmarks may be found in Mr. Field's article.
Execution Times
(In Seconds)
Rascal SoftWorks C Mac C Aztec C Hippo-C MegaMax C
Frame
Normal 0.11 0.13 0.13 0.10 0.25 0.10
Register 0.07 0.08 n.a. 0.05 n.a. 0.05
Pointer
Normal 37.93 24.33 26.60 25.50 33.23 30.02
Register 27.51 11.15 n.a. 13.15 n.a. 18.93
Intmath
Normal 5.33 30.05 5.03 5.10 15.93 5.05
Register 3.12 26.73 n.a. 2.70 n.a. 2.78
Sieve
Normal 6.77 8.83 7.98 6.20 12.65 6.20
Register 4.51 4.73 n.a. 3.88 n.a. 4.17
Qsort
Normal * 157.08 63.92 68.43 * 93.38
Register * 93.72 n.a. 50.87 n.a. 70.80
Float
Normal 149.75** 332.77 n.a. 268.22 n.a. 334.32
Fib
Normal 36.66 28.60 31.67 24.72 47.22 25.97
Interface
Normal 74.20 59.18 71.40 56.22 78.47 72.00
* BenchMark failed, due to insufficient stack space (>24K stack required)
** The Rascal Compiler does not support direct floating point calculations:
this benchmark was performed using SANE floating point calls (see
source, and notes below).
Benchmark Notes
All Benchmark programs were:
1) Compiled, Linked and Executed with a pre-release Rascal Development system
(Dated v.85.10.28.spg). This version of the development system is frozen
(except for minor cosmetic changes), and will be released as the 'final'
version as soon as we finish the documentation.
2) Executed with Rascal's built-in millisecond timer disabled (the timer may
be disabled with Rascal's new configuration dialog box), since the millisecond
timer steals a fair chunk of processor time (~10%).
3) Written to mimic, as much as possible, not only the functionality of the
given C benchmarks, but also their appearance and structure. This included
using curly-braces ( { } ) in place of begin/end (either set is allowed in
Rascal).
Individual Benchmark Notes
Below are a few individual comments on some of the benchmarks -- for a
complete description of all benchmarks and for an explanation of what element
of the development system each tests, see the Byte article.
Qsort
Both Rascal and Hippo-C failed this benchmark due to insufficient stack
space. It should be noted that the particular sorting algorithm given
requires 1 level of recursion for every element, therefore 1000 levels of
recursion are required for Qsort -- Rascal was able to perform the Qsort test
on a smaller array, which required fewer recursion levels. Also note that
there are several disastrous typos in the Byte listing of this benchmark, so
take care when trying to duplicate it in your own favorite language!
Float
The given benchmark assumed built-in Compiler support for floating point
calculations -- neither Mac C, nor Hippo-C nor Rascal provide this amenity, so
the Rascal version of this benchmark should not be used for direct speed
comparison with the other C compilers. Rascal floating point calculations are
carried out using SANE (Standard Apple Numeric Environment) procedure calls
(see the Rascal source). We might point out, however, that although
convenience is sacrificed with this method, you will note that the Rascal
benchmark was twice as fast as the average of the other C compiler versions.
Also, the C floating point numbers are 32 bit double floats, and Rascal's SANE
floats are 80 bit extended precision floats...
Interface
The intent of this benchmark is to test the effectiveness of a language's
interface calls to Macintosh ROM routines. The particular ROM routine chosen
for this test (GetNextEvent) is not well suited for benchmark testing -- the
amount of time spent within GetNextEvent can vary widely between different
systems and Macintosh configurations: the presence of drivers and desk
accessories may slow down the benchmark considerably, so beware.
Benchmark Listings
All benchmark sources were created using an outline source file containing
three procedures: Bench(), _Init() and _Key(c,mods: integer). _Init and _Key
are Rascal entry points. _Init is the initial entry point (_Init is called
before any other Rascal procedure), and _Key is called whenever the user
presses a key on the keyboard. Both _Init and _Key, in the benchmark sources,
call the procedure Bench(): within procedure Bench(), each benchmark was
written. Therefore, the Bench() of a Rascal benchmark is analogous to the
main() of the C benchmarks. The _Key procedure is there so that the benchmark
may be repeated by pressing a key.
Within the Bench() procedure, are calls Start() and Stop() (analogous to the
'startup' and 'done' include files in the C benchmarks). Start() and Stop()
are library routines which were written for these benchmark tests, and are
located in __BenchMarkLib (listing is included here). Start and Stop time
with both Rascal's millisecond timer, and the Macintosh's built-in tick count:
since the millisecond timer was disabled for the actual benchmark tests, the
millisecond value was ignored.
Each benchmark source contains time information at the top, as well as the
final file size. These file sizes, you will note, are extremely small
compared to the file sizes given in the Byte article: this is because we did
not follow the Byte requirement that all benchmarks be stand-alone
applications. For a proper comparison with the file sizes in Byte, add about
18.5K to each of the file sizes given in the Rascal benchmark sources -- all
of these benchmarks may be made into stand-alone applications with no source
modifications (another method of creating Rascal stand-alone applications
would add about 3.5K, instead of 18.5K, but would require source
modification).
Where times are given for register versions of the Rascal benchmarks, the
changes made to the source are very simple. To declare a local register
variable in Rascal (up to 5 are allowed), one just puts the keyword 'register'
between the identifer-list and the type, in the variable declaration:
Var
x,y,z: register integer;
The Sources:
(*________________________FRAME_____________________________*)
Program Frame;
(*
.11 seconds -- Non-Register
.07 seconds -- Register
File Size: 536 bytes
*)
Link __BenchMarkLib, __IO, __ExtendIO :;
Procedure Bench();
Const
Count = 10000;
Var
i: Integer;
{
Start();
loop(,i:=0,++i,i=10000);
Stop();
};
Procedure _Key(c,mods: integer);
{
Bench();
};
Procedure _Init();
{
Bench();
};
(*_________________________POINTER___________________________*)
Program Pointer;
(*
37.93 seconds -- Non-Register
27.51 seconds -- Register
File Size: 622 bytes
*)
Link __BenchMarkLib, __IO, __ExtendIO :;
Procedure Bench();
Const
Count = 10000;
Allotted = 128;
Var
WorkArea: Byte[Allotted];
Ptr: Ptrb;
i: Integer;
{
Start();
loop(,i:=0,++i,i=Count) {
ptr := @WorkArea;
loop(ptr<(workarea+allotted),,,!(ptr<(workarea+allotted))) {
ptr^ := ' ';
++ptr;
};
};
Stop();
};
Procedure _Key(c,mods: integer);
{
Bench();
};
Procedure _Init();
{
Bench();
};
(*___________________________INTMATH__________________________*)
Program IntMath;
(*
5.33 seconds -- Non-Register
3.12 seconds -- Register
File Size: 1148 bytes
*)
Link __BenchMarkLib, __IO, __ExtendIO :;
Procedure Bench();
Const
Count = 10000;
Var
i,j,k: Integer;
{
Start();
Loop(,i:=0,++i,i=Count) {
j := 240; k := 15;
j := (k*(j/k));
j := (k*(j/k));
j := (k+k+k+k+k+k+k+k+k+k+k+k+k+k+k+k);
k := (j-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k);
j := (j<<4); k := (k<<4);
j := (k*(j/k));
j := (k*(j/k));
j := (k+k+k+k+k+k+k+k+k+k+k+k+k+k+k+k);
k := (j-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k);
j := (j<<4); k := (k<<4);
j := (k*(j/k));
j := (k*(j/k));
j := (k+k+k+k+k+k+k+k+k+k+k+k+k+k+k+k);
k := (j-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k);
};
Stop();
};
Procedure _Key(c,mods: integer);
{
Bench();
};
Procedure _Init();
{
Bench();
};
(*__________________________SIEVE___________________________*)
Program Sieve;
(*
6.77 seconds -- Non-Register
4.51 seconds -- Register
File Size: 762 bytes
*)
Uses
(*$U+*) uMemtypes;
Link __BenchMarkLib, __IO, __ExtendIO :;
Const
Size = 8190;
Var
Flags: Byte[8191 ];
Procedure Bench();
Var
i,prime,k,count,iter: Integer;
{
Start();
Loop(,iter:=1,,++iter>10) {
count := 0;
loop(,i:=0,++i,i>Size)
flags[i]:=True;
loop(,i:=0,++i,i>Size)
if flags[i] Then {
prime := i+i+3;
k:=i+prime;
loop(k=<size,,k+=prime,k>Size)
flags[k]:=False;
++count;
};
};
Stop();
Writeln();
WriteIntNs(Count);
WriteString(" : Primes.");
Writeln();
};
Procedure _Key(c,mods: integer);
{
Bench();
};
Procedure _Init();
{
Bench();
};
(*___________________________FLOAT__________________________*)
Program Float;
(*
149.75 seconds -- Non-Register
File Size: 1062 bytes
*)
Link __BenchMarkLib, __IO, __ExtendIO, __Sane :;
const COUNT=10000;
var a,b,c:Float;
Procedure Bench();
var i:integer;
{
Start();
Lqfillf(3141597,-6,a);
Lqfillf(17839032,-3,b);
loop(,i:=0,++i,i=count)
{
c:=a;
mulf(b,c); (* c = c*b *)
divf(a,c); (* c = c/a *)
c:=a; mulf(b,c); divf(a,c);
c:=a; mulf(b,c); divf(a,c);
c:=a; mulf(b,c); divf(a,c);
c:=a; mulf(b,c); divf(a,c);
c:=a; mulf(b,c); divf(a,c);
c:=a; mulf(b,c); divf(a,c);
};
Stop();
};
Procedure _Key(c,mods: integer);
{
Bench();
};
Procedure _Init();
{
Bench();
};
(*____________________________FIB_________________________*)
Program FIB;
(*
36.66 seconds -- Non-Register
File Size: 678 bytes
*)
Link __BenchMarkLib, __IO, __ExtendIO :;
Function Fib(x: Integer): Longint;
{
If x>2 Then
Return(Fib(x-1) + Fib(x-2))
Else
Return(1);
};
Procedure Bench();
Const
NTimes = 10;
Number = 24;
Var
i: Integer;
Value: Longint;
{
Start();
Loop(,i:=0,++i,i=NTimes)
Value := Fib(Number);
Stop();
Writeln();
Writestring("Value: ");
WriteLongNS(Value);
Writeln();
};
Procedure _Key(c,mods: integer);
{
Bench();
};
Procedure _Init();
{
Bench();
};
(*__________________________INTERFACE___________________________*)
Program Interface;
(*
74.20 seconds -- Non-Register
File Size: 696 bytes
*)
Uses __ToolTraps,
(*$U+*) uToolIntf;
Link __BenchMarkLib, __IO, __ExtendIO :;
Procedure Bench();
Const
Count = 10000;
Var
i, M1,M2 : Integer;
Bool1,Bool2 : Boolean;
ER1,ER2 : EventRecord;
{
Start();
M1 := -1;
M2 := -1;
Loop(,i:=0,++i,i=Count) {
Bool1 := GetNextEvent(M1,@ER1);
Bool2 := GetNextEvent(m2,@ER2);
Bool1 := GetNextEvent(M1,@ER1);
Bool2 := GetNextEvent(m2,@ER2);
Bool1 := GetNextEvent(M1,@ER1);
Bool2 := GetNextEvent(m2,@ER2);
Bool1 := GetNextEvent(M1,@ER1);
Bool2 := GetNextEvent(m2,@ER2);
};
Stop();
};
Procedure _Key(c,mods: integer);
{
Bench();
};
(*___________________________ __BENCHMARKLIB __________________________*)
Program __BenchMarkLib;
Uses __ToolBox;
Link __IO, __ExtendIO :;
Var
MTime: Longint;
TTime: Longint;
Proc Start();
{
Writeln();
WriteString("Starting Timers...");
TTime:= -TickCount();
ResetTimer();
};
Proc Stop();
Var t: integer;
{
TimerVal(@MTime);
TTime += TickCount();
Writeln();
WriteString(" Milliseconds: ");
Writelongns(MTime);
Writeln();
WriteString(" Seconds: ");
Writeintns(TTime/60);
WriteChar('.');
T := ((TTime mod 60)*100)/60;
If T<10 Then WriteChar('0');
Writeintns(T);
Writeln();
Writeln();
};
(*___________________________BENCH__________________________*)
Program Bench;
(*
seconds -- Non-Register
seconds -- Register
File Size: bytes
*)
Link __BenchMarkLib, __IO, __ExtendIO :;
Procedure Bench();
{
Start();
Stop();
};
Procedure _Key(c,mods: integer);
{
Bench();
};
Procedure _Init();
{
Bench();
};
End of Article...