[net.sources.mac] Rascal Benchmarks

maclab@reed.UUCP (S.Gillespie/Mac Dev. Lab) (11/06/85)

As a follow-up to the recent Byte review of 5 Macintosh C compilers, I have
compared Rascal's performance on the same benchmarks.  Included are
time comparisons, notes, and Rascal sources.  This is by no means a 
stand-alone article -- I strongly recommend that you use this information
*in conjunction* with the Byte article, should you wish to draw 
specific conclusions about these benchmarks.

Rascal is certainly not the fastest of the development systems, in terms
of these benchmarks, but I think it compares rather favorably with the C's.


Scott Gillespie
Reed College

Claimer:  As a member of the Rascal development team, I am biased
    beyond a doubt, but I hope you will find this to be a fair
    and honest evaluation.


{decvax, ucbvax, pur-ee, uw-beaver, masscomp, cbosg, aat,  mit-ems, 
  psu-cs, uoregon, orstcs, ihnp4, uf-cgrl, ssc-vax}!tektronix 
                                                             \
                                                              +--!reed!maclab 
{teneron, ogcvax, muddcs, cadic, oresoft, grpwre,            /
          harvard, psu-cs, omen, isonvax, nsc-pdc}----------+


---------------------------------------------------------------------------


Rascal Benchmarks

In a recent issue of Byte magazine (November 1985 Vol. 10, No. 12),  Tim Field 
reviewed 5 Macintosh development systems ("Five C Compilers for the 
Macintosh," page 275), and included a number of benchmarks for timing 
comparison.  Because C is part of Rascal's heritage, we found it extremely 
easy to translate the C benchmark sources into Rascal.  What follows are 
benchmark times, sources and notes which put Rascal's performance in 
perspective with the C compilers.  Complete C sources and useful descriptions 
of the various benchmarks may be found in Mr. Field's article.



                               Execution Times
                                (In Seconds)


            Rascal  SoftWorks C     Mac C   Aztec C     Hippo-C     MegaMax C
Frame
 Normal     0.11    0.13            0.13    0.10        0.25        0.10
 Register   0.07    0.08            n.a.    0.05        n.a.        0.05

Pointer
 Normal     37.93   24.33           26.60   25.50       33.23       30.02
 Register   27.51   11.15           n.a.    13.15       n.a.        18.93

Intmath
 Normal     5.33    30.05           5.03    5.10        15.93       5.05
 Register   3.12    26.73           n.a.    2.70        n.a.        2.78

Sieve
 Normal     6.77    8.83            7.98    6.20        12.65       6.20
 Register   4.51    4.73            n.a.    3.88        n.a.        4.17

Qsort
 Normal     *       157.08          63.92   68.43       *           93.38
 Register   *       93.72           n.a.    50.87       n.a.        70.80

Float
 Normal   149.75**  332.77          n.a.    268.22      n.a.        334.32

Fib
 Normal     36.66   28.60           31.67   24.72       47.22       25.97

Interface
 Normal     74.20   59.18           71.40   56.22       78.47       72.00


   *  BenchMark failed, due to insufficient stack space (>24K stack required)
  **  The Rascal Compiler does not support direct floating point calculations:
      this benchmark was performed using  SANE floating point calls (see
      source, and notes below).
      
      
      
Benchmark Notes

All Benchmark programs were:

1) Compiled, Linked and Executed with a pre-release Rascal Development system 
(Dated  v.85.10.28.spg).  This version of the development system is frozen 
(except for minor cosmetic changes), and will be released as the 'final' 
version as soon as we finish the documentation.

2)  Executed with Rascal's built-in millisecond timer disabled  (the timer may 
be disabled with Rascal's new configuration dialog box), since the millisecond 
timer steals a fair chunk of processor time (~10%).

3) Written to mimic, as much as possible, not only the functionality of the 
given C benchmarks, but also their appearance and structure.  This included 
using curly-braces ( { } ) in place of  begin/end  (either set is allowed in 
Rascal).



Individual Benchmark Notes

Below are a few individual comments on some of the benchmarks -- for a 
complete description of all benchmarks and for an explanation of what element 
of the development system each tests, see the Byte article.

Qsort
    Both Rascal and Hippo-C failed this benchmark due to insufficient stack 
space.  It should be noted that the particular sorting algorithm given 
requires 1 level of recursion for every element, therefore 1000 levels of 
recursion are required for Qsort -- Rascal was able to perform the Qsort test 
on a smaller array, which required fewer recursion levels.  Also note that 
there are several disastrous typos in the Byte listing of this benchmark, so 
take care when trying to duplicate it in your own favorite language!

Float
    The given benchmark assumed built-in Compiler support for floating point 
calculations -- neither Mac C, nor Hippo-C nor Rascal provide this amenity, so 
the Rascal version of this benchmark should not be used for direct speed 
comparison with the other C compilers.  Rascal floating point calculations are 
carried out using SANE (Standard Apple Numeric Environment) procedure calls 
(see the Rascal source).  We might point out, however, that although 
convenience is sacrificed with this method, you will note that the Rascal 
benchmark was twice as fast as the average of the other C compiler versions.  
Also, the C floating point numbers are 32 bit double floats, and Rascal's SANE 
floats are 80 bit extended precision floats...

Interface
    The intent of this benchmark is to test the effectiveness of a language's 
interface calls to Macintosh ROM routines.  The particular ROM routine chosen 
for this test (GetNextEvent) is not well suited for benchmark testing -- the 
amount of time spent within GetNextEvent can vary widely  between different 
systems and Macintosh configurations:  the presence of drivers and desk 
accessories may slow down the benchmark considerably, so beware.



Benchmark Listings


All benchmark sources were created using an outline source file containing 
three procedures:  Bench(), _Init() and _Key(c,mods: integer).  _Init and _Key 
are Rascal entry points.  _Init is the initial entry point (_Init is called 
before any other Rascal procedure), and _Key is called whenever the user 
presses a key on the keyboard.  Both _Init and _Key, in the benchmark sources, 
call the procedure Bench():  within procedure Bench(), each benchmark was 
written.  Therefore, the Bench() of a Rascal benchmark is analogous to the 
main() of the C benchmarks.  The _Key procedure is there so that the benchmark 
may be repeated by pressing a key.

Within the Bench() procedure, are calls Start() and Stop() (analogous to the 
'startup' and 'done' include files in the C benchmarks).  Start() and Stop() 
are library routines which were written for these benchmark tests, and are 
located in __BenchMarkLib  (listing is included here).  Start and Stop time 
with both Rascal's millisecond timer, and the Macintosh's built-in tick count:  
since the millisecond timer was disabled for the actual benchmark tests, the 
millisecond value was ignored.

Each benchmark source contains time information at the top, as well as the 
final file size.  These file sizes, you will note, are extremely small 
compared to the file sizes given in the Byte article:  this is because we did 
not follow the Byte requirement that all benchmarks be stand-alone 
applications.  For a proper comparison with the file sizes in Byte, add about 
18.5K to each of the file sizes given in the Rascal benchmark sources -- all 
of these benchmarks may be made into stand-alone applications with no source 
modifications (another method of creating Rascal stand-alone applications 
would add about 3.5K, instead of 18.5K, but would require source 
modification).
 
Where times are given for register versions of the Rascal benchmarks, the
changes made to the source are very simple.  To declare a local register
variable in Rascal (up to 5 are allowed), one just puts the keyword 'register'
between the identifer-list and the type, in the variable declaration:

Var
  x,y,z: register integer;


The Sources:


(*________________________FRAME_____________________________*)

Program Frame;

(*
   .11 seconds -- Non-Register
   .07 seconds -- Register
   
   File Size:  536 bytes
*)

Link  __BenchMarkLib, __IO, __ExtendIO :;


Procedure Bench();
Const
  Count = 10000;
Var
  i:  Integer;
{
  Start();
  loop(,i:=0,++i,i=10000);
  Stop();
};

Procedure _Key(c,mods: integer);
{
  Bench();
};

Procedure _Init();
{
  Bench();
};

(*_________________________POINTER___________________________*)

Program Pointer;

(*
      37.93 seconds -- Non-Register
      27.51 seconds -- Register
   
   File Size:    622 bytes
*)

Link  __BenchMarkLib, __IO, __ExtendIO :;


Procedure Bench();
Const
  Count = 10000;
  Allotted = 128;
Var
  WorkArea: Byte[Allotted];
  Ptr: Ptrb;
  i: Integer;
{
  Start();
  
  loop(,i:=0,++i,i=Count) {
    ptr := @WorkArea;
    loop(ptr<(workarea+allotted),,,!(ptr<(workarea+allotted))) {
      ptr^ := ' ';
      ++ptr;
      };
    };
  
  Stop();
};

Procedure _Key(c,mods: integer);
{
  Bench();
};

Procedure _Init();
{
  Bench();
};


(*___________________________INTMATH__________________________*)

Program IntMath;

(*
      5.33 seconds -- Non-Register
      3.12 seconds -- Register
   
   File Size:    1148 bytes
*)

Link  __BenchMarkLib, __IO, __ExtendIO :;


Procedure Bench();
Const
  Count = 10000;
Var
  i,j,k: Integer;
{
  Start();
  
  Loop(,i:=0,++i,i=Count) {
    j := 240; k := 15;
    
    j := (k*(j/k));
    j := (k*(j/k));    
    j := (k+k+k+k+k+k+k+k+k+k+k+k+k+k+k+k);
    k := (j-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k);
    
    j := (j<<4); k := (k<<4);
    j := (k*(j/k));
    j := (k*(j/k));    
    j := (k+k+k+k+k+k+k+k+k+k+k+k+k+k+k+k);
    k := (j-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k);
    
    j := (j<<4); k := (k<<4);
    j := (k*(j/k));
    j := (k*(j/k));    
    j := (k+k+k+k+k+k+k+k+k+k+k+k+k+k+k+k);
    k := (j-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k);    
    };  
    
  Stop();
};

Procedure _Key(c,mods: integer);
{
  Bench();
};

Procedure _Init();
{
  Bench();
};

(*__________________________SIEVE___________________________*)


Program Sieve;

(*
      6.77 seconds -- Non-Register
      4.51 seconds -- Register
   
   File Size:    762 bytes
*)

Uses 
(*$U+*) uMemtypes;

Link  __BenchMarkLib, __IO, __ExtendIO :;

Const
  Size = 8190;
Var
  Flags: Byte[8191 ];

Procedure Bench();
Var
  i,prime,k,count,iter:  Integer;
{
  Start();  
  Loop(,iter:=1,,++iter>10) {
    count := 0;
    loop(,i:=0,++i,i>Size)
      flags[i]:=True;
    loop(,i:=0,++i,i>Size) 
      if flags[i] Then {
        prime := i+i+3;
        k:=i+prime;
        loop(k=<size,,k+=prime,k>Size)
          flags[k]:=False;
        ++count;
        };
    };  
  Stop();
  Writeln();
  WriteIntNs(Count);
  WriteString(" : Primes.");
  Writeln();
};

Procedure _Key(c,mods: integer);
{
  Bench();
};
Procedure _Init();
{
  Bench();
};



(*___________________________FLOAT__________________________*)

Program Float;

(*
   149.75 seconds -- Non-Register
   
   File Size:   1062  bytes
*)

Link  __BenchMarkLib, __IO, __ExtendIO, __Sane :;

const COUNT=10000;

var a,b,c:Float;

Procedure Bench();
var i:integer;
{
  Start();
   Lqfillf(3141597,-6,a);
   Lqfillf(17839032,-3,b);
   loop(,i:=0,++i,i=count)
     {
       c:=a;    
       mulf(b,c);     (* c = c*b *)
       divf(a,c);     (* c = c/a *)
       
       c:=a;  mulf(b,c); divf(a,c);
       
       c:=a;  mulf(b,c); divf(a,c);
       
       c:=a;  mulf(b,c); divf(a,c);
       
       c:=a;  mulf(b,c); divf(a,c);
       
       c:=a;  mulf(b,c); divf(a,c);
       
       c:=a;  mulf(b,c); divf(a,c);
      };
  Stop();
};

Procedure _Key(c,mods: integer);
{
  Bench();
};

Procedure _Init();
{
  Bench();
};




(*____________________________FIB_________________________*)


Program FIB;

(*
      36.66 seconds -- Non-Register
   
   File Size:   678 bytes
*)

Link  __BenchMarkLib, __IO, __ExtendIO :;

Function Fib(x: Integer): Longint;
{
  If x>2 Then
    Return(Fib(x-1) + Fib(x-2))
  Else
    Return(1);
};

Procedure Bench();
Const
  NTimes = 10;
  Number = 24;
Var
  i: Integer;
  Value: Longint;
{
  Start();
  Loop(,i:=0,++i,i=NTimes)
    Value := Fib(Number);
  Stop();
  Writeln();
  Writestring("Value: ");
  WriteLongNS(Value);
  Writeln();
};

Procedure _Key(c,mods: integer);
{
  Bench();
};

Procedure _Init();
{
  Bench();
};


(*__________________________INTERFACE___________________________*)


Program Interface;

(*
      74.20 seconds -- Non-Register
   
   File Size:    696 bytes
*)

Uses    __ToolTraps,
(*$U+*) uToolIntf;

Link  __BenchMarkLib, __IO, __ExtendIO :;


Procedure Bench();
Const
  Count = 10000;
Var
  i, M1,M2    : Integer;
  Bool1,Bool2 : Boolean;
  ER1,ER2     : EventRecord;
{
  Start();
  
  M1 := -1;
  M2 := -1;  
  Loop(,i:=0,++i,i=Count) {
    Bool1 := GetNextEvent(M1,@ER1);
    Bool2 := GetNextEvent(m2,@ER2);
    
    Bool1 := GetNextEvent(M1,@ER1);
    Bool2 := GetNextEvent(m2,@ER2);
    
    Bool1 := GetNextEvent(M1,@ER1);
    Bool2 := GetNextEvent(m2,@ER2);
    
    Bool1 := GetNextEvent(M1,@ER1);
    Bool2 := GetNextEvent(m2,@ER2); 
    };
  
  Stop();
};

Procedure _Key(c,mods: integer);
{
  Bench();
};



(*___________________________ __BENCHMARKLIB __________________________*)


Program __BenchMarkLib;

Uses     __ToolBox;

Link __IO, __ExtendIO :;

Var
  MTime: Longint;
  TTime: Longint;
  
Proc Start();
{
  Writeln();
  WriteString("Starting Timers...");
  TTime:= -TickCount();
  ResetTimer();
};

Proc Stop();
Var t: integer;
{
  TimerVal(@MTime);
  TTime += TickCount();
  Writeln();
  WriteString("  Milliseconds: ");
  Writelongns(MTime);
  Writeln();
  WriteString("  Seconds: ");
  Writeintns(TTime/60);
  WriteChar('.');
  T := ((TTime mod 60)*100)/60;
  If T<10 Then WriteChar('0');
  Writeintns(T);
  Writeln();
  Writeln();
};


(*___________________________BENCH__________________________*)


Program Bench;

(*
      seconds -- Non-Register
      seconds -- Register
   
   File Size:    bytes
*)

Link  __BenchMarkLib, __IO, __ExtendIO :;


Procedure Bench();
{
  Start();
  Stop();
};

Procedure _Key(c,mods: integer);
{
  Bench();
};

Procedure _Init();
{
  Bench();
};



End of Article...