[comp.lang.eiffel] Eiffel/C++/Smalltalk `Benchmark' is COMPLETELY INVALID

cline@cheetah.ece.clarkson.edu (Marshall Cline) (03/30/90)

In article <274@eiffel.UUCP>, kimr@eiffel.UUCP (Kim Rochat) concludes:
> any perception that C++ has superior performance to Eiffel may
> be invalid.

I enjoy solid technical articles, but unfortunately this was neither.
I claim the conclusions the authors draw are Completely Invalid:

(1) The algorithm is implemented DIFFERENTLY in C++ as in Eiffel.
In particular, the authors ADDED FUNCTION CALLS to the C++ version!
All the lowest level (highest call rate!) actions are non-inlined
functions in C++, but they aren't function calls at all in Eiffel.

(2) The `benchmark' problem (`mosiac') doesn't exercise ANY of the OO
features of the languages.  The authors even admit this, but imply it
is a `feature' -- they claim it compares how the compilers handle
array dereferencing -- but see the next point.

(3) The bottleneck for mosiac is `random()', a common library function.
Thus `mosiac' is a TERRIBLE benchmark -- it might be a valid benchmark
for random() implementations, but certainly NOT for OOP.

(4) The I/O was stripped, supposedly to avoid biasing results due to
I/O on a workstation.  But they compared Sun with Sun, so that makes
no sense.  (I wonder how Eiffel and C++ compare in their I/O?  Anyone
in netland with Eiffel/g++/Cfront want to do it?)

CONCLUSIONS: Mosaic is just a big array calculation with lots of
random numbers, hardly comparing the speed of Eiffel and C++ code.
It's a pathetic benchmark, demonstrating Nothing about the languages
in question since most of its time was spent inside a "C" library
routine.

Marshall Cline
--
===============================================================================
Marshall Cline/ECE Department/Clarkson University/Potsdam NY 13676/315-268-3868
 cline@sun.soe.clarkson.edu, bitnet: BH0W@CLUTX, uunet!clutx.clarkson.edu!bh0w
  Career search in progress: ECE faculty. Research oriented. Will send vitae.
===============================================================================