[comp.arch] Common Compilers for benchmarks

shs@aldebaran.Berkeley.EDU (Steve Schoettler) (10/31/88)

In article <7352@wright.mips.COM> earl@mips.COM (Earl Killian) writes:
>I think it would be interesting to benchmark various different
>machines using gcc as the compiler.  This partially removes one
>variable: how much performance is due to the compiler and how much to
>the hardware.

That might get you somewhere, but I don't think it will what you
want to find out about the architectures, and it won't completely
remove the compiler as a variable.

Consider that gcc compiles down into an intermediate RTL description
of the original C code.  The RTL describes an idealized abstract machine,
and is designed to be abstract enough so that it can be mapped into
a variety of processors: 68K,386,370, etc.  How efficiently this RTL
description is compiled into the target machine code reflects how
close the abstract machine is to the actual target machine.

So, I think what you'll find from such a study is which machine most
closely resembles the abstract machine Richard Stallman et al 
had in mind when the RTL was designed.

I am currently working on some silicon compiler software that is
designed to give instruction set designers rapid feedback about
design choices.  One thing I have been considering is automatically
generating a gcc machine description file, which can be used to generate
a C compiler, which can be used to compile and run benchmarks. 
In other words, with the right software, you should be able to input
an instruction set and get out dhrystones (etc).

If you used only results from these C benchmarks to design your processor,
I believe that you would, after a few iterations, converge on an
instruction set that very closely matched the intermediate abstract
machine used by gcc.  

Maybe that's not such a bad idea.  Perhaps the best instruction set for
the GNU C compiler is one that uses as instructions the same basic
blocks that are described by the RTL.  Of course, you'd have to do 
something about the infinite number of registers, etc, but then
you'd have a fantastic C compiler for it!

But what about the tradeoffs and assumptions made in the design of
the abstract machine?   Certainly there were intended source languages,
and intended target machines, so how ideal is it?
Let me know if you have any answers.

I believe that using gcc for benchmarks across machines is a good idea,
as it is sort of a "constant" and removes some of the variation we do
see across machines.  I just don't know how meaningful the results
will be.

For a demonstration of how much compiler technology affects "performance",
I'd ask the MIPS folks what the difference is between typical unix code
compiled with no optimizations and with all optimizations turned on.

Steve
shs@ji.Berkeley.EDU

ok@quintus.uucp (Richard A. O'Keefe) (10/31/88)

In article <26627@ucbvax.BERKELEY.EDU> shs@ji.Berkeley.EDU (Steve Schoettler) writes:
>In article <7352@wright.mips.COM> earl@mips.COM (Earl Killian) writes:
>>I think it would be interesting to benchmark various different
>>machines using gcc as the compiler.

>So, I think what you'll find from such a study is which machine most
>closely resembles the abstract machine Richard Stallman et al 
>had in mind when the RTL was designed.

It looks to me as though many of the newer machines are designed with
C and Fortran *only* in mind.  That's just a comment from someone who
reads the manuals:  could anyone who really knows say to what extent
ADA and COBOL were studied _when the arhictectures were being designed_?
(And no, I am not suggesting that RISCs should have special EDIT
instructions like the /370, B6700, or VAX.  I just don't see discussion
in the manuals along the lines of here are the pieces you need for the
following ADA or COBOL constructs.)

fotland@hpihoah.HP.COM (Dave Fotland) (11/02/88)

At HP we use the precision architecture for machines running MPE, a
commercial operating system, as well as Unix.  We studied the impact on
COBOL and database manager code extensively when we were designing the
architecture.  Support includes:

Decimal Correct (DCOR and IDCOR) to correct 8 decimal digits in one cycle
	when doing decimal arithmetic.  This allows decimal arithmetic
	at 2 instructions per 8 digits (an add and a correct).

Store bytes (STBYS) This implements the beginning and end cases in byte
	string moves, storing zero to 4 bytes depending on the alignment
	of the address and whether the beginning case or end case is
	specified.  Fast byte string moves are very important in
	commercial applications.

Unit instructions and conditions.  For example, Unit Xor with the condition
	Some Digit Zero can do a very fast BCD string search (8 digits
	per instruction).  Specifying Some Byte Zero gets you a fast
	charcter string search.

Add and trap on overflow (ADDO) supports fast range checking.

The 3/87 HP Journal and the 1/86 HP Journal have articles on how these
instructions are used by the compilers.  COBOL runs extremely well on
HP-PA.  Most benchmarks shown in the article run faster on an HP-PA
machine than on a CISC machine with the same MIPS rating.

-David Fotland

(The HP-PA instruction set manual is part number 09740-90014)

steve@obed.uucp (stephen Samuel) (11/03/88)

In article <609@quintus.UUCP>, ok@quintus.uucp (Richard A. O'Keefe) writes:
> In article <26627@ucbvax.BERKELEY.EDU> shs@ji.Berkeley.EDU (Steve Schoettler) writes:
> >In article <7352@wright.mips.COM> earl@mips.COM (Earl Killian) writes:
> >>machines using gcc as the compiler.
> 
> It looks to me as though many of the newer machines are designed with
> C and Fortran *only* in mind.  That's just a comment from someone who
> reads the manuals:  could anyone who really knows say to what extent
> ADA and COBOL were studied _when the arhictectures were being designed_?

I don't know about ADA, but my feeling is that the people who really
worry about COBOL are a bit more worried about moving their code over
in one piece (I mean, who wants to wrestle with 4.5MB worth of 
   ADD A TO B GIVING C.  <<- Don't forget the period!!!
(guh!)
Most of these people tend to stay on their nice big mainframes with
hords of I/O bandwidth and a compiler they know and love.

Most of the people who are interested in the newer machines are either
scientific types who tend to avoid COBOL or business people who believe
that MIPS is all that it takes to make a good machine (snicker, snicker).

-- 
Stephen samuel !alberta!{obed,edm}!steve
Look on the bright side... It might have worked!

csimmons@hqpyr1.oracle.UUCP (Charles Simmons) (11/03/88)

In article <26627@ucbvax.BERKELEY.EDU> shs@ji.Berkeley.EDU (Steve Schoettler) writes:
>In article <7352@wright.mips.COM> earl@mips.COM (Earl Killian) writes:
>>I think it would be interesting to benchmark various different
>>machines using gcc as the compiler.  This partially removes one
>>variable: how much performance is due to the compiler and how much to
>>the hardware.
>
>Consider that gcc compiles down into an intermediate RTL description
>of the original C code.  The RTL describes an idealized abstract machine,
>and is designed to be abstract enough so that it can be mapped into
>a variety of processors: 68K,386,370, etc.  How efficiently this RTL
>description is compiled into the target machine code reflects how
>close the abstract machine is to the actual target machine.

Um...  Seeing how I've worked on the GCC 370 compiler, I'd argue
with this point of view.  One of the really neat aspects of GCC is
that you can, in some sense, generate machine-dependent RTL code.

For example, the original RTL code doesn't have any support at all
for double word integers (or 'long long's).  But, in an early pass
of the compiler, the compiler calls a machine dependent routine to
generate intermediate RTL code.  By performing this pass, the generated
intermediate RTL code tends to provide an abstract machine that very closely
mimics the target machine.

>So, I think what you'll find from such a study is which machine most
>closely resembles the abstract machine Richard Stallman et al 
>had in mind when the RTL was designed.

There are, of course, other issues.  The 370 has various aspects
that make writing a compiler for it difficult.  In particular,
allowing subroutines that require more than 4K bytes of instructions
is somewhat tricky; and the fact that the 370 doesn't have negative
offsets makes it difficult to implement a stack on the 370 that
is compatible with the type of stack that GCC would like to implement.
(I guess what I'm saying here is that GCC does contain an abstract
machine that doesn't map real well onto the 370, and the abstract
machine is partially described by portions of the compiler that
have nothing to do with the RTL code.)

Of course, even using GCC to run all benchmarks still leaves you
open to differences in the skill of a compiler writer.  An untuned
implementation of GCC might not contain as many peephole optimizations
as would be desirable, or one of the instructions in the instruction
set may not have been described.  On the other hand, if you give me
a benchmark, it becomes relatively easy to tune GCC to run that particular
benchmark quickly.

>Steve

-- Chuck

ok@quintus.uucp (Richard A. O'Keefe) (11/04/88)

In article <1791@pembina.UUCP> steve@obed.uucp (stephen Samuel) writes:
>In article <609@quintus.UUCP>, ok@quintus.uucp (Richard A. O'Keefe) writes:
>> In article <26627@ucbvax.BERKELEY.EDU> shs@ji.Berkeley.EDU (Steve Schoettler) writes:
>> It looks to me as though many of the newer machines are designed with
>> C and Fortran *only* in mind.  That's just a comment from someone who
>> reads the manuals:  could anyone who really knows say to what extent
>> ADA and COBOL were studied _when the arhictectures were being designed_?
>
>I don't know about ADA, but my feeling is that the people who really
>worry about COBOL are a bit more worried about moving their code over
>in one piece
>Most of these people tend to stay on their nice big mainframes with
>hords[sic] of I/O bandwidth and a compiler they know and love.

Well, that's not strictly true.  There are quite a few companies that
make a living translating one dialect of COBOL into another.  (Amongst
other things, the COBOL standard keeps changing.)  Moving from an IBM
mainframe to a RISC machine such as the MC88000 shouldn't be any more
of a headache than moving from one mainframe to another, assuming the
COBOL compiler writer has done a decent job.  I have one hand, while
I am typing this sentence, on IBM's "System Applications Architecture
Common Programming Interface COBOL Reference, document SC26-4354-0",
which gives a synoptic view of COBOL under MVS, VM, and ***OS/2***.
And one keeps seeing requests in comp.sources.wanted "how can I get a
COBOL compiler for UNIX machine foobaz, or a COBOL to C translator,
or ...".

Don't misread my earlier posting as saying that RISCs are not suited
to COBOL.  For example, converting an 18-digit DISPLAY number to
64-bit COMPUTATIONAL form can be done _rather_ fast by a decent RISC
with no multiplies at all, so a COBOL compiler should be able to hack
text->binary by generating a little special-purpose subroutine for
each different PICTURE.  (Though I have yet to see a RISC manual which
bothers to explain how.)  What do you do going the other way?

It's not the text->binary conversion, or the requirement for 18 digits
at least (64 bits is enough), or PICTUREs, or even the more exotic
features of the PERFORM statement which make me think that RISC
designers have ignored COBOL, but ON SIZE ERROR.  [It is not good
enough to generate traps; UNIX takes them away from you.]

yuval@taux02.UUCP (Gideon Yuval) (11/05/88)

In article <626@quintus.UUCP> ok@quintus.UUCP (Richard A. O'Keefe) writes:
>with no multiplies at all, so a COBOL compiler should be able to hack
>text->binary by generating a little special-purpose subroutine for
>each different PICTURE.  (Though I have yet to see a RISC manual which
>bothers to explain how.)  

I never saw such a manual either. Could the relevant juju be posted on netnews?

Thanks



-- 
Gideon Yuval, yuval@taux01.nsc.com, +972-2-690992 (home) ,-52-522255(work)
 Paper-mail: National Semiconductor, 6 Maskit St., Herzliyah, Israel
                                                TWX: 33691, fax: +972-52-558322

aglew@urbsdc.Urbana.Gould.COM (11/06/88)

>It looks to me as though many of the newer machines are designed with
>C and Fortran *only* in mind.  That's just a comment from someone who
>reads the manuals:  could anyone who really knows say to what extent
>ADA and COBOL were studied _when the arhictectures were being designed_?
>(And no, I am not suggesting that RISCs should have special EDIT
>instructions like the /370, B6700, or VAX.  I just don't see discussion
>in the manuals along the lines of here are the pieces you need for the
>following ADA or COBOL constructs.)

Well, I know of one company that did a study of what would be necessary
to support COBOL well on a hypothetical 64 bit RISC. (No, it's not
my current my employer, nor was it the last machine my last employer
built).

The guy who did the study was ex-NCR, and pretty familiar
with commercial and TP issues.

His conclusions? No need for special EDITPC instructions. BCD might
have been good, but arithmetic on BCD treated as binary could be done
and corrected fast enough. But the biggest win was simply that
most of the commercial data items, strings et al., fit into a 64 bit
register, and so could be manipulated all at once. Fixed field length
COBOL records can be handled quite naturally by a vector architecture
that wants things to be fixed length - better than C variable length
strings.

Overall: a large word size scientific processor is a *damned* good
COBOL engine, if you can spend the money to get a good COBOL 
compiler written, and if you have the I/O and O/S facilities.


Andy "Krazy" Glew. 
at: Motorola Microcomputer Division, Champaign-Urbana Development Center
    (formerly Gould CSD Urbana Software Development Center).
mail: 1101 E. University, Urbana, Illinois 61801, USA.
email: (Gould addresses will persist for a while)
    aglew@gould.com     	    - preferred, if you have MX records
    aglew@fang.gould.com     	    - if you don't
    ...!uunet!uiucuxc!ccvaxa!aglew  - paths may still be the only way
   
My opinions are my own, and are not the opinions of my employer, or any
other organisation. I indicate my company only so that the reader may
account for any possible bias I may have towards our products.

PS. I promise to shorten this .signature soon.

daveb@geaclib.UUCP (David Collier-Brown) (11/08/88)

|  In article <626@quintus.UUCP> ok@quintus.UUCP (Richard A. O'Keefe) writes:
| with no multiplies at all, so a COBOL compiler should be able to hack
| text->binary by generating a little special-purpose subroutine for
| each different PICTURE.  (Though I have yet to see a RISC manual which
| bothers to explain how.)  

  Or you could use a mini-interpreter: generate a vector of codes
for each character in the picture (8 bits is enough, 6 used to be
sufficient), and let them reference the entry points to the
corresponding routines. (A technique also used to build some of the
CISC's edit-via-tally instructions).
  Say, pic (999v.9) becomes { 'v4', '%d', '%d', '%d', '.', '%d' }, or
whatever applies, and the convert-to-decimal subroutine uses it as a
pattern for producing character output. (the leading v4 is an indicator
about where to put the logical decimal-point).
  A friend once did this kind of thing in ratfor, as a fast,
fixed-format replacement for printf/format statements.

--dave
-- 
 David Collier-Brown.  | yunexus!lethe!dave
 Interleaf Canada Inc. |
 1550 Enterprise Rd.   | HE's so smart he's dumb.
 Mississauga, Ontario  |       --Joyce C-B

anand@amax.npac.syr.edu (Anand Rangachari) (11/08/88)

In article <474@oracle.UUCP> csimmons@oracle.UUCP (Charles Simmons) writes:
[...]
>Of course, even using GCC to run all benchmarks still leaves you
>open to differences in the skill of a compiler writer.  An untuned
>implementation of GCC might not contain as many peephole optimizations
>as would be desirable, or one of the instructions in the instruction
>set may not have been described.  On the other hand, if you give me
>a benchmark, it becomes relatively easy to tune GCC to run that particular
>benchmark quickly.

  I was just wondering if that was such a bad thing after all. After all, a
benchmark is supposed to be representative of the typical programs a user
may want to run. Thus in improving the speed of a benchmark, you may 
actually improve the speed of a sizeable number of programs.

  An excellent argument against this is of course is that we dont have such 
benchmarks available (So I have gathered from the discussions on this
group).

                                                R. Anand
Internet: anand@amax.npac.syr.edu
Bitnet:   ranand@sunrise

rick@svedberg.bcm.tmc.edu (Richard H. Miller) (11/26/88)

In article <1791@pembina.UUCP>, steve@obed.uucp (stephen Samuel) writes:
> 
> I don't know about ADA, but my feeling is that the people who really
> worry about COBOL are a bit more worried about moving their code over
> in one piece (I mean, who wants to wrestle with 4.5MB worth of 
>    ADD A TO B GIVING C.  <<- Don't forget the period!!!
> (guh!)
> Most of these people tend to stay on their nice big mainframes with
> hords of I/O bandwidth and a compiler they know and love.
> 
> Most of the people who are interested in the newer machines are either
> scientific types who tend to avoid COBOL or business people who believe
> that MIPS is all that it takes to make a good machine (snicker, snicker).
 

Your prejudices are showing. As a manager of technical support for a COBOL
shop, I can assure you that we DO look for architectual support for COBOL when
we are trying to acquire new hardware. A vendor that does provide support for
business DP is a plus. We also tend to ignore (and have for a LONG time) MIPS
as a standard of comparison in anything but the grossest measurement. WE
develop specific benchmarks for our applications when we are looking for a
vendor and hold them to it.





Richard H. Miller                 Email: rick@bcm.tmc.edu
Asst. Dir. for Technical Support  Voice: (713)799-4511
Baylor College of Medicine        US Mail: One Baylor Plaza, 302H
                                           Houston, Texas 77030

henry@utzoo.uucp (Henry Spencer) (11/27/88)

In article <1347@gazette.bcm.tmc.edu> rick@svedberg.bcm.tmc.edu (Richard H. Miller) writes:
>In article <1791@pembina.UUCP>, steve@obed.uucp (stephen Samuel) writes:
>> ...Most of the people who are interested in the newer machines are either
>> scientific types who tend to avoid COBOL or business people who believe
>> that MIPS is all that it takes to make a good machine (snicker, snicker).
> 
>Your prejudices are showing...

They are indeed.  Note that MIPSco has invested significant effort in making
good COBOL, PL/I, and Ada compilers available on its machines.  (Optimizing
Ada is lots of fun, I'm told.)  Since MIPSco seems to run simulations of
the alternatives before deciding when to have lunch :-), I doubt that
they did this on a whim.
-- 
Sendmail is a bug,             |     Henry Spencer at U of Toronto Zoology
not a feature.                 | uunet!attcan!utzoo!henry henry@zoo.toronto.edu