chase@Ozona.orc.olivetti.com (David Chase) (10/14/88)
In article <1710@eos.UUCP> eugene@eos.UUCP (Eugene Miya) writes: > I discovered it was an >APL compiler not a Fortran compiler. The benchmark was a simple >Gaussian sum (3 APL characters). The benchmark adds 1 thru n, the compiler >did what Gauss did: you know n(n+1)/2. It was placed there by the compiler >writer who knew the person in the APL community who did this as a >benchmark. Actually, some people (Perlis; Cheng, Omdahl, & Strawn) have proposed the use of "idiom recognition" in APL compilation. It is claimed that most APL programs are can be covered by a fairly small set (50 or 100) of APL idioms, which can in turn be specially compiled. I don't know what ever happened along those lines -- Cheng et al had problems with slow table generation, but (plug for own work) new techniques are several hundred times faster. Anyhow, given such compilation technology it is fairly easy to recognize small stupid benchmarks and deal with them appropriately. If it were me (and I was using pattern-directed compilation) I'd do it in a flash; it's an easy way to part fools (those who rely on simple benchmarks) and their dollars. You don't win by being too clever, either. I can imagine a diabolical benchmarker writing code full of multiple-entry (irreducible) loops, or code that forces all variables into memory; unfortunately, such code is not the usual case, and so such pathological benchmarks tell you nothing about the everyday performance of a compiler + architecture. David
prins@cs.unc.edu (Jan Prins) (10/15/88)
> In article <1710@eos.UUCP> eugene@eos.UUCP (Eugene Miya) writes: >> I discovered it was an >>APL compiler not a Fortran compiler. The benchmark was a simple >>Gaussian sum (3 APL characters). The benchmark adds 1 thru n, the compiler >>did what Gauss did: you know n(n+1)/2. It was placed there by the compiler >>writer who knew the person in the APL community who did this as a >>benchmark. I don't know the details of this case, but "benchmark pandering" is not neccessarily implicated here. In several APL interpreters (less likely with compilers) vectors of values in an arithmetic progression are represented by generators (e.g. first, increment, length). This makes sense because arithmetic progressions are closed under several common operations (addition, multiplication by scalar) and are often used as an index for array slicing whose implementation can be substantially improved for this case. Consequently when a (sum) reduction is applied across a vector, vectors in this representation must be accomodated. A slight generalization of Gauss' observation governs the sum of any arithmetic progression, and is the natural candidate to use in this particular evaluation. No "special case" examination of the source is necessary to lead to this choice. I would also think that anyone benchmarking APL systems would be aware of this strategy, and would be expecting (testing?) this behavior in some systems, but, again, I don't know the details. Even though this example need not involve recognition of a particular idiom, I agree with David Chase and his citations that such a thing might be potentially useful: In article <30653@oliveb.olivetti.com>, chase@Ozona.orc.olivetti.com writes: > Actually, some people (Perlis; Cheng, Omdahl, & Strawn) have proposed > the use of "idiom recognition" in APL compilation. It is claimed that > most APL programs are can be covered by a fairly small set (50 or 100) of > APL idioms, which can in turn be specially compiled. Inasmuch as such idioms really represent typical program constructs, it seems fair to include them in benchmarks and analyze their performance. Summing arithmetic progression vectors is probably not very typical. Jan Prins (prins@cs.unc.edu)