CCHD@lure.latrobe.edu.au (Huw Davies - La Trobe University Computer Centre) (10/04/90)
I have just got a copy of the September 1990 SPECwatch and I am a bit concerned about the following paragraph: "There also seems to be a problem with replicating IBM's RS6000 SPECmark results, and with achieving the expected levels of performance with other code. It's known that IBM extensively modified the compilers used to compile the benchmarks. If these "knobs and dials" turn out to be not readily accessible to users of the production compilers shipped with the systems, SPEC will be faced with its first serious cheating problem. The usual prize (a trial subscription or four month extension of an existing subscription) for the first person to provide independent RS/6000 SPECmark results using the compilers shipped with the products." Would anyone from IBM (or anywhere else for that matter) like to comment? Has anyone with an RS/6000 and the SPEC benchmark tape tried/succeeded in running them? I have a 530 on site and expect two 540's this month. I'd be happy to run the benchmarks on an idle machine but I don't have access to the tape. Huw Davies Computing Services La Trobe University Melbourne Australia
madd@world.std.com (jim frost) (10/04/90)
CCHD@lure.latrobe.edu.au (Huw Davies - La Trobe University Computer Centre) writes: >I have just got a copy of the September 1990 SPECwatch and I am >a bit concerned about the following paragraph: I was wondering when that would come out. Given the stability of the system when IBM announced the SPECmarks, and given that the performance today is still not too good, I doubted the results. On pure benchmarks my system hits what IBM says it should right on the nose or close enough not to matter. But when it comes to running actual applications the thing is much slower than expected, usually slower feeling than a SparcStation and in compiles it's often slower than our microVAX. I'd like to know what kind of tuning they did to get their numbers so I could do it to my machine! jim frost saber software jimf@saber.com
abe@mace.cc.purdue.edu (Vic Abell) (10/06/90)
In article <4733@lure.latrobe.edu.au>, CCHD@lure.latrobe.edu.au (Huw Davies - La Trobe University Computer Centre) writes: > I have just got a copy of the September 1990 SPECwatch and I am > a bit concerned about the following paragraph: > > "There also seems to be a problem with replicating IBM's RS6000 > SPECmark results, and with achieving the expected levels of > performance with other code. While I haven't had a chance today to run the entire SPEC 1.0 suite on my RS/6000-520 (GA code), I did run the following benchmarks from the suite. 008.espresso 013.spice2g6 023.eqntott 030.matrix300 047.tomcatv The SPEC ratios I got differ from IBM's by no more than 0.4. I got the same result for one test; one of my results was higher by 0.3, another by 0.4; and two were lower by 0.3. (I think SPEC reporting rules keep me from publishing an incomplete report.) Vic Abell Assistant Director Purdue University Computing Center SPEC license 310
abe@mentor.cc.purdue.edu (Vic Abell) (10/07/90)
In article <4733@lure.latrobe.edu.au>, CCHD@lure.latrobe.edu.au (Huw Davies - La Trobe University Computer Centre) writes: > I have just got a copy of the September 1990 SPECwatch and I am > a bit concerned about the following paragraph: > > "There also seems to be a problem with replicating IBM's RS6000 > SPECmark results, I have been able to reproduce the SPEC ratio for the RS/6000-520 with only minor changes needed to enable gcc. (See the Notes/Summary of Changes section in the report for gcc details.) Here is a complete SPEC report for the RISC System/6000, model 520 POWERserver. Its SPEC ratio is 0.1 larger than what IBM reported to me. SPEC Throughput Method A# Results for Release 1.0 Benchmarks Results: SPEC IBM RS/6000 IBM Corporation Ref. 520 Benchmark Time Time SPEC IBM RISC System/6000 No. & Name (sec.) (sec.) Ratio POWERserver 520 001.gcc 1482 102 14.5* Hardware 008.espresso 2266 139 16.3 Model Number: POWERserver 520 013.spice2g6 23951 1189 20.1 CPU: IBM POWER 20MHz 015.doduc 1863 88 21.2 FPU: Integrated 020.nasa7 20093 771 26.1 Number of CPU's: 1 022.li 6206 398 15.6 Cache Size/CPU: 32 KB data, 8 KB ins. 023.eqntott 1101 60 18.4 Memory: 32 MB 030.matrix300 4525 263 17.2 Disk Subsystem: 2-857 MB SCSI 042.fpppp 3038 71 42.8 Network Interface: 1 Ethernet Controller 047.tomcatv 2649 47 56.4 Geometric Mean 3867.7 173.0 22.4* Software System O/S Type and Rev: AIX 3.1, GA Tuning Parameters: None in use Compiler Rev: XL Fortran 1.1 Background Load: Normal Unix daemons Other Software: XL C 1.1 System State: Multi-user, lightly File System Type: IBM Journaled loaded File System Firmware Level: N/A Tested in: October 1990 By: Victor A. Abell <abe@mace.cc.purdue.edu> Purdue University Computing Center Mathematical Sciences Building West Lafayette, IN 47907 (317) 494-1787 SPEC License # 310 Notes/Summary of Changes: # Method A: Homogeneous Load * Portability changes were required: gcc: o Modified alloca.o rule in Makefile to use XL C compiler. o Activated #include of <time.h> in cccp.c. o Disabled redefinition of ptrdiff_t and size_t in stddef.h. o Modified erroneous assignment of enumerated variables to bit fields in rtl.h, tree.h, and varasm.c. o Enabled vfork to fork redefinition in gcc.c. Copyright 1990 Purdue Research Foundation, West Lafayette, Indiana 47907. All rights reserved.