tamir@CS.UCLA.EDU (07/30/87)
I would like to find out which machines are currently
available on the market whose performance is significantly
better than a Sun-4 (>> "10 MIPS") but whose cost is
significantly lower than a Cray-2 (<= $1M).
I am *NOT* interested in multiprocessors, such as Sequent,
where the performance of each processor is less than
or equal to a Sun-4.
I am particularly (but not exclusively) interested in high performance
*integer/string* operations as opposed to vectorized floating point.
Are there any such machines available on the market ?
Yuval Tamir
Internet: tamir@cs.ucla.edu
UUCP: ...!{ihnp4,ucbvax,sdcrdcf,trwspp,randvax,ism780}!ucla-cs!tamirmash@mips.UUCP (John Mashey) (07/30/87)
In article <7500@shemp.UCLA.EDU> tamir@CS.UCLA.EDU (Yuval Tamir) writes: >I would like to find out which machines are currently >available on the market whose performance is significantly >better than a Sun-4 (>> "10 MIPS") but whose cost is >significantly lower than a Cray-2 (<= $1M).... >I am particularly (but not exclusively) interested in high performance >*integer/string* operations as opposed to vectorized floating point. 1) As a side note, we're not sure where the 10-mips number for the Sun-4 came from. It looks more like 6.5-7.0 mips integer (on the VAX 11/780 == 1 ... VAX 8700 == 6 scale). We got this from looking carefully at the published data, filling in some holes in the tables, and cross-comparing with other RISC systems. [We've got a 40-pager that analyzes the data in detail, if anybody is interested.] 2) It would be helpful to know some more about the kinds of applications you want to run. Does it include scalar floating point, or no FP at all? Performance ratios and cost/performance ratios can vary all over the map according to the sorts of programs you want to run. Even integer versus string performance can vary wildly. 3) If you're looking for [for example] real 20-mips uniprocessors, this year, the only things that do that are real mainframes, like Amdahls & such, and the costs rapidly get up in the $1M ballpark or significantly over. It won't be for another year that you can get real 20-mips micros. Structurally, the computer business has tended NOT to have ultra-high-performance integer engines that aren't also, or at least capable of being, serious about floating point. -- -john mashey DISCLAIMER: <generic disclaimer, I speak for me only, etc> UUCP: {decvax,ucbvax,ihnp4}!decwrl!mips!mash OR mash@mips.com DDD: 408-991-0253 or 408-720-1700, x253 USPS: MIPS Computer Systems, 930 E. Arques, Sunnyvale, CA 94086
maiden@sdcsvax.UCSD.EDU (VLSI Layout Project) (08/17/87)
I was wondering if anybody from BBN or a qualified user could
mail or post (preferable mail) information about the BBN
Butterfly performance specifications (fully loaded system).
These specifications would ideally include "best case" parallel
FP performance as well as typical case FP and integer specs.
Thank you very much.
--
Edward K. Y. Jung
The Deep Thought Group: Searching for a better way to think.
UUCP: {seismo|decwrl}!sdcsvax!maiden
ARPA: maiden@beowulf.ucsd.eduwhm@arizona.edu (Bill Mitchell) (08/19/87)
A couple of weeks ago in article <552@winchester.UUCP>, mash@mips.UUCP (John Mashey) wrote: > 1) As a side note, we're not sure where the 10-mips number > for the Sun-4 came from. It looks more like 6.5-7.0 mips integer > (on the VAX 11/780 == 1 ... VAX 8700 == 6 scale). > We got this from looking carefully at the published data, > filling in some holes in the tables, and cross-comparing with other > RISC systems. [We've got a 40-pager that analyzes the data in detail, > if anybody is interested.] We ran our IBS (Inscrutable Benchmark Suite) on a Sun-4/260 and ... [Before I say what we found, I must point out that Sun said that the machine that the tests were run on was (1) a prototype (2) running an early version of the software (3) had a 65ns rather than 60ns clock (4) had something wrong with the floating point (5) several other things of this ilk that amount to the suite being run under adverse conditions.] ... found that at the cpu level the Sun-4 was about 6 (six) mips. The same tests had been run on an 8600 and a Sun-3/280 and we got around 4 mips for both of these machines. I've yet to see a non-floating point benchmark that shows 10-mip speed on a Sun-4, using VAXs as a yardstick. Bill Mitchell whm@arizona.edu {allegra,cmcl2,ihnp4,noao}!arizona!whm
roy@phri.UUCP (Roy Smith) (08/21/87)
In article <552@winchester.UUCP> mash@winchester.UUCP (John Mashey) writes: > We got this [[7 or so mips rating for Sun-4 as opposed to the 10 mips Sun > claims -- RHS]] from looking carefully at the published data, filling in > some holes in the tables, and cross-comparing with other RISC systems. > [We've got a 40-pager that analyzes the data in detail, if anybody is > interested.] John send me that 40-pager ("A Sun-4 Benchmark Analysis", July 23, 1987) at my request, and I've got a few comments. First off, is I have learned to take with a grain of salt *anything* a computer manufacturer says about any other manufacturer's products. With this disclaimer out of the way, the report seems to make a lot of sense, at least on the surface. The problem is that Sun claims 10 mips performance for the Sun-4, while MIPS insists it's more like 7 mips. That's a 30% difference, which in my book just isn't that much for this type of calculation. Also, I don't like the performance graphs in the MIPS report. The graphs have "Performance" on the Y-axis and "Benchmark Type" on the X-axis, with lines connecting the data points. This is wrong, since it indicates that "Benchmark Type" is a continious function, and implies that you can interpolate between the data points to find values for intermediate types of benchmarks. These graphs should have been bar graphs, or scatter plots, or some other type of plot which is more suitable to discrete data. BTW, the graphs look like they were done using CricketGraph on a Mac. Very pleasing to the eye, even if they are misleading. :-) One point made in the report, which I feel is vitally important, is that floating point speed doesn't scale well with integer speed. For us, floating point is where it's at. I believe the Sun-3/100, 3/200, and 4/200 series machines all use the same FPA, based on the Wietek chip set. What that means is that the 3/100 series has the "best" floating/integer speed ratio, and the 4/200 series the "worst". You get the same kind of non-linearity with the Vax-750/780/785/8600/8[78]00 progression, I think. -- Roy Smith, {allegra,cmcl2,philabs}!phri!roy System Administrator, Public Health Research Institute 455 First Avenue, New AM serinalo A=
roy@phri.UUCP (Roy Smith) (08/23/87)
In article <2866@phri.UUCP> I wrote: > the Sun-3/100, 3/200, and 4/200 series machines all use the same FPA, based > on the Wietek chip set. What that means is that the 3/100 series has the > "best" floating/integer speed ratio, and the 4/200 series the "worst". It has since been pointed out to me that while the first sentence is true, the second one is misleading, if not actually false. Yes, all three machines use the same FPA (not clear if it's actually the same board, or a similar board based on the same chip set), but on the 4/2xx machines, the integer cpu and the FPA are closer coupled than on the 3/2xx's, the 4/2xx has better (presumably this means faster) memory, and better compilers. All this adds up to a 1.5-2X increase in floating point speed. Personally, I think the "better compilers" reason is a crock since with appropriate effort, the 3/2xx could have better compilers too, but I can understand why Sun would want to devote more programmer time to working on the compilers for their new machines than for their old ones. On the other hand, since the Sun-4 series is a RISC machine, it's probably easier to write really good compilers for it (or at least, that's the theory), so maybe I'm being unfair (again :-)). -- Roy Smith, {allegra,cmcl2,philabs}!phri!roy System Administrator, Public Health Research Institute 455 First Avenue, New York, NY 10016
mash@mips.UUCP (John Mashey) (08/23/87)
In article <2866@phri.UUCP> roy@phri.UUCP (Roy Smith) writes: > John send me that 40-pager ... a few comments. First off, is I have >learned to take with a grain of salt *anything* a computer manufacturer >says about any other manufacturer's products... ALWAYS! healthy skepticism is a Good Thing. >...The problem is that Sun claims 10 mips performance for the Sun-4, while >MIPS insists it's more like 7 mips. That's a 30% difference, which in my >book just isn't that much for this type of calculation. Besides the usual issue that single-metric performance metrics are bad, and that mips-numbers only get used because everybody gets forced into it, this comment raises the interesting issue of human perception of differences. To look at the general case, assume that a vendor claims one thing, and reality is something else (assuming the reality is something you can measure, i.e., NOT something like unspecified mips). Here are some interesting questions: 1) Do you think of the difference as ((claim - real)/claim * 100) % ? (i.e., the percentage of the claim that is missing) 2) Do you think of it as ((claim - real)/real * 100) % ? (i.e., the percentage overstatement) 3) For your choice of 1), or 2), at what percentage do you think it's worth bothering thinking that X & Y are different? 4) Do any of you ideas change as mips ratings go up? For example, maybe it's sensible to distinguish between 3.5 and 4 mips (12-14%), but maybe not between 9.5 and 10 (5%), and it seems irrational to distinguish between 59.5 and 60. Obviously, all of these are a matter of opinion. What I'm curious about is how human perception works when applied to computer performance characterization. > Also, I don't like the performance graphs in the MIPS report. The >graphs have "Performance" on the Y-axis and "Benchmark Type" on the X-axis, >with lines connecting the data points. This is wrong, since it indicates >that "Benchmark Type" is a continious function, and implies that you can >interpolate between the data points to find values for intermediate types >of benchmarks. These graphs should have been bar graphs, or scatter plots, >or some other type of plot which is more suitable to discrete data. BTW, >the graphs look like they were done using CricketGraph on a Mac. Very >pleasing to the eye, even if they are misleading. :-) Actually, they're a mixture of Cricket, Excel, and MacDraft. There was absolutely no intention of implying interpolattion between points. The good thing about computer graphics is that you can redraw graphs quickly. The bad thing about it is that you can redraw them OFTEN. We tried numerous different ways to present the data, including various flavors of bar graphs and scatter plots, and the ones we ended up with, somewhat to my surprise, were much easier to understand than the others. Here is the reasoning: 1) The goal is to show performance of machines and benchmarks, with enough of both together to see how they compare on those benchmarks, especially to get overall "performance envelopes". 2) If you use the bar graph that shows one benchmark versus a bunch of machines, you get a good view of that, but you get no idea of overall comparisons. 3) If you use the bar graph that shows several benchmarks versus a bunch of machines, the graph gets cluttered very quickly, and it is much harder to see a-b comparisons. The most complex graph had 42 data points, and it is just incomprehensible if done this way. 4) If you use scatter plots, with no lines connecting them, but little symbols instead, you again find it gets cluttered quickly. I've seen people end up connecting the symbols to figure out what's happening. 5) If you use "stem-and-leaf" charts (which I like, akin to the way I've seen some 8700-780 comparisons done), you get a real good idea of overall performance envelopes, but it's hard to do a-b comparisons for specific benchmarks, and it's hard to do more than about 2 machines at once without really baroque graphics. 6) The line chart makes a few things clear, if you're lucky: a) machine A is always faster than machine B, for the set of benchmarks (the lines never cross). b) machine A is usually faster than B, except on a few benchmarks, which are made obvious by line-crossings. c) Missing data points are obvious [they're not as obvious on most bargraphs]. Point 6c) is important: one of the easiest ways to be mislead by some numbers you get is by having partial sets of data, and getting excited about extreme cases, without looking at overall patterns. For example, if all I saw was Dhrystone and character-pushing, I'd sure think a Cray was a slow machine. [Well, not slow, but not as fast as the price.] For this whole area, I cannot recommend the following book highly enough. In my opinion, it's in a league with "The Elements of Style": Edward R. Tufte, "The Visual Display of Quantitative Information", Graphics Press, Cheshire, Connecticut, 1983. About $35, worth more. -- -john mashey DISCLAIMER: <generic disclaimer, I speak for me only, etc> UUCP: {decvax,ucbvax,ihnp4}!decwrl!mips!mash OR mash@mips.com DDD: 408-991-0253 or 408-720-1700, x253 USPS: MIPS Computer Systems, 930 E. Arques, Sunnyvale, CA 94086
root@hobbes.UUCP (John Plocher) (08/26/87)
+---- John Mashey writes in <616@winchester.UUCP> ---- | 4) Do any of you ideas change as mips ratings go up? For example, | maybe it's sensible to distinguish between 3.5 and 4 mips (12-14%), | but maybe not between 9.5 and 10 (5%), and it seems irrational to | distinguish between 59.5 and 60. +---- Not talking about mips, but clock speeds: IBM PC-AT Clones (no flames, please) are very price competetive, and there is mucho advertizing of clock rates. The current prices for a 10 MHz machine with 0 wait states is about $2000 while machines running at 12 MHz and 1 wait state go for about $2500. Not too bad a difference until you realize that the 10/0 machine runs FASTER than the 12/1 machine! It's sobering to think that people are buying the more expensive one just because of the numbers... I suppose this observation does hand in hand with the $49.99 -vs- $50 price comparisons... :-) Most of *us* wouldn't admit being swayed by the higher number, but isn't a 50 mips machine "better" than a 45 mips one? :-( -- John Plocher uwvax!geowhiz!uwspan!plocher plocher%uwspan.UUCP@uwvax.CS.WISC.EDU
eugene@pioneer.arpa (Eugene Miya N.) (08/26/87)
In article <198@hobbes.UUCP> root@hobbes.UUCP (John Plocher) writes: >but isn't a 50 mips machine "better" than a 45 mips one? :-( Ah! wait until the Multiflow Trace machine gets wider circulation! From the Rock of Ages Home for Retired Hackers: --eugene miya NASA Ames Research Center eugene@ames-aurora.ARPA "You trust the `reply' command with all those different mailers out there?" "Send mail, avoid follow-ups. If enough, I'll summarize." {hplabs,hao,ihnp4,decwrl,allegra,tektronix,menlo70}!ames!aurora!eugene