hedrick@geneva.rutgers.edu (Charles Hedrick) (09/26/89)
In the Unix marketplace, MIPS normally means performance that many times that of a VAX 11/780, which is essentially the same as a MicroVAX 1. This led to talk of gold-plating a 780 and giving it to the NBS as the Offical MIP. MIPS are normally evaluated by trying a set of benchmarks on both a VAX and the machine involved. (I suspect that these days they probably do the VAX tests on a newer VAX and then divide by DEC's MVUP rating. MVUP is performance related to the MicroVAX 1.) Thus MIPS is not an actual count of instructions per second. Of course this only gives you a ballpark idea about a machine, since the ratio will be different for different tasks. In general it seems like RISC machines tend to perform somewhat better on simple tests than in reality. MIPS will also be different for different operating systems, so a VAX 8800 running VMS may be faster than a Sun that has the same number of "MIPS". Sun's official rating for the Sun 4/280 was 10 MIPS. That was based on some early tests, but no real user experience. I think many people now believe that for real tasks a more typical number is 8 MIPS, with 6 MIPS for Fortran (both integer and floating point). Presumably Sun is keeping its definition of MIPS constant, so you might want to multiply figures for their newer systems by the same .8 fudge factor. As far as I can tell "RISC MIPS" is used mostly by IBM for the RT. It seems to be an actual instructions per sec count. Since RISC instructions do less than the VAX instructions, "RISC MIPS" are worth less than the MIPS you normally hear quoted. The ratio is probably between 1.5 and 2.0. Nobody is very enthusiastic about MIPS, since nobody really believes you can characterize a machine by one number. Furthermore, MIPS are usually evaluated by vendors, and no two vendors use the same tests. Digital Review does a lot of testing of machines in the Unix marketplace and seems to use a consistent methodology. However their tests are almost entirely in Fortran. Because VMS has a particularly good Fortran compiler, this tends to understate the MIPS of non-DEC machines.
datri@uunet.uu.net (09/27/89)
>In the Unix marketplace, MIPS normally means performance that many times >that of a VAX 11/780, which is essentially the same as a MicroVAX 1. This I'm pretty sure you're thinking of the MVII, which is usually rated at about .9 of a 780. As I remember, the MVI was significantly slower. >Thus MIPS is not an actual count of instructions per second. Million Instructions Per Second. The problem is, as you noted, that many people compare those figures across architectures. It may also be significant how they come up with that number -- do they just average out all the instruction times and divide by n, or do they weight it according to the instructions that people actually use? On the MVII chipset, DEC implemented some of the "lesser-used" instructions in software, which could skew the meaning of an MVII "MIP" according to what you do on it. An example is the CRC instruction, which it seems DEC did in software, but of which the VMS BACKUP program makes heavy use. I prefer the habit of saying "10 times a 780" or "10 VAX MIPS", since that's a more useful number. Another thing to be careful about is that there is a manufacturer named MIPS who make RISCish processors (and machines), and that the DECstation 3100 uses them.
abstine@image.soe.clarkson.edu (Arthur Stine) (09/27/89)
In article <1688@brazos.Rice.edu>, by hedrick@geneva.rutgers.edu (Charles Hedrick): > In the Unix marketplace, MIPS normally means performance that many times > that of a VAX 11/780, which is essentially the same as a MicroVAX 1. This > led to talk of gold-plating a 780 and giving it to the NBS as the Offical > MIP. MIPS are normally evaluated by trying a set of benchmarks on both a > VAX and the machine involved. (I suspect that these days they probably do > the VAX tests on a newer VAX and then divide by DEC's MVUP rating. MVUP > is performance related to the MicroVAX 1.) a 780 is NOT equivalent to a MVI, a MVII is about 90% of a 11/780. There are both VUP and MVUP ratings being used, just to confuse things Art Stine Sr Network Engineer Clarkson U ABStine@CLVMS.Clarkson.Edu
rwood@vajra.dec.com (Richard Wood) (09/29/89)
One significant point of contention about the meaning of "MIPS" is the fact that no commonly used test or benchmark measures only the hardware of the system. Each will measure a different blend of raw hardware speed (which combines many related subsystems) as well as OS performance and compiler efficiency. Most of the earlier MIPS ratings cited by vendors in the UNIX market benchmarked their machines against a VAX 11/780 running 4.1BSD or 4.2BSD. Since those are both quite a few years old, they aren't really valid measurements now. Thus the performance of an 11/780 changes over time - today's software will deliver more power than yesterday's. Since any VAX running a contemporary copy of VMS and it's compilers will out-perform the equivalent hardware running 4.2, there is a distinct bias. Digital realized this bias was being used as a marketing weapon, and started using the term "VUPS", for VAX Units of Processing, defined as the performance of a VAX 11/780 running the most recent VMS software. The most recent Ultrix software will perform roughly the same (a little better in C-based programs, a little worse in Fortran-based - due mostly to the design intentions of the OS, as opposed to compiler quality). Digital Review uses MVUPS as the basis for their quite well thought out suite of tests. Of course, now most vendors that compete directly with Digital use the term "VAX MIPS" (or even VUPS) when claiming performance numbers. Whether these figures are adjusted for the differences in OS/Compilers can only be known by that companies marketing people. The only sure way of testing the performance of any machine is to run the application mix you intend to use on the machines you are considering. Failing that, pay more attention to independent tests such as those in industry magazines. Vendor claims should be thought of as the propaganda they usually are.