[comp.sys.mac] More on Byte Benchmarks

rs4u+@ANDREW.CMU.EDU (Richard Siegel) (08/12/87)

There were also some technical inaccuracies in the benchmarks. Notice
that they listed the Mac SE with HyperCharger having a 68881 clocked
at 7.833 MHZ. This is not true -- the '881 in the HyperCharger is clocked
at 12 MHz, and I understand GCC's going to kick it up to 16 MHz real soon.

Also, they listed the plain Macintosh as having a math chip.

Do these guys not know what they're talking about, or what?

		--Rich

hugo@gnome.cs.cmu.edu (Peter Su) (08/13/87)

BYTE is a rag, and I have no respect for the bozos that put it out.
They haven't had anytihng worth soap for years on end, and they publish
that idiot Pournelle who is still living in the 70s with his 8 inch
disks.  Gawd!

Anyway, I wouldn't trust their benchamarks (or the ones in the august
issue with the Mac II vs. PS/2 model 80) as far as one of their
magazines would fly.

But, what do I know?

Pete

-- 
ARPA: hugo@gnome.cs.cmu.edu
UUCP: ...!{ucbvax,ihnp4,cmucspt}!hugo@gnome.cs.cmu.edu

	"There are reports that many executives make their decisions by
	 flipping coins or by throwing darts, etc.  It is also rumored that 
	 some college professors prepare their grades on such a basis."
				- Donald Knuth

steven@lakesys.UUCP (Steven Goodman) (08/13/87)

In article <AV87t=y00V4IyRA2MW@andrew.cmu.edu> rs4u+@ANDREW.CMU.EDU (Richard Siegel) writes:
>
>There were also some technical inaccuracies in the benchmarks. Notice
>that they listed the Mac SE with HyperCharger having a 68881 clocked
>at 7.833 MHZ. This is not true -- the '881 in the HyperCharger is clocked
>at 12 MHz, and I understand GCC's going to kick it up to 16 MHz real soon.
>
>Also, they listed the plain Macintosh as having a math chip.
>
>Do these guys not know what they're talking about, or what?
>
>		--Rich

The 68881 was trimmed down to about 8Mhz in order to have it a valid 
comparison seeing as the 80287 ran at 8Mhz.  But even then it was unfair
to the Compaq seeing as they were using the "old" model which used this
older FPU.  Also Mac had cacheing where new Deskpro's has integrated.
If this was a fair comparison should have had the 80387 in the new deskpro
(6 times faster than 80287) and configured compaq with equal cache.
Code on the Mac was compiled using a 68020 C compiler.  So the Compaq was
actually at a disadvantage seeing as it was not able to use their nice
new 32 bit FPU where the Mac was and the Mac could speed up some of the
disk action with it's cache and the Compaq was unable because they use
older daekpro.  I would say that if this would done you would even be
more disapointed in the resaults.  Must remember that Mac's have a 
definate disadvantage when it comes to many benchmarks, putting the 
CPU asside most systems offload the processes conserning gaining data from
the disk, Mac's do not.  Saving grace on a Mac II is you have slots and 
can add a card to do this off load processing.  
	What it comes down to is Intel has FINALLY made a good chip, it
DOES outperform the 68020 at equal clock speed.   I am more interested in 
the next generation the 486 vs the 030, that should be very interesting.

 
-- 
Steven Goodman
Lake Systems   Milwaukee, Wisconsin
UUCP:  {ihnp4,uwvax}!uwmcsd1!lakesys!steven

chow@monet.Berkeley.EDU (Christopher Chow) (08/13/87)

In article <170@lakesys.UUCP> steven@.UUCP (Steven Goodman) writes:
>Also Mac had cacheing where new Deskpro's has integrated.
>If this was a fair comparison should have had the 80387 in the new deskpro
>(6 times faster than 80287) and configured compaq with equal cache.

Equal cache size?  What do you mean - the processor cache size is fixed as its
a function of the chip design.  You can't possibly mean disk cache -- it has
nothing to do with floating point benchmarks.

>Code on the Mac was compiled using a 68020 C compiler.  So the Compaq was
>actually at a disadvantage seeing as it was not able to use their nice
>new 32 bit FPU where the Mac was and the Mac could speed up some of the
>disk action with it's cache and the Compaq was unable because they use

What makes you think the Mac code was compiled on a 68020 compiler?  The
article didn't mention which compilers which they used, not did they give any
details to what compiler options were enabled.  It was my _impression_ that
they used a non-68020 compiler w/o direct 68881 calls.  But I can't be sure
since they didn't provide this information.

>	What it comes down to is Intel has FINALLY made a good chip, it
>DOES outperform the 68020 at equal clock speed.   I am more interested in 
>the next generation the 486 vs the 030, that should be very interesting.

Has it really?  I hope you didn't use that article as your only source of
information.

Anyway, there's no point to continue debating the contents of this two page
article in Byte.  I think we can all agree that the Byte article was not worth
the paper it was printed on as it didn't give alot of information (compilers,
compile options, etc).  Perhaps someone could write Byte a nasty letter and
tell them to perform another benchmark and report all the relevant information.


Christopher Chow
/---------------------------------------------------------------------------\
| Internet:  chow@tcgould.tn.cornell.edu (128.84.253.35)                    |
| Usenet:    ...{uw-beaver|ihnp4|decvax|vax135}!cornell!batcomputer!chow    |
| Bitnet:    chow@crnlthry.bitnet                                           |
| Phone:     1-415-643-2953,  USPS:    2299 Piedmount Av, Berkeley CA 94720 |
| Delphi:    chow2            PAN:  chow                                    |
\---------------------------------------------------------------------------/
Christopher Chow
/---------------------------------------------------------------------------\
| Internet:  chow@tcgould.tn.cornell.edu (128.84.253.35)                    |
| Usenet:    ...{uw-beaver|ihnp4|decvax|vax135}!cornell!batcomputer!chow    |

twleung@picasso.mit.edu (Theodore W. (Ted) Leung) (08/14/87)

>	What it comes down to is Intel has FINALLY made a good chip, it
>DOES outperform the 68020 at equal clock speed.   I am more interested in 
>the next generation the 486 vs the 030, that should be very interesting.

Having just returned from Macworld Expo, I have a different
perspective on this entire debate.  It may be true that the 386 is a
"faster chip", but faster is not necesssarily better.  The key to all
computers is the quality of the application software (whatever the
application is).  If you happen to be crunching numbers, or doing
something that is really computationally intensive, then maybe you
really do care if the the 386 is faster than the '020 at the same
clock speed.  Most people are only concerned with getting their work
done as quickly as possible.  For a good many people, machines "as
slow as the Mac II" running software like Hypercard (no jibes about
the quasi multi-tasking Multifinder), are much better tools for
getting their work done, than "fast machines like the Deskpro 386"
running whatever.  I don't mean to demean the engineering and
scientific markets, as I'm at a school where > 60% of the students are
engineers.  But even here, I think that many students who are using
VaxStation II's running 4.3BSD with X would give them up if they could
have Mac II's with the current generation of Mac software, because the
applications are that much better.  Fast hardware is nice, but I'd
rather have second place hardware and first place software.

					Ted Leung

					MIT/Project Athena

Disclaimer: My employers don't even CARE about personal computers.....

radford@calgary.UUCP (Radford Neal) (08/15/87)

In article <170@lakesys.UUCP>, steven@lakesys.UUCP (Steven Goodman) writes:

> The 68881 was trimmed down to about 8Mhz in order to have it a valid 
> comparison seeing as the 80287 ran at 8Mhz...

> 	What it comes down to is Intel has FINALLY made a good chip, it
> DOES outperform the 68020 at equal clock speed...

This is ridiculous. Do you think that users get a nice warm feeling from
knowing their chip is getting "more for the megahertz" than the next
guy's? Would you buy a $10000 Intel machine that ran at 8MHz in preference
to a $10000 Motorola machine that ran at 24MHz, just because the Intel
machine does more per cycle? Crippling one machine to make a "fair" 
comparison does not help answer the real question - "Which machine should
I buy (for my application)?".

    Radford Neal

dchen@rainier.UUCP (08/15/87)

	First of all, all of the benchmarking I have seen comparing the '020
to the '386 has been pretty lame.  Everything (so far, I should say.  Optimism
in action...) is either a) biased on purpose (run by Motorola or Intel),
b) biased by accident (i.e., comparing a real machine against a paper
machine, or comparing radically different configurations), or c) pretty
useless (millions of nops per second, mega-linpack-i'm-sorry-but-99%-
of-comp.sys.mac-inverts-less-than-one-matrix-a-year-anyway, megachars-
compared-to-a-null-string-per-second, etc.).  Most benchmarks commit more
than one of the above.
	I suspect most people's interest in benchmarking is not based on
a need to answer the question 'how should I design my next microprocessor'
(In which case bus-cycles-per-light-furlong is probably an interesting
statistic) but instead on the question 'gee, I wonder how fast a '386 box feels?
I wonder just how well a Mac II justifies the cost difference over a
Mac +/-?  I'm going to write a CPU-bound program to do <x>, which machine
should I buy?'  Another words, it's the application speed that matters.
	So I kinda sorta think that an application benchmark would be
pretty interesting.  You know, Word on a Mac II, Word on a Compaq '386.
Excel.  PageMaker (I heard somewhere that the first PageMaker release
running running on a '386 was slower than on a Mac Plus.  Hey, it's just
a rumor, don't believe it til you see it yourself).
	Anyway, I think the jury is pretty far out right now.  We'll start
to get the results when the applications start getting ported, til then,
well, I think the two machines, running with the same speed DRAMs and
compiler technology, run at about the same speed.
	My personal bias is that Motorola machines can be made to run
faster (the 25 MHz '020 beat the 20 MHz '386 to market by about a year)
if you've got the bucks for a serious cache or fast memories, and that
the 030 will tip the scales to Motorola's side, around the end of '88.
Don't hold your breath for the '486 to come out, I hear they're trying
to get it to support Ada in silicon :-) 
	And, yes, I do think comp.sys.mac is an appropriate forum for
benchmarking discussions.  Applications tend to dominate in
this industry, but speed is important too.  Also, the software
markets are beginning to converge, or at least overlap, so that we
are beginning to have a choice (Make Mine Mac).

David Chenevert

breck@aimt.UUCP (08/15/87)

I've watched this "debate" go on for, what, a hundred messges?  It seems like
forever.  Of course I only "arrived" on the net a couple days ago.  So for me it
is effectively forever.

Anyway I just broke out laughing at the hilarity of the whole issue.  Byte
published the results of their benchmarks.  One guy (sorry, I forget his name
and don't want to go looking for the message) even posted the results from an
ieee benchmark.  Yet still with these numbers no one can even agree which chip
is "faster".  One message interprets the results as indicating that the 386 is
the faster chip.  The next message rejoices that the 020 is faster.  Sort of
goes to show the real value of benchmarks. :-)

By the way, none of this is in any way the opinion of AIM Technology or any
of its employees or officers.... except me that is.
-- 
Breck Beatie
uunet!aimt!breck

alan@pdn.UUCP (Alan Lovejoy) (08/19/87)

In article <53@gnome.cs.cmu.edu> hugo@gnome.cs.cmu.edu (Peter Su) writes:
>BYTE is a rag, and I have no respect for the bozos that put it out.
>
>Anyway, I wouldn't trust their benchamarks (or the ones in the august
>issue with the Mac II vs. PS/2 model 80) as far as one of their
>magazines would fly.
>
>But, what do I know?
>
>Pete
>

More than you think.  Carefull consideration of the results of the two
infamous articles in question will reveal various inconsistencies.
I leave the details as an exercise for the reader (I sent a letter to
BYTE detailing the gaffs; perhaps they'll print it).

But what can you expect from people who think that systems should be
benchmarked using an interpreted language (or any HLL--assembly is the
only fair way to benchmark iron)?  At the very least, they should use
a compiled language and publish the dissasembled machine code for the
benchmark (on BIX, if not in the mag itself).

By the way, did anyone notice that Infoworld rated the Mac II as 3.72
times faster than an 8Mhz IBM AT (faster than the Deskpro 386, slower
only than the Intel Inboard (3.78) and PC's LTD 386 (4.00))?
Unfortunately, Infoworld provides almost no information as to how they
generated these numbers.  At least you can't PROVE that their methods
are whacko.

Alan "Check out those WEIRD Fibonacci results!" Lovejoy

alan@pdn.UUCP (Alan Lovejoy) (08/19/87)

In article <170@lakesys.UUCP> steven@.UUCP (Steven Goodman) writes:
>The 68881 was trimmed down to about 8Mhz in order to have it a valid 
>comparison seeing as the 80287 ran at 8Mhz.  But even then it was unfair
>to the Compaq seeing as they were using the "old" model which used this
>older FPU.  Also Mac had cacheing where new Deskpro's has integrated.

Just a minute here.  Is this a benchmark of CPU chips or computer
systems?  On the surface, BYTE made it appear that they were
benchmarking CPU chips, but the text of the article made it clear
they were benchmarking specific computer systems.  If they're
benchmarking systems, then they should have left the chip speeds alone,
and "fair" can only be defined as comparing shipped product to shipped
product.  If they really were benchmarking CPU chips, then the whole
think is so drastically unfair to everyone that it's beyond fixing.

Alan "you can't argue from both sides of an issue at the same time"
Lovejoy