[comp.sys.mac] Byte benchmark articles

rhsu@topaz.rutgers.edu (Robert Hsu) (08/12/87)

In article <3560@cit-vax.Caltech.Edu> newton@cit-vax.Caltech.Edu (Mike Newton) writes:

> Hi --
> ... 
> Probably like a lot of Mac II buyers, when I saw the latest issue of Byte,
> I was very disappointed.  The article causing this disapointment was the
> one comparing the Mac II vs. the 80386 based PS2/80. 

I, too, have read the byte benchmark articles, and am of the opinion that
the Byte people are biased towards the 386.  I guess that's to be expected;
since they have a history of belittling the 68k (remember the 68k special
issue that was SUPPOSED to be a separate issue?  Or how about the IBM Piece
of Crap special issue that comes out every year?)

Anyway, here is what I find hard to swallow about the two articles I've
seen (Mac SE/68020 acc. board vs. a Compaq 386, and Mac II against Piece
of Shit 2/80 [that's a lot of shit! :-]):

	First, the article cautions that the benchmarks test compilers
as well as the hardware.  But their cover screams: Benchmarking the 386
and the 020!  If I were totally unfamiliar with microprocessors, or if
I simply looked at the table of results without reading the article, I
would have concluded that the 386 beats the 020 hands down, and run out
to buy a Compaq the next day.

	Next, note how sly they are.  They claim that the code
has not been optimized for any processor, and they have not used
register variables.  The reason was that doing so would not be fair to
any particular processor.  But in not using register variables, are they
not actually putting the 020 at a disadvantage?  Would the fact that 
the 386 has only 2 (two! despite Intel claims otherwise) general purpose
registers and the 68k has 8D+7A=15 general purpose regs have made a
difference?  Why the hell did the 68k designers put those regs there?
So they can be not used for benchmarks?

	I also question their using such a strange configuration (a
Mac SE with an accelerator board) for comparison.  The Compaq is the
top of the line 386 machine available today, so why not get a top of
the line 68020 machine?  Why not a Sun/3, Apollo, or even an NCR Tower!
Never mind the cost difference, if you want to compare processors,
compare processors. (I hope Mac fans don't feel too bad about this.)

	There are other subtle things that people may not notice. E.g.
the benchmarks were compiled using MacC from Consulair (I believe.  I'm
thinking of the first article.)  I don't know how that compiler works,
but if it is half decent, it would produce a double-clickable appl. and
print the results in a window.  Now, if they started timing at the double
click, they would have timed the initialization of ROM and the creation
of a window before getting to the benchmark itself.  I doubt they are
that stupid, but you never know.  Or how about the fact that the 68k in
the Mac has stop whatever it's doing and read out the screen buffer 
60 times a second.  On the Compaq they probably did something like:
	C>sieve
	  10000000 primes
	C>
(They sure know how to exaggerate, don't they?) The point is, due to
its inherent operating mode, the Mac (and almost all 68k based machines)
has to take care of many other things (mostly maintaining the graphics
on the screen) while running a benchmark.  The 386 does not.  It doesn't
have to, it can't even handle graphics.

	After degrading the 68k to their hearts content, the benchmarkers
sit back and inform us the the only conclusion that can be drawn from
their results is that the Compaq runs these benchmarks faster than the
Mac SE with a 020 accelerator.  They had to say that, because they know
those 386 machines aren't good for anything else.  Intel probably built
them so the can give 10000000 primes on a sieve benchmark.

	You can probably tell that I'm eagerly awaiting Byte's next
enlightening article with a critical and objective analysis of the 68k
and the 386, in which they will probably match the PS2/80 against a
Mac 128K, because its only fair to do such a comparision, in order to avoid
being unfair to any particular processor.  In the mean time, I will
go play Uninvited on the Mac.

(This article was written without the Byte articles for reference.  I only
used my memory, so there may be factual inaccuracies.  I deeply regret
any such errors.  I also disclaim any responsibility for them.)

Also, why don't one of you technical fellows out there who, unlike me, really
knows what he is talking about write a fuming letter to Byte and blast
the shit out of them?

-Rob

------------------------------------------------------------------------
Robert Hsu		  DISCLAIMER: I disclaim any responsibility for 
rhsu@topaz.rutgers.edu 	  inaccuracies, misinformation, and fabrications
RPO 6768  CN 5063	  that appeared in the preceding article.
New Brunswick, NJ 08903   {...!rutgers!topaz.rutgers.edu!rhsu}
------------------------------------------------------------------------
"Vidi, Vici, Veni"					"Coito ergo sum"
-- 
------------------------------------------------------------------------
Robert Hsu		  DISCLAIMER: I disclaim any responsibility for 
rhsu@topaz.rutgers.edu 	  inaccuracies, misinformation, and fabrications
RPO 6768  CN 5063	  that appeared in the preceding article.
New Brunswick, NJ 08903
------------------------------------------------------------------------
"Vidi, Vici, Veni"					"Coito ergo sum"

brian@casemo.UUCP (Brian Cuthie ) (08/12/87)

In article <13885@topaz.rutgers.edu>, rhsu@topaz.rutgers.edu (Robert Hsu) writes:
> I, too, have read the byte benchmark articles, and am of the opinion that
> the Byte people are biased towards the 386.  I guess that's to be expected;
> since they have a history of belittling the 68k (remember the 68k special
> issue that was SUPPOSED to be a separate issue?  Or how about the IBM Piece
> of Crap special issue that comes out every year?)
     ^^^^(note 1)
> 
> Anyway, here is what I find hard to swallow about the two articles I've
> seen (Mac SE/68020 acc. board vs. a Compaq 386, and Mac II against Piece
> of Shit 2/80 [that's a lot of shit! :-]):
     ^^^^ (note 1)
> 
...
> ... would the fact that 
> the 386 has only 2 (two! despite Intel claims otherwise) general purpose
> registers and the 68k has 8D+7A=15 general purpose regs have made a
                            ^^^^^^^ 
How is it that the Mr. Hsu can claim that the Intel processor only has
two general purpose registers and then lump the 68k's Address and Data
registers into one 'general purpose' goup. This is *highly* biased.

> difference?  Why the hell did the 68k designers put those regs there?
                       ^^^^ (note 1)

...
> Mac SE with an accelerator board) for comparison.  The Compaq is the
> top of the line 386 machine available today, so why not get a top of
> the line 68020 machine?  Why not a Sun/3, Apollo, or even an NCR Tower!

I suspect because they were actually comparing implimentations likely
to be owned by most people.  Granted, the compaq 386 is less likely to be
a home computer than the Mac.  Still, it is more likely than Sun et al.

> 
> 	There are other subtle things that people may not notice. E.g.
> the benchmarks were compiled using MacC from Consulair (I believe.  I'm
> thinking of the first article.)  I don't know how that compiler works,
  ^^^^^^^^
  You can't be sure ?????
...
> the Mac has stop whatever it's doing and read out the screen buffer 
> 60 times a second.  On the Compaq they probably did something like:
> 	C>sieve
> 	  10000000 primes
> 	C>
> (They sure know how to exaggerate, don't they?) The point is, due to
> its inherent operating mode, the Mac (and almost all 68k based machines)
                                            ^^^^^^^^^^^^^^
I would *really* like to see you back this up !

> has to take care of many other things (mostly maintaining the graphics
> on the screen) while running a benchmark.  The 386 does not.  It doesn't
> have to, it can't even handle graphics.
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This is plainly irresponsible !!!!!  Any processor can handle graphics. 
Some are better suited to it than others.  It would be hard to make any
claims that the 68k was more suited to graphics than the 80X86.

> 
> 	After degrading the 68k to their hearts content, the benchmarkers

While you degrade the 80x86.

...
> (This article was written without the Byte articles for reference.  I only
> used my memory, so there may be factual inaccuracies.  I deeply regret
                                  ^^^^^^^^^^^^^^^^^^^^

Then why waste our (the net's) time with this meaningless dribble !!

> any such errors.  I also disclaim any responsibility for them.)

I would too if I were you.

> 
> Also, why don't one of you technical fellows out there who, unlike me, really
                                                              ^^^^^^^^^

That would be just about anyone.

> knows what he is talking about write a fuming letter to Byte and blast
> the shit out of them?
      ^^^^ (note 1)

> 
> -Rob
> 
> ------------------------------------------------------------------------
> Robert Hsu		  DISCLAIMER: I disclaim any responsibility for 
> rhsu@topaz.rutgers.edu 	  inaccuracies, misinformation, and fabrications
> RPO 6768  CN 5063	  that appeared in the preceding article.
> New Brunswick, NJ 08903   {...!rutgers!topaz.rutgers.edu!rhsu}
> ------------------------------------------------------------------------
> "Vidi, Vici, Veni"					"Coito ergo sum"
> -- 
> ------------------------------------------------------------------------
> Robert Hsu		  DISCLAIMER: I disclaim any responsibility for 
> rhsu@topaz.rutgers.edu 	  inaccuracies, misinformation, and fabrications
> RPO 6768  CN 5063	  that appeared in the preceding article.
> New Brunswick, NJ 08903
> ------------------------------------------------------------------------
> "Vidi, Vici, Veni"					"Coito ergo sum"

Nice touch. Two signatures for the price of one...


All in all I find Mr. Hsu's comments to be ill founded, irresponsible and
offensive.  He obviosuly does not have any real understaning of the issues
involved and is just as highly biased towards the 68k as he claims byte
to be towards the 80x86.

I did not read the byte article so I can't defend or attack it. Obviously
Mr. Hsu can't remember it either.  So who is he to attack it so violently ?
I also have no intention of defending or attacking either processor. They
both have the architectural strengths and weaknesses.  I currently own
a full blown AT and am trying to sell it for a Mac II.  I have previously
owned Macs and a Lisa 2/10.


-Brian

Note 1:	There is *absolutely* no reason for profanity in a well presented
argument.  Mr. Hsu has obviously replace four letter words for reasoning.
This type of argument style is nothing more than offensive and has no
place on the net.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Brian Cuthie
CASE Communications Corp.
Columbia, Md. 21046
(301) 290 - 7443

home ARPA:	brian@umbc3.umd.edu
     UUCP:	...seismo!mimsy!brian@umbc3.umd.edu

work UUCP:	...seismo!mimsy!aplcen!casemo!brian

msf@amelia (Michael S. Fischbein) (08/13/87)

In article <13885@topaz.rutgers.edu> rhsu@topaz.rutgers.edu (Robert Hsu) writes:

Stuff elided concerning Byte's alledged bias, IBM bashing, etc.

>	I also question their using such a strange configuration (a
>Mac SE with an accelerator board) for comparison.  The Compaq is the
>top of the line 386 machine available today, so why not get a top of
>the line 68020 machine?  Why not a Sun/3, Apollo, or even an NCR Tower!
>Never mind the cost difference, if you want to compare processors,
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Check out the prices available; compare list to list or discount to discount
and don't be too surprised!  By the time you pay for an SE and accelerator
board, you are pretty close to the SUN, Apollo, or even HP price area.
Further, the designed-in-from-the-start machines are much faster, even if
the clock, coprocessors, etc are identical.

I would have really liked to have seen a comparison against one of the $5k
68020 Unix boxes coming out for the business market.  The use of the odd
hardware combination begs for an explanation, but was not provided in the
article (to my recollection).  If workstation type graphics and capabilities
are desired, they should have been present on both machines.  If that is
wanted, the Mac II is probably the best buy on the market today for a
standalone machine.  (Oh no.  I'm about to get flamed to death.....)
		mike

Michael Fischbein                 msf@prandtl.nas.nasa.gov
                                  ...!seismo!decuac!csmunix!icase!msf
These are my opinions and not necessarily official views of any
organization.

rhsu@topaz.rutgers.edu (Robert Hsu) (08/14/87)

In reply to Mr. Cuthie's highly responsible, intelligent, and
dispassionate retort of my previous posting, I offer the following
petty attempt of self defense.  As I find his criticism of my
overusing expletives to be valid, I will try to avoid them, albeit
with great regret (note 1).

In article <213@casemo.UUCP> brian@casemo.UUCP (Brian Cuthie ) writes:

>This is *highly* biased.

To be absolutely honest, I am indeed biased.  How can one not be
biased on the highly emotional issue of "which processor is
better"?  Mr. Cuthie claims to be impartial on the subject; if
so he is missing out on two of the strongest of human feelings:
love and hate.  But I strongly suspect that, in replying so harshly
to an article lauding the 68000, he has revealed a secret
affection for the 80x86.

> > thinking
>   ^^^^^^^^
>   You can't be sure ?????

If my article appears to you to be the work of some mindless
idiot who can't think, surely you will not have troubled yourself
to honor it with such a dignified, critical, and rational reply.
As far as I remember (I'm sure Mr. Cuthie seriously doubts the
reliability of my memory), I was really thinking at the time
I wrote the article.  I base this on the fact that I was
conscious of electro-chemical activity in my brain while writing.

> > have to, it can't even handle graphics.
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> This is plainly irresponsible !!!!!  Any processor can handle graphics. 

From what I've heard, the 386 does not have bit-manipulation
instructions. Perhaps that's why Intel is developing a graphics
coprocessor.

> While you degrade the 80x86.

Surely Mr. Cuthie will not deny me of such a great pleasure.
I would not call it degrading, however. I would call it telling
the truth and nothing but the truth.

> Then why waste our (the net's) time with this meaningless dribble !!

Call it what you will, but this meaningless dribble is to me the
consummate expression of the highest level of human wisdom. As for
wasting your time, I offer my sincerest apologies.

> > don't one of you technical fellows out there who, unlike me, really
>                                                     ^^^^^^^^^
> That would be just about anyone.

I surely hope so.  Being a person of extreme vanity, I would be
devastated to learn otherwise.

> Nice touch. Two signatures for the price of one...

Thank you.  As I have already noted, I have great self pride,
and love to see my name as much as possible.
 
> He obviosuly does not have any real understaning of the issues

My God! Is it really that obvious?  I tried my best to conceal
this grave flaw, but apparently it cannot escape the acute and 
the observing.

> 
> -Brian

> Note 1: There is *absolutely* no reason for profanity in a well presented
> argument.  Mr. Hsu has obviously replace four letter words for reasoning.
> This type of argument style is nothing more than offensive and has no
> place on the net.

Note 1: With all due respect to Mr. Cuthie and all God-fearing,
clean-living people everywhere, I would like to propose that 
profanity is in the eye of the beholder (listener).  What you
call profanity are mere interjections to me, such as Wow! Great! etc.
I used them to make my article livelier and more vivid. In fact I
recommend their usage for dramatic impact. Apparently it worked, 
since Mr. Cuthie was able to note every one of them, except one.

And besides, I didn't even use the F word!

> 
> Brian Cuthie
>

-Rob
------------------------------------------------------------------------
Robert Hsu		  DISCLAIMER: I disclaim any responsibility for 
rhsu@topaz.rutgers.edu 	  inaccuracies, misinformation, and fabrications
RPO 6768  CN 5063	  that appeared in the preceding article.
New Brunswick, NJ 08903   {...!rutgers!topaz.rutgers.edu!rhsu}
------------------------------------------------------------------------
"Vidi, Vici, Veni"					"Coito ergo sum"

One more time:
-- 
------------------------------------------------------------------------
Robert Hsu		  DISCLAIMER: I disclaim any responsibility for 
rhsu@topaz.rutgers.edu 	  inaccuracies, misinformation, and fabrications
RPO 6768  CN 5063	  that appeared in the preceding article.
New Brunswick, NJ 08903   {...!rutgers!topaz.rutgers.edu!rhsu}
------------------------------------------------------------------------
"Vidi, Vici, Veni"					"Coito ergo sum"

paul@aucs.UUCP (08/14/87)

> ... stuff deleted ...
>
>All in all I find Mr. Hsu's comments to be ill founded, irresponsible and
>offensive.  He obviosuly does not have any real understaning of the issues
>involved and is just as highly biased towards the 68k as he claims byte
>to be towards the 80x86.
>
I agree with Brian, although I felt Byte's choice for a 68020-based micro
was very poor.  A Macintosh spends something like 25% of its time updating
the screen, and you can bet that the Compaq 386 does not.  And Brian's argument
that more people are likely to own the enhanced Mac than a Sun or Apollo is
probably true, but I know more people who own/use Sun's/Apollo's than 68020-
enhanced Mac's.  Incidently, I own both a Compaq Deskpro (8088) and a Mac SE
and use both equally.  I prefer the Mac, but I am realistic about the presence
of PC's in the computer industry.
>
>-Brian
>
>Note 1:  There is *absolutely* no reason for profanity in a well presented
>argument.  Mr. Hsu has obviously replace four letter words for reasoning.
>This type of argument style is nothing more than offensive and has no
>place on the net.
>
I totally agree with Brian.  Certain people tend to use profanity to get a
point across, but if they actually tried, they probably could find a better
way to say the same thing and not make a fool of themselves in the process!


Paul H. Steele      UUCP:      {uunet|watmath|utai|garfield}!dalcs!aucs!Paul
Acadia University   BITNET:    Paul@Acadia
Wolfville, NS       Internet:  Paul%Acadia.BITNET@WISCVM.WISC.EDU
CANADA  B0P 1X0     PHONEnet:  (902) 542-2201x587

jww@sdcsvax.UCSD.EDU (Joel West) (08/19/87)

There was a huge flame war on Bix with all the Macintosh people
dumping on Byte.  The Byte response was:
	1. Using a high-level language is fair, since people write
	   in high-level languages.
	2. We used what compilers were available.
	3. Anyone can gripe about results and if we listened
	   to them, all benchmarks would be in assembler.
There will be some sort of clarification/follow-up in a future
issue, if only a synopsis of the similar griping under
'Best of Bix'.

I'm not much on conspiracy theories (Byte's after the 68000),
but from what I've seen, they have less Mac/68000 experience
on staff than IBM/8086.  I'm not sure I could test the 80386
fairly, since I don't know much about it.
-- 
	Joel West  (c/o UCSD)
	Palomar Software, Inc., P.O. Box 2635, Vista, CA  92083
	{ucbvax,ihnp4}!sdcsvax!jww 	jww@sdcsvax.ucsd.edu
   or	ihnp4!crash!palomar!joel	joel@palomar.cts.com

wjb@goanna.oz (Warwick Bolam) (08/28/87)

I think the offensiveness of the Byte article stems from the fact that
it is heades "68020 vs 80386" when it should have been headed "Machine X
using 68020 vs. 80386".  If you want to compare raw processor speed,
then don't do what BYTE did.

Warwick Bolam
Dept of Computing
Royal Melbourne Institute of Technology
124 La Trobe St
Melbourne, Victoria 3000
AUSTRALIA

PHONE:  (03) 660 2348                 	OVERSEAS: +61 3 660 2348

ARPA:   wjb%goanna.oz@seismo.css.gov    BITNET: wjb%goanna.oz@CSNET-RELAY
CSNET:  wjb@goanna.oz                   JANET:  goanna.oz!wjb@ukc
UUCP:  {enea,hplabs,mcvax,prlb2,seismo,ubc-vision,ukc}!munnari!goanna.oz!wjb

matthews@dartvax.UUCP (08/31/87)

In article <1069@goanna.oz> wjb@goanna.oz (Warwick Bolam) writes:
>
>I think the offensiveness of the Byte article stems from the fact that
>it is heades "68020 vs 80386" when it should have been headed "Machine X
>using 68020 vs. 80386".  If you want to compare raw processor speed,
>then don't do what BYTE did.
>

And if you want to compare commonly used systems, you still don't want
to do what Byte did.  Byte's benchmarks for the 80386 systems were
compiled with a 80386-specific compiler, whereas 95+% of the 80386
systems in this world are spending their time running software developed
for the 8086.  This makes a big difference, since running old software
reduces the 80386 to more or less the performance of an 80286 at equivalent
clock speed.  By contrast, in the 680x0 world there is not much penalty
for running 68000 software.  In fact, the September Byte article pointed
out that a 68000 compiler (Lightspeed C) produced code that was 10-15%
*faster* than that produced by Consulair's 68020 compiler.

So Byte's approach would only be useful to someone who is using special
software or developing their own, and those sorts of people are likely
to run their own, more specific benchmarks.

On the subject of compilers, has anyone benchmarked MPW Pascal 2.0 versus
version 1.0?  It would be interesting to see how much difference 2.0's
peephole optimization makes on the standard benchmarks (sieve, dhrystones,
etc.).

Jim Matthews
matthews@dartvax.uucp