[comp.sys.amiga] GUI Benchmark

rjtatz@hpuxa.ircc.ohio-state.edu (Robert J. Tatz) (04/10/90)

     We all know that statistics lie and that benchmarks can be mis-used.
For example, MIPS cannot be used to compare CISC and RISC chips.  Due to
different version numbers, quoting dhrystones can be questionable.  And
checking out hard drives speeds can be dependant on how optimized or how
full the hard drive is.  However, it is also abundantly clear that the 
graphical user interface (GUI) is here to stay no matter who is selling
the hardware.

     It is very typical for high-end graphics workstations (CAD-CAM, 3-d
modeling) to quote speed for pixels/vectors drawn, polygons filled, etc.
The CPU's involved tend to different in design and speed - the only thing
in common is what's going to the screen in at least comparable resolution.
So why isn't there a benchmark to determine speed of the graphics on
the various hardware platforms currently (or soon to be) available?

     One drawback to testing GUI speed is the great variation in the OS
available.  So what I propose is the following benchmark.  Unix and
X-windows is a standard which is or will be supported on IBM, Mac, Amiga,
Sun, NeXT, etc.  A program written in c for X-windows running under Unix
should open  a) a window to draw text,  b) a window to draw circles or
squares,  c) a window to animate a cube,  d) a window to ??? .  These
windows should then be cycled so the screen needs to be updated.  Of
course, all of these operations would be timed.  Also, putting menus up
should be automated and timed.  Finally, an option would have some CPU
intensive calculation running in the background.  

     Ideally, the benchmark program would then be run on the top end of
each manufacturer's machine, keeping the resolution and # of colors as
similar as possible and clearly noting the resolution/# of colors used.
This would provide a comparison between manufacturers top of the line,
relatively free of the personal bias we have recently heard so much of.
Of course, if the manufacturer had a brand new machine with a 16Mhz and
a 25Mhz version of the 68030 running Unix, these could be compared to
each other.  And if the manufacturer came out with a drop-in 68040 card,
this new high-end machine could be tested.  In addition, comparisons
could be made for a single computer on changing the resolution or # of
colors used.  The effect of installing a special video card could also
be tested.

     Finally, it would be useful if the benchmark program could be ported
to Amiga Workbench, Mac OS, OS/2, NeXT-Step, MS-DOS w/Windows, etc. to
see what these GUI's can do.

     No easy task?  Ideas, suggestions, volunteers?  Can it be shown
that a set of 16 bit, 7.14 Mhz custom graphics chips can make a 25 Mhz
68030 Amiga compete with a 40 Mhz 68030 Mac?  I hope we can generate a
little light with this project without all the heat.

     If you want to email me -> rjtatz@hpuxa.ircc.ohio-state.edu .

BTW, I have a room full of 24 A1000's at the Ohio State University
Dept. of Chemistry running proprietary chemistry intructional software.
Over the past three years with the machines running 45 hrs./week, I
have put only one monitor and one machine in for repair - not a bad
track record.

Regards,
Bob          rjtatz@hpuxa.ircc.ohio-state.edu

dcr3567@ultb.isc.rit.edu (D.C. Richardson) (04/11/90)

In article <972@nisca.ircc.ohio-state.edu> rjtatz@hpuxa.ircc.ohio-state.edu (Robert J. Tatz) writes:
>
>BTW, I have a room full of 24 A1000's at the Ohio State University
>Dept. of Chemistry running proprietary chemistry intructional software.
>Over the past three years with the machines running 45 hrs./week, I
>have put only one monitor and one machine in for repair - not a bad
>track record.
>Bob          rjtatz@hpuxa.ircc.ohio-state.edu

Yep, that *IS* an impressive track record!  We have a CAD Lab running
the newest version of CadKey Software (With *MANY* bugs! This software
is at once AWESOME and PITIFUL!  More later), but the HARDWARE is
GARBAGE!  I lost 8 hours of cad dawings (Ok, so it would be 2 hours to a
experienced CAD person.. I'm learning! :) because the software didn't
like my 360k 5 1/4" disks.  Out of 20 or so '286 machines, usually 2 are
out of service, maybe more.

What I'm saying (Maybe C= should take note) is that RIT is in DIRE need
of QUALITY <ugh> IBM or <Better> CAD systems. (We have a Professional
CAD lab, which *IS* quality, but many hundreds of students have to train
on JUNK)  If someone could bid right, they could make a great deal with
a school that really needs help. (How can a peice of software have
trouble writing to a disk when the disk is fine 100% and only has 20k
on it??   And WE thought we had bad software.... sigh)

-Random Burnt Out Babble Mode... of
-Dan


-- 
Daniel C. Richardson
Rochester Institute Of Technology     /    Mechanical Engineering Dept.
"Immaturity Is The Essence Of Humanity.  Children Shall Be Our Saviors"
-Red's Dream

rjtatz@hpuxa.ircc.ohio-state.edu (Robert J. Tatz) (04/14/90)

WARNING : The following post does not contain any flames or reams of
          quoted material.  It does contain a suggestion on how to make
          comparisons of various hardware platforms and is intended to
          provoke some thought.  It was posted on c.s.a. and had a single
          response (from an Apple student rep.).  Can the A3000/A3500
          beat out a top-end Mac?  Read on, please ...

     We all know that statistics lie and that benchmarks can be mis-used.
For example, MIPS cannot be used to compare CISC and RISC chips.  Due to
different version numbers, quoting dhrystones can be questionable.  And
checking out hard drives speeds can be dependent on how optimized or how
full the hard drive is.  However, it is also abundantly clear that the 
graphical user interface (GUI) is here to stay no matter who is selling
the hardware.

     It is very typical for high-end graphics workstations (CAD-CAM, 3-d
modeling) to quote speed for pixels/vectors drawn, polygons filled, etc.
The CPU's involved tend to different in design and speed - the only thing
in common is what's going to the screen in at least comparable resolution.
So why isn't there a benchmark to determine speed of the graphics on
the various hardware platforms currently (or soon to be) available?

     One drawback to testing GUI speed is the great variation in the OS
available.  So what I propose is the following benchmark.  Unix and
X-windows is a standard which is or will be supported on IBM, Mac, Amiga,
Sun, NeXT, etc.  A program written in c for X-windows running under Unix
should open  a) a window to draw text,  b) a window to draw circles or
squares,  c) a window to animate a cube,  d) a window to ??? .  These
windows should then be cycled so the screen needs to be updated.  Of
course, all of these operations would be timed.  Also, putting menus up
should be automated and timed.  Finally, an option would have some CPU
intensive calculation running in the background.  

     Ideally, the benchmark program would then be run on the top end of
each manufacturer's machine, keeping the resolution and # of colors as
similar as possible and clearly noting the resolution/# of colors used.
This would provide a comparison between manufacturers top of the line,
relatively free of the personal bias we have recently heard so much of.
Of course, if the manufacturer had a brand new machine with a 16Mhz and
a 25Mhz version of the 68030 running Unix, these could be compared to
each other.  And if the manufacturer came out with a drop-in 68040 card,
this new high-end machine could be tested.  In addition, comparisons
could be made for a single computer on changing the resolution or # of
colors used.  The effect of installing a special video card could also
be tested.

     Finally, it would be useful if the benchmark program could be ported
to Amiga Workbench, Mac OS, OS/2, NeXT-Step, MS-DOS w/Windows, etc. to
see what these GUI's can do.

     No easy task?  Ideas, suggestions, volunteers?  Can it be shown
that a set of 16 bit, 7.14 Mhz custom graphics chips can make a 25 Mhz
68030 Amiga compete with a 40 Mhz 68030 Mac?  I hope we can generate a
little light with this project without all the heat.

     If you want to email me -> rjtatz@hpuxa.ircc.ohio-state.edu .

BTW, I have a room full of 24 A1000's at the Ohio State University
Dept. of Chemistry running proprietary chemistry intructional software.
Over the past three years with the machines running 45 hrs./week, I
have put only one monitor and one machine in for repair - not a bad
track record.

Regards,
Bob          rjtatz@hpuxa.ircc.ohio-state.edu

pmorris@bbn.com (phil morris) (04/14/90)

>From: "Robert J. Tatz" <rjtatz@hpuxa.ircc.ohio-state.edu>
>Subject: GUI Benchmark (long)
>Date: 13 Apr 90 19:03:22 GMT
>To:       amiga-relay@udel.edu
>
>     We all know that statistics lie and that benchmarks can be mis-used.
>For example, MIPS cannot be used to compare CISC and RISC chips.  Due to
>different version numbers, quoting dhrystones can be questionable.  And
>checking out hard drives speeds can be dependent on how optimized or how
>full the hard drive is.  However, it is also abundantly clear that the 
>graphical user interface (GUI) is here to stay no matter who is selling
>the hardware.
>
>Regards,
>Bob          rjtatz@hpuxa.ircc.ohio-state.edu


It's in there!  NCGA has just come up with a specification for doing
just this (as seen in INFO World, Vol 12, issue 15 (April 9, 1990) on
pages 23 & 27.

Contact them at:

	National Computer Graphics Association
	Standards and Technical Services Department
	2722 Merrilee Drive, #200
	Fairfax, VA  22031

	(703) 698-9600
	(800) 225-NCGA ext. 318

Phil

--------
Phil Morris (pmorris@dgi0.bbn.com)
Disclaimer: ME? I'm only a non-smoking cat; can't believe a word I meow.