[comp.windows.misc] text benchmarks ???

boots@motorboat.WV.TEK.COM (Darrell Boots Irvin) (08/30/89)

In recent postings to comp.windows.x and comp.windows.misc
Jef Poskanzer posts "Yet Another Benchmark to measure text speed".

Please note that system benchmarking and performance profiling
is a much more complex process than it might seem at first glance.
It is very difficult to determine exactly what you are measuring.

Mr. Poskanzer's text benchmark is not a true measure of the terminal
device's character performance.  It is more a measure of the single
character I/O performance and other bottlenecks of the system running
the benchmark.  I have included some changes to the program to illustrate
this problem "of not measuring what you think you are measuring".

The origional benchmark did its character I/O using putc as shown
below.  The inner loop using putc is very inefficient.

---------------------------------Origional Function---------------------

chargen( fd, chars, charsperline )
FILE *fd;
int chars, charsperline;
    {
    int i, j;

    for ( i = chars; i > 0; i -= charsperline + 1 )
	{
	for ( j = min( charsperline, i - 1 ); j > 0; j-- )
	    putc( '@', fd );
	putc( '\n', fd );
	}
    fflush( fd );
    }

---------------------------------------------------------------------------

Creating a "new" character output function using fprintf to replace 
the calls when timing "Characters Per Second" results in a great
"improvement in character performance" with no other changes to the system,
in particular the speed of the display device/software.

-----------------New Character Print Function (70 chars/line)--------------

chargen1( fd, chars, charsperline )
FILE *fd;
int chars, charsperline;
    {
    int i, j;

    for ( i = chars; i > 0; i -= charsperline + 1 )
	{
	    fprintf (fd,"@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\n");
	}
    fflush( fd );
    }

---------------------------------------------------------------------------


    SYSTEM CONFIGURATION		BEFORE		    AFTER

Tektronix 4325 workstation UTEK 3.1   1115 chars/sec      10345 chars/sec
(internal console)		      952 scrolls/sec     1034 scrolls/sec

Tektronix 4325 workstation UTEK 3.1   728 chars/sec       8451 chars/sec
(X11.3 34x80 xterm)		      458 scrolls/sec      469 scrolls/sec

Tektronix 4325 workstation UTEK 3.1   492 chars/sec       4959 chars/sec
(telnet to Tek 4211 Netstation)       417 scrolls/sec      833 scrolls/sec

Tektronix 4325 workstation UTEK 3.1   633 chars/sec       2778 chars/sec
(38.4 Kbaud Rs232 to Tek 4207)        326 scrolls/sec      319 scrolls/sec

Sun 386i SunOS 4.0.1                  240 chars/sec        743 chars/sec
(internal console)                     95 scrolls/sec       94 scrolls/sec

Sun 386i SunOS 4.0.1                  241 chars/sec       1376 chars/sec
(Daisy X11.2 34x80 xterm)	      135 scrolls/sec      135 scrolls/sec

Sun 386i SunOS 4.0.1                  403 chars/sec       4478 chars/sec
(telnet to Tek 4211 Netstation)       347 scrolls/sec      458 scrolls/sec

Sun 386i SunOS 4.0.1                 1003 chars/sec       2542 chars/sec
(38.4 Kbaud Rs232 to Tek 4207)        273 scrolls/sec      311 scrolls/sec

---------------------------------------------------------------------------

 Note that there was a general increase of about 10x in character performance,
except where "external" influences started to appear.  However we still do not
know what was the limiting factor.  Some possible causes could be:

 The telnet connections were probably influenced by net load factor.

 The Rs232 device may be limited to around 25 to 30 Kbaud under the
conditions it was run. 

 The Daisy X11.2 server and 386i console probably have character performance
problems, most likely in bitblt code used to scroll the screen between
character lines.

 In none of the cases was the system isolated from "outside" influences
such as other users, uneeded processes or net traffic.  Thus we simply
can't say what the performance really is.

MORAL OF THE STORY......."Make sure you know what you are measuring before
                          you claim to have measured it!"

pokey@well.UUCP (Jef Poskanzer) (08/30/89)

In the referenced message, boots@motorboat.WV.TEK.COM (Darrell Boots Irvin) wrote:
}The origional benchmark did its character I/O using putc as shown
}below.  The inner loop using putc is very inefficient.

You're right.  I have a reputation for being moderately bad at optimizing.
Now I have a bigger reputation.

I'll take another look at this benchmark some other time.
---
Jef

    Jef Poskanzer  pokey@well.sf.ca.us  {ucbvax, apple, hplabs}!well!pokey
                           "Why me, John Bigboote?"