[comp.sys.ibm.pc] More disk cache tests

reisert@ricks.enet.dec.com (Jim Reisert) (07/01/90)

Hi,

I did some more cache measurements.  These were all done on a 20 MHz 386
machine with a 42 KB hard disk running MS-DOS 4.01.  I used QEMM as my
EXPAnded memory manager.  All tests were run at the DOS prompt (as opposed
to under Windows or DESQview).  This note includes all measurements made to
date.

The following tests were made using the PC-Labs Benchmark Tests, v5.0.  The
tests are as follows:

	TEST1 - 512 records of   512 bytes each
	TEST2 -  64 records of  4096 bytes each
	TEST3 -  16 records of 16384 bytes each
	TEST4 -   8 records of 32768 bytes each

Each test had 5 stages: 1) file creation, 2) sequential write, 3) sequential
read, 4) random write and 5) random read.  The results are in seconds.

These tests were made using a 384K cache in EXPAnded memory:

		TEST1	TEST2	TEST3	TEST4

no cache	58.50	8.86	5.44	3.99	secs.
SMARTDRV	35.06	7.26	4.00	3.33	 "
PC-CACHE 5.5	30.29	6.70	3.73	3.32	 "
PC-CACHE 6.03	13.13	8.53	3.14	2.75	 "
SuperPCK 3.56	 8.52	4.94	1.83	2.45	 "
SuperPCK 3.19	    - not available -

These tests were made using a 384K cache in EXTEnded memory:

		TEST1	TEST2	TEST3	TEST4

no cache	58.73	9.14	5.50	4.18
SMARTDRV	34.85	7.25	4.01	3.37
PC-CACHE 5.5	29.56	6.51	3.61	3.01
PC-CACHE 6.03	12.50	8.64	4.88	3.87
SuperPCK 3.56	11.07	5.21	3.56	2.36
SuperPCK 3.19	 6.00	5.79	1.60	2.54

Here are some results from CORETEST v2.8.  CORETEST writes 42KB of data and
measures the access speed in KB/sec.  All caches are 384K.

		EXTEnded	EXPAnded

no cache	   537	KB/sec	   537	KB/sec
PC-CACHE 6.03	  2153	  "	  5369    "
SuperPCK 3.56	  4193	  "	  3994	  "
SuperPCK 3.19	  4609	  "	   N/A	  "
PC-CACHE 5.5	  4816	  "	  4368	  "
SMARTDRV	  5479	  "	  3994	  "

I don't know what to make of these measurements.  These say that if you're
using EXTEnded memory, SMARTDRV is the way to go, but if you're using
EXPAnded, then use PC-CACHE 6.03.  The PC-Labs Benchmark Tests clearly show
Super PC-Kwik to be superior in all categories.  Furthermore, for the
PC-Labs tests, the results were similar for both EXPAnded and EXTEnded
memory caches.  CORETEST showed much more variation depending on the cache
and type of memory.

I merely offer this up for discussion, since I'm not sure what to make of it
all.  I'm going to stick with Super PC-Kwik, though.  Anyone care to comment
on any of this?

jim

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

"The opinions expressed here in no way represent the views of Digital
 Equipment Corporation."

James J. Reisert                Internet: reisert@ricks.enet.dec.com
Digital Equipment Corp.         UUCP:     ...decwrl!ricks.enet!reisert
77 Reed Road
Hudson, MA  01749-2895

silver@xrtll.uucp (Hi Ho Silver) (07/01/90)

In <12988@shlump.nac.dec.com> reisert@ricks.enet.dec.com (Jim Reisert) writes:
[...]
$These tests were made using a 384K cache in EXPAnded memory:
$SMARTDRV	35.06	7.26	4.00	3.33	 "
$PC-CACHE 6.03	13.13	8.53	3.14	2.75	 "
$SuperPCK 3.56	 8.52	4.94	1.83	2.45	 "
[...]
$These tests were made using a 384K cache in EXTEnded memory:
$SMARTDRV	34.85	7.25	4.01	3.37
$PC-CACHE 5.5	29.56	6.51	3.61	3.01
$PC-CACHE 6.03	12.50	8.64	4.88	3.87
$SuperPCK 3.56	11.07	5.21	3.56	2.36
$SuperPCK 3.19	 6.00	5.79	1.60	2.54

[CORETEST results deleted]

$I don't know what to make of these measurements.  These say that if you're
$using EXTEnded memory, SMARTDRV is the way to go, but if you're using
$EXPAnded, then use PC-CACHE 6.03.  The PC-Labs Benchmark Tests clearly show

   It's apparent that you don't know what to make of these measurements.
In extended memory, SuperPCK (both versions) beat SMARTDRV in all cases,
so why do you say SMARTDRV is better?  And in expanded memory, SuperPCK
beat PC-Cache in all cases, so why do you say PC-Cache is better?

$Super PC-Kwik to be superior in all categories.  Furthermore, for the

   Your tests also show this.

$I merely offer this up for discussion, since I'm not sure what to make of it
$all.  I'm going to stick with Super PC-Kwik, though.  Anyone care to comment
$on any of this?

   Sure.  Here are a few more things to consider:

- Using a utility such as CORETEST to measure the performance of a disk
  cache is futile, since the activity it performs (repeatedly reading the
  same information) is not a typical task in the real world and, furthermore,
  is the situation in which disk caches excel the most.

- The relative performance of various caches will depend on your hardware.
  For example, of the caches mentioned above, some may be aware when they
  are running on a 386 while others may not.  If using extended memory,
  this will make a huge performance difference since the 386-aware caches
  will realize they don't have to reset the processor to return to real
  mode like they do on a 286.  Other factors which vary the performance
  in varying degrees include the speed of your memory and the data
  transfer rate and seek time of your hard drive.

- Changing the size of your cache, as well as tweaking some of the
  parameters which better caches allow you to do, will alter the
  relative performance levels of the caches.

- One advantage which SMARTDRV has, if you're running Windows, is that
  it will dynamically alter the amount of memory it's using in response
  to Windows' memory requirements.  This is about the only reason I can
  think of why anyone would want to use SMARTDRV.

- Every user will benefit differently from a cache, since every user
  performs a different mix of activity.  As you can see from the
  tables, some caches perform better on some tests and worse on others.
  The best cache for someone who edits, compiles and tests programs may
  or may not be the same as the best cache for someone who does heavy
  database work, even on the same hardware.
-- 
   /Nikebo \ Nikebo says "Nikebo knows how to post.  Just do it."\silver@xrtll/
  /---------\_____________________________________________________\----------/
 /yunexus!xrtll!silver (L, not 1)\ Hi Ho Silver \   just silver for short   /
/Silver:  Ever Searching for SNTF \  Life sucks. \  someone buy me a BEER! /

reisert@ricks.enet.dec.com (Jim Reisert) (07/01/90)

In article <1990Jul1.003125.18399@xrtll.uucp>, silver@xrtll.uucp (Hi Ho Silver) writes...
>In <12988@shlump.nac.dec.com> reisert@ricks.enet.dec.com (Jim Reisert) writes:
> 
>[CORETEST results deleted]
> 
>$I don't know what to make of these measurements.  These say that if you're
>$using EXTEnded memory, SMARTDRV is the way to go, but if you're using
>$EXPAnded, then use PC-CACHE 6.03.  The PC-Labs Benchmark Tests clearly show
> 
>   It's apparent that you don't know what to make of these measurements.
>In extended memory, SuperPCK (both versions) beat SMARTDRV in all cases,
>so why do you say SMARTDRV is better?  And in expanded memory, SuperPCK
>beat PC-Cache in all cases, so why do you say PC-Cache is better?

I knew what I wanted to say, I just didn't say it right.  I meant these
comments to refer to the results of CORETEST only.  Of course it's clear
that using the PC Labs benchmarks, Super PC-Kwik comes out on top.  Sorry
for any confusion this might have caused.  The comments about CORETEST being
a poor test are informative and noted.

jim

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

"The opinions expressed here in no way represent the views of Digital
 Equipment Corporation."

James J. Reisert                Internet: reisert@ricks.enet.dec.com
Digital Equipment Corp.         UUCP:     ...decwrl!ricks.enet!reisert
77 Reed Road
Hudson, MA  01749-2895