[comp.sys.amiga] Disk cache

rokicki@rocky.STANFORD.EDU (Tomas Rokicki) (05/30/87)

Now that my hard disk is gone and I'm back to floppies, I've decided
I need a disk cache.  A good one.  One with the following features:

	- Uses *available memory* exclusively, from the end of the
		free list to the front
	- Intercepts AllocMem to invalidate some cache buffers
	- Is public domain
	- Is *fast*
	- Is write through
	- Works with multiple drives; invalidate on eject

Perhaps I am wrong, but I estimate a few days should be all it takes.
I can handle everything off the top of my head but the intercept of
the trackdisk.device calls, which I've never done before.  Perhaps
someone would like to help me with this?

I know about Perry's facc, but does that use free memory, or is it
like vd0: in requiring a certain amount of memory be put aside?

Are there other programs which intercept AllocMem that I should be
careful of, so I don't step on their toes?

I mean, this could make things a *lot* faster than vd0:.  Comments?

-tom

cjp@vax135.UUCP (Charles Poirier) (06/01/87)

In article <333@rocky.STANFORD.EDU> rokicki@rocky.STANFORD.EDU (Tomas Rokicki) writes:
>I know about Perry's facc, but does that use free memory, or is it
>like vd0: in requiring a certain amount of memory be put aside?

Don't know about facc, but Perry's vd0: RRD does *not* put aside
memory.  You tell it the maximum size but it allocates and frees
memory as needed.
-- 
	Charles Poirier   (decvax,ucbvax,ihnp4,attmail)!vax135!cjp

	"The road to Hell is paved with good opinions."

perry@well.UUCP (Perry S. Kivolowitz) (06/01/87)

In article <333@rocky.STANFORD.EDU>, rokicki@rocky.STANFORD.EDU (Tomas Rokicki) writes:
> Now that my hard disk is gone and I'm back to floppies, I've decided
> I need a disk cache.  A good one.  One with the following features:
> 
> 	- Uses *available memory* exclusively, from the end of the
> 		free list to the front
> 	- Intercepts AllocMem to invalidate some cache buffers
> 	- Is public domain
> 	- Is *fast*
> 	- Is write through
> 	- Works with multiple drives; invalidate on eject
> 
> Perhaps I am wrong, but I estimate a few days should be all it takes.
> I can handle everything off the top of my head but the intercept of
> the trackdisk.device calls, which I've never done before.  Perhaps
> someone would like to help me with this?
> 
> I know about Perry's facc, but does that use free memory, or is it
> like vd0: in requiring a certain amount of memory be put aside?
> 

Tomas,
	Facc does fit your requirements and is available. Forty dealers
picked it up at Comdex, and several distributors as well. 

Here's some interesting news: Diskperf shows df1: (with Facc) to be
the fastest drive available anywhere - at 137,000 bytes per second.

I've made a list of things I want to add to facc and will do so. Anyone
owning Facc simply send inthe original disk with a stamped self addressed
envelope and get the lastest version (for ever).

Oh, you should definitely NOT patch allocmem to discard buffers during tight
times. There are plenty of other ways that are more compatible and more
portable.

Perry

carolyn@cbmvax.cbm.UUCP (Carolyn Scheppner CATS) (06/02/87)

In article <333@rocky.STANFORD.EDU> rokicki@rocky.STANFORD.EDU (Tomas Rokicki) writes:
>
>I can handle everything off the top of my head but the intercept of
>the trackdisk.device calls, which I've never done before.  Perhaps
>someone would like to help me with this?

  See my Cmd.c/Cmda.asm posted here a few days ago for an example of
  SetFunction()ing some of a device's function vectors.

>Are there other programs which intercept AllocMem that I should be
>careful of, so I don't step on their toes?

  YES!  You must be very careful when using SetFunction() and especially
careful not to uninstall yourself if YOUR vectors have since been
replaced by another program.  Again, see Cmd.c for an example of
safe de-installation of SetFunction()'d vectors.  Otherwise, if more
than one program has SetFunction()'d the same vector, the original
true vector can be lost if de-installations are done in the wrong
order.  The code insures that de-installations are done in the
reverse order of the installations.

  The de-installation code is by Andy Finkel and was written for an
important program that SetFunction()s both AllocMem() and AvailMem().
The program is NoFastMem which will appear on the A500/A2000 Workbench.
It is used to temporarily force all memory allocation and availablity
requests to CHIP so that people with internal FAST ram will still be
able to load and run programs that are not written properly for FAST ram.

-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Carolyn Scheppner -- CBM   >>Amiga Technical Support<<
                     UUCP  ...{allegra,caip,ihnp4,seismo}!cbmvax!carolyn 
                     PHONE 215-431-9180
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

andy@cbmvax.UUCP (06/02/87)

In article <333@rocky.STANFORD.EDU> rokicki@rocky.STANFORD.EDU (Tomas Rokicki) writes:
>Now that my hard disk is gone and I'm back to floppies, I've decided
>I need a disk cache.  A good one.  One with the following features:
>
>	- Uses *available memory* exclusively, from the end of the
>		free list to the front
>	- Intercepts AllocMem to invalidate some cache buffers
This method sounds like a bad thing...you will have memory that
isn't really free, and isn't really used.  (tristate memory ?)
Its usually dangerous to take control of memory allocation away
from the OS.

How about something like this instead...do a library (called memory.library
or something) that keeps track of memory for the buffers.  Give it at least
4 calls: AllocateBuffer, FreeBuffer, AvailBuffers, and Expunge (as
the normal Expunge entry point).  Expunge, when called causes
the library to invalidate all buffers and throw them away.  Exec will
use this in a low memory situation.  Then, to improve performance,
fire up a task which uses the first 3 calls to give finer control.
That task could be running at a fairly high priority, checking memory usage,
and adding and/or subtracting buffers as memory requirements change.
You could use AbsAllocate to make it use memory from the top down.
>
>-tom
			good luck,
			andy
-- 
andy finkel		{ihnp4|seismo|allegra}!cbmvax!andy 
Commodore/Amiga		

"An end is always a new beginning." - Captain Cloud

Any expressed opinions are mine; but feel free to share.
I disclaim all responsibilities, all shapes, all sizes, all colors.

bryce@COGSCI.BERKELEY.EDU.UUCP (06/03/87)

In article <> cbmvax!andy@harvard.harvard.EDU typed:
>>	- Intercepts AllocMem to invalidate some cache buffers
> This method sounds like a bad thing...you will have memory that
> isn't really free, and isn't really used.  (tristate memory ?)

Agreed.  There is a problem with other solutions (read on)


>---------
> [make] a library (called memory.library or something) that keeps track of
> memory for the buffers.
> [when exec needs memory it will call the expunge vector which]
> when called causes the library to invalidate all buffers and throw them away.

Here is a major gripe I have had with expunge.  Exec will call it when it is
low on memory, but does not say HOW low.  Without some fancy stuff one would
have to flush all the cache blocks... waseful.   If the exec call to
expunge would specify HOW much memory is needed, a deal could be struck.
(1.3? Please?!)


>---------
> Then, to improve performance, fire up a task which uses the [allocation]
> calls to give finer control.
> That task could be running at a fairly high priority, checking memory usage,
> and adding and/or subtracting buffers as memory requirements change.

Ok, that will do for now.  But the system really has a better idea of what the
memory situation is like.   It can also prevent thrashing between competing
ideas of "high and low water marks".
In addition to the size mentioned above
I would like to see a PRIORITY attached to expungeable stuff.  For example
512 cache buffers are quite expendible, a cache of just a directory is somewhat
less and an actual library less still.  Non-libraries may want to get into the
act also, say a wordprocessor that can edit any size file but uses an area
of RAM to hold of the part that is currently active.  (ie. WORDPERFECT etc.).


My pick would be a set of extensions to exec to allow all this dynamic memory
mangement (sic).  PRIORITIES, VOLATILITIES and AMOUNTS.
(How volatile is that request? Will it be deallocated in a few micros or does
that plan on hang{_ing around for hours/days?)


>-------
>You could use AbsAllocate to make it use memory from the top down.

Semi-related.  Could AllocMem initially refuse to break up a large block with
a small request, and instead skip ahead to a smaller block later on?  From
the standpoint of fragmentation large and small requests would seem to want
to be kept somewhat separate.


>-------
>andy finkel		{ihnp4|seismo|allegra}!cbmvax!andy 

cbmvax is invisible from here unless  cmbvax!user@harvard.harvard.EDU  is used.


-------------
         Ack!  (NAK,EOT,SOH)
 |\ /|  .
 {o O} .  bryce@cogsci.berkeley.EDU -or- ucbvax!cogsci!bryce
 ( " ) 
   U      Single tasking?  Just say *NO!*