dillon@CORY.BERKELEY.EDU (Matt Dillon) (05/23/87)
>Well, I've been thinking again (when will I learn my lesson). What I >was thinking was that it would be great to have a trackdisk.device that >used up all of available (chip?) ram for track buffers, and released the buffers >on a least recently used basis when memory was allocated (in sort of the >same way that libraries stick around until the memory is needed). What is needed is another couple of Exec calls allowing one to install a "memory low" signal to task X (allowing any number of tasks to do this). You would specify some sort of low-water mark below which you want to be notified that memory is running low, and a high-water mark above which you want to be notified that there is plenty of memory. The marks should be placed far enough away so that you don't get into any infinite oscillation loops (could occur when several tasks are using the feature). Then, Exec would signal the specified tasks (with either the high water or low water signal) when the specified condition occurs. The end result is that the system could dynamically utilize the available memory without those tasks (like RAM:) having to poll the amount of free memory available every so often. For instance, This feature could be used to automatically choose buffer sizes for stdio file pointers and dynamically change them if the memory load changes. -Matt
hamilton@uxc.cso.uiuc.edu (05/24/87)
dillon@CORY.BERKELEY.EDU says: > What is needed is another couple of Exec calls allowing one to > install a "memory low" signal to task X (allowing any number of tasks to do > this). You would specify some sort of low-water mark below which you > want to be notified that memory is running low, and a high-water mark > above which you want to be notified that there is plenty of memory. The marks > should be placed far enough away so that you don't get into any infinite > oscillation loops (could occur when several tasks are using the feature). > Then, Exec would signal the specified tasks (with either the high > water or low water signal) when the specified condition occurs. nice idea, but it doesn't require "another couple of Exec calls" (beware creeping featurism). simply create a background task with a "well-known" port name, send "add-me-to-your-distribution-list" and "drop-me..." messages to it from your application, and monitor your own port for status bulletins. you could take it further, creating a "memory broker". some applications need large quantities of RAM on an all-or-nothing basis (bitmaps, waveforms, etc), while others can take it or leave it (huge stdio buffers, auxiliary track buffers, etc). the broker would allow the first class of application to request that the second class relinquish some resources. sorta like what sometimes happens on overbooked airline flights - asking for volunteers to fly later. we already have a simple form of this: the Expunge entry of libraries and devices. wayne hamilton U of Il and US Army Corps of Engineers CERL UUCP: {ihnp4,seismo,pur-ee,convex}!uiucuxc!hamilton ARPA: hamilton@uxc.cso.uiuc.edu USMail: Box 476, Urbana, IL 61801 CSNET: hamilton%uxc@uiuc.csnet Phone: (217)333-8703 CIS: [73047,544] PLink: w hamilton
scotty@l5comp.UUCP (05/25/87)
In article <8705222220.AA29170@cory.Berkeley.EDU> dillon@CORY.BERKELEY.EDU (Matt Dillon) writes: >>Well, I've been thinking again (when will I learn my lesson). What I >>was thinking was that it would be great to have a trackdisk.device that >>used up all of available (chip?) ram for track buffers, and released the buffers >>on a least recently used basis when memory was allocated (in sort of the >>same way that libraries stick around until the memory is needed). > > What is needed is another couple of Exec calls allowing one to >install a "memory low" signal to task X (allowing any number of tasks to do >this). You would specify some sort of low-water mark below which you Track buffers, like any buffers, have this inherent problem. How to determine when enough is enough. The simplest way to see what I mean is to go find a Mac with the disk cache software installed. And play with it. You should find, like I did, that there comes a point where making the cache bigger doesn't return any benifit. It's all well and good to throw all the memory you can lay your hands on at the buffer scheme and then hand it back as needed. BUT it's far better to make a "smart" buffering system that only uses a buffer when it knows that this will result in a net gain in performance. It's a little tricky for us to do this since we're on the outside looking into the dog house. The dog has all sorts of data that would make these determinations much easier to make. Alas we have to do it through the back door, by watching the calls to trackdisk.device. This is imperfect because the dog only calls trackdisk.device for ONE block at a time! We can't get a feel for how much data it needs off one track till it's too late. So we have to throw a buffer at it. And if only one block was needed off that track we may have just wasted a buffer, and if we had to flush the buffer before we could use it for this track we may have just wasted TWO tracks worth of buffering. The dog however knows two things: 1. How big the I/O request is that it is sitting on (ie how many bytes did we ask the dog to read for us.) 2. How many of those blocks that it needs to read are IN this track. What is needed is a CACHE_DISABLE flag from the dog. When the dog determines that it's only going to read/write one block from the track it could set this bit in the I/O request block to advise us of this. Now some of you are saying "But what if we only read one block at a time from the dog? We won't get cached!" Two comments apply here: 1. Perform your I/O with bigger requests. :) 2. The cache could be setup to cache blocks even if they are marked disabled, but only if it doesn't mean flushing a track buffer being used for a NON-DISABLED request... Ideally the dog would just perform all the I/O at one bang to ANY disk.device. But thanks to the #$@! checksum in EACH AND EVERY BLOCK it can only chew one block at a time. When addbuffers was added I thought "Oh BOY!! The dog is going to start making multiple block requests!" My thinking was that by having these buffers, which we could tune the number of, the dog would be able to read all the data it needed for one request (or as much as it had buffer space for) into these buffers and THEN crack the checksums off and copy them to the user buffer. But that isn't happening, so whoever implemented the dog's buffering scheme really blew it. So what I would like to see is the dog use these buffers, that it ALREADY HAS, PROPERLY! Then WE wouldn't have to fool around with replacing or intercepting calls to the disk.devices. No need for complex schemes of grabbing ram and then giving it back. It would be nice if addbuffers could take back some of those buffers as well as add them. :) That way we could tune things on the fly with greater control. But this wouldn't be REQUIRED. The elegance of my suggestion is that it doesn't call for any .library additions. Matt's scheme sounds great, but would require C-A to add calls/ documention/and SUPPORT for a whole new concept. And as Matt pointed out, if used incorrectly bad things could happen. My concept doesn't require ANY extra calls/dox/support. It just requires C-A to use the buffers already there properly. EVERYTHING needed is ALREADY in place. But alas, it requires work from C-A so we'd see it oh, when HELL FREEZES OVER. And alas I can't see any way for US to do it since it requires work on the kickstart... So all those people out there with ROM'd kicks wouldn't get much use from this change. Maybe a massive write in campaign? This BS with misusing the buffers already there IS one of the key reasons the dog is the dog. I know my hard disk system would perk up ALOT if multi-block requests started coming in from the dog. Me, I'm going to keep pluggin' away at my own filesystem handler. Scott Turner -- L5 Computing, the home of Merlin, Arthur, Excalibur and the CRAM. GEnie: JST | UUCP: stride!l5comp!scotty | 12311 Maplewood Ave; Edmonds WA 98020 If Motorola had wanted us to use BPTR's they'd have built in shifts on A regs [ BCPL? Just say *NO*! ] (I don't smoke, send flames to /dev/null)