ranjit@eniac.seas.upenn.edu (Ranjit Bhatnagar) (02/25/89)
What is the appropriate behavior for Examine/ExNext if the contents of a directory change between one request and the next? Should it always report the most up-to-date information possible, even though this may lead to inconsistent lists, or should it buffer the entire directory at the time Examine is called, even though this may lead to out-of-date information, and the buffer may be stuck forever if the Examining program is aborted before it finishes? I have a feeling that no existing filesystem buffers the directory list - but how do current filesystems handle this problem, then? What if the directory entry last examined is deleted before ExNext is invoked on it? What if the next entry (the one the hypothetical 'next' pointer in the previous entry) is deleted? What if EVERYTHING is deleted? The only solution I can think of is to search the entire directory every time its datestamp changes, and skip all those files which we've already seen. I suppose that it's possible to do an ExNext on a directory entry that not only doesn't exist, but never did, because the filesystem must assume that it did exist at some time. - ranjit "Trespassers w" ranjit@eniac.seas.upenn.edu mailrus!eecae!netnews!eniac!... Where your eyes don't go a filthy scarecrow waves its broomstick arms and does a parody of each unconscious thing you do. When you turn around to look it's gone behind you; on its face it's wearing your confused expression...
w-colinp@microsoft.UUCP (Colin Plumb) (02/27/89)
ranjit@eniac.seas.upenn.edu.UUCP (Ranjit Bhatnagar) wrote: > What is the appropriate behavior for Examine/ExNext if the contents > of a directory change between one request and the next? This is a problem I've been dealing with. The current system does some very strange things under certain circumstances. Anyway, the general rule I've gone by is that a file that is created or destroyed during an ExNext chain may be reported zero or one times. If files a and b are created, in that order, it is acceptable to report b but not a. > I have a feeling that no existing filesystem buffers the directory > list - but how do current filesystems handle this problem, then? > What if the directory entry last examined is deleted before ExNext > is invoked on it? What if the next entry (the one the hypothetical > 'next' pointer in the previous entry) is deleted? What if EVERYTHING > is deleted? If the entry the "current" fib "points" to is deleted, the file system is required to handle it correctly. The current FS's just read the block anyway. If it hasn't been reallocated, its next pointer will still be intact. If it has been reallocated, the old FS follows its next pointer anyway, resulting in a very confused directory listing. The FFS notices if something has gone wrong, and tries to do something sane. It does not always succeed. The next pointer is not stored in the fib; it only exists on disk, and is always (or at least whenever another process can examine it) correct. If everything is deleted, ExNext() may return information about no-longer-extant files, but will probably return ERROR_NO_MORE-ENTRIES. > The only solution I can think of is to search the entire directory > every time its datestamp changes, and skip all those files which we've > already seen. I suppose that it's possible to do an ExNext on a > directory entry that not only doesn't exist, but never did, because > the filesystem must assume that it did exist at some time. Well, how do you skip them? Is there an order imposed on the entries? I'm having fun becasue I can't do that in my filesystem. Anyway, ExNext() may be passed anything that may have ever been returned by an Examine() of the lock or by an ExNext() on the same lock. A few notes: apparently, there exists a wildcard-matching utility that unlocks a directory between calls to ExNext(). THIS IS A BAD THING. The FFS has (and my filesystem will have) kludges to make this work most of the time, but people who do this are EVIL. The kludges slow things down dramatically (when they're invoked). The standard way to maintain more state information than you can fit into a FIB is to add it to the accompanying lock. When the lock is UnLock()ed, you can throw away the state. Thus, it is considered good practice to UnLock a lock after you're done ExNext()ing on it. This is usually easy, but if you're ExNext()ing a long-term lock of yours, please DupLock() it beforehand. Thusm the Posix opendir corresponds to Lock() or DupLock() and Examine(), readdir() corresponds to ExNext(), and closedir() corresponds to UnLock(). Other information: FFS can't cope well with more than one ExNext() chain scanning a single lock. I don't like this (I'm going to fix it in my FS), bit it's something to keep in mind. Does anyone have any opinions on what a directory scanning system *should* do? Do you mind if I completely ignore files created since the first ExNext() of a chain? -- -Colin (uunet!microsoft!w-colinp) "Don't listen to me. I never do."
jbwaters@bsu-cs.UUCP (J. Brian Waters) (03/01/89)
In article <757@microsoft.UUCP>, w-colinp@microsoft.UUCP (Colin Plumb) writes: > A few notes: apparently, there exists a wildcard-matching utility that > unlocks a directory between calls to ExNext(). THIS IS A BAD THING. > The FFS has (and my filesystem will have) kludges to make this work > most of the time, but people who do this are EVIL. The kludges slow > things down dramatically (when they're invoked). The Manx scdir() function does this.... I have written a replacment for it that I link in with most of my programs. Unfortantly, my replacment requires the arp library. I think they do this to avoid leaving a lock around if someone aborts the program between calls to scdir(), my replacment gets around this by using arp's resource tracking. Does anyone have a replacment that avoids leaving a lock if aborted, and avoids calling UnLock() after every call? Sigh, I hope the next Manx fixes this, and also includes a atexit() which would make fixing this easier. -- Brian Waters <backbone>!{iuvax|pur-ee}!bsu-cs!jbwaters
edwin@hcr.UUCP (Edwin Hoogerbeets) (03/08/89)
In article <5977@bsu-cs.UUCP> jbwaters@bsu-cs.UUCP writes: >In article <757@microsoft.UUCP>, w-colinp@microsoft.UUCP (Colin Plumb) writes: >> A few notes: apparently, there exists a wildcard-matching utility that >> unlocks a directory between calls to ExNext(). THIS IS A BAD THING. >The Manx scdir() function does this.... I have written a replacment for it >that I link in with most of my programs. [etc] >Does anyone have a replacment that avoids >leaving a lock if aborted, and avoids calling UnLock() after every call? Manx allows you to define the function _abort() which is called from the function Chk_Abort if ^C was pressed. Chk_Abort is called in the i/o routines. In any case, this allows you to clean up temporary files, free memory, and other neat things when someone tries to send a break to the program. So you could go ahead and leave your locks around, but clean them up with _abort(). Unfortunately, there is no way to put this into a library, is there? If you try to link your library containing the revised scdir and your _abort before the standard c.lib, _abort won't be needed and won't get linked in. Also, if you link your library after the c.lib, the linker will pick up the _abort already in c.lib, and will ignore yours! You can't win. Maybe you could rewrite the new scdir type function to take a lock as one of its arguments. This way you could do a scdir on any directory you want, and let the user worry about Locking and UnLocking the [directory] file. It is also my guess that this method would be re-entrant too. (added bonus!) Edwin
jbwaters@bsu-cs.UUCP (J. Brian Waters) (03/13/89)
In article <5307@hcr.UUCP>, edwin@hcr.UUCP (Edwin Hoogerbeets) writes: > Unfortunately, there is no way to put this into a library, is there? If you > try to link your library containing the revised scdir and your _abort > before the standard c.lib, _abort won't be needed and won't get linked > in. Also, if you link your library after the c.lib, the linker will > pick up the _abort already in c.lib, and will ignore yours! You can put it into a library by having the scdir replacment and _abort load together as one object module. This would not solve the problem though as a program could call exit and skip the _abort routine... it really needs to be part of the cleanup routines. -- Brian Waters <backbone>!{iuvax|pur-ee}!bsu-cs!jbwaters
peter@sugar.hackercorp.com (Peter da Silva) (03/15/89)
The only safe thing to do is have scdir() read all the file names right off into malloced ram. That solves both problems: loose locks and changing directories %. Then later calls to scdir() would just step down the list and free the names... or not. -- % Yes, I know this just narrows the window of vulnerability. -- Peter "Have you hugged your wolf today" da Silva `-_-' ...texbell!sugar!peter, or peter@sugar.hackercorp.com 'U`