amos@taux01.UUCP (Amos Shapir) (04/27/88)
Here is something new for a change: When a file is removed, its blocks are released to the free list. If any writes were done, some of these blocks may still be in the buffer pool, waiting for a delayed write. When such buffers are reallocated, they will be written to the disk, even though their contents are no longer relevant. The systems I have checked - 4.2 and sysV.3 - make no effort to prevent this. It seems that in an environment in which many files are created, written to, and then discarded (e.g. compilation) preventing unnecessary writes to disk may have a significant effect on overall system performance. Does anyone know of a system that does this, and how? -- Amos Shapir (My other cpu is a NS32532) National Semiconductor (Israel) 6 Maskit st. P.O.B. 3007, Herzlia 46104, Israel Tel. +972 52 522261 amos%taux01@nsc.com 34 48 E / 32 10 N
edler@cmcl2.NYU.EDU (Jan Edler) (04/29/88)
In article <595@taux01.UUCP> amos@taux01.UUCP (Amos Shapir) asks
if any unix system bothers to cancel pending delayed writes of disk blocks
belonging to files that are being removed.
The current production operating system for the NYU Ultracomputer prototype
(symunix 1) does this. The buffers are also moved to the front of the
buffer cache free list, but in the opposite order of the coresponding
blocks on the disk block free list (so that the buffer most likely
to be reassigned is for the block least likely to be needed soon).
The reason I implemented this feature is that the hardware has fairly
poor i/o performance, and I was looking for ways to avoid unnecessary
i/o operations. The only data I have on the effectiveness of this
technique is numbers like the following, collected over the last 12 days:
number of file blocks freed: 112277
number of those that were in the buffer cache: 29844
number of those with pending delayed write: 12727
The disk blocks are fixed size, 2K bytes. The filesystem is close
to the v7 (or system V) design.
Subjectively, the improvement in overall performance due to this feature
is somewhat noticable, but not staggering.
Jan Edler
NYU Ultracomputer Project
edler@nyu.edurbj@icst-cmr.arpa (Root Boy Jim) (04/29/88)
From: Amos Shapir <amos@taux01.uucp> Here is something new for a change: When a file is removed, its blocks are released to the free list. If any writes were done, some of these blocks may still be in the buffer pool, waiting for a delayed write. When such buffers are reallocated, they will be written to the disk, even though their contents are no longer relevant. I find this hard to believe. If there *are* blocks waiting to be read, they must be associated with an inode, which is in core, and is therefore open by someone. If the file is removed, the link count in core will still be (at least) one, and I/O should happen normally, even tho the file has no directory entrys. The systems I have checked - 4.2 and sysV.3 - make no effort to prevent this. It seems that in an environment in which many files are created, written to, and then discarded (e.g. compilation) preventing unnecessary writes to disk may have a significant effect on overall system performance. Does anyone know of a system that does this, and how? When the file *is* finally closed, the inode ref count becomes zero, and the system should just ignore the delayed write flag and move the blocks to the free list. Seems simple enuf, but I have not read the code to check if it in fact does this. Perhaps it isn't even necessary, as blocks are only allocated on writes, which then overwrite the old data. I can't imagine that the old sata would be written first, but truth is stranger than fiction. Chris, say it ain't so! Amos Shapir (My other cpu is a NS32532) National Semiconductor (Israel) 6 Maskit st. P.O.B. 3007, Herzlia 46104, Israel Tel. +972 52 522261 amos%taux01@nsc.com 34 48 E / 32 10 N (Root Boy) Jim Cottrell <rbj@icst-cmr.arpa> National Bureau of Standards Flamer's Hotline: (301) 975-5688 The opinions expressed are solely my own and do not reflect NBS policy or agreement .. If I had heart failure right now, I couldn't be a more fortunate man!!
chris@mimsy.UUCP (Chris Torek) (05/03/88)
>Amos Shapir <amos@taux01.uucp>: >>... When a file is removed ... [and its] delayed write buffers are >>reallocated, they will be written to the disk, even though their >>contents are no longer relevant. In article <13188@brl-adm.ARPA> rbj@icst-cmr.arpa (Root Boy Jim) writes: >I find this hard to believe. Believe it. In 4.3BSD, though, full blocks are written immediately; only partial blocks are written with B_DELWRI. >If there *are* blocks waiting to be read, Not read---written. >When the file *is* finally closed, ... the system should just ignore >the delayed write flag and move the blocks to the free list. It cannot; the blocks are tagged only by <dev,blkno> pairs, and the truncation code does not search the cache for each such pair. >Chris, say it ain't so! (Why me?) Anyway, given that writes on full blocks are started immediately via bawrite() (see rwip(), when n+on==bsize), only partial block delayed writes could be avoided, and these are probably rare. Instead of trying to fix it, I say, wait for the `virtual memory filesystem' in 4.4BSD. -- In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 454 7163) Domain: chris@mimsy.umd.edu Path: uunet!mimsy!chris
df@nud.UUCP (Dale Farnsworth) (05/04/88)
Amos Shapir (amos@taux01.UUCP) writes: > Here is something new for a change: When a file is removed, its blocks > are released to the free list. If any writes were done, some of these > blocks may still be in the buffer pool, waiting for a delayed write. > When such buffers are reallocated, they will be written to the disk, > even though their contents are no longer relevant. > > The systems I have checked - 4.2 and sysV.3 - make no effort to prevent > this. It seems that in an environment in which many files are created, > written to, and then discarded (e.g. compilation) preventing unnecessary > writes to disk may have a significant effect on overall system > performance. Does anyone know of a system that does this, and how? Yes, a few years ago we made the same observation. When blocks belonging to a file are removed, the cached block list is searched and any blocks belonging to the file are moved to the front of the queue (to be allocated next) and any delayed write flags are cleared. -Dale -- Dale Farnsworth 602-438-3092 uunet!unisoft!nud!df