czei@osupyr.mast.ohio-state.edu (Michael S Czeiszperger) (04/04/88)
I'm managing some suns for the first time, and had quite a shock when the disk read 106% full. After reading the manual, it explained that the 'df' command adjusts the reading so that when it says 100% full, the disk really is only 90% full. It goes on to say that having the disk over 90% would affect its performance. Does anyone know exactly how the performance is affected, and if there are any reasons not to keep a disk at 96-99%?????? Michael S. Czeiszperger | Guest account courtesy of Math Dept. Systems Analyst | Snail: Room 568 Dreese Labs (614) Dept. of Electrical Engineering | 1971 Neil Avenue 292- The Ohio State University | Columbus, OH 43210 0161 cbosgd!osu-cis!osupyr!czei | czei@osupyr.mast.ohio-state.edu
fsinf@iravcl.ira.uka.de (04/10/88)
In article <460@osupyr.mast.ohio-state.edu>, czei@osupyr.mast.ohio-state.edu (Michael S Czeiszperger) writes: > [...] the 'df' command adjusts the reading so that when it says 100% full, > the disk really is only 90% full. It goes on to say that having the > disk over 90% would affect its performance. Does anyone know [...] > if there are any reasons not to keep a disk at 96-99%?????? Maybe the reasons are ... I've heard (but not verified) you can crash *every* unix-system using the CP-command when there is not enough space on disk. CP will not check whether the disk is full and overwrite blocks which are not free. The original data will be lost; also the machine is likely to go down. If this isn't right, please correct. Regards, Elmar ------------------------------------------------------------------------------- Everything should be made as easy as possible, but not easier. --Albert Einstein
pardo@june.cs.washington.edu (David Keppel) (04/12/88)
In article <92@iravcl.ira.uka.de> fsinf@iravcl.ira.uka.de writes: >czei@osupyr.mast.ohio-state.edu (Michael S Czeiszperger) writes: >>[ df says 100%, really only 90%. Rationalle: performance ] >[ crash if full ] There is a tech report from Berkeley about the FFS (fast file system) and some mumbling about performance vs. full. I have the paper at home but it has been a while since I read it. I think it is worthwile reading if you can find a copy. My local guru says that the packing algorithm for multi-block files tries to put things together in a pattern that give fast sequential access. If there are very few free blocks, then the file system manager is forced to allocate whatever blocks might happen to be around, even if they are a large seek (+settling+latency) time away. ;-D on ( But then there's always paper tape ) Pardo
mike@turing.UNM.EDU (Michael I. Bushnell) (04/12/88)
In article <92@iravcl.ira.uka.de> fsinf@iravcl.ira.uka.de writes: >In article <460@osupyr.mast.ohio-state.edu>, czei@osupyr.mast.ohio-state.edu (Michael S Czeiszperger) writes: >> [...] the 'df' command adjusts the reading so that when it says 100% full, >> the disk really is only 90% full. It goes on to say that having the >> disk over 90% would affect its performance. Does anyone know [...] >> if there are any reasons not to keep a disk at 96-99%?????? >Maybe the reasons are ... >I've heard (but not verified) you can crash *every* unix-system using the >CP-command when there is not enough space on disk. CP will not check >whether the disk is full and overwrite blocks which are not free. The original >data will be lost; also the machine is likely to go down. Total nonsense. CP doesn't care beans which disk blocks are free. The kernel does, however. When the kernel can't find a free block, it doesn't just pick an in-use one. That would be REALLY stupid. Instead, it just returns the ENOSPC (No space left on device) error code. And usually prints something like "/usr: file system full" on the console. (4.3 BSD uses the system error logger instead of the console). The reason for the behavior of df is more subtle. In the 4.2 Fast File System, an attempt is made to keep every file "close together" on the disk. The disk is divided into "Cylinder groups" which represent a group of cylinders on the disk. Each file is stored in exactly one cylinder group if possible. (Simplification: large files are split between cylinder groups.) This significantly improved disk access times in Unix. However, as the disk fills up, it becomes harder and harder to allocate files in single cylinder groups. Experimentation revealed that the deprication happened are roughly 90% full. As a result, the kernel supports the feature of reserving most of the disk for use only by superuser...maintaining the 10% margin for performance. If you want the actual statistics, look at the document "A Fast File System for Unix". If you want to change the free space percentage, you can use the tunefs command. >If this isn't right, please correct. Consider it done. N u m q u a m G l o r i a D e o Michael I. Bushnell HASA - "A" division 14308 Skyline Rd NE Computer Science Dept. Albuquerque, NM 87123 OR Farris Engineering Ctr. OR University of New Mexico mike@turing.unm.edu Albuquerque, NM 87131 {ucbvax,gatech}!unmvax!turing.unm.edu!mike
gwyn@brl-smoke.ARPA (Doug Gwyn ) (04/12/88)
In article <92@iravcl.ira.uka.de> fsinf@iravcl.ira.uka.de writes: >I've heard (but not verified) you can crash *every* unix-system using the >CP-command when there is not enough space on disk. CP will not check >whether the disk is full and overwrite blocks which are not free. The original >data will be lost; also the machine is likely to go down. This is totally off-the-wall. "cp" is just a user-mode utility; it is the KERNEL that maintains file system integrity. It is true that "cp" does not check whether there is enough space to hold the copy, but it cannot know that in advance (there are other processes asynchronously modifying the file system). I know of only two UNIX utilities that even attempt to check for sufficient space before writing a file (one is "ed"). The only real problems I know of with user-mode programs running a filesystem out of space are: If the filesystem contains /tmp, several critical system utilities may malfunction. Other applications that don't check for successful system call operation may not realize that their output is being truncated. This is an application design error.
smb@ulysses.homer.nj.att.com (Steven Bellovin) (04/13/88)
... } >I've heard (but not verified) you can crash *every* unix-system using the } >CP-command when there is not enough space on disk. CP will not check } >whether the disk is full and overwrite blocks which are not free. The original } >data will be lost; also the machine is likely to go down. } } Total nonsense. CP doesn't care beans which disk blocks are free. } The kernel does, however. When the kernel can't find a free block, it } doesn't just pick an in-use one. That would be REALLY stupid. } Instead, it just returns the ENOSPC (No space left on device) error } code. And usually prints something like "/usr: file system full" on } the console. (4.3 BSD uses the system error logger instead of the } console). There's actually some truth to the claim; a remarkable number of systems will not survive the experience. They should, of course, but it's hard to get right. And resource exhaustion is often not tested well, since it's hard to fake. I've seen systems where a debugging variable could be set to simulate failure of arbitrary percentages of requests -- deny 30% of the disk block requests, 10% of the mbuf requests, etc. More developers should do that.
rns@se-sd.sandiego.NCR.COM (Rick Schubert) (04/13/88)
In article <92@iravcl.ira.uka.de> fsinf@iravcl.ira.uka.de writes: >In article <460@osupyr.mast.ohio-state.edu>, czei@osupyr.mast.ohio-state.edu (Michael S Czeiszperger) writes: >I've heard (but not verified) you can crash *every* unix-system using the >CP-command when there is not enough space on disk. CP will not check >whether the disk is full and overwrite blocks which are not free. The original >data will be lost; also the machine is likely to go down. > >If this isn't right, please correct. You will probably get tons of corrections on this, but I'd like to add some information that I think may be useful in this and other situations. Commands like "cp" are just programs. In general (i.e. unless the file is setuid to "root"), such programs cannot perform magic. That is, they cannot do anything that you couldn't write a program to do if you knew how. "cp" may be "dumb" (not meant to be derogatory to "cp" or to people who are unable to speak) enough not to check whether or not a write was successful (I haven't bothered to test whether or not it is because that's not my point), but it doesn't have the permission to start writing over blocks that are not free, just as you could not naively (or maliciously) write a program that would do that kind of damage to the system (you could write programs that would do damage to your files and to other unprotected files, however) -- UNIX(tm) does provide a reasonable (no flames, please) amount of security in this area. One thing I think is nice about UNIX in terms of understanding and using it IS that all commands are just programs (even the various shells), and, except for things that need to run as root (or some other userid), any program can do anything that a "command" can do. -- Rick Schubert
madd@bu-cs.BU.EDU (Jim Frost) (04/13/88)
In article <92@iravcl.ira.uka.de> fsinf@iravcl.ira.uka.de writes: |In article <460@osupyr.mast.ohio-state.edu>, czei@osupyr.mast.ohio-state.edu (Michael S Czeiszperger) writes: |> [...] the 'df' command adjusts the reading so that when it says 100% full, |> the disk really is only 90% full. It goes on to say that having the |> disk over 90% would affect its performance. Does anyone know [...] |> if there are any reasons not to keep a disk at 96-99%?????? | |Maybe the reasons are ... |I've heard (but not verified) you can crash *every* unix-system using the |CP-command when there is not enough space on disk. CP will not check |whether the disk is full and overwrite blocks which are not free. The original |data will be lost; also the machine is likely to go down. Well, I've seen filesystems hit 100% on various occasions here. The only time I've seen them crash was when /tmp resided on the same partition. Presumably the system can stay up until some critical process needs to write to a full filesystem; when that happens you've got real trouble and the system crashes. On one of the machines here, I've seen /tmp fill up and the system run for 5 or 10 minutes before crashing for whatever reason. Even this might be a bad indicator, since /tmp and /usr/spool lived on the same partition. From a user point of view, things are a little different. When the file system denies writes, all kinds of programs break and the system is unusable, even if it *is* still running. Obviously you want to avoid this. For performance reasons, you should keep the filesystem within the boundaries. About the only time I can imagine where you wouldn't care is with a static filesystem -- one that contains files that are not going to change for quite some time. Performance isn't a question since you can set it up for optimum performance in advance, and since no writes are going to take place, you're not at risk of a crash. jim frost madd@bu-it.bu.edu
mbr@aoa.UUCP (Mark Rosenthal) (04/14/88)
In article <1508@se-sd.sandiego.NCR.COM> rns@se-sd.sandiego.NCR.COM (Rick Schubert) writes: >"cp" may be "dumb" (not meant to be derogatory to "cp" or to people who >are unable to speak) enough not to check whether or not a write was successful >(I haven't bothered to test whether or not it is because that's not my point), >but it doesn't have the permission to start writing over blocks that are not >free Just a side note. As I understand it, checking the return value of write() does not really tell you what you want to know, due to the fact that write's happen asynchronously. All the return value can tell you is whether the data was successfully copied into the kernel's buffer. Then when the disk is not busy and kernel gets around to it, the actual write happens, by which time your process is off doing other things, or perhaps has already exited. -- Mark of the Valley of Roses ...!{harvard,ima}!bbn!aoa!mbr
brankley@usfvax2.EDU (Bob Brankley) (04/14/88)
In article <92@iravcl.ira.uka.de>, fsinf@iravcl.ira.uka.de writes: > > ... It goes on to say that having the > > disk over 90% would affect its performance. Does anyone know [...] > > if there are any reasons not to keep a disk at 96-99%?????? > > Maybe the reasons are ... > I've heard (but not verified) you can crash *every* unix-system using the > CP-command when there is not enough space on disk. CP will not check > whether the disk is full and overwrite blocks which are not free. The original > data will be lost; also the machine is likely to go down. > > If this isn't right, please correct. > According to the paper, "A Fast File System for UNIX" the reason for keeping a 10% of a file system free is to improve performance. The original studies at Berkeley suggested that reserving 10% of any generic file system will keep the BFF(Berkeley Fast File System) efficient. Filling a BFF above this 10% free mark causes most BFF's to switch from minimizing seek time to minimizing disk storage. This is the only reason why "newfs" on Berkeley systems automatically reserves 10% of a file system unless otherwise instructed. Also, the only person who is allowed to fill a BFF above the high water mark (10% free) is "root." When the program "df" reports a file system as 100% capacity, root can still fill the system up an additional 10%. The only thing that filling the file system does is to cause many "file system full" messages on the system console. Bob Brankley CSNET: usfvax1!brankley@usf.edu UUCP: {ihnp4!codas, gatech}!usfvax2!usfvax1!brankley
mike@turing.UNM.EDU (Michael I. Bushnell) (04/15/88)
In article <182@aoa.UUCP> mbr@aoa.UUCP (Mark Rosenthal) writes: >Just a side note. As I understand it, checking the return value of write() >does not really tell you what you want to know, due to the fact that write's >happen asynchronously. All the return value can tell you is whether the data >was successfully copied into the kernel's buffer. Then when the disk is not >busy and kernel gets around to it, the actual write happens, by which time your >process is off doing other things, or perhaps has already exited. Nope. When your write returns, the only thing left to be done is write the block to disk. Specifically, the decision about which hardware block to use, and the marking of the block used in the map, has already been done. No one else will get the block. Once the write returns, the write will finish without error (except for possible hard errors). Note that this guarantee is missing in NFS. Sigh. N u m q u a m G l o r i a D e o Michael I. Bushnell HASA - "A" division 14308 Skyline Rd NE Computer Science Dept. Albuquerque, NM 87123 OR Farris Engineering Ctr. OR University of New Mexico mike@turing.unm.edu Albuquerque, NM 87131 {ucbvax,gatech}!unmvax!turing.unm.edu!mike