davidsen@sixhub.UUCP (Wm E. Davidsen Jr) (05/04/90)
Most flavors of SysV and some BSD systems have a process which does a
sync from time to time, such as "update." When the kernel is configured
to have a large number of buffers the result is a slowdown in
performance for disk intensive tasks.
Many versions provide a tuning parameter which allows this to happen
more or less often, but it still happens all at once. What I would like
is a way to force or at least encourage the system to write dirty
buffers out as a idle task, so that if the CPU is available and there
are dirty buffers they are queued, but only at some faitly slow rate,
perhaps 3-5% of the buffers per second.
Obviously the best place for this is in the kernel code, but realizing
that it isn't going to go there for a number of systems, is there
another way to get this effect? I have tried having a very low priority
process which slowly reads through a very large file. This helps on some
systems, but on others the clean (read) beffers are reused rather than
write out a dirty buffer.
I also tried a process (low priority again) which does a sync every
three seconds. This works somewhat better, in that I trade a big
slowdown one a minute or so for a small slowdown all the time. Still not
satisfactory.
Okay, now how about suggestions.
--
bill davidsen - davidsen@sixhub.uucp (uunet!crdgw1!sixhub!davidsen)
sysop *IX BBS and Public Access UNIX
moderator of comp.binaries.ibm.pc and 80386 mailing list
"Stupidity, like virtue, is its own reward" -mewsinpdb@lso.win.tue.nl (Paul de Bra) (05/04/90)
In article <911@sixhub.UUCP> davidsen@sixhub.UUCP (bill davidsen) writes: > > Most flavors of SysV and some BSD systems have a process which does a >sync from time to time, such as "update." When the kernel is configured >to have a large number of buffers the result is a slowdown in >performance for disk intensive tasks. > > Many versions provide a tuning parameter which allows this to happen >more or less often, but it still happens all at once. What I would like >is a way to force or at least encourage the system to write dirty >buffers out as a idle task, so that if the CPU is available and there >are dirty buffers they are queued, but only at some faitly slow rate, >perhaps 3-5% of the buffers per second. >... SysVr3.2 comes close to this, except that the kernel does not check whether the system was idle. There are 2 parameters for buffer flushing: BDFLUSHR and NAUTOUP Every BDFLUSHR seconds the kernel looks for buffers that are dirty for at least NAUTOUP seconds and writes those to disk. This means that the buffers are not written "all at once", but at a rate similar to the rate at which they were made dirty. By increasing NAUTOUP one can delay the update process, at the risk of losing more buffers on panic or power fail. My rule of thumb is to have NAUTOUP be large enough for most short-time actions to finish before NAUTOUP seconds have elapsed. (If most compiles take 30 seconds or less you could set NAUTOUP to 30, and the temporary intermediate files will never be actually written to disk since they are deleted before NAUTOUP seconds pass). Increasing BDFLUSHR improves overall system performance a little by reducing the number of times the buffer pool has to be scanned, but it means that probably more buffers will be written to disk all at once, which may be undesirable for the person who asked the question. Paul. (debra@research.att.com)
meissner@osf.org (Michael Meissner) (05/07/90)
In article <911@sixhub.UUCP> davidsen@sixhub.UUCP (Wm E. Davidsen Jr) writes: | Most flavors of SysV and some BSD systems have a process which does a | sync from time to time, such as "update." When the kernel is configured | to have a large number of buffers the result is a slowdown in | performance for disk intensive tasks. | | Many versions provide a tuning parameter which allows this to happen | more or less often, but it still happens all at once. What I would like | is a way to force or at least encourage the system to write dirty | buffers out as a idle task, so that if the CPU is available and there | are dirty buffers they are queued, but only at some faitly slow rate, | perhaps 3-5% of the buffers per second. The only problem with doing this only as an idle task, is that you risk losing lots of data if you have a crash when the system hasn't been idle for awhile (remember murphy after all). You could of course have a general sync, say every 5 minutes or so in addition to doing it on idle cycles. -- Michael Meissner email: meissner@osf.org phone: 617-621-8861 Open Software Foundation, 11 Cambridge Center, Cambridge, MA Catproof is an oxymoron, Childproof is nearly so