16012_3705@uwovax.uwo.ca (Jeff Racine) (04/17/89)
Can anybody offer some insight? I have been told that if you use a disk caching program then you can reduce the number of buffers specified in the `Buffers= ' entry in config.sys. Question: By how much can you reduce the number? Is there any way of `optimizing' the number of buffers? Since each buffer takes up 512 bytes of memory, then reducing buffers from 20 to 2 would free up a scarce resource, lower memory. Currently I am using a 20 Mhz 286 clone with 2 MB of ram on the mother board. The caching program is PC Kwic Cache which is supplied with PCTOOLS V 5.0. Any insight anyone could shed on this problem would be greatly appreciated. Thank you in advance for your time. P.S. Individuals of moronic proclivities and their pecuniary resources are disassociated with celerity.
silver@eniac.seas.upenn.edu (Andy Silverman) (04/18/89)
In article <2082@uwovax.uwo.ca> 16012_3705@uwovax.uwo.ca (Jeff Racine) writes: > > Can anybody offer some insight? I have been told that if you use a >disk caching program then you can reduce the number of buffers specified in >the `Buffers= ' entry in config.sys. > Question: By how much can you reduce the number? Is there any way of >`optimizing' the number of buffers? Since each buffer takes up 512 bytes of >memory, then reducing buffers from 20 to 2 would free up a scarce resource, >lower memory. If you have FTP access to Simtel20, there are a few files in the MSDOS.DSKUTL section, called THRASHER, and a better one called BUFsomething or other, I think, which play with your autoexec and config files and run a test program, then they modify the buffers and reboot and it runs again, for a specified range of buffer sizes. When it's done the good one prints out a graph which shows the performance at different buffer sizes. I used it on my XT clone, which I had previously set at 30 buffers, and the thing showed me that the best value was in fact five buffers. So I switched it and noticed a substantial performance increase, not to mention saved memory. So I recommend these programs highly. +----------------+-----------------------------------------+ | Andy Silverman | Internet: silver@eniac.seas.upenn.edu | | "Why?" | Compu$erve: 72261,531 | +----------------+-----------------------------------------+
bobmon@iuvax.cs.indiana.edu (RAMontante) (04/18/89)
silver@eniac.seas.upenn.edu.UUCP (Andy Silverman) <10004@netnews.upenn.edu> :
-In article <2082@uwovax.uwo.ca> 16012_3705@uwovax.uwo.ca (Jeff Racine) writes:
->
-> Can anybody offer some insight? I have been told that if you use a
->disk caching program then you can reduce the number of buffers specified in
->the `Buffers= ' entry in config.sys.
-
- [ Simtel20 buffer-testing programs... ] I used it on my XT
-clone, which I had previously set at 30 buffers, and the thing showed me
-that the best value was in fact five buffers. So I switched it and noticed
-a substantial performance increase, not to mention saved memory. So I
-recommend these programs highly.
I came to exactly the same conclusion "The old-fashioned way: I EARNED
it." [-: Dub in snooty voice. :-] 30+ buffers down to FIVE.
Actually I just did a lot of manual config.sys changing and file-access
timing. Incidentally, old versions of DOS defaulted to 5 buffers, but
recent ones (> 3.1 ?) default to a larger number like 15, based on
memory size -- so you have to explicitly set the buffers. This caused
quite a blip in my data until I discovered the change in defaults.
(Flame time: There's still one relic I'd like to cut down that would save
me a cool 30K -- anybody got some ideas for tossing COMMAND.COM? :-)ralf@b.gp.cs.cmu.edu (Ralf Brown) (04/18/89)
In article <19767@iuvax.cs.indiana.edu> bobmon@iuvax.cs.indiana.edu (RAMontante) writes: }timing. Incidentally, old versions of DOS defaulted to 5 buffers, but }recent ones (> 3.1 ?) default to a larger number like 15, based on }memory size -- so you have to explicitly set the buffers. This caused }quite a blip in my data until I discovered the change in defaults. DOS 2.x defaults to 2 buffers, 3.0 through 3.21 default to 2 on a PC or XT and 3 on an AT, 3.3+ default varies with available memory. }(Flame time: There's still one relic I'd like to cut down that would save }me a cool 30K -- anybody got some ideas for tossing COMMAND.COM? :-) Tossing it may save 30K disk space, but it will save only about 3K of memory, since most of COMMAND.COM is transient and may be overwritten by applications. -- {harvard,uunet,ucbvax}!b.gp.cs.cmu.edu!ralf -=-=- AT&T: (412)268-3053 (school) ARPA: RALF@B.GP.CS.CMU.EDU |"Tolerance means excusing the mistakes others make. FIDO: Ralf Brown at 129/31 | Tact means not noticing them." --Arthur Schnitzler BITnet: RALF%B.GP.CS.CMU.EDU@CMUCCVMA -=-=- DISCLAIMER? I claimed something? --
nelson@sun.soe.clarkson.edu (Russ Nelson) (04/18/89)
I concur with five buffers. My research showed an exponentially decreasing curve that met the linearly increasing curve at five or six buffers. Since five gives equal performance as six, use five and save some memory! -- --russ (nelson@clutx [.bitnet | .clarkson.edu]) America -- Socialism for the rich people, Capitalism for the rest of us. -- Michael Harrington, Co-Chair, Democratic Socialists of America
Ed.Maurer@f6.n135.z1.fidonet.org (Ed Maurer) (04/18/89)
> Can anybody offer some insight? I have been told that if you > use a disk caching program then you can reduce the number of buffers > specified in the `Buffers= ' entry in config.sys. > Question: By how much can you reduce the number? Is there > any way of optimizing' the number of buffers? You are correct to assume that you should decrease the buffer size when using a cache, but not down as far as you might think. I'm attaching the following test run on my 386 with ESDI 44Mb Disk. Note that under all cache conditions, optimum buffers is between 13-16. Without a cache, optimum buffer size appears to be around 40. This is, of course, one test, (write 1000 1K records, swap and shuffle records); but the results are in general agreement with other published data. The PD program 'Thrasher' was used to create the following table: TIMES IN SECONDS COMPAQ DISK CACHE MicroSoft MicroSoft No Cache 256K 128K 256K in ext mem in ext mem in ext mem ---------------- --------------- --------------- -------------- BUFFERS=05= 8 BUFFERS=05= 26 BUFFERS=05= 25 BUFFERS=10= 33 BUFFERS=06= 7 BUFFERS=06= 23 BUFFERS=06= 21 BUFFERS=11= 34 BUFFERS=07= 7 BUFFERS=07= 20 BUFFERS=07= 19 BUFFERS=12= 34 BUFFERS=08= 5 BUFFERS=08= 15 BUFFERS=08= 14 BUFFERS=13= 33 BUFFERS=09= 5 BUFFERS=09= 12 BUFFERS=09= 11 BUFFERS=14= 33 BUFFERS=10= 5 BUFFERS=10= 12 BUFFERS=10= 11 BUFFERS=15= 33 BUFFERS=11= 5 BUFFERS=11= 12 BUFFERS=11= 11 BUFFERS=16= 33 BUFFERS=12= 5 BUFFERS=12= 12 BUFFERS=12= 11 BUFFERS=17= 33 BUFFERS=13= 4 BUFFERS=13= 12 BUFFERS=13= 11 BUFFERS=18= 34 BUFFERS=14= 4 BUFFERS=14= 12 BUFFERS=14= 11 BUFFERS=19= 33 BUFFERS=15= 5 BUFFERS=15= 12 BUFFERS=15= 11 BUFFERS=20= 32 BUFFERS=16= 5 BUFFERS=16= 12 BUFFERS=16= 11 BUFFERS=21= 33 BUFFERS=17= 5 BUFFERS=17= 12 BUFFERS=17= 11 BUFFERS=22= 32 BUFFERS=18= 5 BUFFERS=18= 12 BUFFERS=18= 11 BUFFERS=23= 32 BUFFERS=19= 5 BUFFERS=19= 12 BUFFERS=19= 12 BUFFERS=24= 32 BUFFERS=20= 5 BUFFERS=20= 12 BUFFERS=20= 11 BUFFERS=25= 33 BUFFERS=21= 5 BUFFERS=21= 13 BUFFERS=21= 12 BUFFERS=26= 33 BUFFERS=22= 5 BUFFERS=22= 13 BUFFERS=22= 11 BUFFERS=27= 33 BUFFERS=23= 5 BUFFERS=23= 13 BUFFERS=23= 12 BUFFERS=28= 33 BUFFERS=24= 5 BUFFERS=24= 13 BUFFERS=24= 13 BUFFERS=29= 33 BUFFERS=25= 5 BUFFERS=25= 13 BUFFERS=25= 12 BUFFERS=30= 33 BUFFERS=26= 5 BUFFERS=26= 13 BUFFERS=26= 11 BUFFERS=31= 33 BUFFERS=27= 5 BUFFERS=27= 13 BUFFERS=27= 12 BUFFERS=32= 33 BUFFERS=28= 5 BUFFERS=28= 13 BUFFERS=28= 12 BUFFERS=33= 33 BUFFERS=29= 5 BUFFERS=29= 14 BUFFERS=29= 12 BUFFERS=34= 33 BUFFERS=30= 5 BUFFERS=30= 14 BUFFERS=30= 14 BUFFERS=35= 33 BUFFERS=31= 5 BUFFERS=31= 14 BUFFERS=31= 14 BUFFERS=36= 33 BUFFERS=32= 5 BUFFERS=32= 14 BUFFERS=32= 14 BUFFERS=37= 33 BUFFERS=33= 6 BUFFERS=33= 15 BUFFERS=33= 15 BUFFERS=38= 33 BUFFERS=34= 6 BUFFERS=34= 14 BUFFERS=34= 14 BUFFERS=39= 31 BUFFERS=35= 6 BUFFERS=35= 14 BUFFERS=35= 14 BUFFERS=40= 32 BUFFERS=36= 6 BUFFERS=36= 15 BUFFERS=36= 15 BUFFERS=41= 32 BUFFERS=37= 7 BUFFERS=37= 14 BUFFERS=37= 16 BUFFERS=42= 33 BUFFERS=38= 7 BUFFERS=38= 14 BUFFERS=38= 16 BUFFERS=43= 32 BUFFERS=39= 8 BUFFERS=39= 17 BUFFERS=39= 17 BUFFERS=44= 34 BUFFERS=40= 8 BUFFERS=40= 18 BUFFERS=40= 17 BUFFERS=45= 33 \\\\\\ ] \^ \\\\\\\\ ]x ]^ Maurer ^ The Micro Group, Inc., Pembroke Pines, FL ___ ___________ (Never replicate a successful test) (....But it worked fine yesterday!) -- Ed Maurer - via FidoNet node 1:135/3 Medical Software Exchange BBS (305) 325-8709 UUCP: ...uunet!gould!umbio!medsoft!6!Ed.Maurer ARPA: Ed.Maurer@f6.n135.z1.fidonet.org
toma@tekgvs.LABS.TEK.COM (Tom Almy) (04/18/89)
In article <2082@uwovax.uwo.ca> 16012_3705@uwovax.uwo.ca (Jeff Racine) writes: > Can anybody offer some insight? I have been told that if you use a >disk caching program then you can reduce the number of buffers specified in >the `Buffers= ' entry in config.sys. > Question: By how much can you reduce the number? Is there any way of >`optimizing' the number of buffers? S The ideal number of buffers depends on: 1. CPU Speed 2. Size of cache in caching program 3. Transfer rate of disk 4. Seek time of disk 5. Disk organization (is it fragmented?, are executables grouped together, are directories clustered at start of disk? Number of files in the directories?) and 6). Your application. Obviously no blanket statment like "use 5" has any meaning, you gotta do your own benchmarking. Time the most disk intensive program, or suite of programs, you normally use (for me this is a large compile and link). Try out different buffer counts and cache sizes (PCTools cache also has a switch for maximum read size to buffer that can also effect results drastically) and come to your own conclusions. Tom Almy toma@tekgvs.labs.tek.com Standard Disclaimers Apply
bobmon@iuvax.cs.indiana.edu (RAMontante) (04/19/89)
ralf@b.gp.cs.cmu.edu (Ralf Brown) <4747@pt.cs.cmu.edu> : - -[quotes me claiming 5 or 15 buffers for DOS before/after v3.1] - -DOS 2.x defaults to 2 buffers, 3.0 through 3.21 default to 2 on a PC or XT -and 3 on an AT, 3.3+ default varies with available memory. Hmmm, well, okay... but Zenith's release of v3.21 adjusts the default depending on memory -- the doc. says so, and sure enough I get 15 with no effort on my part. Perhaps this is another PC-DOS / MS-DOS difference.
zeeff@b-tech.ann-arbor.mi.us (Jon Zeeff) (04/19/89)
A simple test is not going to tell you how many buffers to use. I have one system that needs > 50 because of a large directory. The system just crawls with fewer buffers. Do some tests that copy files around in your largest directory. -- Jon Zeeff zeeff@b-tech.ann-arbor.mi.us Ann Arbor, MI sharkey!b-tech!zeeff
kevinc@auvax.UUCP (Kevin Crocker) (04/21/89)
A few items ago someone posted an answer that there were some good
buffer testing programs on SIMTEL. Also someone posted that if you
can't get to SIMTEL with ftp then you could use the mail servers that
are set up to handle this. Now for the bad news, it seems that both
these solutions don't work for us and we are only one hop away from a
very well connected site. I also know of grape but my phone bill has
been a tad high since starting to use grape. Can anyone point me in
the direction of a BBS or a list of BBS's out west and/ or preferably
in Canada. Our telephone system hasn't managed to pass on an
efficiencies in price drops lately.
Thanks in advance.
P.S. the two programs I really want are THRASHER and the other BUFFER
testing program.
Kevin Crocker
--
Kevin "auric" Crocker Athabasca University
UUCP: ...!{alberta,ncc,attvcr}!atha!kevinc
Inet: kevinc@cs.AthabascaU.CAcs3b3aj@maccs.McMaster.CA (Stephen M. Dunn) (04/21/89)
In article <9235@b-tech.ann-arbor.mi.us> zeeff@b-tech.ann-arbor.mi.us (Jon Zeeff) writes: >A simple test is not going to tell you how many buffers to use. I have one >system that needs > 50 because of a large directory. The system just crawls >with fewer buffers. The only thing is, if you don't need that many, you shouldn't have that many (which I guess is why you said that the user should test it out with varying numbers of buffers). On machines with slow processors (XTs, mostly), the overhead in searching excessive numbers of buffers can actually slow the machine down in disk accesses. I don't know the magnitude of this effect, but I imagine it would be noticeable in very disk-intensive programs. The rule of thumb I've heard is that unless you have reasons to do otherwise, 12-15 buffers is about right. Of course, the other argument against using lots of buffers is that they eat up memory, but many people have more memory than they know what to do with (software developers excluded). -- ====================================================================== ! Stephen M. Dunn, cs3b3aj@maccs.McMaster.CA ! DISCLAIMER: ! ! This space left unintentionally blank - vi ! I'm only an undergrad ! ======================================================================