aab@cichlid.com (Andy Burgess) (11/20/90)
In article <1990Nov18.182135.17954@scuzzy.in-berlin.de> src@scuzzy.in-berlin.de (Heiko Blume) writes: ... > >p.s.: i want a 512Byte filesystem for news, so speed is not an issue. I just did an ls -l /usr/spool/news/comp/unix/sysv386 and I didn't see one file less then 512 bytes long. Most (estimate 90%) were over 1024 bytes long. Doesn't this mean 512 byte block buys you nothing? Just wondering -- I'm no file system guru. Andy -- Andy Burgess Independent Consultant aab@cichlid.com uunet!silma!cichlid!aab
tim@delluk.uucp (Tim Wright) (11/21/90)
In <1990Nov19.232124.7802@cichlid.com> aab@cichlid.com (Andy Burgess) writes: >In article <1990Nov18.182135.17954@scuzzy.in-berlin.de> src@scuzzy.in-berlin.de (Heiko Blume) writes: >... >> >>p.s.: i want a 512Byte filesystem for news, so speed is not an issue. >I just did an ls -l /usr/spool/news/comp/unix/sysv386 and I didn't see one >file less then 512 bytes long. Most (estimate 90%) were over 1024 bytes >long. Doesn't this mean 512 byte block buys you nothing? >Just wondering -- I'm no file system guru. Reduced waste of space due to wasting on average 1/4K in the last block instead of 1/2K. I think the original problem may have been down to not specifying the FSTYPE to fsck and mount - they may not autodetect - I can't remember for sure. I might try it some time if I get a disk with some room on it :-) Tim -- Tim Wright, Dell Computer Corp. (UK) | Email address Bracknell, Berkshire, RG12 1RW | Domain: tim@dell.co.uk Tel: +44-344-860456 | Uucp: ...!ukc!delluk!tim "What's the problem? You've got an IQ of six thousand, haven't you?"
cpcahil@virtech.uucp (Conor P. Cahill) (11/21/90)
In article <1990Nov19.232124.7802@cichlid.com> aab@cichlid.com (Andy Burgess) writes: >I just did an ls -l /usr/spool/news/comp/unix/sysv386 and I didn't see one >file less then 512 bytes long. Most (estimate 90%) were over 1024 bytes >long. Doesn't this mean 512 byte block buys you nothing? No. What matters is how many files fall between a 1K boundry and the next 512byte boundry. If that number is high, then a 512 byte file system may save you some space, but at the expense of performance. If most of the files are between 0 and 512 bytes long then the 512 byte file system will save you both space and performance. If most of the files are between 513 and 1024 bytes long, then the 1K file system wins both. If most are between 1025 and 1536, then the 512 byte file system will save you some space, but will probably loose in the performance arena. and this goes on ad nausium. NOTE that the space savings of a 512 byte file system decreases linearly with the average size of the files while the performance hit for using the smaller block increases linearly. So don't use a 512 byte file system unless your files all range between 0-512, or possibly 1025-1536 ( although I think the latter range will end up being a wash). -- Conor P. Cahill (703)430-9247 Virtual Technologies, Inc., uunet!virtech!cpcahil 46030 Manekin Plaza, Suite 160 Sterling, VA 22170
src@scuzzy.in-berlin.de (Heiko Blume) (11/22/90)
tim@delluk.uucp (Tim Wright) writes: >aab@cichlid.com (Andy Burgess) writes: >>src@scuzzy.in-berlin.de (Heiko Blume) writes: >>>p.s.: i want a 512Byte filesystem for news, so speed is not an issue. >>I just did an ls -l /usr/spool/news/comp/unix/sysv386 and I didn't see one >>file less then 512 bytes long. Most (estimate 90%) were over 1024 bytes >>long. Doesn't this mean 512 byte block buys you nothing? >Reduced waste of space due to wasting on average 1/4K in the last block >instead of 1/2K. you bet! i did a little statistic: size | count -------+------ <= 512 : 3665 <= 1024: 10524 <= 1536: 12610 <= 2048: 7784 >= 2048: 12411 total : 46994 savings on 23790 files no savings on 23204 files i'd have >12MB more space with a 512Byte filesystem! which yields quite a percentage (~15%) for a 76MB filesystem... >I think the original problem may have been down to not >specifying the FSTYPE to fsck and mount - they may not autodetect - I can't >remember for sure. the manual says: if you don't specify the type and it fails to mount it as the root fstype it will try to find out with /etc/fstyp. but that yields S51K.... as i haven't found any fs types other than S51K, S52K, XENIX and DOS, what shall i tell mount ?? -- Heiko Blume <-+-> src@scuzzy.in-berlin.de <-+-> (+49 30) 691 88 93 public source archive [HST V.42bis]: scuzzy Any ACU,f 38400 6919520 gin:--gin: nuucp sword: nuucp uucp scuzzy!/src/README /your/home
johnl@iecc.cambridge.ma.us (John R. Levine) (11/22/90)
In article <1990Nov21.134043.25732@virtech.uucp> cpcahil@virtech.UUCP (Conor P. Cahill) writes: >If most of the files are between 0 and 512 bytes long then the 512 byte >file system will save you both space and performance. > >If most of the files are between 513 and 1024 bytes long, then the 1K >file system wins both. > >If most are between 1025 and 1536, then the 512 byte file system will >save you some space, but will probably loose in the performance arena. Here's some actual data. I wrote a little awk script that looked at all of the 20,000 files in my news partition. It figured out how many blocks, including indirect blocks, the files would take in a 512 or a 1K file system. Multiply linked files are double counted, but I don't think that affected the results much. And the results, in 512 byte blocks are: 512 total 192220 1024 total 210866 That is, a 512 byte file system saves about 10% of the total space. Not overwhelming, but significant. Personally, I'll take the extra performance of a 1K FFS partition. -- John R. Levine, IECC, POB 349, Cambridge MA 02238, +1 617 864 9650 johnl@iecc.cambridge.ma.us, {ima|spdcc|world}!iecc!johnl "Typically supercomputers use a single microprocessor." -Boston Globe
tneff@bfmny0.BFM.COM (Tom Neff) (11/26/90)
I hesitate to suggest someone else write a program I don't have the time
to write myself, but perhaps that awk script could be posted in some
form? It would be Real Nice if people could run something that told
them, numerically, what they currently stood to gain and/or lose by
installing a given file tree onto filesystems of various blocksizes.
--
I'm a Leo. Leos don't believe * * * Tom Neff
in this astrology stuff. * * * tneff@bfmny0.BFM.COMshwake@raysnec.UUCP (Ray Shwake) (11/28/90)
tim@delluk.uucp (Tim Wright) writes: >Reduced waste of space due to wasting on average 1/4K in the last block >instead of 1/2K. I think the original problem may have been down to not >specifying the FSTYPE to fsck and mount - they may not autodetect - I can't >remember for sure. I might try it some time if I get a disk with some >room on it :-) While the 1/4K figure might appear to be intuitive, running fsanalyze on my small ISC 2.2 system showed waste due to unfilled blocks to be less than 4% for root, 7% for /usr and /usr2. I'd done a few spot checks some time back to validate these figures, and they seemed to be on target. Assuming these figures are anywhere near the actual, the performance boost from a 1K block file system makes for a clear win.
tim@delluk.uucp (Tim Wright) (11/29/90)
In <158@raysnec.UUCP> shwake@raysnec.UUCP (Ray Shwake) writes: >tim@delluk.uucp (Tim Wright) writes: >>Reduced waste of space due to wasting on average 1/4K in the last block >>instead of 1/2K. ... > While the 1/4K figure might appear to be intuitive, running fsanalyze >on my small ISC 2.2 system showed waste due to unfilled blocks to be less >than 4% for root, 7% for /usr and /usr2. I'd done a few spot checks some >time back to validate these figures, and they seemed to be on target. >Assuming these figures are anywhere near the actual, the performance boost >from a 1K block file system makes for a clear win. Agreed, but a news partition is not a normal filesystem !! As has been pointed out, a significant space saving can be achieved by using a smaller blocksize. Having said that I'm still not sure I'd bother. Anybody have any comments on using 'ufs' (BSD FFS) on a news partition. Is it possible to make it use say 4K blocks and 1/2K frags even if this is not the default ? Inquiring minds want to know ! Tim -- Tim Wright, Dell Computer Corp. (UK) | Email address Bracknell, Berkshire, RG12 1RW | Domain: tim@dell.co.uk Tel: +44-344-860456 | Uucp: ...!ukc!delluk!tim "What's the problem? You've got an IQ of six thousand, haven't you?"
src@scuzzy.in-berlin.de (Heiko Blume) (11/30/90)
tneff@bfmny0.BFM.COM (Tom Neff) writes: >I hesitate to suggest someone else write a program I don't have the time >to write myself, but perhaps that awk script could be posted in some >form? It would be Real Nice if people could run something that told >them, numerically, what they currently stood to gain and/or lose by >installing a given file tree onto filesystems of various blocksizes. i rather hesitate to post something as simple as this little c program, but what the heck... main() { int x,w,n,s512,s1,s2,s3,s4,i; x=w=n=s512=s1=s2=s3=i=0; for(;;) { if((scanf("%d",&i))==-1) break;; x++; if(i % 1024 <= 512) w++; else n++; if(i <= 512) s512++; else if(i <= 1024) s1++; else if(i <= 1536) s2++; else if(i <= 2048) s3++; else s4++; } printf("<= 512: %d\n",s512); printf("<= 1024: %d\n",s1); /* but > 512 */ printf("<= 1536: %d\n",s2); /* but > 1024 */ printf("<= 2048: %d\n",s3); /* etc */ printf(">= 2048: %d\n",s4); printf("total : %d\n",x); printf("savings on %d files\n",w); printf("no savings on %d files\n",n); } to use this do a find /usr/spool/news -print | xargs ls -l | cut -c32-40 | sort > bla edit out the blank lines from bla and then do a <the program above> < bla that yields the little statistic. feel free to do whatever you want to with this Great Piece Of Software (tm) :-) -- Heiko Blume <-+-> src@scuzzy.in-berlin.de <-+-> (+49 30) 691 88 93 public source archive [HST V.42bis]: scuzzy Any ACU,f 38400 6919520 gin:--gin: nuucp sword: nuucp uucp scuzzy!/src/README /your/home