rcd@ico.isc.com (Dick Dunn) (11/01/90)
larry@nstar.uucp (Larry Snyder) writes: > I am wondering just how fast the FFS is as compared against other fast > file systems -- Not too surprisingly, we've looked at that a bit. One answer stands out like a sore thumb: If you're running V.4, *don't* use the old-style s5 file system unless you really need it. The BSD-style file system is a far better performer. Terminology: The main two file system types in V.4 are called "s5" and "ufs"--these are the old-style System V (V 7 style, if you will:-), and the BSD-style, respectively. (There are others like the boot file system, proc, etc. They're special-purpose.) Looking at performance of V.4 ufs vs s5, ufs wins hands-down. For example, Tim Bray's "bonnie" takes about 50% longer, overall, on s5. There's a lot more CPU time burned in s5. It's fair to say that the V.4 s5 file system has the same performance as a stock V.3 file system. Comparing V.4 ufs to ISC's FFS (our V.3.2 "fast file system") is a much more interesting exercise. They're really pretty close. This isn't surprising, since they both get their major performance gains from two factors: intelligent allocation and dealing in larger chunks. The interesting part is that FFS performs better on a smaller machine; ufs pulls ahead of it a bit on a machine with lots of memory. The reason, it turns out, is not really related to the file system implementation. Rather, V.3 has a statically-allocated buffer cache, while V.4 allocates memory dynamically. On a V.4 system, if the memory load due to program space is low, the buffer cache will grab more memory...so if you've got oodles of memory to start with, you can get a pretty substantial cache going. On V.3, you can allocate a huge cache, of course, but it stays allocated; if the machine load shifts away from I/O and toward large processes, you're wasting memory. I suspect (but haven't really con- firmed) that the reason the V.3.2 FFS does better on non-huge machines is just that the code is cleaner and spends a little less time fiddling with special cases that turn out not to matter. I suspect that cylinder groups (present in ufs but not FFS) don't help much; they may actually be the hindrance that makes the performance difference. -- Dick Dunn rcd@ico.isc.com -or- ico!rcd Boulder, CO (303)449-2870 ...but Meatball doesn't work that way!
larry@nstar.uucp (Larry Snyder) (11/01/90)
rcd@ico.isc.com (Dick Dunn) writes: >Not too surprisingly, we've looked at that a bit. One answer stands out >like a sore thumb: If you're running V.4, *don't* use the old-style s5 >file system unless you really need it. The BSD-style file system is a far >better performer. Yes - but 8K blocks - just think what that would do to your partition with all the news articles -- -- Larry Snyder, Northern Star Communications, Notre Dame, IN USA {larry@nstar, uunet!sco!romed!nstar!larry, nstar%larry@iuvax.cs.indiana.edu} backbone usenet newsfeeds available Public Access Unix Site (219) 289-0282 (5 high speed lines)
larry@nstar.uucp (Larry Snyder) (11/01/90)
rcd@ico.isc.com (Dick Dunn) writes: >just that the code is cleaner and spends a little less time fiddling with >special cases that turn out not to matter. I suspect that cylinder groups >(present in ufs but not FFS) don't help much; they may actually be the >hindrance that makes the performance difference. The Intel release 4 manuals go to great details to explain the gap/cylinder paramters and their "sample setting" whening making filesystems - but their examples are basically for 3b2 drives (all the way up to a big 72 meg drive). -- Larry Snyder, Northern Star Communications, Notre Dame, IN USA {larry@nstar, uunet!sco!romed!nstar!larry, nstar%larry@iuvax.cs.indiana.edu} backbone usenet newsfeeds available Public Access Unix Site (219) 289-0282 (5 high speed lines)
arnold@audiofax.com (Arnold Robbins) (11/02/90)
rcd@ico.isc.com (Dick Dunn) writes: > The BSD-style file system is a far better performer. In article <1990Nov01.114726.14348@nstar.uucp> larry@nstar.uucp (Larry Snyder) writes: >Yes - but 8K blocks - just think what that would do to your partition >with all the news articles -- The BSD file system has both blocks and fragments. On an 8K file system the fragment size will be 1K; files that are less than that in size only take up a single 1K fragment. This is even more efficient than the 2K file system which uses 2K blocks. So, go right ahead and put your news partition on BSD file system'ed partition. I have to testify as to the speed of the BSD filesystem. When we switched to it with ESIX rev D our systems sped up very noticeably! -- Arnold Robbins AudioFAX, Inc. | Laundry increases 2000 Powers Ferry Road, #200 / Marietta, GA. 30067 | exponentially in the INTERNET: arnold@audiofax.com Phone: +1 404 933 7612 | number of children. UUCP: emory!audfax!arnold Fax-box: +1 404 618 4581 | -- Miriam Robbins
cpcahil@virtech.uucp (Conor P. Cahill) (11/02/90)
In article <1990Nov01.114726.14348@nstar.uucp> larry@nstar.uucp (Larry Snyder) writes: >Yes - but 8K blocks - just think what that would do to your partition >with all the news articles -- Read a little more on the structure of the BSD file system. While the major allocation segment is 8K, the BSD file system has a fragment block for the last segment of a file. The size of this block is also configurable, but it is usually on the order of 1K. So a file system with thousands of small files will eat up thousands of 1K blocks (just like in the 1K system V file system). > >-- > Larry Snyder, Northern Star Communications, Notre Dame, IN USA > {larry@nstar, uunet!sco!romed!nstar!larry, nstar%larry@iuvax.cs.indiana.edu} > backbone usenet newsfeeds available > Public Access Unix Site (219) 289-0282 (5 high speed lines) -- Conor P. Cahill (703)430-9247 Virtual Technologies, Inc., uunet!virtech!cpcahil 46030 Manekin Plaza, Suite 160 Sterling, VA 22170
larry@nstar.uucp (Larry Snyder) (11/02/90)
arnold@audiofax.com (Arnold Robbins) writes: >I have to testify as to the speed of the BSD filesystem. When we switched >to it with ESIX rev D our systems sped up very noticeably! How long (since what release) has ESIX supported the BSD file system? -- Larry Snyder, Northern Star Communications, Notre Dame, IN USA {larry@nstar, uunet!sco!romed!nstar!larry, nstar%larry@iuvax.cs.indiana.edu} backbone usenet newsfeeds available Public Access Unix Site (219) 289-0282 (5 high speed lines)
tim@delluk.uucp (Tim Wright) (11/03/90)
In <1990Nov01.114726.14348@nstar.uucp> larry@nstar.uucp (Larry Snyder) writes: >rcd@ico.isc.com (Dick Dunn) writes: >>Not too surprisingly, we've looked at that a bit. One answer stands out >>like a sore thumb: If you're running V.4, *don't* use the old-style s5 >>file system unless you really need it. The BSD-style file system is a far >>better performer. >Yes - but 8K blocks - just think what that would do to your partition >with all the news articles -- Can you say 1/2K frags ? I knew you could :-) :-) -- Tim Wright, Dell Computer Corp. (UK) | Email address Dell Computer Corp. (UK), Bracknell | Domain: tim@dell.co.uk Tel: +44-344-860456 | Uucp: ...!ukc!delluk!tim "What's the problem? You've got an IQ of six thousand, haven't you?"
pcg@cs.aber.ac.uk (Piercarlo Grandi) (11/03/90)
On 1 Nov 90 11:47:26 GMT, larry@nstar.uucp (Larry Snyder) said:
larry> rcd@ico.isc.com (Dick Dunn) writes:
rcd> Not too surprisingly, we've looked at that a bit. One answer stands out
rcd> like a sore thumb: If you're running V.4, *don't* use the old-style s5
rcd> file system unless you really need it. The BSD-style file system is a far
rcd> better performer.
larry> Yes - but 8K blocks - just think what that would do to your partition
larry> with all the news articles --
What about fragments? They cure the wasted space problem. Naturally it
would have been better to just do allocations of runs of small blocks,
like the ISC FFS does, but the fragment approach works almost equally
well, excep that it is much more complex.
There is some recent paper in some conference proceedings on how to make
even faster file systems.
--
Piercarlo "Peter" Grandi | ARPA: pcg%uk.ac.aber.cs@nsfnet-relay.ac.uk
Dept of CS, UCW Aberystwyth | UUCP: ...!mcsun!ukc!aber-cs!pcg
Penglais, Aberystwyth SY23 3BZ, UK | INET: pcg@cs.aber.ac.ukpcg@cs.aber.ac.uk (Piercarlo Grandi) (11/03/90)
Note that foloowups have been redirected to comp.unix.internals, because
this is a discussion not limited to the 386 case.
On 1 Nov 90 11:51:48 GMT, larry@nstar.uucp (Larry Snyder) said:
larry> rcd@ico.isc.com (Dick Dunn) writes:
rcd> just that the code is cleaner and spends a little less time
rcd> fiddling with special cases that turn out not to matter. I suspect
rcd> that cylinder groups (present in ufs but not FFS) don't help much;
rcd> they may actually be the hindrance that makes the performance
rcd> difference.
Well, actually the purpose of cylinder groups is to spread out the
ilist, and to put related inodes together and nearer the file blocks
rooted on them, not to improve IO rates for files; it does not help
bandwidth or file-based applications, but it does help interactive
performance, i.e. applications that are directory based, and open lots
of files with little IO (e.g. 'file *') or of inodes (e.g. 'ls -F').
The real problem with the BSD FFS is that the block/fragment scheme is
equivalent to allocating runs of blocks and doing synchronous 8 block
read ahead all the time, only it limits the length of the runs to 8, and
forces all reads, even random reads, to do 8 blocks of synchronous read
ahead. The authors simply did not understand the design principles of
the Unix V7 filesystem, and its author's well motivated reluctance to
souble the block size to 1024 bytes, and their use of asynchronous read
ahead instead.
Another win would have been to put the first page of the file just after
the inode.
larry> The Intel release 4 manuals go to great details to explain the
larry> gap/cylinder paramters and their "sample setting" whening making
larry> filesystems - but their examples are basically for 3b2 drives
larry> (all the way up to a big 72 meg drive).
Note that the gap should be dependent on the *CPU* and ther kernel
speed, not on the drive (for a given rotational speed). The filesystem
gap is there to compensate for interrupt and processing time.
--
Piercarlo "Peter" Grandi | ARPA: pcg%uk.ac.aber.cs@nsfnet-relay.ac.uk
Dept of CS, UCW Aberystwyth | UUCP: ...!mcsun!ukc!aber-cs!pcg
Penglais, Aberystwyth SY23 3BZ, UK | INET: pcg@cs.aber.ac.ukbill@unixland.uucp (Bill Heiser) (11/03/90)
In article <1990Nov02.124206.5434@nstar.uucp> larry@nstar.uucp (Larry Snyder) writes: > >How long (since what release) has ESIX supported the BSD file system? Rev D of Esix has what they call the FFS (Fast File System). I don't know if that is a BSD file system or not... -- home: ...!{uunet,bloom-beacon,esegue}!world!unixland!bill bill@unixland.uucp, bill%unixland.uucp@world.std.com Public Access Unix - Esix SYSVR3 - (508) 655-3848 other: heiser@world.std.com Public Access Unix (617) 739-9753
dawes@suphys.physics.su.OZ.AU (David Dawes) (11/03/90)
From article <294@audfax.audiofax.com>, by arnold@audiofax.com (Arnold Robbins): > rcd@ico.isc.com (Dick Dunn) writes: >> The BSD-style file system is a far better performer. > > In article <1990Nov01.114726.14348@nstar.uucp> larry@nstar.uucp (Larry Snyder) writes: >>Yes - but 8K blocks - just think what that would do to your partition >>with all the news articles -- > > The BSD file system has both blocks and fragments. On an 8K file system > the fragment size will be 1K; files that are less than that in size only > take up a single 1K fragment. This is even more efficient than the 2K > file system which uses 2K blocks. So, go right ahead and put your news > partition on BSD file system'ed partition. > > I have to testify as to the speed of the BSD filesystem. When we switched > to it with ESIX rev D our systems sped up very noticeably! I too am using ESIX rev D with ffs. One of my file systems became heavily fragmented, and got to the point where there were 0 free blocks, and 5000 free frags. df reported 10000 blocks free, but attempting to write to the file system resulted in "Disk full" errors. This meant that I had an unusable 5MB on a 65MB file system. (BTW, there were plenty of free inodes.) Is this how FFS is supposed to work, or is there a problem with the ESIX implementation? David -- ------------------------------------------------------------------------------ David Dawes (dawes@suphys.physics.su.oz.au) DoD#210 | Phone: +612 692 2639 School of Physics, University of Sydney, Australia | Fax: +612 660 2903 ------------------------------------------------------------------------------
cpcahil@virtech.uucp (Conor P. Cahill) (11/04/90)
In article <1990Nov3.124110.2155@metro.ucc.su.OZ.AU> dawes@suphys.physics.su.OZ.AU (David Dawes) writes: >I too am using ESIX rev D with ffs. One of my file systems became heavily >fragmented, and got to the point where there were 0 free blocks, and >5000 free frags. df reported 10000 blocks free, but attempting to write >to the file system resulted in "Disk full" errors. This meant that I >had an unusable 5MB on a 65MB file system. (BTW, there were plenty of free >inodes.) If the ESIX ffs is a BSD file system, the file system code will only allow the super user to write to the last 10% of the partition. This is because the performance of the system is supposed to degrade substantially when you go over 90% utilization. -- Conor P. Cahill (703)430-9247 Virtual Technologies, Inc., uunet!virtech!cpcahil 46030 Manekin Plaza, Suite 160 Sterling, VA 22170
rmtodd@servalan.uucp (Richard Todd) (11/04/90)
dawes@suphys.physics.su.OZ.AU (David Dawes) writes: >I too am using ESIX rev D with ffs. One of my file systems became heavily >fragmented, and got to the point where there were 0 free blocks, and >5000 free frags. df reported 10000 blocks free, but attempting to write >to the file system resulted in "Disk full" errors. This meant that I >had an unusable 5MB on a 65MB file system. (BTW, there were plenty of free >inodes.) >Is this how FFS is supposed to work, or is there a problem with the ESIX >implementation? Well, that's how FFS works on other machines (I've hit it on Apple Unix 2.0, which includes Berkeley FFS). Hitting the no-free-blocks limit when you've got 5M free in fragments is a little unusual; I was hitting the limit with only ~250K of fragments free on a 282MB filesystem, and on the (frequent) occasions when the local campus BSD machine fills a partition it gets down to fairly small values of free-fragment-space (again, in the neighborhood of a couple hundred K on ~200MB filesystems). Evidently you managed to stumble on a pathological worst-case.... -- Richard Todd rmtodd@uokmax.ecn.uoknor.edu rmtodd@chinet.chi.il.us rmtodd@servalan.uucp 7 days until the season premiere of WISEGUY!
john@jwt.UUCP (John Temples) (11/05/90)
In article <1990Nov03.193936.665@virtech.uucp> cpcahil@virtech.UUCP (Conor P. Cahill) writes: >If the ESIX ffs is a BSD file system, the file system code will only >allow the super user to write to the last 10% of the partition. Yes, this is how the ESIX FFS works. The 10% figure is a default which can be configured when you make the file system. -- John W. Temples -- john@jwt.UUCP (uunet!jwt!john)
dawes@suphys.physics.su.OZ.AU (David Dawes) (11/05/90)
In article <2239@jwt.UUCP> john@jwt.UUCP (John Temples) writes: >In article <1990Nov03.193936.665@virtech.uucp> cpcahil@virtech.UUCP (Conor P. Cahill) writes: >>If the ESIX ffs is a BSD file system, the file system code will only >>allow the super user to write to the last 10% of the partition. > >Yes, this is how the ESIX FFS works. The 10% figure is a default >which can be configured when you make the file system. I have found that ESIX FFS with less than 10% free space can still be written to by a non-super user. ffsdump shows the minfree parameter, but it doesn't seem to be doing anything. Also all BSD df's I've seen show disk usage as a percentage of (total space - min free space) -- so the max usage is 110% for the default minfree. Esix df doesn't work like this. David -- ------------------------------------------------------------------------------ David Dawes (dawes@suphys.physics.su.oz.au) DoD#210 | Phone: +612 692 2639 School of Physics, University of Sydney, Australia | Fax: +612 660 2903 ------------------------------------------------------------------------------
richard@pegasus.com (Richard Foulk) (11/05/90)
>>How long (since what release) has ESIX supported the BSD file system? > >Rev D of Esix has what they call the FFS (Fast File System). I don't >know if that is a BSD file system or not... > It has long file names doesn't it? -- Richard Foulk richard@pegasus.com
bill@unixland.uucp (Bill Heiser) (11/06/90)
In article <1990Nov3.222929.2164@servalan.uucp> rmtodd@servalan.uucp (Richard Todd) writes: > > Well, that's how FFS works on other machines (I've hit it on Apple Unix >2.0, which includes Berkeley FFS). Hitting the no-free-blocks limit when >you've got 5M free in fragments is a little unusual; I was hitting the Is there something that can be done "on-line" to de-fragment Unix partitions? Or is the best bet to just backup and restore the partition using something other than 'dump'? My /usr partition is at 6% fragmentation (according to fsck) -- The system has only been up for a couple of months, and I've been getting usenet news for a few weeks. That seems like a lot for such a short time. I'm using the Esix ffs. (I haven't done anything with the Esix "2K file system utility" yet -- I'm not sure what is on that disk-- been too busy to experiment...) -- home: ...!{uunet,bloom-beacon,esegue}!world!unixland!bill bill@unixland.uucp, bill%unixland.uucp@world.std.com Public Access Unix - Esix SYSVR3 - (508) 655-3848 other: heiser@world.std.com Public Access Unix (617) 739-9753
bill@unixland.uucp (Bill Heiser) (11/06/90)
In article <1990Nov5.154056.4737@pegasus.com> richard@pegasus.com (Richard Foulk) writes: >>Rev D of Esix has what they call the FFS (Fast File System). I don't >>know if that is a BSD file system or not... >> >It has long file names doesn't it? > No, Esix has a 14 character limit on filename length. -- home: ...!{uunet,bloom-beacon,esegue}!world!unixland!bill bill@unixland.uucp, bill%unixland.uucp@world.std.com Public Access Unix - Esix SYSVR3 - (508) 655-3848 other: heiser@world.std.com Public Access Unix (617) 739-9753
john@karnak.cactus.org (John B. Meaders Jr.) (11/06/90)
In article <1990Nov5.154056.4737@pegasus.com> richard@pegasus.com (Richard Foulk) writes: >> >>Rev D of Esix has what they call the FFS (Fast File System). I don't >>know if that is a BSD file system or not... >> > >It has long file names doesn't it? No; for compatibility. -- 2LT John B. Meaders, Jr. 510 Manchester Ct., Hopewell, VA 23860 Voice: 804-458-2983 Net: john@karnak.cactus.org or john@karnak.sigma.com Uucp: ...!{sequoia,letni,ditka}!karnak!john "Cowabunga dude" - TMNT Disclaimer: These are my opinions, not those of DA or DOD.
larry@nstar.uucp (Larry Snyder) (11/06/90)
bill@unixland.uucp (Bill Heiser) writes: >Is there something that can be done "on-line" to de-fragment Unix partitions? >Or is the best bet to just backup and restore the partition using something karl@ddsw1 put together a diskpack package a year or so ago - I don't know if it runs under ESIX - or if it works on the FFS - but in any case, it's available on nstar. -- Larry Snyder, Northern Star Communications, Notre Dame, IN USA {larry@nstar, uunet!sco!romed!nstar!larry, nstar%larry@iuvax.cs.indiana.edu} backbone usenet newsfeeds available Public Access Unix Site (219) 289-0282 (5 high speed lines)
john@jwt.UUCP (John Temples) (11/07/90)
In article <1990Nov5.154056.4737@pegasus.com> richard@pegasus.com (Richard Foulk) writes: >It [ESIX FFS] has long file names doesn't it? No, it doesn't. -- John W. Temples -- john@jwt.UUCP (uunet!jwt!john)
larry@nstar.uucp (Larry Snyder) (11/07/90)
bill@unixland.uucp (Bill Heiser) writes: >No, Esix has a 14 character limit on filename length. that's a real shame - I believe that Intel release 4 with the FFS supports long file names - but the manuals don't mention how long is long... -- Larry Snyder, Northern Star Communications, Notre Dame, IN USA {larry@nstar, {uunet|backbone}!nstar!larry, larry%nstar@iuvax.cs.indiana.edu} backbone usenet newsfeeds available Public Access Unix Site (219) 289-0282 (5 high speed lines)
bill@bilver.UUCP (Bill Vermillion) (11/08/90)
In article <1990Nov3.124110.2155@metro.ucc.su.OZ.AU> dawes@suphys.physics.su.OZ.AU (David Dawes) writes: >From article <294@audfax.audiofax.com>, by arnold@audiofax.com (Arnold Robbins): >> rcd@ico.isc.com (Dick Dunn) writes: >>> The BSD-style file system is a far better performer. >> >> In article <1990Nov01.114726.14348@nstar.uucp> larry@nstar.uucp (Larry Snyder) writes: >>>Yes - but 8K blocks - just think what that would do to your partition >>>with all the news articles -- >> I have to testify as to the speed of the BSD filesystem. When we switched >> to it with ESIX rev D our systems sped up very noticeably! >I too am using ESIX rev D with ffs. One of my file systems became heavily >fragmented, and got to the point where there were 0 free blocks, and >5000 free frags. df reported 10000 blocks free, but attempting to write >to the file system resulted in "Disk full" errors. This meant that I >had an unusable 5MB on a 65MB file system. (BTW, there were plenty of free >inodes.) >Is this how FFS is supposed to work, or is there a problem with the ESIX >implementation? That's about the right amount. Check your manual in the newfs entry or the ffsmkfs entry, and you will see that 10% of the disk is reserved and can NOT be used by anyone except the super user. That disk is full 0%, as far as a regular user is concerned. I just remade a fs last night, and found one other interesting thing. You can set how many inodes there are in the system, but the -i flag doesn't work, but changing the number of cylinder groups does. I made a fs with 95000+ inodes, and got a warning installing it saying it may (it did not say would, just may) cause a system PANIC. So I decided to take a look. I had cpio'ed the entire news hierarchy onto another partition to be able to add more inodes (I took default the first time and didn't have enough). When I cpio'ed them back, many attempts to put files in their proper place came back with "not a directory" type error messages. Sometimes directories would be created, other times files with directory names would be created and then subsequent writes went awry. I remade the fs with just about 65000 inodes using a smaller numbers of cylinders per cylinder group, and it came out to about 6.5 megs per c/g, and appears to be working just fine. There is a description of the file system in the manual. Talking with a friend of mine about this problem last night, it appears the Esix ffs is only partially BSD, because he said I should have no inode limits on a pure BSD file system. Other than that the ffs system seems to be a good performer. -- Bill Vermillion - UUCP: uunet!tarpit!bilver!bill : bill@bilver.UUCP
pcg@cs.aber.ac.uk (Piercarlo Grandi) (11/08/90)
On 5 Nov 90 22:27:33 GMT, bill@unixland.uucp (Bill Heiser) said:
bill> In article <1990Nov3.222929.2164@servalan.uucp> rmtodd@servalan.uucp (Richard Todd) writes:
rmtodd> Well, that's how FFS works on other machines (I've hit it on
rmtodd> Apple Unix 2.0, which includes Berkeley FFS). Hitting the
rmtodd> no-free-blocks limit when you've got 5M free in fragments is a
rmtodd> little unusual;
Very unusual.
bill> Is there something that can be done "on-line" to de-fragment Unix
bill> partitions?
You are confusing the BSd with the V7 filesystem designs. The BSD
filesystem does not get fragmented. You cannot unfragment it.
bill> Or is the best bet to just backup and restore the partition using
bill> something other than 'dump'?
This will not change anything. The BSD filesystem has several block
sizes; a file is made up of 4/8KB blocks, up to the last one, which is
0.5/1/2/3/4/5/6/7KB long (a fragment) if the filesize is not a multiple
of 4/8KB. If a fragment of the right size cannot be found, a larger one
will be split. Eventually you can get to the situation where all full
4/8KB blocks have been split, and so you cannot write any new file large
than that.
A partial solution is to use as the large block size the smallest that
you can define, e.g. 4KB instead of 8KB, which is in general advisable
anyhow on a 386. So if you have a choice define your filesystems to have
a 4KB fundamental size and a 1KB fragment size, not 8KB/1KB or 4KB/512B.
bill> My /usr partition is at 6% fragmentation (according to fsck)
This means that 6% of files have a length that is not a multiple of
4/8KB. This seems very low for a news partition.
bill> -- The system has only been up for a couple of months, and I've
bill> been getting usenet news for a few weeks. That seems like a lot
bill> for such a short time.
In the BSD filesystem fragementation is not a function of time, but only
of the file sizes.
In the V7 filesystem fragmentation has a completely different meaning,
which could be better described with the word "scattering". As time goes
by free blocks and therefore the free list and so new files tend to get
scattered around the disc. This often takes very little time (two weeks)
to manifest itself.
It does not happen with the BSD (or the ISC) filesystems, which keep the
free blocks well packed and the free list in order.
--
Piercarlo "Peter" Grandi | ARPA: pcg%uk.ac.aber.cs@nsfnet-relay.ac.uk
Dept of CS, UCW Aberystwyth | UUCP: ...!mcsun!ukc!aber-cs!pcg
Penglais, Aberystwyth SY23 3BZ, UK | INET: pcg@cs.aber.ac.ukmburg@unix386.Convergent.COM (Mike Burg) (11/08/90)
In article <1990Nov5.225213.11920@unixland.uucp> bill@unixland.uucp (Bill Heiser) writes: >In article <1990Nov5.154056.4737@pegasus.com> richard@pegasus.com (Richard Foulk) writes: >>>Rev D of Esix has what they call the FFS (Fast File System). I don't >>>know if that is a BSD file system or not... >>It has long file names doesn't it? >No, Esix has a 14 character limit on filename length. Well, when I was still working at ESIX and porting the FFS to System V, I did add a "directory flag" into the super block stating which directory style to use. Either the System V style of 14-characters, or the original BSD 255 character style. However, that was over a year ago and I haven't really kept in contact with them to see what was the final turnout. (It seems that either some notes of mine got lost, or someone didn't like that feature after I left :-) :-)) The reason for the two directory layouts was for those programs that still were using the old open("."), read(.., &dir, sizeof(struct dir)), method instead of using opendir/readdir to retrieve the filenames in a directory. I was very surpised to see a mass of system utilities in 3.2 that still used the former method. -- ---------------------------------- Michael Burg - Unisys/Convergent Corp. Unix Intel Platforms Division San Jose Phone: (408) 456-5934 UUCP: uunet!pyramid!ctnews!unix386.Convergent.com!mburg
cpcahil@virtech.uucp (Conor P. Cahill) (11/08/90)
In article <1990Nov07.120135.3300@nstar.uucp> larry@nstar.uucp (Larry Snyder) writes: >that's a real shame - I believe that Intel release 4 with >the FFS supports long file names - but the manuals don't >mention how long is long... System VR4 has a UFS file system (otherwise known as a BSD FFS) that will support directory entries of 255 bytes. -- Conor P. Cahill (703)430-9247 Virtual Technologies, Inc., uunet!virtech!cpcahil 46030 Manekin Plaza, Suite 160 Sterling, VA 22170
davidsen@sixhub.UUCP (Wm E. Davidsen Jr) (11/08/90)
In article <1990Nov07.120135.3300@nstar.uucp> larry@nstar.uucp (Larry Snyder) writes: | that's a real shame - I believe that Intel release 4 with | the FFS supports long file names - but the manuals don't | mention how long is long... The beta was compiled for 256. -- bill davidsen - davidsen@sixhub.uucp (uunet!crdgw1!sixhub!davidsen) sysop *IX BBS and Public Access UNIX moderator of comp.binaries.ibm.pc and 80386 mailing list "Stupidity, like virtue, is its own reward" -me
pb@idca.tds.PHILIPS.nl (Peter Brouwer) (11/08/90)
In article <1990Nov06.120947.13551@nstar.uucp> larry@nstar.uucp (Larry Snyder) writes: >bill@unixland.uucp (Bill Heiser) writes: >>Is there something that can be done "on-line" to de-fragment Unix partitions? >>Or is the best bet to just backup and restore the partition using something >karl@ddsw1 put together a diskpack package a year or so ago - I don't >know if it runs under ESIX - or if it works on the FFS - but in any >case, it's available on nstar. I saw source of a diskpack utility posted sometime ago that will defragment your disk but will use and set the gap size to 1. Be aware that this will not always be the optimal value and it might decrease yoyr file system performance. -- # Peter Brouwer, | Philips Information Systems, # # NET : pb@idca.tds.philips.nl | Department P9000-i Building V2, # # UUCP : ....!mcsun!philapd!pb | P.O.Box 245,7300AE Apeldoorn,The Netherlands# # PHONE:ext [+31] [0]55 432992, | FAX :ext [+31] [0]55 433488 #
jde@everex.uucp (-Jeff Ellis()) (11/10/90)
In article <1990Nov5.225213.11920@unixland.uucp> bill@unixland.uucp (Bill Heiser) writes: >No, Esix has a 14 character limit on filename length. Yes and no. If the ESIX FFS partion is made with the -sysv option then you are limited to 14 characters. Otherwise its 256 characters. Our default in our install script is with the -sysv. -- Jeff Ellis ESIX SYSTEM/V UUCP:uunet!zardoz!everex!jde US Mail: 1923 St. Andrew Place, Santa Ana, CA 92705
david@twg.com (David S. Herron) (11/11/90)
In article <1990Nov5.222733.11408@unixland.uucp> bill@unixland.uucp (Bill Heiser) writes: >In article <1990Nov3.222929.2164@servalan.uucp> rmtodd@servalan.uucp (Richard Todd) writes: >> Well, that's how FFS works on other machines (I've hit it on Apple Unix >>2.0, which includes Berkeley FFS). Hitting the no-free-blocks limit when >>you've got 5M free in fragments is a little unusual; I was hitting the That 10% limit is a heuristic invented when Berkeley invented FFS designed to help keep fragmentation down. >Is there something that can be done "on-line" to de-fragment Unix partitions? Theoretically, yes. It would work best if the disk were `unmounted' first which is easiest to do if the system were brought to single user. It would also require writing software which would first sort the free list, then reorder all the data blocks into as contiguous an order as possible. The method is left as an excercise to the reader. If Unix were multitasking internally (as Mach is) then I can imagine an addition to either the disk driver or file system driver which would run as a seperate `thread', at very low priority, and do file system optimization in the background. Since it would run at low priority it would only snarf up idle time ... >Or is the best bet to just backup and restore the partition using something >other than 'dump'? This will work -- assuming you're using `tar' or `cpio'. > (I haven't done anything with the Esix >"2K file system utility" yet -- I'm not sure what is on that disk-- been too >busy to experiment...) All the 2K file system is is the same old V7 file system with some constants changed to confuse the innocent. It's no great stride forward in understanding the Universe .. it has most of the same performance features & limitations (eg, it fragments quickly and doesn't work well with largeish partitions since the inodes are all stuck at one end of the partition) as the .5K and 1K file systems. The difference is that the basic unit of allocation is a 2Kbyte block. Therefore each each data block in an inode refers to twice as much data as a 1K file system. Therefore you can have more data in a file before you get to indirect or doubly indirect blocks. Therefore a read returns more data (inside the kernel) so you tend to get a little more read ahead effect. Therefore it's suitable for file systems where you intend to store lots of largeish files. Perhaps (guess) it would also be good for file systems where you intend to store lots of 1K<"file_size"<2K files since then all the data in the file would be stored in one data block. This might have a positive effect on fragmentation .. -- <- David Herron, an MMDF & WIN/MHS guy, <david@twg.com> <- Formerly: David Herron -- NonResident E-Mail Hack <david@ms.uky.edu> <- <- Use the force Wes!
bill@unixland.uucp (Bill Heiser) (11/11/90)
In article <1990Nov9.220931.2091@everex.uucp> jde@everex.uucp (-Jeff Ellis()) writes: >In article <1990Nov5.225213.11920@unixland.uucp> bill@unixland.uucp (Bill Heiser) writes: >>No, Esix has a 14 character limit on filename length. > >Yes and no. If the ESIX FFS partion is made with the -sysv option then >you are limited to 14 characters. Otherwise its 256 characters. Our >default in our install script is with the -sysv. > If I were to re-install, making sure the -sysv option is NOT used, would there be any side effects? Would the FFS still work properly, for example? -- home: ...!{uunet,bloom-beacon,esegue}!world!unixland!bill bill@unixland.uucp, bill%unixland.uucp@world.std.com Public Access Unix - Esix SYSVR3 - (508) 655-3848 other: heiser@world.std.com Public Access Unix (617) 739-9753
dawes@suphys.physics.su.OZ.AU (David Dawes) (11/11/90)
In article <1990Nov9.220931.2091@everex.uucp> jde@everex.uucp (-Jeff Ellis()) writes: >In article <1990Nov5.225213.11920@unixland.uucp> bill@unixland.uucp (Bill Heiser) writes: >>No, Esix has a 14 character limit on filename length. > >Yes and no. If the ESIX FFS partion is made with the -sysv option then >you are limited to 14 characters. Otherwise its 256 characters. Our >default in our install script is with the -sysv. I have just been playing around with different ffs parameters. I assume the install script you are referring to is /etc/diskadd. It calls /etc/ffs/newfs with the `-sysv' option. I tried running /etc/ffs/newfs -v /dev/rdsk/1s1 and it created a sysv (ie 14 char file names) file system. I assume then that -sysv is the default. In this case what option should be used for bsd type file names? The -v options revealed that newfs was calling /etc/ffs/mkfs -S -R 4 /dev/rdsk/1s1 [list of parameters] I tried running this without -S, and got a bsd type file system. What does the -R 4 option do? I couldn't see any difference between using it or omitting it. There seem to be a number of undocumented options for ffsmkfs, newfs. Even a few options shown in the SYNOPSIS part of the man page are not described. Is it possible to get some more complete documentation? >Jeff Ellis ESIX SYSTEM/V UUCP:uunet!zardoz!everex!jde > US Mail: 1923 St. Andrew Place, Santa Ana, CA 92705 David -- ------------------------------------------------------------------------------ David Dawes (dawes@suphys.physics.su.oz.au) DoD#210 | Phone: +612 692 2639 School of Physics, University of Sydney, Australia | Fax: +612 660 2903 ------------------------------------------------------------------------------
john@jwt.UUCP (John Temples) (11/12/90)
In article <1990Nov11.013732.1263@unixland.uucp> bill@unixland.uucp (Bill Heiser) writes: >If I were to re-install, making sure the -sysv option is NOT used, would >there be any side effects? Would the FFS still work properly, for example? I just built a BSD directory format file system on a floppy (with /etc/ffs/mkfs /dev/rdsk/f0t 2400) and it seems to work. I only played with it for a few minutes, but I was able to edit files with long names, make directories, and list files. I wonder if anything would break by using BSD directory format on the hard drive? -- John W. Temples -- john@jwt.UUCP (uunet!jwt!john)
bill@unixland.uucp (Bill Heiser) (11/12/90)
In article <2258@jwt.UUCP> john@jwt.UUCP (John Temples) writes: >I just built a BSD directory format file system on a floppy (with >/etc/ffs/mkfs /dev/rdsk/f0t 2400) and it seems to work. I only >played with it for a few minutes, but I was able to edit files with >long names, make directories, and list files. I wonder if anything >would break by using BSD directory format on the hard drive? This is interesting. I, too, would be interested in knowing what the results would be of using the BSD directory format ... bill -- home: ...!{uunet,bloom-beacon,esegue}!world!unixland!bill bill@unixland.uucp, bill%unixland.uucp@world.std.com Public Access Unix - Esix SYSVR3 - (508) 655-3848 other: heiser@world.std.com Public Access Unix (617) 739-9753
pcg@cs.aber.ac.uk (Piercarlo Grandi) (11/13/90)
On 10 Nov 90 21:43:05 GMT, david@twg.com (David S. Herron) said: [ ... on the BSD FFS and its ESIX incarnation ... ] david> That 10% limit is a heuristic invented when Berkeley invented FFS david> designed to help keep fragmentation down. No, it is designed to prevent the hashed quadratic search for a cylinder group with free blocks to be repeated too many times. It has *nothing* to do with fragmentation, only with the distribution of free blocks. [ ... on defragmenting the BSD FFS online ... ] david> Theoretically, yes. It would work best if the disk were david> `unmounted' first which is easiest to do if the system were david> brought to single user. It would also require writing software david> which would first sort the free list, then reorder all the data david> blocks into as contiguous an order as possible. The method is david> left as an excercise to the reader. But it would be pointless. The BSD FFS already keeps the blocks in nearly optimal order, without rewuiring reorganization. I doubt very much that in a FFS partition with sufficient free space (>10%) reorganization would buy a lot of extra thruput, whether the reoganization is online or offline. Moreover again: the issue we are discussing here is *scattering*, not fragmentation. In the BSD FFS fragmentation is the splitting of large blocks into small blocks for file tails. -- Piercarlo Grandi | ARPA: pcg%uk.ac.aber.cs@nsfnet-relay.ac.uk Dept of CS, UCW Aberystwyth | UUCP: ...!mcsun!ukc!aber-cs!pcg Penglais, Aberystwyth SY23 3BZ, UK | INET: pcg@cs.aber.ac.uk
dawes@suphys.physics.su.OZ.AU (David Dawes) (11/18/90)
In article <1305@bilver.UUCP> bill@bilver.UUCP (Bill Vermillion) writes: >In article <1990Nov3.124110.2155@metro.ucc.su.OZ.AU> dawes@suphys.physics.su.OZ.AU (David Dawes) writes: > >>I too am using ESIX rev D with ffs. One of my file systems became heavily >>fragmented, and got to the point where there were 0 free blocks, and >>5000 free frags. df reported 10000 blocks free, but attempting to write >>to the file system resulted in "Disk full" errors. This meant that I >>had an unusable 5MB on a 65MB file system. (BTW, there were plenty of free >>inodes.) > >>Is this how FFS is supposed to work, or is there a problem with the ESIX >>implementation? > >That's about the right amount. Check your manual in the newfs entry or the >ffsmkfs entry, and you will see that 10% of the disk is reserved and can >NOT be used by anyone except the super user. That disk is full 0%, as far >as a regular user is concerned. I forgot to mention that I checked this angle, and found that it wasn't the problem. I've not seen any evidence that the min-free parameter actually does anything on an Esix FFS. My problem was that there were just too many frags. I have been able to improve this by using 4k blocks (with 1k frags). David -- ------------------------------------------------------------------------------ David Dawes (dawes@suphys.physics.su.oz.au) DoD#210 | Phone: +612 692 2639 School of Physics, University of Sydney, Australia | Fax: +612 660 2903 ------------------------------------------------------------------------------