sef@kithrup.COM (Sean Eric Fagan) (03/04/91)
In article <1991Mar4.001026.3043@athena.mit.edu> jik@athena.mit.edu (Jonathan I. Kamens) writes: > OK, so how do you find out the number for a particular filesystem? We've >seen one answer that says it should be in terms of 512-byte blocks on any >systems, and that systems that use something other than 512-byte blocks are >broken. SysVr3.2 (kithrup, at least) has a system call called 'statfs'. It returns lots of information about the filesystem, including: struct statfs { short f_fstyp; /* File system type */ long f_bsize; /* Block size */ long f_frsize; /* Fragment size (if supported) */ long f_blocks; /* Total number of blocks on file system */ long f_bfree; /* Total number of free blocks */ long f_files; /* Total number of file nodes (inodes) */ long f_ffree; /* Total number of free file nodes */ char f_fname[6]; /* Volume name */ char f_fpack[6]; /* Pack name */ }; If the 3.2 stat() returned st_blocks, then it would take two system calls to find out the desired information. (Side note: in SCO's implementation, pathconf and fpathconf are based on statfs(). It's a neat system call. 8-)) > Now you say that in fact it's OK for different systems to use different >block sizes when calculating st_blocks. Yep. The fact that there is no way in SunOS (prior to 4.0 or 4.1, since those are supposed to be SysVr4 compatible, which is SysVr3.2 compatible, meaning it should have the statfs() call) or BSD to get the information is a defect in the OS. However, it should be relatively easy to fix. *If* POSIX ever decides to add the st_blocks feature (which might or might not be a good thing; I've not decided yet), then I would hope it would also adds something similar to the statfs() call... -- Sean Eric Fagan | "I made the universe, but please don't blame me for it; sef@kithrup.COM | I had a bellyache at the time." -----------------+ -- The Turtle (Stephen King, _It_) Any opinions expressed are my own, and generally unpopular with others.
jik@athena.mit.edu (Jonathan I. Kamens) (03/05/91)
I know about statfs; I did, after all, mention it in the subject line of my original message (which you may not have seen, since it was posted to comp.unix.wizards). The statfs I've got on my system (which came with the vnode code which came with our NFS code) doesn't appear to have a f_frsize element of the statfs structure. So I can't use it on my system. As you pointed out, statfs isn't a global thing. So I can't use it in general in my program. So, the question still remains -- is there any *reliable* way to find out what st_blocks counts in terms of? Is the answer, "If statfs exists on the system, AND it has an f_frsize field, then use that field and do statfs on every filesystem you encounter. Otherwise, use 512." Or is there something more I can do? Sigh. This all seems like such a muddled mess of different interfaces (or lacks of interface!) to the same information that I'm tempted to forget about the whole thing and not make any attempt at all to count blocks. Yuck. -- Jonathan Kamens USnail: MIT Project Athena 11 Ashford Terrace jik@Athena.MIT.EDU Allston, MA 02134 Office: 617-253-8085 Home: 617-782-0710
guy@auspex.auspex.com (Guy Harris) (03/05/91)
>Yep. The fact that there is no way in SunOS (prior to 4.0 or 4.1, since >those are supposed to be SysVr4 compatible, Umm, no, they're not. Some Sun Marketoons may have claimed, without proper qualification, that they're completely compatible; they're pretty close, but they're *not* completely compatible. SunOS 4.x has the same "statfs()" call that SunOS releases prior to 4.x did. >which is SysVr3.2 compatible, meaning it should have the statfs() call) S5R4 does, I presume, have the same "statfs()" call as S5R3; however the call you're *supposed* to use in S5R4 is "statvfs()", which is a new call in S5R4, atoning for S5R3's error in having a "statfs()" call with the same name as SunOS's but incompatible semantics. You don't need to use that call to find out the units of "st_blocks", though; the S5R4 manual states that "st_blocks" is "the total number of physical blocks of size 512 bytes actually allocated on disk." Presumably, if the allocation granularity of the disk, or of the file system, is in blocks that are some multiple of 512 bytes in size, the number-of-blocks will be multiplied by that multiple to get the number of 512-byte chunks. If it's the units are less than 512 bytes, the system'd have to *divide* by the ratio of sizes, and if the ratio of sizes isn't integral, you'd have a problem; fortunately, I suspect most UNIX systems have disks whose sector size is some multiple of 512 bytes (the only exceptions may be assorted mainframes, such as IBM mainframes, although the UNIXes on those systems may, when not working on fixed-block-size disks, use some standard block size - I don't think anybody's ported UNIX to the IBM 1130, so that doesn't count... :-)). The SunOS documentation doesn't say what the units are, but "st_blocks" is in units of 512 bytes on UFS and on NFS to most if not all UNIX servers, and should be in units of 512 bytes on any other file systems plugged into your system. The NFS protocol spec does not, alas, do a very good job of explaining the fields of the "fattr" structure, so it's conceivable that a server writer may not return a file size in 512-byte units in the "blocks" field. It *should* have indicated that the "blocksize" field is the block size *recommended for I/O*, rather than the block size used as units for the "blocks" field, as the "blocksize" field is what gets turned into the "st_blksize" field, and that field - in BSD, in SunOS, and in S5R4 - is the recommended block size for I/O operations. It should also have indicated that the "blocks" field should be in units of 512 bytes. As for "statfs()" a la SunOS, "statfs()" a la S5R3, and "statvfs()" a la S5R4 (including, I assume, Sun's current "developer's release" of S5R4, and any future Sun releases of S5R4): SunOS "struct statfs" includes: long f_bsize; /* fundamental file system block size */ long f_blocks; /* total blocks in file system */ long f_bfree; /* free blocks */ long f_bavail; /* free blocks available to non-super-user */ where "f_bsize" is the unit of allocation of the particular file system - i.e., on a BSD/UFS file system, the "fragment size", which may be bigger than the sector size; it's typically 1K on Sun systems, even though the sector size is 512 bytes. "st_bfree" and "st_bavail" are in units of "f_bsize", not units of 512 bytes. (Determined by checking the code; it's not well-documented.) S5R3's "struct statfs" includes: long f_bsize; /* Block size */ long f_frsize; /* Fragment size (if supported) */ long f_blocks; /* Total number of blocks on file system */ long f_bfree; /* Total number of free blocks */ where "f_bsize" is, on S5 file systems, the unit of allocation of the particular file system - i.e., on a 1K file system, it's 1K, and on a 512-byte file system, it's 512 bytes. "f_frsize" is 0. (All as of S5R3.1, from checking the code.) Dunno what's done on S5 systems that support a BSD file system; one presumes the idea is that "fs_bsize" is the "block size", and "fs_fsize" is the "fragment size", the *latter* being the true unit of allocation, and the former being more the suggested unit of I/O. "f_blocks" and "f_bfree" are in units of 512-byte chunks. S5R4's "struct statvfs" includes: ulong f_bsize; /* preferred file system block size */ ulong f_frsize; /* fundamental filesystem block size (if supported) */ ulong f_blocks; /* total # of blocks on file system in units of f_frsize */ ulong f_bfree; /* total # of free blocks */ ulong f_bavail; /* # of free blocks avail to non-superuser */ I'm not sure what this "(if supported)" nonsense is for "f_frsize"; if it's not "supported", the "f_blocks" and, presumably, "f_bfree" and "f_bavail" fields are meaningless, as the former is documented as being in units of "f_frsize" and the other two are, I assume, also in those units. I suspect that, for an S5 file system, both "f_bsize" and "f_frsize" are supported; dunno whether the latter is 512 or the block size of the file system. Dunno whether "f_bsize" is the block size of the file system, or 512, or 1024, or what. For a BSD/UFS file system, "f_bsize" is probably the block size (recommended I/O size) and "f_frsize" is probably the fragment size (unit of allocation). (Determined from reading the documentation.)
paul@cs.edinburgh.ac.uk (Paul Anderson) (03/05/91)
In article <6428@auspex.auspex.com>, guy@auspex.auspex.com (Guy Harris) writes: > The SunOS documentation doesn't say what the units are, but "st_blocks" > is in units of 512 bytes on UFS and on NFS to most if not all UNIX > servers, and should be in units of 512 bytes on any other file systems > plugged into your system. Things might have changed, but this certainly didn't used to be true of the Pyramid system we had - st_blocks was in units of f_bsize. Things like "du" gave results that were out by a factor of two (or more) depending on the NFS filesystem that your files lived on! I agree with you that this was wrong, but I have never seen the correct units actually stated anywhere. -- Paul Anderson JANET: paul@uk.ac.ed.lfcs LFCS, Dept. of Computer Science UUCP: ..!mcvax!ukc!lfcs!paul University of Edinburgh ARPA: paul%lfcs.ed.ac.uk@nsfnet-relay.ac.uk Edinburgh EH9 3JZ, UK. Tel: 031-650-5193
dhesi%cirrusl@oliveb.ATC.olivetti.com (Rahul Dhesi) (03/06/91)
I think it's safe to say that most users who use "du" and "df" care about the number of kilobytes and not much about the block size. When 4.3BSD seemed to be standardizing on a block size of 1K in such reports, I was happy, especially because the existence of fragments meant, to those wondering about disk space wasted due to fragmentation, that the block size was no longer as important. Then POSIX came along, and SVR4 reared its ugly head too. Now I'm really depressed. Maybe it's time for a new field in the stat struct called something like st_kbytes, and fields in the statfs struct called something like f_kbfree and f_kbavail. -- Rahul Dhesi <dhesi%cirrusl@oliveb.ATC.olivetti.com> UUCP: oliveb!cirrusl!dhesi
guy@auspex.auspex.com (Guy Harris) (03/08/91)
>I think it's safe to say that most users who use "du" and "df" care >about the number of kilobytes and not much about the block size. When >4.3BSD seemed to be standardizing on a block size of 1K in such >reports, I was happy, especially because the existence of fragments >meant, to those wondering about disk space wasted due to fragmentation, >that the block size was no longer as important. > >Then POSIX came along, and SVR4 reared its ugly head too. Now I'm >really depressed. > >Maybe it's time for a new field in the stat struct called something >like st_kbytes, and fields in the statfs struct called something like >f_kbfree and f_kbavail. I don't see how what you're complaining about, namely the behavior of the "du" and "df" *utilities*, would at all call for new fields in the aforementioned structures. All you have to do to make a version of those utilities that reports in 1K units on S5R4 is to divide "st_blocks" by 2 in "du", and divide "f_bfree"/"f_bavail" by 512 and multiply them by "f_frsize" in "df". "st_blocks" is specified as being in units of 1/2K, and "f_bfree"/"f_bavail" are specified as being in units of "f_frsize". In fact, S5R4 makes things *better*, in some ways, at the programmer's level; it nails down what those units are, instead of just waving its hands. The fact that POSIX seems to be tending towards making the utilities report in units other than kbytes is annoying, but that definitely does *not* make it time to add redundant fields of the sort you suggest. Any programmer with such rigid thought patterns that they're *obliged* to make those programs report in units other than kbytes, just because the calls the program uses to *retrieve* that information report in units other than kbytes, requires adult supervision....
dhesi%cirrusl@oliveb.ATC.olivetti.com (Rahul Dhesi) (03/10/91)
Guy Harris writes: All you have to do to make a version of those utilities that reports in 1K units on S5R4 is to divide "st_blocks" by 2 in "du", and divide "f_bfree"/"f_bavail" by 512 and multiply them by "f_frsize" in "df". Yes! The smart programmer was never prevented from reporting disk usage in consistent units, preferably kilobytes. Even under non-SVR4, it should have been possible for the smart programmer to figure out just what block sizes were and to scale calculations accordingly, so the user was always reported figures in kilobytes. The problem, however, is, that smart programmers are few and far between. When they get one thing right, they get another wrong...for example, Sun's df program is good about reporting disk usage in one kilobyte units, but it's terrible (pre-4.1.1) about not trying to do stats on filesystems that aren't mounted, thus causing the automounter to go crazy, and it doesn't attempt any sort of timeout on statfs, thus causing or a user's login session to hang if a server isn't responding. AT&T's "df" (SVR2 and SVR3) seems to report space in units of 512 bytes. But give an intuitive command like "df ." and AT&T's "df" just exits, without any error message. These problems are not hard-to-avoid bugs caused by somebody overlooking something subtle; they are very obvious fundamental flaws that a little careful thinking would completely avoid. >The fact that POSIX seems to be tending towards making the utilities >report in units other than kbytes is annoying, but that definitely does >*not* make it time to add redundant fields of the sort you suggest. Probably true...my comment was 25% sarcasm (directed at the programmers that you said need adult supervision), 25% humor, and 50% despair at the state of the art in UNIX implementations. -- Rahul Dhesi <dhesi%cirrusl@oliveb.ATC.olivetti.com> UUCP: oliveb!cirrusl!dhesi
jik@athena.mit.edu (Jonathan I. Kamens) (03/11/91)
I'm sorry to be so insistent, but I still haven't seen one clear, unequivocal answer to this question: If a system has st_blocks in its stat structure, then how can I tell what the count in st_blocks is in terms of? Do I assume it's 512 everywhere? Do I do statfs on each file? Or something else? I'm willing to use #ifdef magic in order to make it work on various different types of systems, but I haven't yet seen one clear answer to my question of *what* the #ifdef magic should do. The theoretical discussion is interesting, but it doesn't really help to solve my particular problem, especially since I don't understand all of it at first glance and don't have time to read it enough times to understand it :-). -- Jonathan Kamens USnail: MIT Project Athena 11 Ashford Terrace jik@Athena.MIT.EDU Allston, MA 02134 Office: 617-253-8085 Home: 617-782-0710
guy@auspex.auspex.com (Guy Harris) (03/12/91)
> I'm sorry to be so insistent, but I still haven't seen one clear, >unequivocal answer to this question: If a system has st_blocks in its stat >structure, then how can I tell what the count in st_blocks is in terms of? Perhaps the *lack* of such an answer *is* the answer - i.e., the way you tell is system-dependent. Sad, but seemingly true; while most systems report in 512-byte units, according to another posting, at least at one point Pyramid systems didn't: From: paul@cs.edinburgh.ac.uk (Paul Anderson) Newsgroups: comp.unix.internals Subject: Re: What, exactly, are stat.st_blocks, statfs.f_b Message-ID: <7235@skye.cs.ed.ac.uk> Date: 5 Mar 91 13:27:30 GMT Organization: Department of Computer Science, University of Edinburgh ... Things might have changed, but this certainly didn't used to be true of the Pyramid system we had - st_blocks was in units of f_bsize. Things like "du" gave results that were out by a factor of two (or more) depending on the NFS filesystem that your files lived on! I agree with you that this was wrong, but I have never seen the correct units actually stated anywhere. S5R4 and 4.3-reno *finally* nail down the units in their documentation (512 bytes), so hopefully Pyramid, and any others whose "stat()" calls and NFS implementations don't use those units will change to report in those units (both in "stat()" *and* in the NFS server code - the idea is not just to fix programs running on the Pyramid, but to fix programs running *elsewhere* accessing files on the Pyramid). > Do I assume it's 512 everywhere? Nope - see above. > Do I do statfs on each file? Nope - not all systems with an "st_blocks" field *have* "statfs()" (e.g., vanilla 4.3BSD and 4.3-tahoe; Reno appears to have it). >Or something else? I'm willing to use #ifdef magic in order to make >it work on various different types of systems, but I haven't yet seen >one clear answer to my question of *what* the #ifdef magic should do. Well, it appears it should do a "statfs()", and multiply "st_blocks" by "f_bsize/512", on a Pyramid, according to the above posting. It should do nothing on 4.3-reno, S5R4 and SunOS (you may be screwed if those systems are mounting file systems from a Pyramid, say, though). I can't speak for other systems; I don't know if *anybody* can enumerate all the systems.