timcc@csv.viccol.edu.au (Tim Cook) (03/08/91)
My promised summary of responses to my survey on Unix without disk quotas: > 1. Have you implemented any other method of controlling disk usage? > If so, what does it entail? Most of those who had, had a set of shell scripts that regularly checked for the greatest over-users of disk space, then hassled these users (or left the hassling to the administrator) to get them to clean up. > 1a. How difficult was the implentation of this alternative? For those who had a homebrew shell-script alternative, the implementation was usually easy. A few mentioned that they had got their system from somewhere else, indicating that there is some (semi?) serious development being put into this area. > 1b. Is it effective? Is it better or worse for the administrator > and for the user? The homebrew systems were more or less effective. They could not prevent a partition from filling up, but most users would respond to any hassling they attracted. Of course, some users (notably, those who are on holiday or otherwise unavailable) can be difficult to budge. > 2. If you have not implemented an alternative, how much more disk > space do you think you use (if any)? Some alternatives put all users in one group onto a common filesystem, with resposiblity for disk usage passed over to someone with some authority in that group. For most this means you need a large number of medium-sized filesystems, preferably of adjustable size, and it still has the same shortcomings of the shell-script-hassle system for the individual groups. Almost all respondents were sure that they were using more disk space. Some noted that disk space is relatively cheap, though. > 3. How much more time does the system administrator spend controlling > disk space usage, either with an alternative method of control, or > "by hand". For some, a very small amount of time. For others a lot. For all, it meant constant (at least daily) attention. > 4. How would you rate the presence of a disk quota system in > importance, compared to other system features (for example, dynamic > disk bad-block re-mapping, an extended access-control mechanism, > or adherence to contemporary Unix and Open Systems standards). I got a very mixed response here. Generally, those who had an alternative that worked well did not see disk quotas as important. Those who were feeling the humdrum of monitoring students who have fits of FTP-ing GIF's and other assorted goodies were a bit cynical about the other things I listed. In summary, I am convinced that a hard quota system would be the most desired ``option'' on a Unix system used by a large number of users. Some mentioned that a hard quota system would not allow a user access to a large chunk of disk on a ``temporary'' basis, such as would be needed to compile a large system, or perhaps for manipulating a large dataset. I would answer this by saying ``They can ring up and ask'', or ``What about /tmp or /usr/tmp?''. The one thing I have learnt from being a Systems Administrator is ``automate or delegate''. All of the alternative systems were not and could not be fully automated. I would like to thank all those who responded. I received about two dozen responses, including one from Elizabeth Zwicky of SRI International, who gave me a copy of a paper entitled ``Disk Space Management Without Quotas'' that she presented to the Third Annual USENIX Workshop on Large Installation Systems Administration. The file pub/non-source/quotas.Z, a compressed copy of my mail folder containing these messages (including this summary), will be available from the anonymous-FTP area on admin.viccol.edu.au by the time you read this. If this doesn't suit you, I can mail it. -- Tim Cook Systems Administrator, Victoria College Computer Services
peter@ficc.ferranti.com (Peter da Silva) (03/09/91)
One point that you have missed bringing up in this summary is that a lot of us consider automated disk quotas a negative value. If you have enough disk space to hold the sum total of all the quotas, all you've done is create the equivalent a bunch of partitions without any of the advantages of the same. If you don't, then you haven't solved the problem of disk space abuse: just narrowed the window. Like ulimits, it's an idea that sounds nice but is really just an attempt to apply a technological quick fix to a social, organizational, or procedural problem. -- Peter da Silva. `-_-' peter@ferranti.com +1 713 274 5180. 'U` "Have you hugged your wolf today?"
zwicky@erg.sri.com (Elizabeth Zwicky) (03/12/91)
In article <1991Mar7.124230.6609@csv.viccol.edu.au> timcc@csv.viccol.edu.au (Tim Cook) writes>> 3. How much more time does the system administrator spend controlling >> disk space usage, either with an alternative method of control, or >> "by hand". >For some, a very small amount of time. For others a lot. For all, it >meant constant (at least daily) attention. I find this extremely startling. It certainly isn't true here, not if you mean from humans anyway. I do run cleanup out of cron every night, but the only actual attention I pay to the matter is removing the e-mail it sends me. >In summary, I am convinced that a hard quota system would be the most >desired ``option'' on a Unix system used by a large number of users. Some >mentioned that a hard quota system would not allow a user access to a large >chunk of disk on a ``temporary'' basis, such as would be needed to compile >a large system, or perhaps for manipulating a large dataset. I would >answer this by saying ``They can ring up and ask'', or ``What about /tmp or >/usr/tmp?''. As someone else has pointed out, there are those of us with large numbers of users (from nearly 200 to nearly 2,000 in my case) who find a hard quota system literally worse than useless. In particular, users here have a tendency to work on projects that have deadlines of the "Finish by this time or we don't pay you" sort, and they tend to work until those deadlines. Nobody is going to be amused if they hit a quota at 4am with free disk space and an 8am deadline. They're going to begrudge the lost time while they call me, and I'm going to begrudge the lost sleep. /tmp and /usr/tmp aren't large enough, don't persist through reboots, and aren't exported from machine to machine. You may be convinced that a hard quota system is the option you desire most; but it would be hard to argue that it's intrinsically desirable, unless you mean by "hard quota system" something rather different from the traditional Berkeley quota system. >The one thing I have learnt from being a Systems Administrator is >``automate or delegate''. All of the alternative systems were not and >could not be fully automated. "Were not" I'll take your word for. But "could not be" is just silly. I can think of two ways to automate Ohio State's system off the top of my head - put in a daemon that watches free disk space and runs the user-warning program when it gets below a certain point, or simply run the user-warning program from cron. Either way you'd want to add some teeth to the user-warning program, maybe making log number of warnings to a user and automatically initiate compression or switch logins to a restricted shell when users racked up too many warnings. The possibilities abound. In any case, one should always automate dealings with programs, but dealings with users are not always wisely automated. Controlling disk space is a matter of managing people, and is a reasonable place to invest some human time and judgement. A machine has trouble telling the difference between the student with the fondness for GIFs and the student who's doing cutting edge graphics research. This (and the availability of cheap student labour) are why Ohio State didn't automate in the first place. I don't have cheap student labour at SRI, but then again I have more disk and no undergraduates. It still works out in favour of managing disk space through semi-automated methods. Elizabeth Zwicky zwicky@erg.sri.com
pack@acd.uucp (Daniel Packman) (03/13/91)
No doubt the best way to manage disk space depends on what the users are doing at each site. Here each user needs some "fixed" disk space for programs and small data files. For running the programs, the user needs large amounts of disk space (from several megabytes to several hundred megabytes). The straightforward way to manage this is via quotas in a "HOME" area governing the "fixed" disk space and no quotas on a large scratch area. My system of preference (to be built) uses hard disk quotas on all disk areas with no area oversubscribed. This way no program (or user) can inadvertantly fill up a partition. The management system will give the user a certain amount of fixed space in a set of partitions and will dynamically allocate space on other partitions on request. The dynamically allocated space (through either a command interface or library call) is managed through the system-wide disk quota system. The user requests so much space for so much time. If allowed, the space is guaranteed for that time. The user may then specify that a set of his files in this dynamic area are valuable and cannot be destroyed. These files are then migrated automatically to offline media (eg, tape) before that disk area is freed and his quota automatically reduced. Files not so marked are assumed scratch and are deleted after the agreed upon time limit expires and disk space is needed.
jfc@athena.mit.edu (John F Carr) (03/15/91)
In article <9DZ98Y8@xds13.ferranti.com> peter@ficc.ferranti.com (Peter da Silva) writes: >If you have enough disk >space to hold the sum total of all the quotas, all you've done is create >the equivalent a bunch of partitions without any of the advantages of the >same. If you don't, then you haven't solved the problem of disk space abuse: >just narrowed the window. Narrowing the window is worth a lot in some environments. There are users who will use all available disk space. With quotas, a small number of users can not fill a partition. We don't have the manpower to monitor disk usage and warn users manually, and the average disk usage requires us to overallocate disk space. We have 10000 users but less than 10 gigabytes of space allocated to user filesystems (as distinct from software development or other multi-user projects). Last time I checked, 50% of the users were using a third of their quota or less. If you don't want to overallocate space, quotas still have two advantages over partitions. Quotas can be easily changed, and if quotas are small, you would need a large number of disk partitions. I haven't used an operating system that allows an unlimited number of partitions. BSD normally allows 7 partitions per drive, AIX allows 31. -- John Carr (jfc@athena.mit.edu)