jgd@Dixie.Com (John G. DeArmond) (01/15/91)
Submitted-by: jgd@Dixie.Com (John G. DeArmond)
Posting-number: Volume 16, Issue 75
Archive-name: ichk/part01
Ichk is a quick program and script that I wrote in order to address the
Interactive (and other) System V Inode bug. Ichk sits as a daemon and
checks the designated filesystem at specified intervals against
specified limits. If a problem is found such as too few inodes, too few
free blocks or the ratio of blocks to inodes becomes too large, then a
child process is started which is passed an argument that indicates the
problem. The child process or script should look at the arguments and
take corrective action. An example script that works against the news
partition on Dixie.com is included.
Though a fix for the inode problem from ISC would be ideal and a patch
for the kernel has been posted to the net, this programs directly
addresses the problem without kernal patches (which indicates a full
backup to any responsible sysadm - you got any idea how long it takes to
backup 1GB to 60 meg tape? :-). Plus it catches other problems such as
the news partition just flat running out of space from a flood of news.
This program is free of any encumbrances on use (see inode.txt for
details) and is free of the GNU General Public Virus. Comments, bug
reports and improvements appreciated. In particular, I'd appreciate
the results of porting this program to Berzerkly or other variant
Unixes.
John De Armond
jgd@dixie.com
---- Cut Here and unpack ----
#!/bin/sh
# This is ichk, a shell archive (shar 3.11)
# made 01/14/1991 06:07 UTC by jgd@dixie.com
# Source directory /usr/local/tools/ichk
#
# existing files WILL be overwritten
#
# This shar contains:
# length mode name
# ------ ---------- ------------------------------------------
# 1963 -rwxr-xr-x Inodes
# 1389 -rw-r--r-- README
# 5937 -r--r--r-- ichk.c
# 7310 -rw-r--r-- ichk.txt
# 567 -rw-rw---- makefile
#
touch 2>&1 | fgrep '[-amc]' > /tmp/s3_touch$$
if [ -s /tmp/s3_touch$$ ]
then
TOUCH=can
else
TOUCH=cannot
fi
rm -f /tmp/s3_touch$$
# ============= Inodes ==============
echo "x - extracting Inodes (Text)"
sed 's/^X//' << 'SHAR_EOF' > Inodes &&
X# @(#) Inodes 1.7 91/01/14 00:29:18";
X
X
X# this shell script is invoked by ichk whenever there is a news filesystem
X# emergency such as if the inodes count or free disk should fall
X# below critical levels
X
XLIBD=/usr/local/news/bin
XSPOOLD=/news/spool
XLOCK=/usr/spool/locks/LCK..news
X#NEWSSLICE=/dev/dsk/c0d1p5
XNEWSSLICE=`grep $1 /etc/fstab | cut -d" " -f1`
XEXPIRE=/usr/local/news/bin/expire
XTMPFILE=/tmp/Clean.tmp
XNNMASTER="/usr/local/news/nnbin/nnmaster -r -E -C"
XDAYS=5
X
XFLAG=0
X# first, check for inode shortage.
X#
Xif [ "$2" = "inode" -o "$2" = "ratio" ]
Xthen
X
X trap "rm -f $LOCK; exit" 0 1 2 3 15
X /bin/touch $LOCK
X echo Cleaning /news: inodes problem ---- `date` >>$LIBD/cleanlog
X echo Cleaning /news: inodes problem >/dev/osm
X
X
X# algorithm changed 09/28/90 JGD. If we are out of Inodes, then
X# we're dumping news on the ground anyway so go ahead and kill rnews
X# and/or any other process running on the partition.
X
X echo "
X ATTENTION:
X The news partition must be cleaned up. All news-related
X processes will be terminated in 30 seconds. Please
X finish your work and clear out of news.
X " | \
X /etc/wall
X
X sleep 30
X touch $LOCK
X /etc/fuser -k $NEWSSLICE >/dev/null 2>&1
X #just to be sure
X /etc/fuser -k $NEWSSLICE >/dev/null 2>&1
X # lock again just in case anything that got "fused" removed a lock"
X touch $LOCK
X
X # then run the fsck
X #$LIBD/fixnewsfs $NEWSSLICE >/dev/null 2>&1
X /etc/umount $NEWSSLICE
X /etc/fsck -y $NEWSSLICE >/dev/null 2>&1
X /etc/mount $NEWSSLICE
X
X # finally, restart nnmaster
X $NNMASTER
X
X echo "Subject: Inode cleanup \nInodes cleaned up at `date` " | mail jgd pda
X
Xfi
X
X# next, check to see if we ran out of blocks
X#
Xif [ "$2" = "block" ]
Xthen
X echo "Freespace hit danger mark, cleaning up - `date`" >>$LIBD/cleanlog
X echo "Freespace hit danger mark, cleaning up" >/dev/osm
X
X find $SPOOLD -type f -mtime +$DAYS -print | xargs rm -f
X FLAG=1
Xfi
X
Xif [ $FLAG -eq 1 ]
Xthen
X $EXPIRE -r -I -e 999999 -E 999999
Xfi
Xrm -f $LOCK
X
SHAR_EOF
chmod 0755 Inodes || echo "restore of Inodes fails"
if [ $TOUCH = can ]
then
touch -am 0114003091 Inodes
fi
set `wc -c Inodes`;Wc_c=$1
if test "$Wc_c" != "1963"
then echo original size 1963, current size $Wc_c;fi
# ============= README ==============
echo "x - extracting README (Text)"
sed 's/^X//' << 'SHAR_EOF' > README &&
X ICHK - The Inode Checker
X
XIchk is a quick program and script that I wrote in order to address the
XInteractive (and other) System V Inode bug. Ichk sits as a daemon and
Xchecks the designated filesystem at specified intervals against
Xspecified limits. If a problem is found such as too few inodes, too few
Xfree blocks or the ratio of blocks to inodes becomes too large, then a
Xchild process is started which is passed an argument that indicates the
Xproblem. The child process or script should look at the arguments and
Xtake corrective action. An example script that works against the news
Xpartition on Dixie.com is included.
X
XThough a fix for the inode problem from ISC would be ideal and a patch
Xfor the kernel has been posted to the net, this programs directly
Xaddresses the problem without kernal patches (which indicates a full
Xbackup to any responsible sysadm - you got any idea how long it takes to
Xbackup 1GB to 60 meg tape? :-). Plus it catches other problems such as
Xthe news partition just flat running out of space from a flood of news.
X
XThis program is free of any encumbrances on use (see inode.txt for
Xdetails) and is free of the GNU General Public Virus. Comments, bug
Xreports and improvements appreciated. In particular, I'd appreciate
Xthe results of porting this program to Berzerkly or other variant
XUnixes.
X
XJohn De Armond
Xjgd@dixie.com
X
SHAR_EOF
chmod 0644 README || echo "restore of README fails"
if [ $TOUCH = can ]
then
touch -am 0114010791 README
fi
set `wc -c README`;Wc_c=$1
if test "$Wc_c" != "1389"
then echo original size 1389, current size $Wc_c;fi
# ============= ichk.c ==============
echo "x - extracting ichk.c (Text)"
sed 's/^X//' << 'SHAR_EOF' > ichk.c &&
X
X#ifdef SCCSID
Xstatic sccsid[] = "@(#) ichk.c 1.4 91/01/14 00:51:20";
X#endif
X
X#include <stdio.h>
X#include <sys/types.h>
X#include <sys/statfs.h>
X#include <sys/signal.h>
X
X/* this program checks vital parameters on the file system it is invoked
X** against and when a problem is found, runs a specified program that
X** remedies the problem (hopefully).
X**
X** This program must be setuid root and/or run from root
X*/
X
Xstruct statfs buf;
Xlong inode_trigger = 200L;
Xlong block_trigger = 1000L;
Xlong ratio = 5L;
Xint i;
Xchar file_sys[256];
Xchar action[256];
Xchar c;
Xint verbose=0;
Xint snooze=10;
Xlong i_ratio;
Xint talk=0;
X
Xextern int optind;
Xextern char *optarg;
Xextern int errno;
Xextern void usage();
Xextern void take_action();
Xextern char *decimalize();
Xextern void terminator();
X
Xmain(argc,argv)
Xint argc;
Xchar **argv;
X{
X
X#ifdef DAEMON
X /* do NOT do this if you run ichk from init. Init gets real pissed */
X /* with processes that try to daemonize themselves */
X if (fork())
X exit(0);
X#endif
X
X setpgrp(); /* make us a little bit independent */
X
X
X signal(SIGTERM,terminator);
X signal(SIGINT,terminator);
X signal(SIGHUP,terminator);
X
X action[0] = '\0';
X
X if (argc < 2) {
X usage();
X }
X
X if ( setuid(0) ) {
X fprintf(stderr,"*** ERROR: Setuid root failed");
X sleep(5);
X exit(2);
X }
X
X if ( setgid(0) ) {
X fprintf(stderr,"*** ERROR: Setgid root failed");
X sleep(5);
X exit(3);
X }
X
X while ( (c = getopt(argc, argv, "a:b:i:r:s:vt")) != -1 ) {
X switch (c) {
X case 'a': /* specifies the action to take */
X strcpy(action,optarg);
X break;
X case 'b': /* low block count threshold */
X block_trigger = atol(optarg);
X if (block_trigger == 0L)
X usage();
X break;
X case 'i':
X inode_trigger=atol(optarg);
X if (inode_trigger == 0L)
X usage();
X break;
X case 'r':
X ratio=atol(optarg);
X if (ratio == 0L)
X usage();
X break;
X case 'v':
X verbose=1;
X break;
X case 's':
X snooze = atoi(optarg);
X if (snooze == 0 )
X usage();
X break;
X case 't': /* give a report only or (t)alk */
X talk=1;
X break;
X case '?':
X usage();
X break;
X }
X }
X if (talk) verbose = 0;
X
X strcpy(file_sys,argv[optind]);
X
X if (verbose)
X printf("pid = %d\n", getpid());
X
X errno=0;
X if ( statfs(file_sys, &buf, sizeof(struct statfs), 0) == -1) {
X fprintf(stderr,"*** ERROR: statfs() failed, errno = %d\n",errno);
X fprintf(stderr,"file_sys = <%s>\n", file_sys);
X fprintf(stderr,"sizeof(struct statfs) = %d \n", sizeof(struct statfs));
X sleep(10); /* keeps init happy */
X exit(4);
X }
X
X if (talk) {
X printf("%s %ld %ld ", buf.f_fname, buf.f_blocks, buf.f_bfree);
X printf("%ld %ld %s\n", buf.f_files, buf.f_ffree,
X decimalize((buf.f_bfree*10L) / (buf.f_ffree)));
X exit(0);
X
X }
X
X
X if (verbose) {
X printf("File system type = %d\n", buf.f_fstyp);
X printf("Total blocks = %ld\n", buf.f_blocks);
X printf("Free blocks = %ld\n", buf.f_bfree);
X printf("Total inodes = %ld\n", buf.f_files);
X printf("Free inodes = %ld\n", buf.f_ffree);
X printf("Volume name = %s\n", buf.f_fname);
X }
X
X while (1) {
X
X i_ratio = (buf.f_bfree*10L) / buf.f_ffree;
X /* preserving one decimal place of rounding */
X
X /* ratio of blocks to inodes will go UP when the system starts losing
X inodes */
X if ( i_ratio >= (ratio*10L) ) {
X if (verbose)
X printf("Block to Inode ratio %s is too high\n", decimalize(i_ratio));
X take_action("ratio");
X goto slumber;
X } else {
X if (verbose)
X printf("Block to Inode ratio is %s\n", decimalize(i_ratio));
X }
X
X if (buf.f_ffree < inode_trigger) {
X if (verbose)
X printf("Inodes low threshold triggered at %ld inodes\n",
X buf.f_ffree);
X take_action("inode");
X goto slumber;
X } else {
X if (verbose)
X printf("Inodes = %ld\n",
X buf.f_ffree);
X }
X
X if (buf.f_bfree < block_trigger) {
X if (verbose)
X printf("Blocks low threshold triggered at %ld blocks\n",
X buf.f_bfree);
X take_action("block");
X goto slumber;
X } else {
X if (verbose)
X printf("Blocks = %ld\n",
X buf.f_bfree);
X }
X
X slumber:
X sleep(snooze);
X
X } /* for (;;) */
X
X /* NOTREACHED */
X sleep(5);
X exit(0);
X} /* main */
X
Xvoid
Xtake_action(x)
Xchar x[]; /* problem to be remedied */
X{
X char buff[256];
X
X if (!strlen(action))
X return;
X
X sprintf(buff,"%s %s %s", action, file_sys, x);
X if (verbose)
X printf("Action taken: Command line to system()\n\t%s\n",buff);
X
X system(buff);
X return;
X}
X
Xchar *
Xdecimalize(x)
Xlong x;
X{
X static char buff[256];
X int len;
X
X memset(buff,'\0',256);
X
X sprintf(buff,"%ld", x);
X len = strlen(buff);
X
X buff[len] = buff[len-1]; /* move last char over one to the right*/
X buff[len-1] = '.';
X
X return(buff);
X}
X
Xvoid
Xusage()
X{
X
X fprintf(stderr,
X "\nUsage:\n\nichk <-a action>\n <-i inode_trig>\n <-r ratio>\n <-v>\n");
X fprintf(stderr,
X " <-s sleep_secs>\n <-b block_count_trigger>\n <-t>\n file_system_name\n");
X fprintf(stderr,
X "\nThe defaults are %ld inodes, %ld:1 block_to_inodes_ratio,\n",
X inode_trigger, ratio);
X fprintf(stderr,
X "%ld blocks, action = nothing and %d seconds between checks.\n",
X block_trigger, snooze);
X fprintf(stderr,
X "\nWhen the <action> program is invoked, the program is passed the\n");
X fprintf(stderr,
X "following arguments:\nargv[1] == the file system\nargv[2] == the problem\n");
X fprintf(stderr,
X "\nargv[2] can be:\nratio = ratio of blocks to inodes exceeded.\n");
X fprintf(stderr,
X "inode = the low inode threshold exceeded.\n");
X fprintf(stderr,
X "block = the low block threshold exceeded.\n");
X fprintf(stderr,
X "If the -t (talk) flag is specified, all other options are ignored and\n");
X fprintf(stderr,
X "ichk reports the status of the file system in the form:\n fs_name ");
X fprintf(stderr,
X "block_count blocks_free inode_count inodes_free ratio_blocks_2_inodes\n");
X
X sleep(5);
X exit(1);
X}
Xvoid
Xterminator(x)
Xint x;
X{
X fprintf(stderr,"Signal caught: ichk going down on signal %d\n", x);
X exit(0);
X}
SHAR_EOF
chmod 0444 ichk.c || echo "restore of ichk.c fails"
if [ $TOUCH = can ]
then
touch -am 0114005191 ichk.c
fi
set `wc -c ichk.c`;Wc_c=$1
if test "$Wc_c" != "5937"
then echo original size 5937, current size $Wc_c;fi
# ============= ichk.txt ==============
echo "x - extracting ichk.txt (Text)"
sed 's/^X//' << 'SHAR_EOF' > ichk.txt &&
X
X Ichk - The Inode Checker.
X Version 1.0 01/13/91
X
XIchk is designed to watch the condition of a file system and if specified
Xbounds are exceeded, to fire off a corrective process or script. Ichk is
Xdesigned to be run from init(1) as a respawned job. The program was originally
Xwritten to address the famous SysV inode bug that typically manifests
Xitself in conjunction with news but it has many more uses.
X
XWhen ichk is invoked, it uses statfs(2) to collect statistics on the
Xfile system specified in the command line. It then compares the number
Xof free blocks, the number of free inodes and the ratio of free blocks
Xto free inodes to specified values or defaults. If any limits are
Xexceeded, a child process is started via system(3S).
X
XThe child process is passed the name of the file system and a keyword
Xthat specifies the problem to be addresses. Typical action would be in
Xthe case of low inodes or increasing blocks to inodes would be to unmount
Xthe affected partition and fsck it and remount it. Typical action for
Xlow blocks might be to run an emergency expire on the news partition.
X
XI have observed that when the inode bug is manifesting itself, the
Xfirst indication is that as the inode count decreases rapidly, the
Xratio of blocks to inodes increases. Thus, it is possible to catch the
Xproblem before the inodes run completely out and data is dropped.
X
XAfter any corrective action is run or if there is no problem, the daemon
Xsleeps for a designated period and then repeats the process. Here at
XDixie Communications, ichk is designated to examine things once a
Xminute.
X
XIchk also has an information, or "talk" mode that only reports statistics
Xand then exits. This makes ichk useful for reporting file system information
Xto a corrective script and is somewhat easier and less resource intensive
Xto use than df.
X
XIchk is equipped with sleeps of 5 seconds at all exit points which should
Xkeep init happy in case something makes the program exit prematurely.
XNote that if ichk is killed while a corrective process is running, ichk
Xmakes no attempt to kill the child. It is assumed that the child may
Xbe executing actions that would be destructive if not run to
Xcompletion.
XTherefore if you want to kill the child process, you'll have to do it
Xmanually.
X
XOPTIONS
X
XIchk has several options that may be specified at invocation. Most options
Xhave reasonable defaults. Following are listed each of the options and
Xan explanation of each.
X
X
XUsage:
X
ichk <-a action> <-i inode_trig> <-r ratio> <-v> <-s sleep_secs>
X <-b block_count_trigger> <-t> file_system_name
X
X
X-a action Specifies the process or script to be invoked in the event a
X limit is exceeded. Multiple options to the child process should
X be quoted to keep the shell happy. When the child process is invoked,
X it receives all commands specified with the -a option plus the
X name of the affected file system and a keyword that specifies the
X problem. The keywords are:
X
X ratio = ratio of blocks to inodes exceeded.
X inode = the low inode threshold exceeded.
X block = the low block threshold exceeded.
X
X The default is to do nothing; ie, there is no default command
X specified and ichk will do nothing without a specification.
X
X-i inode_trigger Specifies the low inode trigger point. When the free
X inode count is found to fall below this point, the <action> child
X process is invoked. If -i is not specified, then the default of 200
X inodes is used.
X
X-r ratio Specifies the integer ratio of blocks to inodes above which the
X <action> child process is invoked. The default value is 5:1.
X
X-s seconds Specifies how many seconds ichk waits between checks of the
X filesystem. The default is 10 seconds.
X
X-b block_count Specifies the minimum block count below which the <action>
X child process is invoked. The default is 1000 blocks.
X
X-t Specifies the "talk mode" which reports the statistics of the file system
X and then exits. The statistics are presented in a format easily used
X in script files. The format is:
X
X fs_name block_count blocks_free inode_count inodes_free \
X ratio_blocks_2_inodes
X
X-v Specifies verbose mode. Quite a bit of information is returned in
X verbose mode so it is suggested that it only be used for debugging
X purposes.
X
Xfilesystem Specifies the filesystem to examine. This filesystem must
X be mounted. The FM is somewhat vague as to how the statfs(2) call
X works but at least with Interactive Unix, the call works correctly
X whether the actual mount point or a file on the file system is
X specified.
X
X
XOPERATIONS
X
XBecause most any corrective action requires root privileges, this program
Xshould be run as root. Because it provides a tremendous opportunity for a
Xsecurity breach, the permission mask should be set as
follows:
X
X-r-s------ 1 root root 6644 Jan 13 13:12 /etc/ichk
X
Xor 04500.
X
XThe typical usage is for ichk to be run from init(1). A typical inittab
Xentry is as follows:
X
Xk2:2345:respawn:/etc/ichk -a /usr/local/news/bin/Inodes -s 60 /news >/dev/null 2>&1
X
XThis line uses the default values for inodes, blocks and inode to blocks ratio
Xand it checks the /news filesystem every 60 seconds. If a problem is
Xfound, then the Inodes script is run.
X
XThe script /usr/local/news/bin/Inodes or whatever your protective program
Xis named must also be owned by root and must have the permissions 0500 in
Xorder to prevent a security hole. It is also suggested that the
Xdirectory that the script is in be set so that users cannot access it.
X
XA copy of a typical Inodes script is included with this package. It
Xshould examine the specified problem and take action as appropriate.
X
XThough most people will want to run ichk from init(1) because it makes it
Xso easy to manage, there are provisions to run ichk as a true daemon. If
Xyou want to run ichk as a daemon, for instance as from /etc/rc2.d/xxx,
Xsimply specify -DDAEMON on the compile line in the makefile and remake
Xichk. It will then daemonize itself when invoked.
X
XOTHER USES
X
XThough ichk has its primary use with news, there are other uses. For
Xexample, if you configure your system such that all log files are kept
Xon a separate partition, you can have ichk monitor the free blocks on
Xthe partition and fire off cleanup scripts to delete old or obese files.
X
XAnother use is to run ichk -t periodically against the news partition and
Xdirect the output to a log file. In this manner, you can collect use
Xstatistics and in particular, monitor the block to inode ratio. Since
Xthe needed block to inode ratio for news is typically much different
Xthan that for normal file systems, you can quickly determine what
Xthe optimum ratio is.
X
XCOPYRIGHTS, ETC.
X
XThis package is copyright 1991 John De Armond, All rights reserved. You
Xmay freely copy, modify and use this program provided that if you modify
Xit, you do not distribute it as ichk. If you distribute ichk, you must
Xdistribute it intact with the source and documentation. You may not
Xsell it or claim you wrote it. I specifically disclaim any association
Xwith the GNU General Public Virus, er, license. I also specifically
Xdisclaim any warranty of any kind.
X
XAny feedback, errors, bugs, nicely worded gripes :-) or other comments
Xwelcome.
X
XJohn De Armond
Xjgd@dixie.com
X
SHAR_EOF
chmod 0644 ichk.txt || echo "restore of ichk.txt fails"
if [ $TOUCH = can ]
then
touch -am 0114004591 ichk.txt
fi
set `wc -c ichk.txt`;Wc_c=$1
if test "$Wc_c" != "7310"
then echo original size 7310, current size $Wc_c;fi
# ============= makefile ==============
echo "x - extracting makefile (Text)"
sed 's/^X//' << 'SHAR_EOF' > makefile &&
X
X# This install must be run as root
X# This makefile links with shared libraries. Remove the "-lc_s" if
X# shared libraries are not desired for some reason.
X# If you want ichk to automatically demonize itself (NOT when run from
X# init(1)), add the string "-DDAEMON" to the cc line.
X
Xichk: ichk.c
X cc -o ichk -O ichk.c -lc_s
X strip ichk
X mcs -d ichk
X
Xinstall: ichk
X chown root ichk
X chgrp other ichk
X mv /etc/ichk /etc/old # so we can move new ichk in with old one running
X mv ichk /etc
X chmod 04750 /etc/ichk
X #"be sure to kill old ichk process and delete /etc/old"
SHAR_EOF
chmod 0660 makefile || echo "restore of makefile fails"
if [ $TOUCH = can ]
then
touch -am 0114003791 makefile
fi
set `wc -c makefile`;Wc_c=$1
if test "$Wc_c" != "567"
then echo original size 567, current size $Wc_c;fi
exit 0
--
John De Armond, WD4OQC | "Purveyors of speed to the Trade" (tm)
Rapid Deployment System, Inc. | Home of the Nidgets (tm)
Marietta, Ga |
{emory,uunet}!rsiatl!jgd |"Politically InCorrect.. And damn proud of it
exit 0 # Just in case...
--
Kent Landfield INTERNET: kent@sparky.IMD.Sterling.COM
Sterling Software, IMD UUCP: uunet!sparky!kent
Phone: (402) 291-8300 FAX: (402) 291-4362
Please send comp.sources.misc-related mail to kent@uunet.uu.net.