dt@yenta.alb.nm.us (David B. Thomas) (03/28/91)
This message contains the README, followed by the source:
README for reap version 1.1 by David B. Thomas (dt@yenta.alb.nm.us).
A Problem:
News volume fluctuates wildly...sites with small disks must either
expire aggresively, often deleting more than necessary, or worry that an
unexpected huge dose of news might overfill the disk.
A Solution:
Implement a tool that expires according to a user-defined scheme
until sufficient freespace is reclaimed, then stops, leaving as much
juicy news online as is feasible. Reap does this.
Expire Does Two Jobs:
Both Bnews and Cnews expires really do two jobs:
1. trim history files
2. delete outdated articles
Thanks to some inspired jootsing (acronym for "jumping out of the system")
by Mike Murphy (mrm@sceard.com) and others, it is more than possible to
separate those two functions. This is, of course, in keeping with the
unix philosophy of one tool doing one job well!
What Reap Does:
Reap only takes care of the second job: deleting old articles.
It works by checking freespace, and processing one line at a time from a
list of expire functions, until the desired freespace is attained.
Each expire function consists of an age limit in days (decimals okay)
and a list of newsgroups to process or not process, sys file style. Ex:
.5 alt.sex.pictures,talk,!talk.bizarre,junk
1 rec,!rec.games,!rec.humor
This example would check freespace, and if more space is needed, expire
to .5 days everything in alt.sex.pictures, talk (except for talk.bizarre)
and junk. Then it would stop and check freespace again. If still more
space is needed, it would expire to 1 day everything in rec except
rec.games.* and rec.humor.*. It's that simple.
Okay...So How Do I Arrange To Trim The History Files, Since Reap Can't:
Included in this distribution is a shell script (mostly written
by Mike Murphy) to handle Cnews history files. It shouldn't be too
difficult to do something similar for Bnews, or you can give in and use
the original expire utility with options that tell it to expire the
history files only...but I think this will be slow. It just comes down
to removing lines from ordinary text files, based on their contents.
Murphy and I used awk.
But Is It Fast:
Yes, largely because it doesn't have to do much. Even "find | rm"
is slower because find is repeatedly exec-ing rm. "rm -rf" has me beat,
though, I'll bet! :-)
Since the functions of deleting articles and trimming history
are separate, I now run reap every six hours, but trim the history list
just once a day. That effectively keeps my disk space up to snuff, but
only thrashes at the history file in the middle of the night.
Credits:
I owe a lot to Mike Murphy for inspiring me with his "trasher"
system. I also owe a lot to all of your netters who will flood me with
more suggestions and improvements in the coming weeks (hint, hint!).
little david
dt@yenta.alb.nm.us
ps - reap is freeware. No strings(1) attached.
#!/bin/sh
# This is a shell archive (shar 3.44)
# made 03/28/1991 02:51 UTC by dt@yenta
# Source directory /u/dt/src/reap
#
# existing files will NOT be overwritten unless -c is specified
#
# This shar contains:
# length mode name
# ------ ---------- ------------------------------------------
# 934 -rw-rw-r-- Install
# 339 -rw-rw-r-- MANIFEST
# 900 -rw-rw-r-- Makefile
# 2918 -rw-rw-r-- README
# 969 -rwxrwxr-x exphist
# 3567 -rw-rw-r-- lib.c
# 1611 -rw-rw-r-- list.sample
# 2497 -rw-rw-r-- main.c
# 6129 -rw-r--r-- reap.8
# 2845 -rw-rw-r-- reap.c
# 1195 -rw-rw-r-- reap.h
#
# ============= Install ==============
if test -f 'Install' -a X"$1" != X"-c"; then
echo 'x - skipping Install (File already exists)'
else
echo 'x - extracting Install (Text)'
sed 's/^X//' << 'SHAR_EOF' > 'Install' &&
X
To install:
X
1. Edit tops of Makefile and reap.h to fit your system.
X [If you are installing this on a non-system-V machine, you will need
X to rewrite the freeblox() function at the top of lib.c. This should
X be very easy. Please email your working mods to me.]
2. make
3. Compose a sample function list file or use the sample provided to
X test reap (use the -n option to avoid really removing files).
4. su and make install.
5. Arrange to trim your news system's history list regularly. If you have
X Cnews, try the "exphist" script provided, run from cron. That
X script requires that a tiny file, /usr/lib/news/histdays, containing
X the number of days of history data to keep, or else you can hardcode
X in your favorite number. If you have Bnews, I don't have a nifty
X suggestion, other than using expire(8) with a ridiculously long
X article expire time, but a sane history expire time.
6. Arrange to run reap from cron.
SHAR_EOF
chmod 0664 Install ||
echo 'restore of Install failed'
Wc_c="`wc -c < 'Install'`"
test 934 -eq "$Wc_c" ||
echo 'Install: original size 934, current size' "$Wc_c"
fi
# ============= MANIFEST ==============
if test -f 'MANIFEST' -a X"$1" != X"-c"; then
echo 'x - skipping MANIFEST (File already exists)'
else
echo 'x - extracting MANIFEST (Text)'
sed 's/^X//' << 'SHAR_EOF' > 'MANIFEST' &&
Included with this distribution are:
X
README - general description and credits
Install - hints on how to install reap and get it going
Xexphist - history trimmer utility for Cnews
list.sample - sample function script file for reap
reap.8 - man page
lib.c - source
main.c - source
reap.c - source
reap.h - source
Makefile - makefile
SHAR_EOF
chmod 0664 MANIFEST ||
echo 'restore of MANIFEST failed'
Wc_c="`wc -c < 'MANIFEST'`"
test 339 -eq "$Wc_c" ||
echo 'MANIFEST: original size 339, current size' "$Wc_c"
fi
# ============= Makefile ==============
if test -f 'Makefile' -a X"$1" != X"-c"; then
echo 'x - skipping Makefile (File already exists)'
else
echo 'x - extracting Makefile (Text)'
sed 's/^X//' << 'SHAR_EOF' > 'Makefile' &&
#----------------------------
# parameters for installation
#----------------------------
X
# directories where manual and executable are to be installed
BINDIR = /usr/lib/newsbin/expire
MANDIR = /usr/man/man8
X
# owner, group and file mode for manual and executable
EXEOWNER = bin
EXEGROUP = bin
EXEMODE = 775
MANOWNER = bin
MANGROUP = bin
MANMODE = 664
X
X
# your favorite C compiler
#CC = cc
CC = shcc
X
# directory access functions library, if not in standard C library.
LIB = -ldirent
X
X
#-------------
# boring stuff
#-------------
X
OBJ = main.o lib.o reap.o
X
reap: $(OBJ)
X $(CC) -s -o reap $(SHLIB) $(OBJ) $(LIB)
X
$(OBJ): reap.h
X
install: reap reap.8
X cp reap $(BINDIR)
X chown $(EXEOWNER) $(BINDIR)/reap
X chgrp $(EXEGROUP) $(BINDIR)/reap
X chmod $(EXEMODE) $(BINDIR)/reap
X cp reap.8 $(MANDIR)
X chmod $(MANMODE) $(MANDIR)/reap.8
X chown $(MANOWNER) $(MANDIR)/reap.8
X chgrp $(MANGROUP) $(MANDIR)/reap.8
SHAR_EOF
chmod 0664 Makefile ||
echo 'restore of Makefile failed'
Wc_c="`wc -c < 'Makefile'`"
test 900 -eq "$Wc_c" ||
echo 'Makefile: original size 900, current size' "$Wc_c"
fi
# ============= README ==============
if test -f 'README' -a X"$1" != X"-c"; then
echo 'x - skipping README (File already exists)'
else
echo 'x - extracting README (Text)'
sed 's/^X//' << 'SHAR_EOF' > 'README' &&
X
README for reap version 1.1 by David B. Thomas (dt@yenta.alb.nm.us).
X
A Problem:
X News volume fluctuates wildly...sites with small disks must either
Xexpire aggresively, often deleting more than necessary, or worry that an
unexpected huge dose of news might overfill the disk.
X
X
A Solution:
X Implement a tool that expires according to a user-defined scheme
until sufficient freespace is reclaimed, then stops, leaving as much
juicy news online as is feasible. Reap does this.
X
X
Expire Does Two Jobs:
X Both Bnews and Cnews expires really do two jobs:
X 1. trim history files
X 2. delete outdated articles
Thanks to some inspired jootsing (acronym for "jumping out of the system")
by Mike Murphy (mrm@sceard.com) and others, it is more than possible to
separate those two functions. This is, of course, in keeping with the
unix philosophy of one tool doing one job well!
X
X
What Reap Does:
X Reap only takes care of the second job: deleting old articles.
It works by checking freespace, and processing one line at a time from a
list of expire functions, until the desired freespace is attained.
Each expire function consists of an age limit in days (decimals okay)
and a list of newsgroups to process or not process, sys file style. Ex:
X
X .5 alt.sex.pictures,talk,!talk.bizarre,junk
X 1 rec,!rec.games,!rec.humor
X
This example would check freespace, and if more space is needed, expire
to .5 days everything in alt.sex.pictures, talk (except for talk.bizarre)
and junk. Then it would stop and check freespace again. If still more
space is needed, it would expire to 1 day everything in rec except
rec.games.* and rec.humor.*. It's that simple.
X
X
Okay...So How Do I Arrange To Trim The History Files, Since Reap Can't:
X Included in this distribution is a shell script (mostly written
by Mike Murphy) to handle Cnews history files. It shouldn't be too
difficult to do something similar for Bnews, or you can give in and use
the original expire utility with options that tell it to expire the
history files only...but I think this will be slow. It just comes down
to removing lines from ordinary text files, based on their contents.
Murphy and I used awk.
X
X
But Is It Fast:
X Yes, largely because it doesn't have to do much. Even "find | rm"
is slower because find is repeatedly exec-ing rm. "rm -rf" has me beat,
though, I'll bet! :-)
X
X Since the functions of deleting articles and trimming history
are separate, I now run reap every six hours, but trim the history list
just once a day. That effectively keeps my disk space up to snuff, but
only thrashes at the history file in the middle of the night.
X
X
Credits:
X I owe a lot to Mike Murphy for inspiring me with his "trasher"
system. I also owe a lot to all of your netters who will flood me with
more suggestions and improvements in the coming weeks (hint, hint!).
X
X little david
X dt@yenta.alb.nm.us
X
ps - reap is freeware. No strings(1) attached.
SHAR_EOF
chmod 0664 README ||
echo 'restore of README failed'
Wc_c="`wc -c < 'README'`"
test 2918 -eq "$Wc_c" ||
echo 'README: original size 2918, current size' "$Wc_c"
fi
# ============= exphist ==============
if test -f 'exphist' -a X"$1" != X"-c"; then
echo 'x - skipping exphist (File already exists)'
else
echo 'x - extracting exphist (Text)'
sed 's/^X//' << 'SHAR_EOF' > 'exphist' &&
#! /bin/sh
# exphist - expire history file
X
# =()<. ${NEWSCONFIG-@<NEWSCONFIG>@}>()=
. ${NEWSCONFIG-/usr/lib/news/bin/config}
X
PATH=$NEWSCTL/bin:$NEWSBIN/expire:$NEWSBIN:$NEWSPATH ; export PATH
umask $NEWSUMASK
X
days=`cat /usr/lib/news/histdays`
X
lock="$NEWSCTL/LOCKexpire"
ltemp="$NEWSCTL/L.$$"
Xecho $$ >$ltemp
trap "rm -f $ltemp ; exit 0" 0 1 2 15
if newslock $ltemp $lock
then
X trap "rm -f $ltemp $lock ; exit 0" 0 1 2 15
Xelse
X echo "$0: expire apparently already running" | mail "$NEWSMASTER"
X exit 1
fi
X
cd $NEWSCTL
rm -f history.n history.n.pag history.n.dir
now=`getdate now`
age=`expr 86400 \* $days`
ago=`expr $now - $age`
awk "{split(\$2,dates,\"~\");if(dates[1]>$ago)print \$0}" history >history.n
mkdbm history.n
[ -s history.n ] &&
mv history history.o && # install new ASCII history file
mv history.n history &&
rm -f history.pag && # and related dbm files
rm -f history.dir &&
mv history.n.pag history.pag &&
mv history.n.dir history.dir
Xexit 0
SHAR_EOF
chmod 0775 exphist ||
echo 'restore of exphist failed'
Wc_c="`wc -c < 'exphist'`"
test 969 -eq "$Wc_c" ||
echo 'exphist: original size 969, current size' "$Wc_c"
fi
# ============= lib.c ==============
if test -f 'lib.c' -a X"$1" != X"-c"; then
echo 'x - skipping lib.c (File already exists)'
else
echo 'x - extracting lib.c (Text)'
sed 's/^X//' << 'SHAR_EOF' > 'lib.c' &&
X
/*
X * lib.c
X * subroutines needed by reap utility
X */
X
#include "reap.h"
X
/*
X * freeblox(devno)
X *
X * THIS WILL REQUIRE MODIFICATION TO WORK UNDER NON-SYSTEM-V UNIX.
X * IT SHOULD BE EASY. PLEASE EMAIL WORKING MODS TO THE AUTHOR:
X * dt@yenta.alb.nm.us
X *
X * assuming devno is the device number of a file system,
X * this function returns the free space on that file system,
X * in blocks (whatever that means for that filesystem) as an int.
X *
X */
X
freeblox(devno)
int devno;
{
X static struct ustat ust;
X
X if (ustat(devno, &ust))
X ouch ("%s: ustat() failed\n");
X
X return (ust.f_tfree);
X
} /* freeblox() */
X
X
X
X
/*
X * parse(cmd)
X * execute a line from the script file
X *
X * valid command lines:
X * a blank line is ignored
X * # comment (ignored)
X * days filespecs
X *
X * filespecs is a comma separated list of any combination of the following:
X * 1. a directory (newsgroup) to include, without recursion.
X * ex: alt.sources.
X * 2. a directory to include, recursively.
X * ex: alt.sources
X * 3. a directory to exclude, without recursion.
X * ex: !alt.sex.
X * 4. a directory to exclude, recursively.
X * ex: !alt.sex
X */
X
parse(cmd)
char *cmd;
{
X register int l;
X int recurseflag;
X double days;
X char *t,
X *p,
X *q,
X *seps = ",\n ";
X
X
/* skip initial whitespace */
X p = cmd;
X while (isspace(*p))
X ++p;
X
/* ignore blank lines or comments */
X if (*p == '\0' || *p == '#')
X return(1);
X
/* read the expire time from the line as a floating point number,
X * then figure out what the timestamp on a file that old would be */
X age = now - (time_t) (atof(p) * SECINDAY);
X
/* skip ahead to the filespec list */
X while (!isspace(*p))
X ++p;
X while (isspace(*p))
X ++p;
X
/* initialize exclusion list to null list */
X preclude();
X
/* for each filespec in list */
X for (t = strtok(p, seps); t; t = strtok(NULL, seps)) {
X
X /* forbid starting with a slash */
X if (*t == '/')
X *t = '.';
X
X /* change dots to slashes except in column 1 */
X while ( (q = strchr(t+1,'.')) != NULL)
X *q = '/';
X
X /* for final dot or slash, remove it, and shut off recursion */
X recurseflag = 1;
X if (t[l = (strlen(t) - 1)] == '/') {
X recurseflag = 0;
X t[l] = '\0';
X }
X
X if (*t == '!')
X exclude (t+1, recurseflag);
X else
X include (t, recurseflag);
X }
X
X reap();
X
X return (0);
X
} /* parse() */
X
X
X
/*
X * newnode,exclude, include, preclude
X *
X * functions to maintain global linked lists:
X * incl include this path in list of dirs to reap
X * excl leave out this directory
X (each entry can be recursive or not)
X *
X * the functions are:
X * include (text,rflag) add to incl list
X * exclude (text,rflag) add to excl list
X * preclude() clear both lists
X */
struct filspec *
newnode(text)
char *text;
{
X struct filspec *f;
X
X if ( (f = (struct filspec *) malloc (sizeof(struct filspec))) == NULL ||
X (f->name = malloc (strlen(text)+1)) == NULL)
X ouch ("%s: out of memory\n");
X
X strcpy (f->name, text);
X return (f);
}
Xexclude(text, rflag)
char *text;
int rflag;
{
X struct filspec *f;
X
X f = newnode(text);
X f->recurse = rflag;
X f->next = excl;
X excl = f;
}
include(text, rflag)
char *text;
int rflag;
{
X struct filspec *f;
X
X f = newnode(text);
X f->recurse = rflag;
X f->next = incl;
X incl = f;
}
preclude()
{
X struct filspec *p;
X
X while (incl != NULL) {
X p = incl;
X incl = incl->next;
X free (p->name);
X free (p);
X }
X while (excl != NULL) {
X p = excl;
X excl = excl->next;
X free (p->name);
X free (p);
X }
}
X
X
/*
X * ouch(fmt, arg)
X * outputs an error message and exits with error status
X */
X
ouch (fmt,arg)
char *fmt, *arg;
{
X fprintf (stderr, fmt, progname, arg);
X exit (-1);
}
SHAR_EOF
chmod 0664 lib.c ||
echo 'restore of lib.c failed'
Wc_c="`wc -c < 'lib.c'`"
test 3567 -eq "$Wc_c" ||
echo 'lib.c: original size 3567, current size' "$Wc_c"
fi
# ============= list.sample ==============
if test -f 'list.sample' -a X"$1" != X"-c"; then
echo 'x - skipping list.sample (File already exists)'
else
echo 'x - extracting list.sample (Text)'
sed 's/^X//' << 'SHAR_EOF' > 'list.sample' &&
# yenta's /usr/lib/news/reaplist function script file.
.1 junk,alt.flame,control,comp.binaries,alt.sex.pictures.,alt.fractals.pictures
.1 alt.desert-storm,alt.desert-shield,soc.motss,soc.culture
1 soc.singles,alt.activism,alt.romance.chat
.5 talk
1 sci,!sci.skeptic
2 sci.skeptic
.5 comp.text,comp.sys.atari,comp.windows,comp.os.vms
.5 comp.unix.sysv386
.5 misc.jobs,misc.handicap
2 soc,!soc.singles,!soc.motss,!soc.culture
.5 rec,!rec.arts.anime,!rec.arts.animation,!rec.arts.movies,!rec.games.hack,!rec.radio.amateur,!rec.autos,!rec.arts.tv,!rec.audio,!rec.video,!rec.music,!rec.skydiving
1 rec.music,!rec.music.synth
3 rec.music.synth,rec.skydiving
2 alt.tv
2 rec.arts.anime,rec.arts.animation,rec.arts.movies,rec.radio.amateur,rec.autos,rec.arts.tv,rec.audio,rec.video
3 rec.games.hack
1 misc.headlines,misc.consumers,misc.kids,misc.legal
1 alt.sex,!alt.sex.bondage,!alt.sex.pictures.
1.5 comp,!comp.dcom.telecom,!comp.binaries,!comp.text,!comp.sys.atari,!comp.windows,!comp.os.vms,!comp.sys.att,!comp.sys.3b1,!comp.sources.3b1
3 comp.dcom.telecom
1 comp.binaries
1 alt.sources
1 news,!news.announce.newusers
3 biz
3 misc
3 alt,!alt.sources,!alt.tv,!alt.desert-shield,!alt.activism,!alt.fractals.pictures,!alt.desert-storm,!alt.romance.chat
3 comp.sys.att
5 comp.sys.3b1
5 comp.sources.3b1
7 general
# up to here is "normal expire"... now let's get desparate
.1 alt.sex.pictures
.1 rec.arts.anime,rec.arts.animation,rec.arts.movies,rec.ham-radio,rec.autos,rec.arts.tv,rec.audio,rec.video
.1 biz
.1 talk
.1 news
.1 soc
.1 sci
.1 comp
.1 alt
.1 misc
# if we got here, we're in big trouble -- wipe it all!
0 .
SHAR_EOF
chmod 0664 list.sample ||
echo 'restore of list.sample failed'
Wc_c="`wc -c < 'list.sample'`"
test 1611 -eq "$Wc_c" ||
echo 'list.sample: original size 1611, current size' "$Wc_c"
fi
# ============= main.c ==============
if test -f 'main.c' -a X"$1" != X"-c"; then
echo 'x - skipping main.c (File already exists)'
else
echo 'x - extracting main.c (Text)'
sed 's/^X//' << 'SHAR_EOF' > 'main.c' &&
X
/*
X * main.c
X * main routine and globals for reap utility, version 1.1
X */
X
X
#include "reap.h"
X
X
/* linked lists for included and excluded file specs */
struct filspec
X *incl = NULL,
X *excl = NULL
;
char
X *scriptfile = SCRIPT, /* path of function script */
X *progname, /* argv[0] for errors */
X *newsdir = NEWSDIR /* news root directory */
;
int
X verbose = 0, /* verbosity (-v) flag */
X dryrun = 0 /* do-not-unlink (-n) flag */
;
time_t
X age, /* unlink files older than this */
X now /* current time */
;
X
X
/************************
X * MAIN FUNCTION *
X ************************/
X
main(argc, argv)
int argc;
char *argv[];
{
X int device, /* device # containing newsdir */
X c,
X lineno = 0,
X summary = 0,
X thenfree,
X errflag = 0;
X long wantblox; /* desired free space */
X char *p;
X static char line[MAXLINE];
X FILE *script; /* file ptr for function script */
X struct stat st;
X
X
/* save argv[0] for error spewings */
X progname = argv[0];
X
/* parse args */
X while ( (c=getopt(argc,argv,"vsnf:")) != EOF)
X switch(c) {
X case 'n':
X ++dryrun;
X break;
X case 'v':
X ++verbose;
X break;
X case 'f':
X scriptfile = optarg;
X break;
X case 's':
X ++summary;
X break;
X default:
X ++errflag;
X break;
X }
X
X if (argc - optind != 1 ||
X sscanf(argv[optind], "%ld", &wantblox) != 1 )
X ++errflag;
X
X if (errflag) {
X fprintf (stderr,
X "usage: %s [-v] [-s] [-n] [-f funclist] freeblocks\n",
X progname);
X exit (-1);
X }
X
X
/* stat newsdir to find the number of the device it's on */
X if (stat(newsdir, &st))
X ouch ("%s: can't stat %s\n", newsdir);
X device = st.st_dev;
X
/* open function script file for reading */
X if ( (script = fopen (scriptfile, "r")) == NULL)
X ouch ("%s: can't read %s\n", scriptfile);
X
/* Record the current time, for deciding what gets the axe. */
X time (&now);
X
/* chdir to newsdir */
X chdir (newsdir);
X
/* record initial freespace if summary is desired */
X if (summary)
X thenfree = freeblox(device);
X
/* main loop ... process until goal reached or end of script */
X
X while (freeblox(device) < wantblox &&
X (p = fgets (line, MAXLINE, script)) != NULL )
X parse (line), ++lineno;
X
X fclose (script);
X
X if (summary) {
X c = freeblox(device);
X printf ("Freespace was %d, now %d. Cleared %d.\n",
X thenfree, c, c - thenfree);
X printf ("Stopped after line %d in %s\n", lineno,
X scriptfile);
X }
X
/* return 0 if freespace goal was met at exit, 1 if not */
X exit (freeblox(device) >= wantblox ? 0 : 1);
X
} /* main() */
X
SHAR_EOF
chmod 0664 main.c ||
echo 'restore of main.c failed'
Wc_c="`wc -c < 'main.c'`"
test 2497 -eq "$Wc_c" ||
echo 'main.c: original size 2497, current size' "$Wc_c"
fi
# ============= reap.8 ==============
if test -f 'reap.8' -a X"$1" != X"-c"; then
echo 'x - skipping reap.8 (File already exists)'
else
echo 'x - extracting reap.8 (Text)'
sed 's/^X//' << 'SHAR_EOF' > 'reap.8' &&
.TH REAP 8 LOCAL
.SH NAME
reap - remove news articles as space needed
.SH SYNOPSIS
.B reap
[-v] [-s] [-n] [-f scriptfile] freeblocks
.SH DESCRIPTION
.I Reap
checks disk freespace and deletes netnews articles
according to a flexible user-specified
scheme until
.I freeblocks
minimum freespace is available.
It does this by sequentially reading commands (expire functions) from a
text file (function script), and executing them one at a time.
.PP
Each expire function consists of an age limit and a list of newsgroups
to expire to that limit.
Before executing
Xeach new expire function, reap checks the freespace on the news spool
device. Reap will exit when either the desired freespace is attained, or
the end of the function script file is reached (the latter case is considered
an unsuccessful exit).
By carefully designing the function script, a news administrator can
automate a wide variety of expiring policies.
.PP
The
.B -n
causes a dry run (as in make(1)). When this
option is specified, reap merely reports which articles it would delete if
the option were left off. This is useful for debugging function scripts.
However, since no space is actually freed in this mode,
it will either do nothing
(there was enough space to begin with) or proceed through the entire function
script file, indicating that reap would delete all applicable articles.
.PP
With the
.B -v
option, reap prints a verbose commentary on its progress to the standard
output.
.PP
The
.B -s
option causes reap to print a brief summary of blocks freed to the standard
output just before exiting. It also indicates how much of the script file
was processed. This is independent of -v.
.PP
By default, reap looks in /usr/lib/news/reaplist for its list of functions.
The
.B -f
option is used to specify an alternate function script file.
.SH FUNCTION SCRIPT FILE FORMAT
A function script file consists of a series of expire functions, one per line.
Each expire function contains an age limit (in days, decimals okay), followed
by a comma-separated list of newsgroup specifications, similar to the
sys file format.
.PP
There are four types of valid newsgroup specifications that can go in the list:
.PP
(1) An ordinary newsgroup name (alt.sex) indicates that any articles in that
newsgroup and any of its descendants (such as alt.sex.pictures), which are
older than the age limit, should be expired.
.PP
(2) A newsgroup name with a trailing dot (alt.sex.) indicates that any articles
in that newsgroup, which are older than the age limit, should be expired.
However, recursion is not implied. Articles in descendants of that group
(such as alt.sex.pictures) are not affected.
.PP
(3) A newsgroup name with a preceding exclamation point (!alt.sex.pictures)
indicates that that newsgroup and all of its descendants should be excluded
from the list.
For example, "1 alt.sex,!alt.sex.pictures" means to expire everything subsumed
under alt.sex to 1 day, but do not touch anything subsumed under
alt.sex.pictures.
.PP
(4) A newsgroup name with both a preceding exclamation point and a trailing
dot (!alt.sex.pictures.) indicates that that newsgroup should be excluded
from the list, but its descendants should remain in the list. For example,
"1 alt.sex,!alt.sex.pictures." means to expire everything subsumed under
alt.sex to 1 day, including the contents of any subgroups of alt.sex.pictures
(like alt.sex.pictures.d), but do not touch the articles in the newsgroup
alt.sex.pictures.
.PP
Blank lines and lines beginning with a "#" are ignored.
A sample function script file:
.PP
.nf
X # my first function script file
X 0.5 talk,junk,alt.sex.pictures.
X 2 rec,!rec.games,!rec.humor
X 2 misc
.fi
.PP
In the example,
if space is needed, reap
Xexpires to .5 days the entire talk and junk
hierarchies, and alt.sex.pictures (not touching any subgroups of
alt.sex.pictures, such as alt.sex.pictures.d).
If still more space is needed, the
rec hierarchy (excepting all of rec.games.* and rec.humor.*) is expired
to 2 days. If freespace is still short, reap then expires all of misc
to 2 days.
.PP
Note that, while it would be possible to consolidate the 2-day
lines, leaving them separate makes it possible for reap to stop
in between them if sufficient space is cleared.
.PP
A practical function script file would directly or indirectly
include every group carried on the system, with some age limit. Otherwise,
the spool directory will inevitably overflow without operator intervention.
.PP
It is valid to specify a dot all by itself (.) in the newsgroup list field.
This is taken to mean the entire spool directory hierarchy, so "14 ." means
to expire all news to 14 days. Be warned that reap must search the entire
news hierarchy when this feature is used. Still, it is probably a good
idea to end all function script files with a "0 .", so that, in absolute
desperation, all news would be removed.
.SH FILES
.TP 25
/usr/spool/news
News spool directory
.TP 25
/usr/lib/news/reaplist
Default function list file
.SH DIAGNOSTICS
Returns zero if sufficient space was free at exit, 1 if the entire function
script was executed without freeing enough space, and -1 on an error
condition.
.SH CAVEATS
What constitutes a disk "block" is implementation-dependent.
.PP
History files are not updated. That must be performed as a separate operation.
.PP
For best performance, exclude recursively (without the dot) whenever
possible. Otherwise, reap will have to search the excluded directory
for any subdirectories which might not have been excluded, which is slow.
.SH BUGS
The use of the ustat() system call is not very portable, and your humble
author isn't aware of its non-system-V equivalents.
.PP
Lines in the function file are limited to 1024 characters.
.PP
The "cleared" figure in the summary merely indicates the difference in
freespace before and after the run, not necessarily the actual number of
blocks liberated by reap.
.SH AUTHOR
Reap is freeware by David B. Thomas (dt@yenta.alb.nm.us). You are free
to copy, distribute, staple, bend, fold or mutilate this package to your
heart's content. Please email any enhancements or ideas for enhancements.
SHAR_EOF
chmod 0644 reap.8 ||
echo 'restore of reap.8 failed'
Wc_c="`wc -c < 'reap.8'`"
test 6129 -eq "$Wc_c" ||
echo 'reap.8: original size 6129, current size' "$Wc_c"
fi
# ============= reap.c ==============
if test -f 'reap.c' -a X"$1" != X"-c"; then
echo 'x - skipping reap.c (File already exists)'
else
echo 'x - extracting reap.c (Text)'
sed 's/^X//' << 'SHAR_EOF' > 'reap.c' &&
X
/*
X * reap.c
X * contains the reap() function.
X *
X * Once the global linked lists incl and excl have been stuffed,
X * reap() actually scans the filesystem for files that meet the specs
X * and unlinks them if they are older than the age global variable.
X */
X
#include "reap.h"
X
X
reap()
{
X struct filspec *f;
X
X for (f = incl; f != NULL; f = f->next) {
X if (verbose)
X printf ("scanning %s ...\n", f->name);
X dodir (f->name, f->recurse);
X }
X
} /* reap() */
X
X
X
dodir(name, rflag)
char *name;
int rflag;
{
X /* the following can be overwritten safely during recursion */
X static struct filspec *e;
X static struct stat st;
X static struct dirent *dp;
X static char thisname[MAXFILENAME+1];
X /* the following must be preserved through recursion */
X char *fullpath;
X int eflag = 0;
X DIR *dirp;
X
X
/* open directory for reading */
X if ( (dirp = opendir(name)) == NULL)
X ouch ("%s: can't read directory %s\n", name);
X
X
/* see if this directory is excluded.
X * If it's excluded recursively, quit here.
X * If it's excluded non-recursively, set a flag, so we won't consider
X * deleting any files in it, but we'll still explore subdirectories.
X */
X for (e = excl; e != NULL; e = e->next)
X if (!strcmp (name, e->name))
X break;
X if (e != NULL) {
X if (e->recurse)
X return;
X else
X ++eflag;
X }
X
X
/* loop for each directory entry */
X while ( (dp = readdir(dirp)) != NULL) {
X
X /* name might be exactly MAXFILENAME characters long, and thus
X * might not be null-terminated. Some insurance:
X */
X strncpy (thisname, dp->d_name, MAXFILENAME);
X thisname[MAXFILENAME] = '\0';
X
X /* skip dot and dotdot */
X if (!strcmp(thisname, ".") || !strcmp(thisname, ".."))
X continue;
X
X /* build the full pathname of current object */
X if ( (fullpath =
X malloc(strlen(name)+strlen(thisname)+2)) == NULL)
X ouch ("%s: out of memory\n");
X
X sprintf (fullpath, "%s/%s", name, thisname);
X
X /* try to stat the object */
X if (stat(fullpath,&st)) {
X fprintf (stderr, "%s: can't stat %s\n",
X progname, fullpath);
X free (fullpath);
X continue;
X }
X
X /* maybe recurse if it's a directory */
X if ( st.st_mode & S_IFDIR ) {
X if (rflag)
X dodir (fullpath, 1);
X free (fullpath);
X continue;
X }
X
X /* it's a file ... is this a non-recursively excluded directory?
X * if so, there's nothing to do to this file
X */
X if (eflag)
X continue;
X
X /* leave it alone if this directory is excluded, or
X * if it's new enough.
X */
X if (eflag || st.st_mtime > age) {
X free (fullpath);
X continue;
X }
X
X /* reap this file! */
X if (dryrun) {
X printf ("Would unlink %s\n", fullpath);
X free (fullpath);
X continue;
X }
X if (verbose)
X printf ("Unlinking %s\n", fullpath);
X
X if (unlink (fullpath) == -1)
X fprintf (stderr,
X "%s: cannot unlink %s\n", progname, fullpath);
X
X free (fullpath);
X
X } /* while */
X
X closedir (dirp);
X
} /* dodir() */
SHAR_EOF
chmod 0664 reap.c ||
echo 'restore of reap.c failed'
Wc_c="`wc -c < 'reap.c'`"
test 2845 -eq "$Wc_c" ||
echo 'reap.c: original size 2845, current size' "$Wc_c"
fi
# ============= reap.h ==============
if test -f 'reap.h' -a X"$1" != X"-c"; then
echo 'x - skipping reap.h (File already exists)'
else
echo 'x - extracting reap.h (Text)'
sed 's/^X//' << 'SHAR_EOF' > 'reap.h' &&
X
/*
X * reap.h
X * header file for reap utility
X */
X
/* set MAXFILENAME to the maximum number of characters in a filename for
X * your system. Typically 14 or infinity, where infinity equals
X * 256 characters. :-)
X */
#define MAXFILENAME 14
X
/* set NEWSDIR to the directory containing news on your system.
X * Very commonly /usr/spool/news
X */
#define NEWSDIR "/usr/spool/news"
X
/* set SCRIPT to the path of the default function script file
X * Usually this is /usr/lib/news/reaplist
X */
#define SCRIPT "/usr/lib/news/reaplist"
X
X
X
X
#include <stdio.h>
#include <string.h>
#include <ctype.h>
#include <time.h>
#include <malloc.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <ustat.h>
#include <dirent.h>
X
#define MAXLINE 1024 /* max len of line in script */
#define SECINDAY (3600 * 24) /* seconds in a day */
X
/* structure for linked lists of included and excluded file specs */
struct filspec {
X char *name;
X int recurse;
X struct filspec *next;
};
X
Xextern struct filspec
X *incl,
X *excl
;
Xextern char
X *progname,
X *scriptfile,
X *newsdir
;
Xextern int
X verbose,
X dryrun,
X optind
;
Xextern char *optarg;
Xextern double atof();
Xextern long freeblox();
Xextern time_t
X age,
X now
;
SHAR_EOF
chmod 0664 reap.h ||
echo 'restore of reap.h failed'
Wc_c="`wc -c < 'reap.h'`"
test 1195 -eq "$Wc_c" ||
echo 'reap.h: original size 1195, current size' "$Wc_c"
fi
exit 0
--
Bottom of stack = 0x40000
Stack pointer = 0x3fffe
Don't push it!admerlev@immd4.informatik.uni-erlangen.de (Arnd Merlevede) (04/02/91)
dt@yenta.alb.nm.us (David B. Thomas) writes: hallo andreas, vielleicht interessiert Dich die follgende Msg falls Dus nicht selbst gelesen hast. Uebrigns, liegt noch die Source von nn irgendwo ? Und ist des schwer die zu installieren ? Ich wuerd sie gerne im ET-CipPool installieren (ala faui09) ? Bis denn Arnd >This message contains the README, followed by the source: >README for reap version 1.1 by David B. Thomas (dt@yenta.alb.nm.us). >A Problem: > News volume fluctuates wildly...sites with small disks must either >expire aggresively, often deleting more than necessary, or worry that an >unexpected huge dose of news might overfill the disk. >A Solution: > Implement a tool that expires according to a user-defined scheme >until sufficient freespace is reclaimed, then stops, leaving as much >juicy news online as is feasible. Reap does this. >Expire Does Two Jobs: > Both Bnews and Cnews expires really do two jobs: > 1. trim history files > 2. delete outdated articles >Thanks to some inspired jootsing (acronym for "jumping out of the system") >by Mike Murphy (mrm@sceard.com) and others, it is more than possible to >separate those two functions. This is, of course, in keeping with the >unix philosophy of one tool doing one job well! >What Reap Does: > Reap only takes care of the second job: deleting old articles. >It works by checking freespace, and processing one line at a time from a >list of expire functions, until the desired freespace is attained. >Each expire function consists of an age limit in days (decimals okay) >and a list of newsgroups to process or not process, sys file style. Ex: > .5 alt.sex.pictures,talk,!talk.bizarre,junk > 1 rec,!rec.games,!rec.humor >This example would check freespace, and if more space is needed, expire >to .5 days everything in alt.sex.pictures, talk (except for talk.bizarre) >and junk. Then it would stop and check freespace again. If still more >space is needed, it would expire to 1 day everything in rec except >rec.games.* and rec.humor.*. It's that simple. >Okay...So How Do I Arrange To Trim The History Files, Since Reap Can't: > Included in this distribution is a shell script (mostly written >by Mike Murphy) to handle Cnews history files. It shouldn't be too >difficult to do something similar for Bnews, or you can give in and use >the original expire utility with options that tell it to expire the >history files only...but I think this will be slow. It just comes down >to removing lines from ordinary text files, based on their contents. >Murphy and I used awk. >But Is It Fast: > Yes, largely because it doesn't have to do much. Even "find | rm" >is slower because find is repeatedly exec-ing rm. "rm -rf" has me beat, >though, I'll bet! :-) > Since the functions of deleting articles and trimming history >are separate, I now run reap every six hours, but trim the history list >just once a day. That effectively keeps my disk space up to snuff, but >only thrashes at the history file in the middle of the night. >Credits: > I owe a lot to Mike Murphy for inspiring me with his "trasher" >system. I also owe a lot to all of your netters who will flood me with >more suggestions and improvements in the coming weeks (hint, hint!). > little david > dt@yenta.alb.nm.us >ps - reap is freeware. No strings(1) attached. >#!/bin/sh ># This is a shell archive (shar 3.44) ># made 03/28/1991 02:51 UTC by dt@yenta ># Source directory /u/dt/src/reap ># ># existing files will NOT be overwritten unless -c is specified ># ># This shar contains: ># length mode name ># ------ ---------- ------------------------------------------ ># 934 -rw-rw-r-- Install ># 339 -rw-rw-r-- MANIFEST ># 900 -rw-rw-r-- Makefile ># 2918 -rw-rw-r-- README ># 969 -rwxrwxr-x exphist ># 3567 -rw-rw-r-- lib.c ># 1611 -rw-rw-r-- list.sample ># 2497 -rw-rw-r-- main.c ># 6129 -rw-r--r-- reap.8 ># 2845 -rw-rw-r-- reap.c ># 1195 -rw-rw-r-- reap.h ># ># ============= Install ============== >if test -f 'Install' -a X"$1" != X"-c"; then > echo 'x - skipping Install (File already exists)' >else >echo 'x - extracting Install (Text)' >sed 's/^X//' << 'SHAR_EOF' > 'Install' && >X >To install: >X >1. Edit tops of Makefile and reap.h to fit your system. >X [If you are installing this on a non-system-V machine, you will need >X to rewrite the freeblox() function at the top of lib.c. This should >X be very easy. Please email your working mods to me.] >2. make >3. Compose a sample function list file or use the sample provided to >X test reap (use the -n option to avoid really removing files). >4. su and make install. >5. Arrange to trim your news system's history list regularly. If you have >X Cnews, try the "exphist" script provided, run from cron. That >X script requires that a tiny file, /usr/lib/news/histdays, containing >X the number of days of history data to keep, or else you can hardcode >X in your favorite number. If you have Bnews, I don't have a nifty >X suggestion, other than using expire(8) with a ridiculously long >X article expire time, but a sane history expire time. >6. Arrange to run reap from cron. >SHAR_EOF >chmod 0664 Install || >echo 'restore of Install failed' >Wc_c="`wc -c < 'Install'`" >test 934 -eq "$Wc_c" || > echo 'Install: original size 934, current size' "$Wc_c" >fi ># ============= MANIFEST ============== >if test -f 'MANIFEST' -a X"$1" != X"-c"; then > echo 'x - skipping MANIFEST (File already exists)' >else >echo 'x - extracting MANIFEST (Text)' >sed 's/^X//' << 'SHAR_EOF' > 'MANIFEST' && >Included with this distribution are: >X >README - general description and credits >Install - hints on how to install reap and get it going >Xexphist - history trimmer utility for Cnews >list.sample - sample function script file for reap >reap.8 - man page >lib.c - source >main.c - source >reap.c - source >reap.h - source >Makefile - makefile >SHAR_EOF >chmod 0664 MANIFEST || >echo 'restore of MANIFEST failed' >Wc_c="`wc -c < 'MANIFEST'`" >test 339 -eq "$Wc_c" || > echo 'MANIFEST: original size 339, current size' "$Wc_c" >fi ># ============= Makefile ============== >if test -f 'Makefile' -a X"$1" != X"-c"; then > echo 'x - skipping Makefile (File already exists)' >else >echo 'x - extracting Makefile (Text)' >sed 's/^X//' << 'SHAR_EOF' > 'Makefile' && >#---------------------------- ># parameters for installation >#---------------------------- >X ># directories where manual and executable are to be installed >BINDIR = /usr/lib/newsbin/expire >MANDIR = /usr/man/man8 >X ># owner, group and file mode for manual and executable >EXEOWNER = bin >EXEGROUP = bin >EXEMODE = 775 >MANOWNER = bin >MANGROUP = bin >MANMODE = 664 >X >X ># your favorite C compiler >#CC = cc >CC = shcc >X ># directory access functions library, if not in standard C library. >LIB = -ldirent >X >X >#------------- ># boring stuff >#------------- >X >OBJ = main.o lib.o reap.o >X >reap: $(OBJ) >X $(CC) -s -o reap $(SHLIB) $(OBJ) $(LIB) >X >$(OBJ): reap.h >X >install: reap reap.8 >X cp reap $(BINDIR) >X chown $(EXEOWNER) $(BINDIR)/reap >X chgrp $(EXEGROUP) $(BINDIR)/reap >X chmod $(EXEMODE) $(BINDIR)/reap >X cp reap.8 $(MANDIR) >X chmod $(MANMODE) $(MANDIR)/reap.8 >X chown $(MANOWNER) $(MANDIR)/reap.8 >X chgrp $(MANGROUP) $(MANDIR)/reap.8 >SHAR_EOF >chmod 0664 Makefile || >echo 'restore of Makefile failed' >Wc_c="`wc -c < 'Makefile'`" >test 900 -eq "$Wc_c" || > echo 'Makefile: original size 900, current size' "$Wc_c" >fi ># ============= README ============== >if test -f 'README' -a X"$1" != X"-c"; then > echo 'x - skipping README (File already exists)' >else >echo 'x - extracting README (Text)' >sed 's/^X//' << 'SHAR_EOF' > 'README' && >X >README for reap version 1.1 by David B. Thomas (dt@yenta.alb.nm.us). >X >A Problem: >X News volume fluctuates wildly...sites with small disks must either >Xexpire aggresively, often deleting more than necessary, or worry that an >unexpected huge dose of news might overfill the disk. >X >X >A Solution: >X Implement a tool that expires according to a user-defined scheme >until sufficient freespace is reclaimed, then stops, leaving as much >juicy news online as is feasible. Reap does this. >X >X >Expire Does Two Jobs: >X Both Bnews and Cnews expires really do two jobs: >X 1. trim history files >X 2. delete outdated articles >Thanks to some inspired jootsing (acronym for "jumping out of the system") >by Mike Murphy (mrm@sceard.com) and others, it is more than possible to >separate those two functions. This is, of course, in keeping with the >unix philosophy of one tool doing one job well! >X >X >What Reap Does: >X Reap only takes care of the second job: deleting old articles. >It works by checking freespace, and processing one line at a time from a >list of expire functions, until the desired freespace is attained. >Each expire function consists of an age limit in days (decimals okay) >and a list of newsgroups to process or not process, sys file style. Ex: >X >X .5 alt.sex.pictures,talk,!talk.bizarre,junk >X 1 rec,!rec.games,!rec.humor >X >This example would check freespace, and if more space is needed, expire >to .5 days everything in alt.sex.pictures, talk (except for talk.bizarre) >and junk. Then it would stop and check freespace again. If still more >space is needed, it would expire to 1 day everything in rec except >rec.games.* and rec.humor.*. It's that simple. >X >X >Okay...So How Do I Arrange To Trim The History Files, Since Reap Can't: >X Included in this distribution is a shell script (mostly written >by Mike Murphy) to handle Cnews history files. It shouldn't be too >difficult to do something similar for Bnews, or you can give in and use >the original expire utility with options that tell it to expire the >history files only...but I think this will be slow. It just comes down >to removing lines from ordinary text files, based on their contents. >Murphy and I used awk. >X >X >But Is It Fast: >X Yes, largely because it doesn't have to do much. Even "find | rm" >is slower because find is repeatedly exec-ing rm. "rm -rf" has me beat, >though, I'll bet! :-) >X >X Since the functions of deleting articles and trimming history >are separate, I now run reap every six hours, but trim the history list >just once a day. That effectively keeps my disk space up to snuff, but >only thrashes at the history file in the middle of the night. >X >X >Credits: >X I owe a lot to Mike Murphy for inspiring me with his "trasher" >system. I also owe a lot to all of your netters who will flood me with >more suggestions and improvements in the coming weeks (hint, hint!). >X >X little david >X dt@yenta.alb.nm.us >X >ps - reap is freeware. No strings(1) attached. >SHAR_EOF >chmod 0664 README || >echo 'restore of README failed' >Wc_c="`wc -c < 'README'`" >test 2918 -eq "$Wc_c" || > echo 'README: original size 2918, current size' "$Wc_c" >fi ># ============= exphist ============== >if test -f 'exphist' -a X"$1" != X"-c"; then > echo 'x - skipping exphist (File already exists)' >else >echo 'x - extracting exphist (Text)' >sed 's/^X//' << 'SHAR_EOF' > 'exphist' && >#! /bin/sh ># exphist - expire history file >X ># =()<. ${NEWSCONFIG-@<NEWSCONFIG>@}>()= >. ${NEWSCONFIG-/usr/lib/news/bin/config} >X >PATH=$NEWSCTL/bin:$NEWSBIN/expire:$NEWSBIN:$NEWSPATH ; export PATH >umask $NEWSUMASK >X >days=`cat /usr/lib/news/histdays` >X >lock="$NEWSCTL/LOCKexpire" >ltemp="$NEWSCTL/L.$$" >Xecho $$ >$ltemp >trap "rm -f $ltemp ; exit 0" 0 1 2 15 >if newslock $ltemp $lock >then >X trap "rm -f $ltemp $lock ; exit 0" 0 1 2 15 >Xelse >X echo "$0: expire apparently already running" | mail "$NEWSMASTER" >X exit 1 >fi >X >cd $NEWSCTL >rm -f history.n history.n.pag history.n.dir >now=`getdate now` >age=`expr 86400 \* $days` >ago=`expr $now - $age` >awk "{split(\$2,dates,\"~\");if(dates[1]>$ago)print \$0}" history >history.n >mkdbm history.n >[ -s history.n ] && >mv history history.o && # install new ASCII history file >mv history.n history && >rm -f history.pag && # and related dbm files >rm -f history.dir && >mv history.n.pag history.pag && >mv history.n.dir history.dir >Xexit 0 >SHAR_EOF >chmod 0775 exphist || >echo 'restore of exphist failed' >Wc_c="`wc -c < 'exphist'`" >test 969 -eq "$Wc_c" || > echo 'exphist: original size 969, current size' "$Wc_c" >fi ># ============= lib.c ============== >if test -f 'lib.c' -a X"$1" != X"-c"; then > echo 'x - skipping lib.c (File already exists)' >else >echo 'x - extracting lib.c (Text)' >sed 's/^X//' << 'SHAR_EOF' > 'lib.c' && >X >/* >X * lib.c >X * subroutines needed by reap utility >X */ >X >#include "reap.h" >X >/* >X * freeblox(devno) >X * >X * THIS WILL REQUIRE MODIFICATION TO WORK UNDER NON-SYSTEM-V UNIX. >X * IT SHOULD BE EASY. PLEASE EMAIL WORKING MODS TO THE AUTHOR: >X * dt@yenta.alb.nm.us >X * >X * assuming devno is the device number of a file system, >X * this function returns the free space on that file system, >X * in blocks (whatever that means for that filesystem) as an int. >X * >X */ >X >freeblox(devno) >int devno; >{ >X static struct ustat ust; >X >X if (ustat(devno, &ust)) >X ouch ("%s: ustat() failed\n"); >X >X return (ust.f_tfree); >X >} /* freeblox() */ >X >X >X >X >/* >X * parse(cmd) >X * execute a line from the script file >X * >X * valid command lines: >X * a blank line is ignored >X * # comment (ignored) >X * days filespecs >X * >X * filespecs is a comma separated list of any combination of the following: >X * 1. a directory (newsgroup) to include, without recursion. >X * ex: alt.sources. >X * 2. a directory to include, recursively. >X * ex: alt.sources >X * 3. a directory to exclude, without recursion. >X * ex: !alt.sex. >X * 4. a directory to exclude, recursively. >X * ex: !alt.sex >X */ >X >parse(cmd) >char *cmd; >{ >X register int l; >X int recurseflag; >X double days; >X char *t, >X *p, >X *q, >X *seps = ",\n "; >X >X >/* skip initial whitespace */ >X p = cmd; >X while (isspace(*p)) >X ++p; >X >/* ignore blank lines or comments */ >X if (*p == '\0' || *p == '#') >X return(1); >X >/* read the expire time from the line as a floating point number, >X * then figure out what the timestamp on a file that old would be */ >X age = now - (time_t) (atof(p) * SECINDAY); >X >/* skip ahead to the filespec list */ >X while (!isspace(*p)) >X ++p; >X while (isspace(*p)) >X ++p; >X >/* initialize exclusion list to null list */ >X preclude(); >X >/* for each filespec in list */ >X for (t = strtok(p, seps); t; t = strtok(NULL, seps)) { >X >X /* forbid starting with a slash */ >X if (*t == '/') >X *t = '.'; >X >X /* change dots to slashes except in column 1 */ >X while ( (q = strchr(t+1,'.')) != NULL) >X *q = '/'; >X >X /* for final dot or slash, remove it, and shut off recursion */ >X recurseflag = 1; >X if (t[l = (strlen(t) - 1)] == '/') { >X recurseflag = 0; >X t[l] = '\0'; >X } >X >X if (*t == '!') >X exclude (t+1, recurseflag); >X else >X include (t, recurseflag); >X } >X >X reap(); >X >X return (0); >X >} /* parse() */ >X >X >X >/* >X * newnode,exclude, include, preclude >X * >X * functions to maintain global linked lists: >X * incl include this path in list of dirs to reap >X * excl leave out this directory >X (each entry can be recursive or not) >X * >X * the functions are: >X * include (text,rflag) add to incl list >X * exclude (text,rflag) add to excl list >X * preclude() clear both lists >X */ >struct filspec * >newnode(text) >char *text; >{ >X struct filspec *f; >X >X if ( (f = (struct filspec *) malloc (sizeof(struct filspec))) == NULL || >X (f->name = malloc (strlen(text)+1)) == NULL) >X ouch ("%s: out of memory\n"); >X >X strcpy (f->name, text); >X return (f); >} >Xexclude(text, rflag) >char *text; >int rflag; >{ >X struct filspec *f; >X >X f = newnode(text); >X f->recurse = rflag; >X f->next = excl; >X excl = f; >} >include(text, rflag) >char *text; >int rflag; >{ >X struct filspec *f; >X >X f = newnode(text); >X f->recurse = rflag; >X f->next = incl; >X incl = f; >} >preclude() >{ >X struct filspec *p; >X >X while (incl != NULL) { >X p = incl; >X incl = incl->next; >X free (p->name); >X free (p); >X } >X while (excl != NULL) { >X p = excl; >X excl = excl->next; >X free (p->name); >X free (p); >X } >} >X >X >/* >X * ouch(fmt, arg) >X * outputs an error message and exits with error status >X */ >X >ouch (fmt,arg) >char *fmt, *arg; >{ >X fprintf (stderr, fmt, progname, arg); >X exit (-1); >} >SHAR_EOF >chmod 0664 lib.c || >echo 'restore of lib.c failed' >Wc_c="`wc -c < 'lib.c'`" >test 3567 -eq "$Wc_c" || > echo 'lib.c: original size 3567, current size' "$Wc_c" >fi ># ============= list.sample ============== >if test -f 'list.sample' -a X"$1" != X"-c"; then > echo 'x - skipping list.sample (File already exists)' >else >echo 'x - extracting list.sample (Text)' >sed 's/^X//' << 'SHAR_EOF' > 'list.sample' && ># yenta's /usr/lib/news/reaplist function script file. >.1 junk,alt.flame,control,comp.binaries,alt.sex.pictures.,alt.fractals.pictures >.1 alt.desert-storm,alt.desert-shield,soc.motss,soc.culture >1 soc.singles,alt.activism,alt.romance.chat >.5 talk >1 sci,!sci.skeptic >2 sci.skeptic >.5 comp.text,comp.sys.atari,comp.windows,comp.os.vms >.5 comp.unix.sysv386 >.5 misc.jobs,misc.handicap >2 soc,!soc.singles,!soc.motss,!soc.culture >.5 rec,!rec.arts.anime,!rec.arts.animation,!rec.arts.movies,!rec.games.hack,!rec.radio.amateur,!rec.autos,!rec.arts.tv,!rec.audio,!rec.video,!rec.music,!rec.skydiving >1 rec.music,!rec.music.synth >3 rec.music.synth,rec.skydiving >2 alt.tv >2 rec.arts.anime,rec.arts.animation,rec.arts.movies,rec.radio.amateur,rec.autos,rec.arts.tv,rec.audio,rec.video >3 rec.games.hack >1 misc.headlines,misc.consumers,misc.kids,misc.legal >1 alt.sex,!alt.sex.bondage,!alt.sex.pictures. >1.5 comp,!comp.dcom.telecom,!comp.binaries,!comp.text,!comp.sys.atari,!comp.windows,!comp.os.vms,!comp.sys.att,!comp.sys.3b1,!comp.sources.3b1 >3 comp.dcom.telecom >1 comp.binaries >1 alt.sources >1 news,!news.announce.newusers >3 biz >3 misc >3 alt,!alt.sources,!alt.tv,!alt.desert-shield,!alt.activism,!alt.fractals.pictures,!alt.desert-storm,!alt.romance.chat >3 comp.sys.att >5 comp.sys.3b1 >5 comp.sources.3b1 >7 general ># up to here is "normal expire"... now let's get desparate >.1 alt.sex.pictures >.1 rec.arts.anime,rec.arts.animation,rec.arts.movies,rec.ham-radio,rec.autos,rec.arts.tv,rec.audio,rec.video >.1 biz >.1 talk >.1 news >.1 soc >.1 sci >.1 comp >.1 alt >.1 misc ># if we got here, we're in big trouble -- wipe it all! >0 . >SHAR_EOF >chmod 0664 list.sample || >echo 'restore of list.sample failed' >Wc_c="`wc -c < 'list.sample'`" >test 1611 -eq "$Wc_c" || > echo 'list.sample: original size 1611, current size' "$Wc_c" >fi ># ============= main.c ============== >if test -f 'main.c' -a X"$1" != X"-c"; then > echo 'x - skipping main.c (File already exists)' >else >echo 'x - extracting main.c (Text)' >sed 's/^X//' << 'SHAR_EOF' > 'main.c' && >X >/* >X * main.c >X * main routine and globals for reap utility, version 1.1 >X */ >X >X >#include "reap.h" >X >X >/* linked lists for included and excluded file specs */ >struct filspec >X *incl = NULL, >X *excl = NULL >; >char >X *scriptfile = SCRIPT, /* path of function script */ >X *progname, /* argv[0] for errors */ >X *newsdir = NEWSDIR /* news root directory */ >; >int >X verbose = 0, /* verbosity (-v) flag */ >X dryrun = 0 /* do-not-unlink (-n) flag */ >; >time_t >X age, /* unlink files older than this */ >X now /* current time */ >; >X >X >/************************ >X * MAIN FUNCTION * >X ************************/ >X >main(argc, argv) >int argc; >char *argv[]; >{ >X int device, /* device # containing newsdir */ >X c, >X lineno = 0, >X summary = 0, >X thenfree, >X errflag = 0; >X long wantblox; /* desired free space */ >X char *p; >X static char line[MAXLINE]; >X FILE *script; /* file ptr for function script */ >X struct stat st; >X >X >/* save argv[0] for error spewings */ >X progname = argv[0]; >X >/* parse args */ >X while ( (c=getopt(argc,argv,"vsnf:")) != EOF) >X switch(c) { >X case 'n': >X ++dryrun; >X break; >X case 'v': >X ++verbose; >X break; >X case 'f': >X scriptfile = optarg; >X break; >X case 's': >X ++summary; >X break; >X default: >X ++errflag; >X break; >X } >X >X if (argc - optind != 1 || >X sscanf(argv[optind], "%ld", &wantblox) != 1 ) >X ++errflag; >X >X if (errflag) { >X fprintf (stderr, >X "usage: %s [-v] [-s] [-n] [-f funclist] freeblocks\n", >X progname); >X exit (-1); >X } >X >X >/* stat newsdir to find the number of the device it's on */ >X if (stat(newsdir, &st)) >X ouch ("%s: can't stat %s\n", newsdir); >X device = st.st_dev; >X >/* open function script file for reading */ >X if ( (script = fopen (scriptfile, "r")) == NULL) >X ouch ("%s: can't read %s\n", scriptfile); >X >/* Record the current time, for deciding what gets the axe. */ >X time (&now); >X >/* chdir to newsdir */ >X chdir (newsdir); >X >/* record initial freespace if summary is desired */ >X if (summary) >X thenfree = freeblox(device); >X >/* main loop ... process until goal reached or end of script */ >X >X while (freeblox(device) < wantblox && >X (p = fgets (line, MAXLINE, script)) != NULL ) >X parse (line), ++lineno; >X >X fclose (script); >X >X if (summary) { >X c = freeblox(device); >X printf ("Freespace was %d, now %d. Cleared %d.\n", >X thenfree, c, c - thenfree); >X printf ("Stopped after line %d in %s\n", lineno, >X scriptfile); >X } >X >/* return 0 if freespace goal was met at exit, 1 if not */ >X exit (freeblox(device) >= wantblox ? 0 : 1); >X >} /* main() */ >X >SHAR_EOF >chmod 0664 main.c || >echo 'restore of main.c failed' >Wc_c="`wc -c < 'main.c'`" >test 2497 -eq "$Wc_c" || > echo 'main.c: original size 2497, current size' "$Wc_c" >fi ># ============= reap.8 ============== >if test -f 'reap.8' -a X"$1" != X"-c"; then > echo 'x - skipping reap.8 (File already exists)' >else >echo 'x - extracting reap.8 (Text)' >sed 's/^X//' << 'SHAR_EOF' > 'reap.8' && >.TH REAP 8 LOCAL >.SH NAME >reap - remove news articles as space needed >.SH SYNOPSIS >.B reap >[-v] [-s] [-n] [-f scriptfile] freeblocks >.SH DESCRIPTION >.I Reap >checks disk freespace and deletes netnews articles >according to a flexible user-specified >scheme until >.I freeblocks >minimum freespace is available. >It does this by sequentially reading commands (expire functions) from a >text file (function script), and executing them one at a time. >.PP >Each expire function consists of an age limit and a list of newsgroups >to expire to that limit. >Before executing >Xeach new expire function, reap checks the freespace on the news spool >device. Reap will exit when either the desired freespace is attained, or >the end of the function script file is reached (the latter case is considered >an unsuccessful exit). >By carefully designing the function script, a news administrator can >automate a wide variety of expiring policies. >.PP >The >.B -n >causes a dry run (as in make(1)). When this >option is specified, reap merely reports which articles it would delete if >the option were left off. This is useful for debugging function scripts. >However, since no space is actually freed in this mode, >it will either do nothing >(there was enough space to begin with) or proceed through the entire function >script file, indicating that reap would delete all applicable articles. >.PP >With the >.B -v >option, reap prints a verbose commentary on its progress to the standard >output. >.PP >The >.B -s >option causes reap to print a brief summary of blocks freed to the standard >output just before exiting. It also indicates how much of the script file >was processed. This is independent of -v. >.PP >By default, reap looks in /usr/lib/news/reaplist for its list of functions. >The >.B -f >option is used to specify an alternate function script file. >.SH FUNCTION SCRIPT FILE FORMAT >A function script file consists of a series of expire functions, one per line. >Each expire function contains an age limit (in days, decimals okay), followed >by a comma-separated list of newsgroup specifications, similar to the >sys file format. >.PP >There are four types of valid newsgroup specifications that can go in the list: >.PP >(1) An ordinary newsgroup name (alt.sex) indicates that any articles in that >newsgroup and any of its descendants (such as alt.sex.pictures), which are >older than the age limit, should be expired. >.PP >(2) A newsgroup name with a trailing dot (alt.sex.) indicates that any articles >in that newsgroup, which are older than the age limit, should be expired. >However, recursion is not implied. Articles in descendants of that group >(such as alt.sex.pictures) are not affected. >.PP >(3) A newsgroup name with a preceding exclamation point (!alt.sex.pictures) >indicates that that newsgroup and all of its descendants should be excluded >from the list. >For example, "1 alt.sex,!alt.sex.pictures" means to expire everything subsumed >under alt.sex to 1 day, but do not touch anything subsumed under >alt.sex.pictures. >.PP >(4) A newsgroup name with both a preceding exclamation point and a trailing >dot (!alt.sex.pictures.) indicates that that newsgroup should be excluded >from the list, but its descendants should remain in the list. For example, >"1 alt.sex,!alt.sex.pictures." means to expire everything subsumed under >alt.sex to 1 day, including the contents of any subgroups of alt.sex.pictures >(like alt.sex.pictures.d), but do not touch the articles in the newsgroup >alt.sex.pictures. >.PP >Blank lines and lines beginning with a "#" are ignored. >A sample function script file: >.PP >.nf >X # my first function script file >X 0.5 talk,junk,alt.sex.pictures. >X 2 rec,!rec.games,!rec.humor >X 2 misc >.fi >.PP >In the example, >if space is needed, reap >Xexpires to .5 days the entire talk and junk >hierarchies, and alt.sex.pictures (not touching any subgroups of >alt.sex.pictures, such as alt.sex.pictures.d). >If still more space is needed, the >rec hierarchy (excepting all of rec.games.* and rec.humor.*) is expired >to 2 days. If freespace is still short, reap then expires all of misc >to 2 days. >.PP >Note that, while it would be possible to consolidate the 2-day >lines, leaving them separate makes it possible for reap to stop >in between them if sufficient space is cleared. >.PP >A practical function script file would directly or indirectly >include every group carried on the system, with some age limit. Otherwise, >the spool directory will inevitably overflow without operator intervention. >.PP >It is valid to specify a dot all by itself (.) in the newsgroup list field. >This is taken to mean the entire spool directory hierarchy, so "14 ." means >to expire all news to 14 days. Be warned that reap must search the entire >news hierarchy when this feature is used. Still, it is probably a good >idea to end all function script files with a "0 .", so that, in absolute >desperation, all news would be removed. >.SH FILES >.TP 25 >/usr/spool/news >News spool directory >.TP 25 >/usr/lib/news/reaplist >Default function list file >.SH DIAGNOSTICS >Returns zero if sufficient space was free at exit, 1 if the entire function >script was executed without freeing enough space, and -1 on an error >condition. >.SH CAVEATS >What constitutes a disk "block" is implementation-dependent. >.PP >History files are not updated. That must be performed as a separate operation. >.PP >For best performance, exclude recursively (without the dot) whenever >possible. Otherwise, reap will have to search the excluded directory >for any subdirectories which might not have been excluded, which is slow. >.SH BUGS >The use of the ustat() system call is not very portable, and your humble >author isn't aware of its non-system-V equivalents. >.PP >Lines in the function file are limited to 1024 characters. >.PP >The "cleared" figure in the summary merely indicates the difference in >freespace before and after the run, not necessarily the actual number of >blocks liberated by reap. >.SH AUTHOR >Reap is freeware by David B. Thomas (dt@yenta.alb.nm.us). You are free >to copy, distribute, staple, bend, fold or mutilate this package to your >heart's content. Please email any enhancements or ideas for enhancements. >SHAR_EOF >chmod 0644 reap.8 || >echo 'restore of reap.8 failed' >Wc_c="`wc -c < 'reap.8'`" >test 6129 -eq "$Wc_c" || > echo 'reap.8: original size 6129, current size' "$Wc_c" >fi ># ============= reap.c ============== >if test -f 'reap.c' -a X"$1" != X"-c"; then > echo 'x - skipping reap.c (File already exists)' >else >echo 'x - extracting reap.c (Text)' >sed 's/^X//' << 'SHAR_EOF' > 'reap.c' && >X >/* >X * reap.c >X * contains the reap() function. >X * >X * Once the global linked lists incl and excl have been stuffed, >X * reap() actually scans the filesystem for files that meet the specs >X * and unlinks them if they are older than the age global variable. >X */ >X >#include "reap.h" >X >X >reap() >{ >X struct filspec *f; >X >X for (f = incl; f != NULL; f = f->next) { >X if (verbose) >X printf ("scanning %s ...\n", f->name); >X dodir (f->name, f->recurse); >X } >X >} /* reap() */ >X >X >X >dodir(name, rflag) >char *name; >int rflag; >{ >X /* the following can be overwritten safely during recursion */ >X static struct filspec *e; >X static struct stat st; >X static struct dirent *dp; >X static char thisname[MAXFILENAME+1]; >X /* the following must be preserved through recursion */ >X char *fullpath; >X int eflag = 0; >X DIR *dirp; >X >X >/* open directory for reading */ >X if ( (dirp = opendir(name)) == NULL) >X ouch ("%s: can't read directory %s\n", name); >X >X >/* see if this directory is excluded. >X * If it's excluded recursively, quit here. >X * If it's excluded non-recursively, set a flag, so we won't consider >X * deleting any files in it, but we'll still explore subdirectories. >X */ >X for (e = excl; e != NULL; e = e->next) >X if (!strcmp (name, e->name)) >X break; >X if (e != NULL) { >X if (e->recurse) >X return; >X else >X ++eflag; >X } >X >X >/* loop for each directory entry */ >X while ( (dp = readdir(dirp)) != NULL) { >X >X /* name might be exactly MAXFILENAME characters long, and thus >X * might not be null-terminated. Some insurance: >X */ >X strncpy (thisname, dp->d_name, MAXFILENAME); >X thisname[MAXFILENAME] = '\0'; >X >X /* skip dot and dotdot */ >X if (!strcmp(thisname, ".") || !strcmp(thisname, "..")) >X continue; >X >X /* build the full pathname of current object */ >X if ( (fullpath = >X malloc(strlen(name)+strlen(thisname)+2)) == NULL) >X ouch ("%s: out of memory\n"); >X >X sprintf (fullpath, "%s/%s", name, thisname); >X >X /* try to stat the object */ >X if (stat(fullpath,&st)) { >X fprintf (stderr, "%s: can't stat %s\n", >X progname, fullpath); >X free (fullpath); >X continue; >X } >X >X /* maybe recurse if it's a directory */ >X if ( st.st_mode & S_IFDIR ) { >X if (rflag) >X dodir (fullpath, 1); >X free (fullpath); >X continue; >X } >X >X /* it's a file ... is this a non-recursively excluded directory? >X * if so, there's nothing to do to this file >X */ >X if (eflag) >X continue; >X >X /* leave it alone if this directory is excluded, or >X * if it's new enough. >X */ >X if (eflag || st.st_mtime > age) { >X free (fullpath); >X continue; >X } >X >X /* reap this file! */ >X if (dryrun) { >X printf ("Would unlink %s\n", fullpath); >X free (fullpath); >X continue; >X } >X if (verbose) >X printf ("Unlinking %s\n", fullpath); >X >X if (unlink (fullpath) == -1) >X fprintf (stderr, >X "%s: cannot unlink %s\n", progname, fullpath); >X >X free (fullpath); >X >X } /* while */ >X >X closedir (dirp); >X >} /* dodir() */ >SHAR_EOF >chmod 0664 reap.c || >echo 'restore of reap.c failed' >Wc_c="`wc -c < 'reap.c'`" >test 2845 -eq "$Wc_c" || > echo 'reap.c: original size 2845, current size' "$Wc_c" >fi ># ============= reap.h ============== >if test -f 'reap.h' -a X"$1" != X"-c"; then > echo 'x - skipping reap.h (File already exists)' >else >echo 'x - extracting reap.h (Text)' >sed 's/^X//' << 'SHAR_EOF' > 'reap.h' && >X >/* >X * reap.h >X * header file for reap utility >X */ >X >/* set MAXFILENAME to the maximum number of characters in a filename for >X * your system. Typically 14 or infinity, where infinity equals >X * 256 characters. :-) >X */ >#define MAXFILENAME 14 >X >/* set NEWSDIR to the directory containing news on your system. >X * Very commonly /usr/spool/news >X */ >#define NEWSDIR "/usr/spool/news" >X >/* set SCRIPT to the path of the default function script file >X * Usually this is /usr/lib/news/reaplist >X */ >#define SCRIPT "/usr/lib/news/reaplist" >X >X >X >X >#include <stdio.h> >#include <string.h> >#include <ctype.h> >#include <time.h> >#include <malloc.h> >#include <sys/types.h> >#include <sys/stat.h> >#include <ustat.h> >#include <dirent.h> >X >#define MAXLINE 1024 /* max len of line in script */ >#define SECINDAY (3600 * 24) /* seconds in a day */ >X >/* structure for linked lists of included and excluded file specs */ >struct filspec { >X char *name; >X int recurse; >X struct filspec *next; >}; >X >Xextern struct filspec >X *incl, >X *excl >; >Xextern char >X *progname, >X *scriptfile, >X *newsdir >; >Xextern int >X verbose, >X dryrun, >X optind >; >Xextern char *optarg; >Xextern double atof(); >Xextern long freeblox(); >Xextern time_t >X age, >X now >; >SHAR_EOF >chmod 0664 reap.h || >echo 'restore of reap.h failed' >Wc_c="`wc -c < 'reap.h'`" >test 1195 -eq "$Wc_c" || > echo 'reap.h: original size 1195, current size' "$Wc_c" >fi >exit 0 >-- >Bottom of stack = 0x40000 >Stack pointer = 0x3fffe >Don't push it! --- reply to : admerlev@faui43.informatik.uni-erlangen.de
dt@yenta.alb.nm.us (David B. Thomas) (04/02/91)
admerlev@immd4.informatik.uni-erlangen.de (Arnd Merlevede) writes: >hallo andreas, vielleicht interessiert Dich >die follgende Msg falls Dus nicht selbst gelesen hast. > [ ... ] >Bis denn Arnd > [ and then includes my entire reap source! ] Arnd, wie das multenposten mit nicht makenchangen? little david -- Robert Thomas, speaking of good software for Unix vs. MsDos: "Quality is either the result of a whole lot of dedication, or it's a thin layer of cream on top of a whole lot of milk."
gh@s5000.UUCP (Sys Admin) (04/17/91)
Hello netters my hard disk went south and the reap program with it could some kind soul send me a copy thanks Glen uunet!uokmax!s5000!gh