[alt.sources] PEXPIRE - a subscription based expire

taylor@hhplabs.HP.COM (Dave Taylor) (09/20/88)

If you're like we are, you have a lot of groups that are
taking up an astounding amount of disk space on your machine
even though no-one is reading them.  Well, we thought about
it for a bit and the end result was "pexpire", a program
that figures out who reads what (like `arbitron') then
sets the expiration time for each group accordingly.

But "pexpire" is much more sophisticated than this, with
the capability to have dozens of rules to set expiration
dates of different groups as you'd like (for example,
all source groups have 4 week expires, but all talk groups,
regardless of being read or not, have 2 day expires).

The only caveat is that I haven't yet tried to even 
compile this program on other than HP-UX machines.  If you
unpack this and have any problems/fix any SysV/HP-UX
dependencies, then please drop me a note and I shall roll
out a revision of the program.

For further information read the man page enclosed.

					-- Dave Taylor

taylor@hplabs.hp.com

-- Attachment: "pexpire.shar":

# This is a shell archive.  Remove anything before this line,
# then unpack it by saving it in a file and typing "sh file".
#
# Wrapped by taylor at hptsug2 on Tue Aug 30 14:43:32 1988
#
# This archive contains:
#	README			pexpire.1		
#	Makefile		pexpire.c		
#	pexpire.defaults	pexpire.h		
#

LANG=""; export LANG

echo x - README
cat >README <<'@EOF'

	     Additional Administrative Notes for Pexpire

			  August 29th, 1988

  It is strongly recommended that you read the man page for the
  netnews expire() command, studying the '-e' and '-E' options,
  then read the pexpire() man page distributed with the pexpire
  package.

  From there, you need to edit the Makefile to ensure that it
  is pointing at the right source directory for netnews, then
  edit the file "pexpire.h". 

  You should notice that pexpire.h expects that the netnews
  "defs.h" file is available -- it grabs the default expiration
  times from that file (see the extensively commented "pexpire.h"
  file for details).

  Finally, this is a first distribution, so there might very well
  be problems and non-portabilities.  If you come across anything,
  please drop me a note.

					Dave Taylor
					
  taylor@hplabs.hp.com
@EOF

chmod 666 README

echo x - pexpire.1
cat >pexpire.1 <<'@EOF'
.TH PEXPIRE 1L Experimental
.ad b
.SH NAME
pexpire - expire netnews groups based on local readership
.SH SYNOPSIS
.B pexpire
[-cgrov] 
[-e cmd] 
[-a N] 
[-h N] 
[-H N] 
.SH HP-UX COMPATIBILITY
.TP 10
Level:
HP-UX/CONTRIBUTED
.TP
Origin:
Hewlett-Packard
.SH DESCRIPTION
.I Pexpire
is intended to offer a finer granularity in the expiration of
netnews articles on a multi-user machine.  The philosophy behind
it is that there are typically a large set of newsgroups that
no-one on the machine reads, which makes them very likely targets
for shorter expiration times.  
.PP
This program allows you to do exactly that \(em it lets you set
default expiration times for all groups on your machine depending
on if they are subscribed to or not, then checks each users
".newsrc" to ascertain this information.  The finaly output of
the program is a set of \fIexpire\fR commands suitable for automatic execution
out of cron.
.PP
The flags understood are:
.TP 8
.B \-a n  	
Set default history expire value to 'n'. 
.sp
There are actually three
flags to do with expiration dates that \fIpexpire\fR understands:
the `-a' flag to set the default history expire value, and the `-h'
and `-H' flags to set a bracketing for when the `-E' flag needs to
be output.
.sp
That doesn't make any sense, I'm sure, so let's look at it this
way: the \fIexpire\fR program uses two different expiration
dates, one for when the article should be removed, and another
for when the entry should be removed from the ``history'' file.
With that in mind, the `-a' flag sets the default history expire
date for the `history' file, and the `-h' and `-H' flags set up
the window (eg. the program checks:
.ft CW
.nf

	min-hist-expire < current-expire < max-hist-expire

.fi
.ft R
for each \fIexpire\fR command output).  Please see the 
\fIexpire\fR man page for more information on the `-e'
and `-E' flags.
.sp
The default value for this is 28 days.
.TP 8
.B \-c    	
Make the groupname list comma separated, rather than space
separated.  This is a cosmetic change to the output, but you
might have a version of \fIexpire\fR that wants one or the
other explicitly.
.TP 8
.B \-e cmd	
Uses 'cmd' for output rather than the default expire program
.sp
The default set to ``/usr/local/lib/news/expire''.
.TP 8
.B \-g n  	
Forces `n' or less groups output per command.
This is because some versions of \fIexpire\fR have a limit as
to the number of groups they'll accept for expiration in a single
invocation.
.sp
The default is 50 groups.
.TP 8
.B \-h n  	
Set the default minimum history expire value to 'n'  (see `-a' above)
.sp
The current default is set to 14 days.
.TP 8
.B \-H n  	
Set the default maximum history expire value to 'n'  (see `-a' above)
.sp
The current default is set to 28 days.
.TP 8
.B \-r    	
Takes user id 0 account ".newsrc" files into account \(em a lot of
sites have multiple roots, with each having their usual home
directory (eg. the one for their non-administrative account).
In a situation like this there is no reason to pay the extra
overhead of checking their ".newsrc" file twice.
.sp
The default is to ignore user id 0 accounts.
.TP 8
.B \-o    	
Forces one-group-per-line output format.
.sp
The default is to use the value of the `-g' flag for groups per 
output line.
.TP 8
.B \-v    	
Turns on verbose output mode.
.PP
In addition, the program allows the administrator to define a file
that contains default expiration times for groups or sets of 
groups.  This file is called ``pexpire.defaults'' and the format
it expects is:
.nf

    pattern	+expire     -expire

.fi
Where the pattern can be any reguler expression accepted by the
regexp(3c) package, the +expire is the number of days to expire
the group if people are reading it, and -expire is the number
of days to expire the group if no-one is reading it currently.
.SH EXAMPLES
The configuration we have locally for ourselves has the following
"pexpire.defaults" file:
.nf
.ft CW

  #
  # This is the "pexpire" default expiration times file.  The 
  # format of this file is:
  #
  #   <regular expression>	<+expire> 	<-expire>
  #
  # where <+expire> is the expiration date for groups that are 
  # currently read by people on this machine, <-expire> are for 
  # those that are unread, and <regular expression> is any regular 
  # expression as per regexp(3c).
  #
  # It is recommended that you have ".*" as the first expression so 
  # that you can set the default expiration for all groups.  The 
  # processing order of this information is: 
  #      for each pattern read in this file:
  #        for each group in the active file:
  #          if the pattern matches, set the dates accordingly.
  #
  # this means that the patterns "^comp.*" and "source" in that 
  # order would result in "comp.unix.sources" having the source 
  # expire times.
  #
  # NOTE: never lead an expression with an asterisk -- assume all 
  #       patterns are unrooted, and use '^' to get them left rooted 
  #       if you want to
  .*                    14      1
  ^hp.*                 30      15
  ^comp.*               10      2
  ^talk.*               7       1
  ^soc.*                7       1
  ^news.*               14      2
  source                14      7
  test                  1       1
  comp.mail.elm         56      28

.ft R
.fi
Notice that the first regular expression, ``.*'', gives us the default
expiration time for all groups on our machine, then we modify it according
to local interests and needs.  Also notice that patterns default to being
able to `float', that is, ``source'' matches all groups that have the word
source in their names, whether left, right, or not rooted at all.
.sp
Additionally, we invoke the following shell script via cron:
.nf
.ft CW

  : Use /bin/sh
  
  # expire news using pexpire()
  # script written by Rob Sartin, HP  (sartin@hplabs.hp.com) 

  expire_script="/tmp/expire$$"
  trap 'rm -f $expire_script' 0 1 2 3 15
  
  # display our disk space utilization before the command ...

  echo "Expiring old news"
  echo "\\nBefore:"
  bdf
  
  # get the netnews home directory by fiddling /etc/passwd:

  eval `awk -F: "/^netnews:/"' { printf "LIBDIR=%s;\\n", $6 }' \\
     < /etc/passwd`
  
  # create the new expire script

  rm -f ${expire_script}
  ${LIBDIR}/pexpire > ${expire_script} 2>/dev/null
  chmod 0755 ${expire_script}

  # and execute it:

  ${expire_script}

  # finally, output disk space utilization after the command

  echo "\\nAfter:"
  bdf

  # and we're done.

  exit 0

.ft R
.fi
Note that you can have a minimal script by having the following
sequence instead, if you choose:
.ft CW
.nf

  : Use /bin/sh

  PEXPIRE=/usr/local/lib/news/pexpire
  TMPFILE=/tmp/expire.$$

  $PEXPIRE > $TMPFILE 

  sh $TMPFILE

  exit 0

.fi
.ft R
Though the former is recommended.
.PP
Also, you can test out the pexpire program to see what it thinks
the expiration time of a specific group is, for example, by a
sequence like:
.nf
.ft CW

  % pexpire -o -e expire | grep \fIgroup you're interested in\fR

.ft R
.fi
For example:
.nf
.ft CW

  % pexpire -o -e expire | grep soc.singles

  expire -e 7 -E 28 -n soc.singles

.ft R
.fi
This tells us that the group is to be expired in 7 days, but that
the actual article entries are to remain in the netnews history
file for 28 days.
.PP
We can also find out what user ``.newsrc'' files are 
checked with:
.nf
.ft CW

  % pexpire > /dev/null

  Checking against ".newsrc" for the following users:
          sartin taylor jin markc

.ft R
.fi
or, with the `-v' verbose option turned on:
.nf
.ft CW

  % pexpire -v | sed '/^$/,$d'

  Read 511 groups out of the active file.
  Checked against 10 patterns in the default-expire file.
  Checking against .newsrc for user "sartin"
  Checking against .newsrc for user "taylor"
  Checking against .newsrc for user "jin"
  Checking against .newsrc for user "markc"
  
.ft R
.fi
.SH AUTHOR
Dave Taylor, Hewlett-Packard Company  (taylor\s-1@\s+1hplabs.hp.com)
.SH FILES
.nf
.if n .ta 26
.if t .ta 20
/etc/passwd	for accounts to check ".newsrc" files
$USER/.newsrc	for each account on the machine, to check
$NETNEWS	usually ``/usr/local/lib/news''
$NETNEWS/active	Where the netnews active file lives
$NETNEWS/expire	The `real' \fIexpire\fR command
$NETNEWS/pexpire.default	home for the ``pexpire.default'' file
/bin/sh	valid login shell for user
/bin/csh	valid login shell for user
/bin/ksh	valid login shell for user
/bin/rsh	valid login shell for user
.SH SEE\ ALSO
expire(1)
@EOF

chmod 644 pexpire.1

echo x - Makefile
cat >Makefile <<'@EOF'
#
#	Makefile for the pexpire program
#      
#  by Dave Taylor, Hewlett-Packard Co.

SHELL    =	/bin/sh

PROGNAME =	pexpire

CFILES   =	pexpire.c
HEADERS  =	pexpire.h
OBJS     =	pexpire.o

# the next is probably the only thing you'll need to locally customize
# to reflect the top level location of the netnews source on your 
# machine...

NEWS_SRC =	/usr/local/src/news.2.11

INCLUDEDIR  =   -I${NEWS_SRC}

LIBS     =   	-lPW
CFLAGS   = 	-O ${INCLUDEDIR}
CC       =	/bin/cc
RM       =	/bin/rm -f

${PROGNAME}: ${OBJS} ${HEADERS} ${NEWS_SRC}/src/defs.h
	${CC} -o ${PROGNAME} -n ${OBJS} ${LIBS}

.c.o:   ${HEADERS}
	${CC} -c ${CFLAGS} $*.c 

clean:
	${RM} ${OBJS} LINT.OUT

lint: LINT.OUT

LINT.OUT: ${CFILES}
	lint ${DEFINE} ${INCLUDEDIR} ${CFILES} ${LIBS} > LINT.OUT
@EOF

chmod 666 Makefile

echo x - pexpire.c
cat >pexpire.c <<'@EOF'
/**				pexpire.c			**/

/** This program is designed to set the expire dates of newsgroups according
    to whether they are read locally or not.  The idea is that it is easy
    to go into users .newsrc files and compile an overall list of who reads
    what groups, then 1-day-expire those groups that no-one is reading.

    This hinges on the availability of other local machines to serve as
    archives for various groups, as well as the understanding of the 
    local users that subscribing to a new newsgroup will more than likely
    net you almost *no* new articles -- but will allow the news to flow
    in normally and then gradually build up to a more reasonable level.

    (C) Copyright 1988 Dave Taylor

    ***************************************************************************
    **  Permission is granted for unlimited modification, use, and dist-     **
    **  ribution, except that this software may not be sold for profit       **
    **  directly nor as part of any software package.  This software is made **
    **  available with no warranty of any kind, express or implied.          **
    ***************************************************************************

**/

#include <stdio.h>
#include <pwd.h>

#include "src/defs.h"			/* from the netnews source! */
#include "pexpire.h"

#define ROOT_UID	0			     /* who's root? */

#define MAX_GROUPS	1024			/* should be sufficient */

#define SLEN		128

#define COLON		':'

#ifndef TRUE
# define TRUE		1
# define FALSE		0
#endif

/** some easy to read and use macro functions **/

#define whitespace(c)	(c == ' ' || c == '\t')
#define matches(re,pat)	(regex(re, pat) != NULL)

#define plural(n)	(n == 1 ? "" : "s")

/** and data structures/global variables for the program **/

struct group_rec {
	char 	*name;
	int     is_read;
	int     read_expire;
	int	unread_expire;
       };

char *login_shells[] = { "/bin/sh", "/bin/csh", "/bin/ksh", "/bin/rsh", "" };

struct group_rec groups[MAX_GROUPS];

int group_count = 0,			/** total number of groups  **/
    include_root = FALSE,		/** include root .newsrc?   **/
    verbose = FALSE,			/** lots of output?         **/
    comma_separated = TRUE,		/** output list format	    **/
    groups_per_cmd,			/**  .. and more too	    **/

    min_history_expire,			/** for the expire() cmd    **/
    max_history_expire,			/**     "   "               **/
    default_history_expire,		/** 	"   "		    **/

    output_one_per_line = FALSE;	/** final output format     **/

char *prog_name,			/** program name for errors **/
     expire_cmd[SLEN];			/** expire cmd we'll use    **/

/** forward definitions and other stuff to keep LINT a happy clam   **/

char *regcmp(), *regex(), *strcpy(), *strcat(), *strchr(), *malloc();
void	exit(), perror(), qsort();

/** The algorithm that we'll be using here is:

     1. Read in the active file to get a list of all newsgroups available

     2. Go through the ``EXPIRE_DEFAULTS'' file to set initial expiration
	dates (typically by high level groups -- it's left rooted).  This
	file typically has two sets of numbers, the first being for groups
	that are being actively read, the second being for those that are
	not.  For example:

		soc.*	24	1

	would set any soc.* group to a 24 day expire if read, and a 1 day
	expire if not.

     3. Then, for each user of the system:

	   if they have a .newsrc
	       tag as 'read' any group that the user subscribes to

     4. Sort the newsgroups by expiration date, then output a shell
	script suitable for automatic execution...

**/

main(argc, argv)
char *argv[];
{
	extern char *optarg;
	int c;

	/** first off let's grab the program name for error messages **/

	prog_name = *argv;

	/** initialize some values that can be changed by the user **/

	groups_per_cmd = DEFAULT_GROUPS_PER_LINE;

        max_history_expire = DEFAULT_MAX_HISTORY_EXPIRE;
	min_history_expire = DEFAULT_MIN_HISTORY_EXPIRE;

	default_history_expire = DEFAULT_HISTORY_EXPIRE;

	(void) strcpy(expire_cmd, EXPIRE);

	/** now process the starting arguments **/

	while ((c = getopt(argc, argv, "a:ch:H:e:g:rov")) != EOF) {
	  switch (c) {

	    case 'a' : default_history_expire = atoi(optarg);
		       break;

	    case 'c' : comma_separated = TRUE;
		       break;

	    case 'e' : (void) strcpy(expire_cmd, optarg);	
		       break;

	    case 'g' : groups_per_cmd = atoi(optarg);
		       break;

	    case 'h' : min_history_expire = atoi(optarg);
		       break;

	    case 'H' : max_history_expire = atoi(optarg);
		       break;

	    case 'r' : include_root = TRUE;
		       break;

	    case 'o' : output_one_per_line = TRUE;
		       break;

	    case 'v' : verbose = TRUE;
		       break;

	    default  : (void) fprintf(stderr,
   "\nUsage: %s [-a n] [-c] [-e cmd] [-g] [-h n] [-H n] [-r] [-o] [-v]\n", 
			      prog_name);
		       (void) fprintf(stderr,"\nWhere:\n");
		       (void) fprintf(stderr,
   "   -a n  \tset default history expire value to 'n' (see the man page)\n");
		       (void) fprintf(stderr,
   "         \t(the current default is set to %d day%s)\n", 
		              DEFAULT_HISTORY_EXPIRE, 
			      plural(DEFAULT_HISTORY_EXPIRE));
		       (void) fprintf(stderr,
   "   -c    \tmake the groupname list comma separated, rather than space\n");
		       (void) fprintf(stderr,
   "   -e cmd\tuses 'cmd' for output rather than the default expire program\n");
		       (void) fprintf(stderr,
   "         \t(current default set to \"%s\")\n", EXPIRE);
		       (void) fprintf(stderr,
   "   -g n  \tforces `n' or less groups output per command (default = %d)\n",
			      DEFAULT_GROUPS_PER_LINE);
		       (void) fprintf(stderr,
   "   -h n  \tset the default minimum history expire value to 'n'\n");
		       (void) fprintf(stderr,
   "         \t(the current default is set to %d day%s)\n", 
		              DEFAULT_MIN_HISTORY_EXPIRE, 
			      plural(DEFAULT_MIN_HISTORY_EXPIRE));
		       (void) fprintf(stderr,
   "   -H n  \tset the default maximum history expire value to 'n'\n");
		       (void) fprintf(stderr,
   "         \t(the current default is set to %d day%s)\n", 
		              DEFAULT_MAX_HISTORY_EXPIRE, 
			      plural(DEFAULT_MAX_HISTORY_EXPIRE));
		       (void) fprintf(stderr,
   "   -r    \ttakes UID 0 account .newsrc files into account\n");
		       (void) fprintf(stderr,
   "   -o    \tforces one-group-per-line output format\n");
		       (void) fprintf(stderr,
   "   -v    \tturns on verbose output mode\n\n");
		       exit(0);
	  }
	}
	
	/** next let's read in the netnews active file **/

	read_active_file();

	/** read in the EXPIRE_DEFAULTS file and set default expires **/

	set_default_expiration_dates();

	/** check each user for a .newsrc and mark groups subscribed **/

	check_each_user();

	/** whip through a quick resort by expiration time **/

	sort_groups_by_expiration();

	/** and finally output the script that we can execute **/

	output_script();

	/** and we're done **/

	return(0);
}

read_active_file()
{
	/** this routine reads in the active file, sorts it, and 
	    returns.  It is assumed that it always works - if something
	    fails it will exit from here..
	**/

	int  compare();

	FILE *fd;
	char buffer[SLEN];
	register int  i;
	
	if ((fd = fopen(ACTIVE_FILE, "r")) == NULL) {
	  (void) fprintf(stderr,"%s: cannot open active file '%s':\n",
		  prog_name, ACTIVE_FILE);
	  perror("fopen");
	  exit(1);
	}

	while (fgets(buffer, SLEN, fd) != NULL) {

	  /** get just the first word ... **/ 

	  for (i=0; ! whitespace(buffer[i]); i++) ;
	  buffer[i] = '\0';

	  if ((groups[group_count].name = malloc((unsigned) i+1)) == NULL) {
 	    (void) fprintf(stderr,"%s: couldn't malloc memory for group '%s'\n",
		    prog_name, buffer);
	    perror("malloc");
	    exit(1);
	  }

	  /** now load up the new record and increment our counter **/

	  (void) strcpy(groups[group_count].name, buffer);
	  groups[group_count].is_read = FALSE;
	  groups[group_count].read_expire = -1;
	  groups[group_count].unread_expire = -1;

	  group_count++;

	  /** and on to the next one... **/
	}
	
	(void) fclose(fd);

	qsort(groups, (unsigned) group_count, 
	      sizeof (struct group_rec), compare);

	if (verbose)
	  (void) printf("Read %d group%s out of the active file.\n",
		  group_count, plural(group_count)); 
}

set_default_expiration_dates()
{
	/** this routine is responsible for reading in the default
	    expire file and setting the default expiration dates on
	    all of the groups in memory.   If there is no file or it
	    is impossible to get to, then the defaults indicated in
	    this program will be used for all groups.

	    The format of the file is quite simple:

	    	<regular expression> < tab > <+expire> <tab> <-expire>

	    where +expire is the expiration time if people are reading
	    the group, and -expire is if they're not.  The regular
	    expression format is that of regex(3c), so you can have
	    structures such as "^comp.*" and so on.
	**/

	FILE *fd;
	char  buffer[SLEN], *regular_expression;
	int   read_expire, unread_expire;
	register int i, count = 0;

	if ((fd = fopen(EXPIRE_DEFAULTS, "r")) == NULL) {
	  (void) fprintf(stderr,"%s: Couldn't read file '%s'\n", 
		  prog_name, EXPIRE_DEFAULTS);
	  (void) fprintf(stderr,
		  "(Using default expirations: read = %d, unread = %d)\n",
		  DEFAULT_READ_EXPIRE, DEFAULT_UNREAD_EXPIRE);
	  perror("fopen");
	  (void) fprintf(stderr,"---\n");

	  /** now spin through setting all expire dates accordingly **/

	  for (i=0; i < group_count; i++) {
	    groups[i].read_expire = DEFAULT_READ_EXPIRE;
	    groups[i].unread_expire = DEFAULT_UNREAD_EXPIRE;
	  }

	  return;
	}

	/** if we've gotten here we've got the file open and ready
	    to work with... **/

	while (fgets(buffer, SLEN, fd) != NULL) {

 	  if (buffer[0] == '#' || strlen(buffer) < 3) continue;

	  (void) sscanf(buffer, "%*s %d %d", &read_expire, &unread_expire);

	  count++;

	  for (i=0;! whitespace(buffer[i]); i++) ; 
	  buffer[i] = '\0';

	  /** now apply this pattern to all groups we've got, setting
	      the expire date as makes sense... **/

	  regular_expression = regcmp(buffer, (char *) 0);

	  for (i=0 ; i < group_count; i++)
	    if (matches(regular_expression, groups[i].name)) {
	      groups[i].unread_expire = unread_expire;
	      groups[i].read_expire = read_expire;
	    }
	}

	(void) fclose(fd);

	if (verbose)
	 (void) printf("Checked against %d pattern%s in default-expire file.\n",
		 count, plural(count)); 
}

check_each_user()
{
	/** this routine goes through the /etc/passwd file to
	    find all the users on the machine.  For each entry
	    found, it will ascertain if they have a login shell
	    then look for a .newsrc file.  If they have one, it
	    will extract all the groups that they currently read,
	    marking each in memory as being read ..
	**/

	FILE   *fd;
	struct passwd	*getpwent(), *pass;
	char   newsrc[SLEN], buffer[SLEN], user_list[SLEN];
	register int i;

	/** initialize **/

	user_list[0] = '\0';

	/** and step through the password file .. **/

	while ((pass = getpwent()) != NULL) {
	  if (has_login_shell(pass->pw_shell)) { 

	    if (pass->pw_uid == ROOT_UID && ! include_root)
	      continue;

	    (void) sprintf(newsrc, "%s/%s", pass->pw_dir, NEWSRC);
	    
	    if ((fd = fopen(newsrc, "r")) == NULL) continue;

	    if (verbose)
	      (void) printf("Checking against %s for user \"%s\"\n", 
		     NEWSRC, pass->pw_name);
	    else {
	      if (user_list[0] != '\0') (void) strcat(user_list, " ");
	      (void) strcat(user_list, pass->pw_name);
	    }

	    while (fgets(buffer, SLEN, fd) != NULL)
	      if (strchr(buffer, COLON) != (char *) NULL) {
	        for (i=0;buffer[i] != COLON; i++);
		buffer[i] = '\0';
	        mark_as_read(buffer);
	      } 

	    (void) fclose(fd);
	  }
	}

	if (! verbose && strlen(user_list) > 0) 
	  (void) fprintf(stderr, 
		  "Checked against \"%s\" for the following users:\n\t%s\n",
		  NEWSRC, user_list);
}

sort_groups_by_expiration()
{
	/** We now resort the list according to the expiration date of
	    the group... 
	**/

	int compare_expirations();

	qsort(groups, (unsigned) group_count, sizeof (struct group_rec), 
	      compare_expirations);
}

output_script()
{
	/** Now that we've gotten the groups sorted by their
	    expiration date, we can output a script that is suitable 
	    for input to the real netnews expire() program...
	**/
	
	register int i; 
	int current_expire_time = 0, expire, 
	    groups_on_line = 0, on_line = 0, in_expiration = 0;

	/** set the current expiration time, then:
	      for each group that has the same date
	      output the group name
	    when we hit a new date output the new format line
	**/

	for (i=0; i < group_count; i++) {

	  /** set the expiration time based on if the group is 
	      currently being read or not... 
	  **/

	  expire = groups[i].is_read ? groups[i].read_expire : 
		     groups[i].unread_expire;

	  if (output_one_per_line) {
	    if (expire > max_history_expire)
 	      (void) printf("%s -e %d -E %d -n %s\n", 
			  expire_cmd, expire, expire, groups[i].name);
 	    else if (expire < min_history_expire)
 	      (void) printf("%s -e %d -E %d -n %s\n", 
			  expire_cmd, expire, default_history_expire, 
			  groups[i].name);
	    else
 	      (void) printf("%s -e %d -n %s\n", 
			  expire_cmd, expire, groups[i].name);
	  }
	  else {
	    if ( expire != current_expire_time || 
		 in_expiration > groups_per_cmd) {
	      if (expire > max_history_expire)
	        (void) printf("\n%s -e %d -E %d -n ", 
			      expire_cmd, expire, expire);
 	      else if (expire < min_history_expire)
	        (void) printf("\n%s -e %d -E %d -n ", 
			      expire_cmd, expire, default_history_expire);
	      else
	        (void) printf("\n%s -e %d -n ", expire_cmd, expire);
	      groups_on_line = 0;
	      current_expire_time = expire;
	      in_expiration = 0;
	    }

	    in_expiration++;

	    on_line += strlen(groups[i].name) + 1;
	    
	    if (on_line > 66) {
	      (void) printf("%c \\\n\t", groups_on_line > 0? ',':' ');
	      on_line = 8 + strlen(groups[i].name);
	      groups_on_line = 0;
	    }

	    if (groups_on_line)
	      (void) printf("%c%s", comma_separated? ',' : ' ', groups[i].name);
	    else
	      (void) printf("%s", groups[i].name);

	    groups_on_line++;
	  }
	}
	(void) printf("\n");
}

int
compare(a,b)
struct group_rec a, b;
{
	/** strcmp() routine for our data structure, rather than the
	    simple expedient of just using strcmp directly.  See the
	    invocation of qsort() above
	**/

	return( strcmp(a.name, b.name) );
}

int 
compare_expirations(a, b)
struct group_rec a, b;
{
	/** strcmp() routine for data for second sort -- this one
	    is a sort by the expiration date of the groups.  To
	    do this we want to look at the is_read flag and from
	    that decide which of the two expiration dates we want to be 
	    looking at.
	**/

	return ( (b.is_read ? b.read_expire : b.unread_expire) - 
		 (a.is_read ? a.read_expire : a.unread_expire) );
}
 
int
has_login_shell(shell_name)
char *shell_name;
{
	/** returns TRUE iff the shell given is contained in the
	    list of possible login shells compiled with.
	**/

	register int i;

	for (i=0; login_shells[i][0] != '\0'; i++)
	  if (strcmp(login_shells[i], shell_name) == 0) return(TRUE);
	
	return(FALSE);
}
	  
mark_as_read(name)
char *name;
{
	/** Mark the group specified as being read -- it's extracted
	    from a users .newsrc file.
	**/

	int index;

	if ((index = find_group(name)) == -1)
	  (void) fprintf(stderr, 
		  "** Couldn't find group '%s' in internal tables?? **\n",
		  name);
	else
	  groups[index].is_read = TRUE;
}

int
find_group(name)
char *name;
{
	/** A binary search of the list to find the group - returns the
	    index into the 'groups' array of the group, or '-1' if not
	    in the list.
	 **/

	register int first = 0, last, middle, difference;

	last = group_count-1;

	while (first <= last) {

          middle = ((first+last) / 2);

          difference = strcmp(name, groups[middle].name);

          if (difference < 0)
            last = middle - 1;
          else if (difference == 0)
            return(middle);
          else  /* greater */
            first = middle + 1;
        }

        return(-1);
}
@EOF

chmod 644 pexpire.c

echo x - pexpire.defaults
cat >pexpire.defaults <<'@EOF'
#
# This is the "pexpire" default expiration times file.  The format of this
# file is:
#
#   <regular expression>	<+expire> 	<-expire>
#
# where <+expire> is the expiration date for groups that are currently
# read by people on this machine, <-expire> are for those that are unread,
# and <regular expression> is any regular expression as per regexp(3c).
#
# It is recommended that you have ".*" as the first expression so that you
# can set the default expiration for all groups.  The processing order of
# this information is: 
#      for each pattern read in this file:
#        for each group in the active file:
#          if the pattern matches, set the dates accordingly.
#
# this means that the patterns "^comp.*" and "source" in that order
# would result in "comp.unix.sources" having the source expire times.
#
# NOTE: never lead an expression with an asterisk -- assume all patterns
#       are unrooted, and use '^' to get them left rooted if you want to
.*		14	1
^HP.*		 3	1
^hp.*		30	15
^comp.*		10	2
^talk.*		7	1
^soc.*		7	1
^news.*		14	2
source		14	7
test		1	1
comp.mail.elm	56	28
@EOF

chmod 644 pexpire.defaults

echo x - pexpire.h
cat >pexpire.h <<'@EOF'
/**				   pexpire.h				   **/

/****************************************************************************
  This set of defines are those that might need to be localized or otherwise 
  customized for your local system and setting.                            
 ****************************************************************************/

/** first off, where's your netnews active file?  It'd be a suprise if
    it wasn't as indicated here, but you can change it if you want.
**/

#define ACTIVE_FILE	"/usr/local/lib/news/active"

/** Next, the pexpire() program has a default set of rules that can be
    applied to the set of groups to determine the expiration dates
    either by top-level newsgroup (eg. "comp.*") or down to the 
    specific group (eg. "comp.sys.hp").  Please see the expire man
    page for more discussion of this file.
**/

#define EXPIRE_DEFAULTS	"/usr/local/lib/news/pexpire.defaults"

/** NEWSRC is simply the name of the file kept in users home directories **/

#define NEWSRC		".newsrc"

/** Finally, this is the command used for expiration of news.  Most likely
    it'll be located in the same directory as the active file (see above).
    If you're running a strange expire command you might want to check to
    ensure it understands "-e", "-E" and "-n" flags... see the man page
    for further details.
**/

#define EXPIRE 		"/usr/local/lib/news/expire"

/** the default history expire is the standard number of days that an article
    is allowed to live in the history file -- regardless of how long it is
    on the machine in actual text form.  (This is different so that you don't
    get into looping trouble with very fast expires and multiple news feeds)

    The netnews source has HISTEXP and DFLTEXP in seconds, and for our
    own use, we'll change those back into days ...
**/

#define DEFAULT_MAX_HISTORY_EXPIRE	(HISTEXP / DAYS)
#define DEFAULT_MIN_HISTORY_EXPIRE	(DFLTEXP / DAYS)

/** if a group is being expired at less than the default minimum history
    exiration time, then we want to ensure that we have the default
    time rather than the one specific to the group.  That is, if we
    have a group with a 1 day expire, we still want to keep the articles
    in the history file for, say, 2 weeks...
**/

#define DEFAULT_HISTORY_EXPIRE		(HISTEXP / DAYS)

/** next, if the program cannot find your pexpire.default file, it will
    us the next two settings as the default for groups that are being
    subscribed to and those that are not.  Recommended that the unread
    expire not be incredibly short here in case the daemon messes up one
    night - you might come back and a major chunk of news is gone!  
**/

#define DEFAULT_READ_EXPIRE	  24
#define DEFAULT_UNREAD_EXPIRE	  3

/** finally, when the program outputs the commands for eventual shell
    execution, it tries to keep them in a format that the netnews 
    expire(UTIL) command can deal with.  One of the problems is that
    it is possible to have all 400 - 500 groups expire at the same
    time, and it's too much for a single invocation.  Instead, you can
    fine tune this to be the largest value possible, but smaller than
    the max limit of expire().
**/

#define DEFAULT_GROUPS_PER_LINE   50

/***********************  end of local customization  **********************/
@EOF

chmod 644 pexpire.h

exit 0