[net.sources] Non compressing news batcher.

lee@kcl-cs.UUCP (Lee McLoughlin) (02/07/85)

I had the situation where I had free links  to  other  sites  but
virtually  no  free  CPU/disc  to  store  the uucp queues in. The
standard batchers seem to make copies of the news  in  the  queue
area  so  I  couldn't run them.  It seems to truck along ok. Feel
free to hack it about, just mail me  back  any  interesting  mods
please. Undoubtly the include files will be wrong for *something*
out there, so look out for them.

In  case  you're  wondering  what  you  gain  bear  in  mind  the
following.  Ten  incoming  news  articles  being  fed to a single
neighbour using the :U: flag to the sys  file  will  generate  10
calls  to  uux  and  involve 20 files in the uucp queue. Normally
with lotsend you'll have a single call to uux and only 2 files in
the  uucp queue. This saves 9 uux calls and 18 blocks.  Now given
that there are something like 200-300 articles  a  day  and  this
represents quite a saving.

However even with this I've come to the conclusion that you can't
fetch  news  and  feed  it  onto  three sites over 2 1200Bd lines
overnight only.  So I'll have to get my  Prime  version  of  UUCP
running.

/*
 * Simple news batcher.
 * Rather than running compress and all that stuff simply submit a
 * command like:
 *	uux -r -c remote!rnewsall !/news/net/oa/1 !/news/net/ai/276
 * where rnewsall is a simple script which calls rnews on all its arguments.
 * Its input can be generated by a news sysfile line like:
 *	ic-cs:uk,eunet,net,mod,fa:F:/usr/lib/news/lots.ic-cs
 * Lotsend itself should be called (by cron) a few minutes before news is
 * due to be sent.  For example the following crontab entry.
0 0,1,2,3,4,5,6,7,8,23 * * * /usr/lib/news/lotsend ic-cs | su news
 *
 * CAREFULL:  lotsend will zap the existing lots list.  It also doesn't
 * run any commands itself but rather prints them so that the above style
 * of crontab entry will work.
 *
 * Note that you do not get the transmission saving that compress style 
 * batchers give you but you do save more blockspace since copies of the data
 * are never stored in the spool area.
 *
 * L.M.McLoughlin.
 *
 * rnewsall:
for i
do	if test -r $i
	then	rnews < $i
		rm -f $i
	else	echo $0: failed to find $i >> $NEWS/rnewsall.log
	fi
done
 */
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#define BLK	512	/* Don't fiddle with this */
#define MAXBLKS 200
#define MAXARGS 20
int maxblks = MAXBLKS;	/* Max blks of data per uux call. */
long maxsize;
int maxargs = MAXARGS;	/* Max no. of args per uux call. */
char *site;		/* Name of site to use */

/* Where the lists the news generates are kept
 * This program MUST have write permission on that directory.
 * Preferably run this program under the news id.
 */
#define LOTSDIR "/usr/lib/news"

main( argc, argv )
int argc;
char **argv;
{
	int ma, mb;
	FILE *lf;
	char *lt, lotname[ BUFSIZ ];
	char filename[ BUFSIZ ];

	while( argv++, --argc ){
		if( argv[ 0 ][ 0 ] != '-' )
			break;
		switch( argv[ 0 ][ 1 ] ){
		case 'a':
			ma = 0;
			sscanf( &argv[ 0 ][ 2 ], "%d", &ma );
			if( 0 < ma && ma <= MAXARGS )
				maxargs = ma;
			break;
		case 'b':
			mb = 0;
			sscanf( &argv[ 0 ][ 2 ], "%d", &mb );
			if( 0 < mb && mb <= MAXBLKS )
				maxblks = mb;
			break;
		}
	}
	maxsize = (long)maxblks * BLK;

	if( argc == 0 ){
		fprintf( stderr, "Usage: lotsend [-aMAXARGS] [-bMAXBLKS] file\n" );
		exit( 1 );
	}

	if( chdir( LOTSDIR ) == -1 ){
		fprintf( stderr, "lotsend: cannot chdir to %s\n", LOTSDIR );
		exit( 2 );
	}

	/* Move the given lot file into a temporary to avoid problems
	 * with inews updates.
	 */
	site = argv[ 0 ];
	sprintf( lotname, "lots.%s", site );
	lt = mktemp( "LOTS.XXXXXX" );
	if( link( lotname, lt ) == -1 ){
		fprintf( stderr, "lotsend: cannot link %s to %s\n",
			lotname, lt );
		exit( 3 );
	}
	unlink( lotname );

	if( (lf = fopen( lt, "r" )) == NULL ){
		fprintf( stderr, "lotsend: cannot open to %s\n", lt );
		exit( 4 );
	}
	
	init();	/* Initialize the command generator */

	while( fgets( filename, BUFSIZ, lf ) != NULL ){
		filename[ strlen( filename ) - 1 ] = '\0';
		addname( filename );
	}
	outcmd();

	fclose( lf );
	unlink( lt );
	exit( 0 );
}

long size_sofar = 0;
int args_sofar = 0;
char cmd[ 5*1024 ];

/* Initialize the cmd.
 */
init()
{
	size_sofar = 0l;
	args_sofar = 0;
	sprintf( cmd, "uux -r -z -c %.7s!rnewsall ", site );
}

/* Add the name to the list to be given to uux.
 * If maxargs  name in the list call outcmd.
 * If the total size of all files > maxblks then call outcmd
 */
addname( name )
char *name;
{
	struct stat sb;
	long tot;

	if( stat( name, &sb ) == -1 )
		return;	/* Its gone! Ignore it */

	if( (tot = sb.st_size + size_sofar) > maxsize )
		outcmd();	/* Send what we have sofar */
	
	size_sofar = tot;

	strcat( cmd, "!" );
	strcat( cmd, name );
	strcat( cmd, " " );
	args_sofar++;

	if( args_sofar >= maxargs )
		outcmd();
}

outcmd()
{
	if( args_sofar == 0 )
		return;
	printf( "%s\n", cmd );
	init();
}
-- 
UKUUCP SUPPORT  Lee McLoughlin	<UK>!ukc!lmcl
		kcl-cs!lee
	"What you once thought was only a nightmare is now a reality!"