[net.sources] news batching with file size limits

chuqui@nsc.UUCP (Chuq Von Rospach) (04/07/84)

In putting up the batching software on my system, I decided to go ahead and
hack in something I've been meaning to do for awhile. This is a version of
batchnews.c that knows enough to start a new file if the old one gets too
big. 

there are a couple of reasons for this. If your uucp lines are a
touch flakey, passing a 200K file is a lot less likely to happen than
passing 4 50K files. If you do die, you don't lose nearly as much. Also, if
you are tight on spool space you don't need to worry about keeping room for
200K of batched news AND 200K of messages, because the batched news will
delete itself as it goes along. 

Here it is, enjoy! (install! use!) Note that the filesize maximum of 50,000
characters is rather arbitrary, so feel free to use whatever you want.

chuq

--- batchnews.c ----
# ifndef NOSCCS
static char *sccsid = "@(#)batchnews.c	1.2	4/7/84";
static char *cpyrid = "@(#)Copyright (C) 1984 by National Semiconductor Corp.";
# endif

/*****************************************************************************
* batchnews:      take many news articles and batch them into one big        *
*                 group                                                      *
*****************************************************************************/

# include <stdio.h>
# include <sys/types.h>
# include <sys/stat.h>

# define INTERVAL       3600    /* one spool file per hour */
# define FILEMAX	50000	/* maximum number of bytes in a file */
# define BATCHDIR	"/usr/spool/bnews" /* where the batching is done */

/* write out a line with the file length, then the file itself */

main(ac, av)
char **av;
{
	long now;
	char fnbuf[100], lckfile[100], buf[BUFSIZ];
	char tmpfile[100];
	register FILE *fi, *fo;
	int i;
	struct stat statbuf;

	sprintf(tmpfile,"%s/.newsXXXXXX",BATCHDIR);
	mktemp(tmpfile);
	if((fi = fopen(tmpfile, "w+")) == NULL) exit(1);
	while((i = fread(buf, 1, BUFSIZ, stdin)) != NULL)
		fwrite(buf, 1, i, fi);
	fflush(fi);
	rewind(fi);
	stat(tmpfile, &statbuf);

	time(&now);
	sprintf(fnbuf, "%s/%s.%ld",BATCHDIR,ac>1? av[1]: "news",now/INTERVAL);
	checksize(fnbuf);
	sprintf(lckfile, "%s/%s.LCK",BATCHDIR,ac>1? av[1]: "news");
	while(close(creat(lckfile, 0)) < 0) sleep(2);
	fo = fopen(fnbuf, "a");
	fprintf(fo, "%ld\n", statbuf.st_size);
	while((i = fread(buf, 1, BUFSIZ, fi)) != NULL)
		fwrite(buf, 1, i, fo);
	fclose(fi);
	fclose(fo);
	unlink(tmpfile);
	unlink(lckfile);
}
 
checksize(fnbuf)
char *fnbuf;
{
    register int i;
    char s[100];
    struct stat sbuf;

    if ((stat(fnbuf,&sbuf) == -1) || (sbuf.st_size < FILEMAX))
	return;

    strcpy(s,fnbuf);
    for (i = 0;; i++)
    {
	sprintf(fnbuf,"%s.%d",s,i);
	if ((stat(fnbuf,&sbuf) == -1) || (sbuf.st_size < FILEMAX))
	{
	    fprintf(stderr,"returning, fnbuf = %s\n",fnbuf);
	    return;
	}
    }
    /* NOTREACHED */
}
-- 
From under the bar at Callahan's:		Chuq Von Rospach
{amd70,fortune,hplabs,menlo70}!nsc!chuqui	(408) 733-2600 x242

A toast! To absent friends... {clink}

mark@cbosgd.UUCP (Mark Horton) (04/10/84)

Here is another way to do it.  This fix is much simpler, although it
doesn't do as much as some of the other fixes.   The problem is that,
if you use =batch, and your neighbor goes down for a few days, you
can get one huge batch, taking hours to transmit, and when they come
back up, it will burp before it can finish one complete transmission.

The fix is to simply move the copy of the file names to a unique filename
each time uux is invoked by crontab.  The spool directories are rearranged
slightly to avoid recompiling uucp, which only permits transfers from
/usr/spool/batch.  So now the sys file F option appends to
/usr/spool/batchnews/sysname instead of /usr/spool/batch/sysname.
The sendbatchednews script moves it to /usr/spool/batch.  A one line
change is also needed to batch, which unstead of truncating the file
(close(creat)), unlinks it.

	Mark
: ' sendbatchednews : transmit news which has been batched'
F=/usr/spool/batchnews
S=/usr/spool/batch
cd $F
for sys in *
do
	mv $F/$sys $S/$sys.$$
	uux -z -gd -r -c $sys!rnews '<' =/usr/lib/news/batch_$S/$sys.$$
done
/usr/lib/uucp/uucico -r1 &