[news.sysadmin] Supersedes: filling my disk

gnu@hoptoad.uucp (John Gilmore) (12/24/87)

I am running an odd mix of pre-alpha C rnews, 2.10.3 inews, 2.11 vnews,
and such on hoptoad and am beginning to think that the best way to cope with
the new regime on map entries is to add ",!comp.mail.maps" to my sys file.

I don't understand why the entries are posted with 1 month expiration
dates, when sites that pathalias the stuff will copy it off somewhere
else and unpack it anyway, and sites that don't pathalias the stuff
will want it to go away (or can expire it with local control).

I could run the new alpha C expire program, which lets you set "min,
default, and max" expiration times on a per-newsgroup basis, in a
control file.  This could easily force comp.mail.maps stuff to expire
in a reasonable amount of time without bothering the other newsgroups.
However, the C expire doesn't yet keep history of expired articles, and
there seem to be a LOT of old articles flooding the net this month...
I could run old expire twice, once just on comp.mail.maps, but it already
takes it most of an hour of bashing hard on the disk every night.

I suppose I could stop doing news software development and just run a
stock 2.11 patchlevel XXX system, but I wanted to voice a protest to the
"Gotcha!  We finally found a way to force the sites running obsolete software
off the net!" syndrome running around -- from a site that pulls its weight
and has good reasons to run odd newsware.
-- 
{pyramid,ptsfa,amdahl,sun,ihnp4}!hoptoad!gnu			  gnu@toad.com
  I forsee a day when there are two kinds of C compilers: standard ones and 
  useful ones ... just like Pascal and Fortran.  Are we making progress yet?
	-- ASC:GUTHERY%slb-test.csnet

avolio@decuac.dec.com (Frederick M. Avolio) (12/24/87)

I am not sure what the problem is.  You needn't trash the maps.  Just 
expire them early...

	expire -i -e2 -n comp.mail.maps

Fred

heiby@falkor.UUCP (Ron Heiby) (12/31/87)

Frederick M. Avolio (avolio@decuac.dec.com) writes:
> I am not sure what the problem is.  You needn't trash the maps.  Just 
> expire them early...
> 	expire -i -e2 -n comp.mail.maps

Perhaps an even simpler method would be something along the lines of:
	find /usr/spool/news/comp/mail/maps -mtime +2 -type f -print |
	xargs rm -f

Yes, I know that "xargs" is not available on every system.  There are
similar things that can be done without it, though.  (It's really just
an efficiency hack.)

Anyone who is not a leaf node should make sure that the map articles
don't get removed until the sites they feed have had a reasonable chance
to get them.  Maybe something like the following script would work.  I
haven't tested it, as I don't currently have a problem with keeping the
map articles around.  You should test it completely before enabling the
"rm" command.  Note:  If you are running ihave/sendme with another
site, then this will almost certainly not work.  You'll have to do something
much more clever, like grepping out Message-Ids and comparing them with
the contents of the .ihave files.  Even that won't do it, though, as
the "ihave" message may have been sent and the "sendme" not yet received.
-----
	SPOOLDIR=/usr/spool/news
	BATCHDIR=/usr/spool/batch

	# Get pathnames of the map articles.
	find $SPOOLDIR/comp/mail/maps -type f -print | sort > /tmp/arts.$$

	# cat all the outgoing batch files together sort out the map articles
	cat $BATCHDIR/* 2>/dev/null |
		grep 'comp/mail/maps' |
		sort >/tmp/outg.$$

	# See if any articles have already been batched and queued.
	uniq -d /tmp/arts.$$ /tmp/outg.$$ >/tmp/del.$$

	# If there are any, then delete them
	if test -s /tmp/del.$$
	then
	#	rm -f `cat /tmp/del.$$`	# un-comment when testing done
		cat /tmp/del.$$		# this line for testing or logging
	else				# else clause for testing or logging
		echo "No map articles to be deleted."
	fi

	# clean up
	# un-comment rm line when testing done
	#rm -f /tmp/arts.$$ /tmp/outg.$$ /tmp/del.$$
	exit 0
-----
-- 
Ron Heiby, heiby@mcdchg.UUCP	Moderator: comp.newprod & comp.unix
"Intel architectures build character."

sfq@bcd-dyn.UUCP (sfq) (12/31/87)

I've set up a shell script, called by cron once in a while, that unshars the
maps, moves them to /users/news/Maps, and zaps the messages themselves.  It's
working so far.

If you don't want to save the maps, it's even easier.

On our system, the news stuff is in /users/news; you'll probably have to
change all the references.  You should also have all your users un-subscribe
to comp.mail.maps.  But they probably have, anyway.  All ours have.

Stanley F. Quayle	UUCP: cbosgd!osu-cis!bcd-dyn!sfq
(614) 424-4052		USPS: 505 King Ave., Columbus, OH  43201
N8SQ @ W8CQK		Fido: Stanley Quayle, Node 1:226/610
My opinions are mine.  What more of a disclaimer could you need?

---------------------

#! /bin/csh

# Moves the UUCP maps to the appropriate directory, un-shars them, and
# pitches the originals.

# Written 30 December 1987 by Stanley F. Quayle

# Exit if news is executing (and probably unbatching)
test -r /usr/spool/uucp/LCK.XQT && exit 0

# Exit if no maps have been received recently
grep -s "comp.mail.maps" /users/news/lib/news/log || exit 0

# Create temporary directory
mkdir /users/news/Maps/tmp

# Change directory
cd /users/news/Maps/tmp

# Move the maps
mv /users/news/spool/news/comp/mail/maps/* .

# Unshar them
/users/news/lib/news/unshar [0-9]* >/dev/null

# Zap the originals, and any shar messages
rm -f [0-9]* s.log s.tmp

# Compress the maps
/users/news/lib/news/compress *

# Move everyone to the Maps directory, replacing as needed
mv * ..

# Zap the temporary directory
cd /users/news/Maps
rmdir /users/news/Maps/tmp
-- 
Stanley F. Quayle	UUCP: cbosgd!osu-cis!bcd-dyn!sfq
(614) 424-4052		USPS: 505 King Ave., Columbus, OH  43201
N8SQ @ W8CQK		Fido: Stanley Quayle, Node 1:226/610
My opinions are mine.  What more of a disclaimer could you need?

esj@beach.cis.ufl.edu (Eric S. Johnson) (01/01/88)

I have found a simple way to deal with the supercedes problem. When first
announced I developed a little awk/sed/shell script combination which 
I would run with crontab. It found the supercedes articles and would 
rm the article it superceded. 

But...

I only used this for a week. I decided to try to bring the news here up to 
the latest revision level. (14) I was worried that this would break things,
but I figured that it was worth a shot. My reasoning was "if the patches 
work without problem, then ill install it and watch carefully". I was
patching from 2.11.3.

So the patches worked, and so far (a month) news 2.11.14 has been running
here fine. Not a single problem. Our news system is based on one host with
real news, and the rest running rrn/remote inews. NNTP et. al. work fine
still.

So.. Before you spend time trying to figure out various ways to hack around
the supercedes problem, try just bringing news up to patchlevel 14 and see
what happens. The guys that bring us news software seem to be bending over
backwards to make it easy to install/patch.. Least we can do is try it their 
way first, eh?

A satisfied customer. :-)


--
In Real Life:			UUCP: ...ihnp4!codas!ufcsv!beach.cis.ufl.edu!esj
Eric S. Johnson II              Internet: esj@beach.cis.ufl.edu
University of Florida           "Your species is always dying and suffering" -Q