blair@obdient.UUCP (Doug Blair) (03/10/88)
Chances are this has come up before we became a usenet
site. I'm posting this as a reminder of a potential pitfall
to other sites.
Relevant configuration info: news 2.11.8 is in
/usr/lib/news like most other sites. The articles are kept
in a separate filesystem, /u1/news. The $BATCHDIR is
/u1/news/batch. obdient is an intermediate node two hops
away from a backbone. We supply a full newsfeed to one other
site and limited groups to two other sites (so far).
It just happened that we hadn't run expire lately and
/u1 was down to less than 2000 blocks. We received a large
batch of news and uuxqt was called from cron while the news
was coming in, feeding files from the spool directory to
/usr/lib/news/rnews. More and more articles came in and were
stored on /u1/news. Cron also calls sendbatch once hourly
for each of the systems we feed. Eventually /u1 got down to
zero blocks left just about the time that sendbatch had
created a foo.work file for one of our downstream sites.
Since the .work file is a list of articles to be sent to a
system (and you will recall that it was now on an unwritable
filesystem) subsequent invocations of sendbatch generated
dozens of identical 51267 byte files to be sent to our down-
stream site, which in turn filled up our entire /usr
filesystem too! Each time sendbatch complained that it had
already created the .work file, but this didn't stop it from
executing the .work file that had been created by a previous
sendbatch run....
Since part of our standard profile involves writing to
a logfile on the /usr filesystem and only the root filesys-
tem had any blocks left on it, all attempts to login failed
halfway through the profile, apparently before a shell could
be established. The technical name for this condition is a
REAL MESS.
In fact the only way we could get out of the jam was to
powerdown, reboot from the installation disk (how many peo-
ple actually make a spare /unix on a floppy for emergencies?
:-) (Yes, we have one NOW, thank you! ;-)), mount /usr, move
some files to the root filesystem, sync, umount /usr, reboot
and thank goodness the system came up more or less normally.
Conclusionss and Comments: uucp is smart enough to
refuse more incoming files if storage space in the spool
directory gets low. So is Microport's sar package and a
(few) space intensive procedures. I haven't found anything
in the 2.11 installation guide that addresses this problem,
but shouldn't there be some way to tell rnews (or whatever
the equivalent C-news program will be) to shut down if there
isn't enough space in the $NEWSDIR and/or to prevent
sendbatch from running if it can't write in the $BATCHDIR?
I leave it to the net wizards to work this one out - my
temporary solution is to move the $BATCHDIR to
/usr/lib/news/batch instead of on the same filesystem as the
articles are stored. The odds of this filesystem filling up
are much smaller ! :-)
Doug Blair
___ _ _ _ _
/ \ | | | | |_| _| |_ Doug Blair_______312-653-5527
| | || |_ ___ _| | _ __ __ |_ _| Obedient Software Corporation
| | || __\ / __\ /__ | | | / __\ | \ | | 1007 Naperville Road_________
\___/ |___/ \___/ \___| |_| \___/ |_|_| |_| Wheaton, Illinois 60187______henry@utzoo.uucp (Henry Spencer) (03/13/88)
> ... shouldn't there be some way to tell rnews (or whatever > the equivalent C-news program will be) to shut down if there > isn't enough space in the $NEWSDIR and/or to prevent > sendbatch from running if it can't write in the $BATCHDIR? C news generally tries to be careful about responding intelligently to full filesystems. It's not perfect, but it generally does okay. In particular, our sendbatches command carefully observes free-space limits (and limits on the length of uucp queues to other sites) when preparing batches. -- Those who do not understand Unix are | Henry Spencer @ U of Toronto Zoology condemned to reinvent it, poorly. | {allegra,ihnp4,decvax,utai}!utzoo!henry