[news.software.nntp] Filesystem full patches?

jwabik@shamash.cdc.com (Jeff Wabik) (03/20/89)

Forgive, as I know this has come up before here ...  I just don't read 
this group as often as I should ..

Are there patches to the NNTP software that make it do something other
than chuck incoming news into never-never land when when the filesystem
news is stored in fills?   The version I have allocs the inode, then just
leaves empty files lying around.

FTP sitenames and version number(s) (if they exist!) would be fabulous..

Thanks ..

	-Jeff

--
Jeff A. Wabik       E/Mail: jwabik@shamash.cdc.com   AT&T: +1 612 853 6811
  ____  ____                                         FAX:  +1 612 853 4789
 / ___||___ \
| |___  ___| |  Control Data Corporation - "Party on, dudes!"
 \____||____/

      " .. You went out, nearly froze, earned a quarter, 
		     then the women took it..  Today you're a man."

tadguy@cs.odu.edu (Tad Guy) (03/21/89)

In article <11896@shamash.cdc.com>, jwabik@shamash (Jeff Wabik) writes:
>Are there patches to the NNTP software that make it do something other
>than chuck incoming news into never-never land when when the filesystem
>news is stored in fills?   The version I have allocs the inode, then just
>leaves empty files lying around.

You don't specify what system you're running NNTP on, but if it's 4BSD
the enclosed diffs and source might help.  The code to check the
remaining space is already provided (and works under USG), but a
routine to find the free space of a filesystem under 4BSD isn't.  I
had such a routine in our library of local code, so I used that.

Below are the diffs to Makefile, serve.c, and misc.c as well as a new
file dfree.c which returns the number of free kilos of a filesystem.
(It's not really done like this here since dfree() is from -local, but
I've doctored the diffs to make it look like we compile dfree.c in the
nntp server directory to make it easier to install these hacks.  This
also means this may not work for you the first time.)

If you use this, keep in mind that nntpd must then be able to read the
disk device in /dev to determine the amount of free space remaining.
We keep our news on its own partition, so making the /dev entry for
that disk world readable isn't a problem here, but if you keep other
stuff on your news spool disk (like mail or uucp traffic), you don't
want to do this.  An alternative is to make nntpd setgid to a group
that can read that disk, etc.

This has been tested only under 4.3BSD on a VAX (``All the world's a VAX'').
Your line numbers will almost certainly be different. 

Enjoy...

	...tad

---- Cut Here and unpack ----
#!/bin/sh
# shar:	Shell Archiver  (v1.22)
#
#	Run the following text with /bin/sh to create:
#	  dfree.c
#	  Makefile.diff
#	  misc.c.diff
#	  serve.c.diff
#
echo "x - extracting dfree.c (Text)"
sed 's/^X//' << 'SHAR_EOF' > dfree.c &&
X#include <sys/types.h>
X#include <sys/param.h>
X#include <sys/stat.h>
X#include <sys/fs.h>
X#include <sys/file.h>
X#include <fstab.h>
X
X/*
X * return the number of free kilobytes remaining on the filesystem where
X * the named file resides.  returns -1 on error.
X */
X
Xoff_t lseek();
X
Xint
Xdfree(name)
Xchar *name;
X{
X    struct stat namest, fsst;
X    struct fstab *fsp;
X    char lname[MAXPATHLEN];
X    int fd;
X    union {
X	struct fs u_fs;
X	char dummy[SBSIZE];
X    } sb;
X#define sblock sb.u_fs
X
X    strcpy(lname,name);
X    do {
X	if (stat(lname,&namest))		/* if stat fails, die */
X	  return -1;			
X	if ((namest.st_mode & S_IFMT) == S_IFLNK) { /* if symlink */
X	    if ((fd = readlink(lname,lname,sizeof(lname))) < 0) 
X	      return -1;
X	    lname[fd] = '\0';
X	}
X    } while ((namest.st_mode & S_IFMT) == S_IFLNK);
X
X    (void) setfsent();
X
X    while (fsp = getfsent()) {
X	if (stat(fsp->fs_spec,&fsst))
X	  continue;
X	if (fsst.st_rdev == namest.st_dev)
X	  break;
X    }
X
X    if (!fsp ||	(fd = open(fsp->fs_spec,O_RDONLY)) < 0) {
X	(void) endfsent();
X	return -1;
X    }
X    (void) endfsent();
X
X    (void) lseek(fd,SBLOCK*DEV_BSIZE,L_SET);
X    if (read(fd,(char *)&sblock,SBSIZE) != SBSIZE ||
X	(sblock.fs_magic != FS_MAGIC))
X      return -1;
X    (void) close(fd);
X
X    return ((((sblock.fs_dsize) * ( 100 - sblock.fs_minfree) / 100)
X	     - ((sblock.fs_dsize) 
X		- (sblock.fs_cstotal.cs_nbfree 
X		   * sblock.fs_frag + sblock.fs_cstotal.cs_nffree))) 
X	    * sblock.fs_fsize / 1024);
X}
X
X
SHAR_EOF
chmod 0444 dfree.c || echo "restore of dfree.c fails"
echo "x - extracting Makefile.diff (Text)"
sed 's/^X//' << 'SHAR_EOF' > Makefile.diff &&
X*** /tmp/,RCSt1020900	Mon Mar 20 14:51:53 1989
X--- Makefile	Mon Mar 20 14:51:42 1989
X***************
X*** 6,18 ****
X  	ahbs.o globals.o group.o help.o ihave.o list.o misc.o netaux.o \
X  	newgroups.o newnews.o nextlast.o ngmatch.o post.o parsit.o scandir.o \
X  	slave.o spawn.o strcasecmp.o subnet.o time.o xhdr.o fakesyslog.o \
X! 	../common/version.o
X  
X  SRVRSRC = main.c serve.c access.c access_inet.c access_dnet.c active.c \
X  	ahbs.c globals.c group.c help.c ihave.c list.c misc.c netaux.c \
X  	newgroups.c newnews.c nextlast.c ngmatch.c post.c parsit.c scandir.c \
X  	slave.c spawn.c strcasecmp.c subnet.c time.c xhdr.c fakesyslog.c \
X! 	../common/version.c
X  
X  SRVRINC = common.h ../common/conf.h ../common/nntp.h
X  
X--- 6,18 ----
X  	ahbs.o globals.o group.o help.o ihave.o list.o misc.o netaux.o \
X  	newgroups.o newnews.o nextlast.o ngmatch.o post.o parsit.o scandir.o \
X  	slave.o spawn.o strcasecmp.o subnet.o time.o xhdr.o fakesyslog.o \
X! 	../common/version.o dfree.o
X  
X  SRVRSRC = main.c serve.c access.c access_inet.c access_dnet.c active.c \
X  	ahbs.c globals.c group.c help.c ihave.c list.c misc.c netaux.c \
X  	newgroups.c newnews.c nextlast.c ngmatch.c post.c parsit.c scandir.c \
X  	slave.c spawn.c strcasecmp.c subnet.c time.c xhdr.c fakesyslog.c \
X! 	../common/version.c dfree.c
X  
X  SRVRINC = common.h ../common/conf.h ../common/nntp.h
X  
X***************
X*** 19,27 ****
X  SRCS	= ${SRVRSRC}
X  
X  # -ldbm here if you've #define'ed DBM in ../common/conf.h
X! LIBS	=
X  
X! CFLAGS	= -O
X  
X  # Where nntpd is going to live
X  
X--- 19,27 ----
X  SRCS	= ${SRVRSRC}
X  
X  # -ldbm here if you've #define'ed DBM in ../common/conf.h
X! LIBS	= 
X  
X! CFLAGS	= -O -DODU -DMINFREE=2048
X  
X  # Where nntpd is going to live
X  
SHAR_EOF
chmod 0644 Makefile.diff || echo "restore of Makefile.diff fails"
echo "x - extracting misc.c.diff (Text)"
sed 's/^X//' << 'SHAR_EOF' > misc.c.diff &&
X*** /tmp/,RCSt1020049	Mon Mar 20 14:17:11 1989
X--- misc.c	Mon Mar 20 14:16:54 1989
X***************
X*** 645,647 ****
X--- 645,659 ----
X  	return(0);
X  }
X  #endif USG
X+ 
X+ #ifdef ODU
X+ /*
X+  * just like space above (uses dfree from -local)
X+  */
X+ int
X+ space()
X+ {
X+ 	if (dfree(SPOOLDIR) < MINFREE) return(-3);
X+ 	return(0);
X+ }
X+ #endif
SHAR_EOF
chmod 0644 misc.c.diff || echo "restore of misc.c.diff fails"
echo "x - extracting serve.c.diff (Text)"
sed 's/^X//' << 'SHAR_EOF' > serve.c.diff &&
X*** /tmp/,RCSt1019981	Mon Mar 20 14:15:48 1989
X--- serve.c	Mon Mar 20 14:15:35 1989
X***************
X*** 74,84 ****
X  	double		user, sys;
X  #ifdef USG
X  	time_t		start, finish;
X- 	extern int	space();
X- 	int		out_of_space;
X  #else not USG
X  	struct timeval	start, finish;
X  #endif not USG
X  	extern char	*ctime();
X  #ifdef POSTER
X  	struct passwd	*pp;
X--- 74,86 ----
X  	double		user, sys;
X  #ifdef USG
X  	time_t		start, finish;
X  #else not USG
X  	struct timeval	start, finish;
X  #endif not USG
X+ #if USG|ODU
X+ 	extern int	space();
X+ 	int		out_of_space;
X+ #endif
X  	extern char	*ctime();
X  #ifdef POSTER
X  	struct passwd	*pp;
X***************
X*** 144,150 ****
X  		exit(1);
X  	}
X  
X! #ifdef USG
X  	if (space() < 0 && !canpost ) {
X  		printf("%d %s NNTP server out of space. Try later.\r\n",
X  			ERR_FAULT, host);
X--- 146,152 ----
X  		exit(1);
X  	}
X  
X! #if USG|ODU
X  	if (space() < 0 && !canpost ) {
X  		printf("%d %s NNTP server out of space. Try later.\r\n",
X  			ERR_FAULT, host);
X***************
X*** 226,232 ****
X  				*cp = '\0';
X  		}
X  
X! #ifdef USG
X  		if (space() < 0 && !canpost ) {
X  			out_of_space++;
X  			break;
X--- 228,234 ----
X  				*cp = '\0';
X  		}
X  
X! #if USG|ODU
X  		if (space() < 0 && !canpost ) {
X  			out_of_space++;
X  			break;
X***************
X*** 258,264 ****
X  #endif TIMEOUT
X  	}
X  
X! #ifdef USG
X  	if(out_of_space) {
X  		printf("%d %s NNTP server out of space. Quitting.\r\n",
X  			ERR_GOODBYE, host);
X--- 260,266 ----
X  #endif TIMEOUT
X  	}
X  
X! #if USG|ODU
X  	if(out_of_space) {
X  		printf("%d %s NNTP server out of space. Quitting.\r\n",
X  			ERR_GOODBYE, host);
SHAR_EOF
chmod 0644 serve.c.diff || echo "restore of serve.c.diff fails"
exit 0




-- 
Tad Guy              Comp Sci, Old Dominion University, Norfolk, VA  23529-0162
tadguy@cs.odu.edu    tadguy@xanth.cs.odu.edu [128.82.8.1]     tadguy@xanth.uucp

"Think twice before speaking, but don't say `think think click click'."
                                                            -- John Owens

per@erix.ericsson.se (Per Hedeland) (04/04/89)

In article <8159@xanth.cs.odu.edu> tadguy@cs.odu.edu (Tad Guy) writes:
>You don't specify what system you're running NNTP on, but if it's 4BSD
>the enclosed diffs and source might help.

I installed Tad's code (with some cosmetic changes: merged dfree() into
space(), made the whole thing #ifndef USG rather than #ifdef ODU, and made both
versions of space() return 0 if MINFREE wasn't #defined) - it works fine (still
on a VAX/4.3BSD), and has had numerous opportunities to prove this during the
last week... May I suggest that this be included in a future "official" patch?

However, I don't quite follow the logic of when this check is made by nntpd (as
per patch #3, not Tad's code), i.e. iff !canpost - wouldn't the opposite be
more appropriate:-)? Seriously, I understand that there is little point in
giving your NNTP feed(s) posting priviliges, but requiring that it doesn't have
them, on the other hand, seems like an unnecessary hassle (we generate the
access file more or less automatically from name server data).  More
importantly perhaps, if you have clients with reading but not posting
priviliges, these suddenly can't even read just because the spool directory is
getting full!

Wouldn't a more reasonable method be to have the out_of_space condition set
canpost to 0, and require posting priviliges for IHAVE? Or, if this breaks
existing setups (probably) or violates the definitions of the permissions,
just have the IHAVE command (and maybe POST) check out_of_space directly?

BTW, patch #3 says MINFREE should be defined in news/defs.h, and indeed, for
those bold enough to install patches #15-17 for 2.11 (which were sent out some
4 months *after* nntp patch #3), there is a reason for having it there, but I
still don't see how such a definition would find it's way into nntpd?

Regards
--Per Hedeland
per@erix.ericsson.se   or   ...uunet!erix.ericsson.se!per

tadguy@cs.odu.edu (Tad Guy) (04/05/89)

In article <1961@erix.ericsson.se>, per@erix (Per Hedeland) writes:
>I installed Tad's code (with some cosmetic changes: merged dfree() into
>space(), made the whole thing #ifndef USG rather than #ifdef ODU, and made
>both versions of space() return 0 if MINFREE wasn't #defined)

Yeah, well, gee.  The posting was ugly because I forced it together.
We have dfree() in a local library here, but for the posting I had to
include it as a separate file (yuck).  The #ifdef ODU's (the second
time I've ever used ODU as a conditional compilation token, something
I hope not to do again) were because of the prejudice of the space()
for USG.  All the space checking code really should be gated off some
other constant, like MINFREE, instead of being tied in with all the
USG stuff.

However, I felt it was more important to get the code out the door
than to make it look great.  I'm glad it works for you.  It's been a
great help here...

>							       - it works
>fine (still on a VAX/4.3BSD), and has had numerous opportunities to prove
>this during the last week... May I suggest that this be included in a
>future "official" patch?

I'd be happy to see it in a future release as well (less local mods
I'll have to merge in).  However, the code does need to be generalized
more...  If someone wants to do it, great!  (I have no incentive to do
it here until our VAX goes away...)

>(we generate the access file more or less automatically from name server 
>data).

There was patch a while back to allow wildcards in the nntp_access
file.  If there's interest, I can repost that patch (we have over a
hundred local machines, but only four lines in our nntp_access file).

	...tad