mark@zok.UUCP (Mark W. Snitily) (08/20/90)
News inherently spools all articles on one filesystem. This poses a problem
when one is running System V since a filesystem is limited to approximately
64K inodes. You might have hundreds of MB's but only 64K inodes. In other
words, when running System V you have a max of 64K spooled articles regardless
of the amount of your disk space.
What follows is a patch to inews.c [B News, patchlevel 19] which allows news
articles to be placed on two separate filesystems. The concept is general but
the following code has only been tested with two filesystems. Bottom line
result is that by patching inews.c you can doubled the number of news articles
that can be spooled.
What I've done is to create two filesystem and mount them on /usr/spool/news
and /usr/spool/news/comp. The filesystems are about 130MB's and were created
with the maximum number of inodes, (65488 to be exact). This works out
nicely for spooling about 3 weeks worth of news. (Actually, the
/usr/spool/news/comp articles are saved for 3 weeks with typically 50MB's
of free space on the filesystem. Most articles on /usr/spool/news are expired
before 21 days -- 3 weeks of everything except comp articles would easily
overflow 130MB's.)
Here's a df of my current news filesystems:
Filesystem kbytes used avail %used iused ifree %iused Mounted on
/dev/dsk/0s4 130560 119190 11370 91% 45446 20042 69% /usr/spool/news
/dev/dsk/0s5 130560 70622 59938 54% 20781 44707 31% /usr/spool/news/comp
As you can see, the % of inodes used is actually less than the % of disk used.
:-) :-) :-)
Here's the basic concept of the patch:
When hard linking an unbatched news article, if the error returned was
"cross-device link" (EXDEV), then this article needs to be placed on the
other filesystem. So...
1) If we have not created a file on the other filesystem, create it.
2) Otherwise we've already created a file, so hard link to it.
After all hard links have been created for the article, if we created
a file on the other filesystem, copy the article from the first filesystem
to the second filesystem.
The result is that at most two copies of the article are created, one for
each filesystem; each copy may have multiple hard links. On the down side,
news unbatches a little slower because of the extra copy.
Though I haven't switched over to C news, I've heard that C news unbatches
articles in a similar way (using hard links), so the above concept should
work in C news also.
In case it's important, this has been tested under System V R3.2.
The patch is only invoked if MULTI_FILESYSTEMS is defined. I just threw the
define into inews.c since it's the only file that's modified. In general
though, it should be placed into defs.h and localize.sh.
I've been running B news patchlevel 17 with this patch for about 6 months
now without any apparent side effects. Have just recently upgraded to
patchlevel 19.
I'd appreciate hearing from any news gurus out there if you see a fundamental
(or even minor) flaw with the patch.
Finally, here's the patch:
*** inews.c.org Sun Aug 5 09:58:43 1990
--- inews.c Sun Aug 19 13:02:49 1990
***************
*** 40,45 ****
--- 40,48 ----
#endif /* !BSD4_2 */
/* local defines for inews */
+ #define MULTI_FILESYSTEMS /* Spool news over two separate filesystems. */
+ /* Mark W. Snitily mark@zok.uucp 19-Aug-1990 */
+
#define OPTION 0 /* pick up an option string */
#define STRING 1 /* pick up a string of arguments */
***************
*** 812,817 ****
--- 815,825 ----
char firstbufname[BUFLEN];
+ #ifdef MULTI_FILESYSTEMS
+ static char *cdlinkfn; /* cross-device link filename */
+ static FILE *cdfp; /* cross-device file pointer */
+ #endif /* MULTI_FILESYSTEMS */
+
/*
* Link ARTICLE into dir for ngname and update active file.
*/
***************
*** 878,883 ****
--- 886,917 ----
#else /* !VMS */
if (link(ARTICLE, bfr) == 0)
break;
+ #ifdef MULTI_FILESYSTEMS
+ /* If error was a "cross-device link", then this article needs to be
+ placed on the other filesystem.
+
+ If we have not created the file on the other filesystem, create it.
+
+ Otherwise, the file already exists on the other filesystem so link to it.
+
+ For either case, if the create or link was successful, break out of
+ the loop, otherwise an error occurred so continue execution following
+ this code.
+ */
+ if (errno == EXDEV) { /* cross-device link error */
+ if (cdfp == NULL) { /* file on other filesystem doesn't exist */
+ if ((cdfp = xfopen(bfr, "w")) != NULL) { /* create file */
+ cdlinkfn = malloc(strlen(bfr)+1);
+ (void) strcpy(cdlinkfn, bfr);
+ break;
+ }
+ }
+ else { /* already have file on other filesystem, link to it */
+ if (link(cdlinkfn, bfr) == 0)
+ break;
+ }
+ }
+ #endif /* MULTI_FILESYSTEMS */
#endif /* !VMS */
e = errno; /* keep log from clobbering it */
log("Cannot install article as %s: %s", bfr, errmsg(errno));
***************
*** 979,984 ****
--- 1013,1023 ----
MKTEMP(ARTICLE);
tfp = xfopen(ARTICLE, "w");
linkcount = 0;
+ #ifdef MULTI_FILESYSTEMS
+ /* Init cross-device file pointer and cross-device link filename. */
+ cdfp = NULL;
+ cdlinkfn = (char *) 0;
+ #endif /* MULTI_FILESYSTEMS */
#ifndef NFSCLIENT
if (is_invalid) {
***************
*** 1186,1194 ****
--- 1225,1253 ----
{
for (c = 0; c < linkcount; c++)
free(artlinks[c]);
+ #ifdef MULTI_FILESYSTEMS
+ /* If cross-device link, copy temp file to file on other
+ filesystem. Opening ARTICLE from scratch because I
+ couldn't get "r+" to work with tfp. */
+ if (cdfp != NULL) {
+ FILE *ttfp = xfopen(ARTICLE, "r");
+ rewind(ttfp);
+ while (fgets(bfr, BUFLEN, ttfp) != NULL)
+ fputs(bfr, cdfp);
+ (void) fclose(ttfp);
+ }
+ #endif /* MULTI_FILESYSTEMS */
}
(void) fclose(tfp);
(void) fclose(infp);
+ #ifdef MULTI_FILESYSTEMS
+ if (cdfp != NULL) { /* perform cleanup */
+ (void) fclose(cdfp); /* close file on other filesystem */
+ cdfp = NULL;
+ free(cdlinkfn); /* free filename string */
+ cdlinkfn = (char *) 0;
+ }
+ #endif /* MULTI_FILESYSTEMS */
if (infpbuf) {
(void) free(infpbuf);
infpbuf = NULL;
-- Mark
Mark W. Snitily Consulting Services:
894 Brookgrove Lane Graphics, Operating Systems, Compilers
Cupertino, CA 95014 (408) 252-0456
mark@zok.uucp West Coast UUCP X11 archive site