det@hawkmoon.MN.ORG (Derek E. Terveer) (07/02/88)
I suppose that this subject has erupted before but, obviously, i don't have the information that was revealed in that thrashing... (sigh...) In B2.11.14 News a "problem" that i have noticed that affects the running of my systems directly is that News seems to demand that /usr/spool/news, if mounted on a seperate file system, not have any seperate file systems mounted as sub- directories. Permit me an example: / (/dev/dsk/0s1 ): 8314 blocks 1900 i-nodes /usr (/dev/dsk/0s3 ): 2658 blocks 3622 i-nodes /usr/spool/news (/dev/dsk/1s3 ): 11336 blocks 10840 i-nodes /usr/spool/news/comp (/dev/dsk/0s4 ): 14090 blocks 8781 i-nodes /mnt/tmp (/dev/dsk/f0q15dt): 1056 blocks 157 i-nodes I had, until i ran into the problem, /dev/dsk0s4 mounted on /usr/spool/news/comp to increase my total /usr/spool/news space. I put it on comp, 'cause that is by far my biggest distribution. However, after about 2 days of running in this configuration the /usr/spool/news fs was trashed. I had to run fsck on it a couple of times to restore it to sanity. Apparently (just guessing here w/o consulting the sources), News forges links beween files without checking that its on a seperate fs and somehow this is screwing up the overall fs tree (understandably). Now, the problem is that from the above it appears that the News spool area is limited to a single fs. This is all very nice, but a number of my systems have enough space to handle the news feeds i'm getting, but not all on one disk! I have, for example, on one system two 40M drives with one of the 40Meggers mounted as /usr/spool/news. Now i can sacrifice about 20M out of / (since / doesn't really need an extra 20M (:-)), but i can't mount it as a sub directory of /usr/spool/news. Any ideas on how to configure News to get around this feature, or am i just stuck? Or have i overlooked something real obvious?? derek -- Derek Terveer det@hawkmoon.MN.ORG uunet!rosevax!elric!hawkmoon!det
fair@ucbarpa.Berkeley.EDU (Erik E. Fair) (07/06/88)
News shouldn't trash filesystems like that, but we'll chalk that up to problems in the UNIX you run. Did you say System V? Anyway, you have a couple of options, none of them especially pretty. 1. buy bigger disks. 5.25 inch winchesters in the 100's of megabytes have been available for quite some time, at continually decreasing prices. 2. pester your vendor into supplying you with a disk device driver that will make several disks look like one big disk; then you can have a filesystem that spans multiple physical disks. It's not hard to do one of these, but few vendors I know are inclined to do special driver hacks without, ahem, financial incentive. 3. Put /usr/spool/news into the biggest partition that you can right now, and live with it. If your vendor is at all reasonable, s/he will have made it possible for you to make a whole disk into a single partition. In fact, really reasonable vendors will have written their disk drivers to allow the user to have a separate partition table for each disk on his system. If you can do this, be very careful, and make lots of verified backups, before and after you change your disk configurations around. Also, bear in mind that netnews traffic has a pretty constant flow per day, and except for occasional variations (and, of course, the usual slow growth) you can control the amount of space netnews uses on your system by twiddling the amount of time you keep netnews around (i.e. by telling expire what age articles should be when they're nuked). You might also consider cutting back on the number of newsgroups you receive; this will allow you keep the remainder longer. 4. Use expire to archive expired netnews into a different partition. I've never done this, but I assume that it is possible (i.e. netnews will copy articles into the archive partition, if a link attempt fails). Note that there is no secondary expire on the archive - it will grow without bound, so you'll have to do something to keep it from taking over the partition you assign to it. Also, none of the netnews user interfaces I know of will look at an archive. They all want to see the articles in /usr/spool/news (or whatever name you compiled in defs.h), and aren't interested in /usr/spool/oldnews (or whatever). Interface Writers! This Is An Opportunity! Welcome to Economics as applied to disk space - the Science of Scarcity. Time to decide what's really important to you, and what you want to pay for it. Erik E. Fair ucbvax!fair fair@ucbarpa.berkeley.edu
haugj@pigs.UUCP (Joe Bob Willie) (07/06/88)
In article <161@hawkmoon.MN.ORG> det@hawkmoon.MN.ORG (Derek E. Terveer) writes: >I suppose that this subject has erupted before but, obviously, i don't >have the information that was revealed in that thrashing... (sigh...) > >In B2.11.14 News a "problem" that i have noticed that affects the running of my >systems directly is that News seems to demand that /usr/spool/news, if mounted >on a seperate file system, not have any seperate file systems mounted as sub- >directories. Permit me an example: if worst comes to worst you could hack the software to believe it was running on eunice which doesn't have links to begin with. i haven't looked at the code to see how news handles eunice systems, but i do know that just like on a unix system, news under eunice does not make multiple copies of an article. check out the config.h and defs.h files for both the news source and the rn source. the appropriate defines should be there. also, thanks to larry wall for the wonderful eunice hacks. - john. ps - please don't include local distributions. the mn.general group gagged my ancient inews. -- John "Evil USENET User" F. Haugh II HECI Exploration Co, Inc., Dallas UUCP: ...!killer!rpp386!jfh jfh@rpp386.UUCP :DOMAIN **** Trivia question of the day: VYARZERZIMANIMORORSEZASSEZANSERAREORSES? **** "You are in a twisty little maze of UUCP connections, all alike" -- fortune
jeff@necntc.nec.com (Jeff Janock) (07/07/88)
In article <4276@pasteur.Berkeley.Edu> fair@ucbarpa.Berkeley.EDU (Erik E. Fair) writes: > >4. Use expire to archive expired netnews into a different partition. > I've never done this, but I assume that it is possible This is what we do on necntc; ie. /usr/spool/news (80MB) is on one FS, /usr/spool (50MB) lives on another and /usr/spool/oldnews yet another. (Ultrix lets us cut up the disk any way we choose; a real win here) [I should note that /usr also lives on a different partition] This all works out quite well. We archive /usr/spool/oldnews (~30MB) (to tape) when the partition reaches a certain capacity (ie. ~90+%) This allows us to make good use of the TU80 1600bpi tape which will write ~30MB on a single tape. This also lets us keep /usr/spool and /usr/spool/news in logically distinct areas for administrative reason such as ease of restore :-) Also, these partitions actually live across several disk drives to reduce the possibility of lossage even more. (and get some opposing disk access as the UDA50 is pretty slow...) Comments on the above welcome. Now a question: Has anyone tried mounting a FS as /usr/spool/news/.rnews to prevent any news loss when spool/news fills up? -- ...!{ames,harvard,mit-eddie,pyramid}!necntc!jeff - jeff@necntc.nec.com
shields@ists.yorku.ca (Paul Shields) (07/07/88)
In article <161@hawkmoon.MN.ORG>, det@hawkmoon.MN.ORG (Derek E. Terveer) writes: > [...] > In B2.11.14 News a "problem" that i have noticed that affects the running of my > systems directly is that News seems to demand that /usr/spool/news, if mounted > on a seperate file system, not have any seperate file systems mounted as sub- > directories. [example deleted] > Apparently [...] News forges links beween files > without checking that its on a seperate fs and somehow this is screwing up the > overall fs tree (understandably). The order is: open a temporary file in /usr/spool/news, link to the eventual locations, write the file, then close and unlink the temp file. Exactly how this trashes the filesystem I don't know. It would be easier if News did the links AFTER it wrote the files, so that you could choose to copy-and-delete if the destination was on a different filesystem. Then, in the case of crossposting, you could use symbolic links, if you have them. (BTW, if you look at the source, you will find ifdef's for VMS, which doesn't have links, so it's kludged using fake symbolic links. If you have REAL symbolic links you're already 70% there.) [...] > Any ideas on how to configure News to get around this feature, or am i just > stuck? > Or have i overlooked something real obvious?? Not that I can see. It looks like a little code writing is necessary. Paul Shields, shields@ists.yorku.CA, shields@yunccn.UUCP (...utzoo!yunexus!ists, ...mnetor!ontmoh!yunccn)!shields It's amazing just how long it takes to get nothing done.
dan@maccs.McMaster.CA (Dan Trottier) (07/07/88)
In article <4276@pasteur.Berkeley.Edu> fair@ucbarpa.Berkeley.EDU (Erik E. Fair) writes: >News shouldn't trash filesystems like that, but we'll chalk that up to >problems in the UNIX you run. Did you say System V? > >Anyway, you have a couple of options, none of them especially pretty. > > [options deleted] Or if your site is a BSD derivative you could always use symbolic links into other filesystems! It actually sounds easier to switch from System V to BSD than do some of the things Erik suggests :-) dan -- A.I. - is a three toed sloth! | ...!uunet!mnetor!maccs!dan -- Official scrabble players dictionary -- | dan@mcmaster.BITNET
jbuck@epimass.EPI.COM (Joe Buck) (07/07/88)
In article <240@pigs.UUCP> haugj@pigs.UUCP (Evil USENET User) writes: >if worst comes to worst you could hack the software to believe it was >running on eunice which doesn't have links to begin with. i haven't >looked at the code to see how news handles eunice systems, but i do >know that just like on a unix system, news under eunice does not make >multiple copies of an article. I did the 2.11 news code for Eunice, using Larry Wall's method of fake symbolic links: groups other than the first group have a file with the name of the article in it. Since I did it Larry's way, "rn" on Eunice works with news. The latest version of Eunice, version 4.3, looks much like 4.3bsd and supports symbolic links. These symbolic links are -- guess what -- little files with the name of the real file in them. Since I did the work under Eunice 4.2, it doesn't use these, but the code still works under 4.3. Unfortunately, there are problems with this scheme, in connection with "expire" when different groups are kept for different amounts of time. The way it works right now, the real article is in the first named group; once it's expired, the article is gone, where on a Unix a hard link in the other group directories will still be present. If you expire everything at the same time this isn't a problem. A workable hybrid scheme would be for inews to do the following: Install the article in the directory corresponding to the first group. Go directly here rather than /usr/spool/news/.arxxxxx. Make links to the positions for all the other group. If any link fails with errno == EXDEV (cross-device link), make a symbolic link instead. -- - Joe Buck {uunet,ucbvax,pyramid,<smart-site>}!epimass.epi.com!jbuck jbuck@epimass.epi.com Old Arpa mailers: jbuck%epimass.epi.com@uunet.uu.net If you leave your fate in the hands of the gods, don't be surprised if they have a few grins at your expense. - Tom Robbins
blair@obdient.UUCP (Doug Blair) (07/11/88)
You were concerned with the amount of storage space used by news. Hang in there for a few more weeks until the beta tests are done on news 3.0. You can store the news articles in a compressed format, which (at the expense of CPU time for decompression) will let you fit a lot more stuff into your present fs. Doug ___ _ _ _ _ / \ | | | | |_| _| |_ Doug Blair_______312-653-5527 | | || |_ ___ _| | _ ___ __ |_ _| Obedient Software Corporation | | || ==\ / ==\ /== | | | / ==\ | \ | | 1007 Naperville Road_________ \___/ |___/ \___/ \___| |_| \___/ |_|_| |_| Wheaton, Illinois 60187______
rick@pcrat.UUCP (Rick Richardson) (07/19/88)
In article <586@obdient.UUCP> blair@obdient.UUCP (Doug Blair) writes: >You were concerned with the amount of storage space used by news. >Hang in there for a few more weeks until the beta tests are done >on news 3.0. You can store the news articles in a compressed Maybe 3.0 already does this, but couldn't you decompress only the first, say, 24 lines, and leave the rest of the article stored compressed? Then you could browse at full speed, while still saving space. -- Rick Richardson, PC Research, Inc. (201) 542-3734 (voice, nights) OR (201) 389-8963 (voice, days) uunet!pcrat!rick (UUCP) rick%pcrat.uucp@uunet.uu.net (INTERNET)
eric@snark.UUCP (Eric S. Raymond) (07/21/88)
In article <535@pcrat.uucp>, rick@pcrat.UUCP (Rick Richardson) writes: > Maybe 3.0 already does this, but couldn't you decompress only > the first, say, 24 lines, and leave the rest of the article > stored compressed? Then you could browse at full speed, while still > saving space. I *like* it! You get the week's prize for interesting feature suggestion. If I can figure out a clean way to implement this without breaking the reader layering too badly, I'll do it. -- Eric S. Raymond (the mad mastermind of TMN-Netnews) UUCP: ..!{uunet,att,rutgers!vu-vlsi}!snark!eric Smail: eric@snark.UUCP Post: 22 South Warren Avenue, Malvern, PA 19355 Phone: (215)-296-5718