nash@ucselx.sdsu.edu (Ron Nash) (01/19/90)
I recently switched from running Bnews 2.11 to Cnews with patches thru Nov 1989. I think I am missing something as my news partition is filling up, even with expire set to 8 days. Under Bnews, I was expiring articles after 12 days and had about 30MBytes free on average. The news spool partition is 200Mbytes. Here is the expire -v output: Thu Jan 18 01:00:03 PST 1990 24420 kept, 1174 expired 9660 residual lines 0 links archived, 1310 junked, 8 missing What are the "residual lines"? This number grows each day. Here is a copy of explist: # hold onto history lines 20 days, nobody gets >90 days /expired/ x 20 - /bounds/ x 0-1-90 - # hold moderated groups longer sci,rec,talk,soc m 14 - # real noise gets thrown away fast junk,control x 2 - # sources and binaries comp.sources,comp.binaries m 20 - alt.sources x 20 - # default: 10 days and no archive all x 8 - Any ideals or suggestions? The system is running BSD4.3. Thanks in advance! -- Ron Nash San Diego State University Internet: nash@ucselx.sdsu.edu UUCP: ucsd!sdsu!ucselx!nash
henry@utzoo.uucp (Henry Spencer) (01/19/90)
In article <1990Jan18.182952.26048@ucselx.sdsu.edu> nash@ucselx.sdsu.edu (Ron Nash) writes: >I recently switched from running Bnews 2.11 to Cnews with patches thru >Nov 1989. I think I am missing something as my news partition is >filling up, even with expire set to 8 days... Bear in mind that traffic has been very heavy during the last couple of months, except for the holidays. We don't know of anything that would make expire fail in such a subtle manner without leaving a trace. My guess is that this expansion is "real". We have 40-50MB of news online here despite rather shorter expiry times than yours; the size goes up during the week and down on the weekend but there is no net increase, so I don't think we've got a leak in expire. You might try "find /usr/spool/news -mtime +8 -print" or something like that to see just how much old stuff you've got. That might shed some light. >What are the "residual lines"? This number grows each day. Here is a copy >of explist: > > # hold onto history lines 20 days, nobody gets >90 days > /expired/ x 20 - Note that "/expired/" line -- you've told expire to hang onto history lines for expired articles for 20 days. That's what "residual" lines are. (Admittedly the message could be clearer; I'll see what I can do.) Note that this does have a cost: it makes the history and history.pag files larger. They, and the residual-lines count, will hit steady state after about 20 days. -- 1972: Saturn V #15 flight-ready| Henry Spencer at U of Toronto Zoology 1990: birds nesting in engines | uunet!attcan!utzoo!henry henry@zoo.toronto.edu
brian@cimage.com (Brian Kelley) (02/21/91)
I've been running Cnews for about 8 months now with no real problems. Recently, I've been looking a little more closely at news disk usage. I started looking at the dates on articles. Taking a totally arbitrary group, rec.gardens, I find the following article: -rw-rw-r-- 1 news 1502 Jan 4 20:22 1190 looking at the header: Path: dgsi!umich!samsung!usc!wuarchive!mit-eddie!apollo!betsyp From: betsyp@apollo.HP.COM (Betsy Perry) Newsgroups: rec.gardens Subject: Re: Wayside Garden's English Roses Message-ID: <4f015001.20b6d@apollo.HP.COM> Date: 4 Jan 91 14:58:00 GMT References: <138882@pyramid.pyramid.com> Sender: root@apollo.HP.COM Reply-To: betsyp@apollo.HP.COM (Betsy Perry) Organization: Hewlett-Packard Company, Apollo Division; Chelmsford, MA Lines: 23 I ask myself, why is this article still here? doing an ls -l on the directory, I get the following: -rw-rw-r-- 1 news 1502 Jan 4 20:22 1190 -rw-rw-r-- 1 news 1708 Jan 10 20:26 1220 -rw-rw-r-- 1 news 590 Jan 10 20:27 1221 -rw-rw-r-- 1 news 759 Jan 10 20:57 1222 -rw-rw-r-- 1 news 1380 Jan 10 21:10 1223 -rw-rw-r-- 1 news 3340 Jan 11 03:24 1224 -rw-rw-r-- 1 news 1108 Jan 11 08:17 1225 -rw-rw-r-- 1 news 1156 Jan 11 08:18 1226 -rw-rw-r-- 1 news 915 Jan 11 20:32 1227 -rw-rw-r-- 1 news 804 Jan 11 20:33 1228 -rw-rw-r-- 1 news 3363 Jan 11 20:39 1229 -rw-rw-r-- 1 news 2677 Jan 11 20:40 1230 -rw-rw-r-- 1 news 1611 Jan 11 20:47 1231 -rw-rw-r-- 1 news 2294 Jan 11 20:48 1232 -rw-rw-r-- 1 news 1484 Jan 11 20:55 1233 -rw-rw-r-- 1 news 492 Jan 12 00:15 1234 -rw-rw-r-- 1 news 1456 Jan 12 20:19 1235 -rw-rw-r-- 1 news 1351 Jan 12 20:21 1236 -rw-rw-r-- 1 news 2203 Jan 12 20:21 1237 -rw-rw-r-- 1 news 3742 Jan 12 20:27 1238 -rw-rw-r-- 1 news 1610 Jan 12 23:16 1239 -rw-rw-r-- 1 news 1491 Jan 13 00:17 1240 -rw-rw-r-- 1 news 2049 Jan 13 23:15 1241 -rw-rw-r-- 1 news 2023 Jan 14 20:19 1242 -rw-rw-r-- 1 news 1769 Jan 14 20:19 1243 -rw-rw-r-- 1 news 757 Jan 14 20:25 1244 -rw-rw-r-- 1 news 1062 Jan 14 20:30 1245 -rw-rw-r-- 1 news 968 Jan 14 20:37 1246 -rw-rw-r-- 1 news 795 Jan 15 02:32 1247 -rw-rw-r-- 1 news 1631 Jan 15 20:18 1248 -rw-rw-r-- 1 news 3600 Jan 15 20:21 1249 -rw-rw-r-- 1 news 1713 Jan 15 20:30 1250 -rw-rw-r-- 1 news 2757 Jan 15 20:30 1251 -rw-rw-r-- 1 news 5047 Jan 15 20:32 1252 -rw-rw-r-- 1 news 883 Jan 16 00:18 1253 -rw-rw-r-- 1 news 968 Jan 16 00:18 1254 -rw-rw-r-- 1 news 508 Jan 16 04:15 1255 -rw-rw-r-- 1 news 967 Jan 16 04:15 1256 -rw-rw-r-- 1 news 1374 Jan 16 04:18 1257 -rw-rw-r-- 1 news 1567 Jan 16 05:15 1258 -rw-rw-r-- 1 news 613 Jan 16 20:18 1259 -rw-rw-r-- 1 news 695 Jan 16 20:18 1260 -rw-rw-r-- 1 news 2116 Jan 16 20:24 1261 -rw-rw-r-- 1 news 1359 Jan 16 20:25 1262 -rw-rw-r-- 1 news 1037 Jan 16 20:29 1263 -rw-rw-r-- 1 news 1149 Jan 16 20:36 1264 -rw-rw-r-- 1 news 1875 Jan 16 20:36 1265 -rw-rw-r-- 1 news 1193 Jan 16 20:45 1266 -rw-rw-r-- 1 news 985 Jan 16 20:45 1267 -rw-rw-r-- 1 news 3759 Jan 17 02:16 1268 -rw-rw-r-- 1 news 1844 Jan 17 04:18 1269 -rw-rw-r-- 1 news 1884 Feb 9 03:16 1425 -rw-rw-r-- 1 news 1090 Feb 9 06:17 1426 -rw-rw-r-- 1 news 1824 Feb 10 20:18 1427 [the articles continue but are irrelevant] My explist file is: # expire fields: field1-field2-field3 # # retention period field1: days before articles are candidates for expiration # dflt expiry date field2: default expiration date # purge date field3: unconditional expiration # hold onto history lines 14 days, nobody gets >90 days /expired/ x 7 - #/bounds/ x 0-1-90 - # local groups - keep forever dsi,cimage,ctps x 9000-9000-9000 - comp x 5-7-10 - # big non-tech groups held long enough for a long weekend #sci,rec,talk,soc,misc,alt x 3-4-5 - # real noise gets thrown away fast junk,tor.news.stats,biz x 2-2-2 - # throw away some technical stuff not worth archiving comp.os.vms,comp.mail.maps x 1-1-1 - # default: 7 days and no archive all x 7 - --------------------- My question is, why are those old articles hanging around? I did have some problems (I believe back in Jan) running low on space. doexpire couldn't find space for it's work files. I don't believe that expire failing back then should effect last night's expire (which ran fine and should have removed those articles). Any thoughts? Thanks for any help, Brian --- brian@cimage.com
henry@zoo.toronto.edu (Henry Spencer) (02/21/91)
In article <1991Feb20.161636.9861@cimage.com> brian@cimage.com (Brian Kelley) writes: >-rw-rw-r-- 1 news 1502 Jan 4 20:22 1190 >... >Message-ID: <4f015001.20b6d@apollo.HP.COM> Have you tried "newshist 4f015001.20b6d@apollo.HP.COM"? If the article is not in your history file -- normally this is the result of something traumatic like a crash or a full filesystem -- then it *cannot* be expired. Dealing with situations like this is the job of the "addmissing" command that was recently added to C News. -- "Read the OSI protocol specifications? | Henry Spencer @ U of Toronto Zoology I can't even *lift* them!" | henry@zoo.toronto.edu utzoo!henry
brian@cimage.com (Brian Kelley) (02/21/91)
In article <1991Feb20.161636.9861@cimage.com> brian@cimage.com (Brian Kelley) writes: >My question is, why are those old articles hanging around? I did have some >problems (I believe back in Jan) running low on space. doexpire couldn't >find space for it's work files. I don't believe that expire failing back >then should effect last night's expire (which ran fine and should have >removed those articles). Any thoughts? Ok, problem solved. When the Cnews expire runs, looking for articles to kill, it looks at the entries in the history file. It does not traverse the file system. Back in January, expire ran out of space a couple of times. I believe the history file wasn't properly updated and therefore the articles were never expired. The solution was to run mkhistory to rebuild the history file based on what is actually in the news spool directory. Simple. Though when I ran mkhistory, I got the following message: mkhistory: <swill@trash> found in history.n -- aborting Cute, eh? I took the brute force approach and edited the history.n file, deleting the offending swill line (and removing the two articles it had a problem with by hand). I am going to complete the remainder of the mkhistory script by hand. Hopefully tonight expire will do the right thing and all will be well. Thanks to the two very quick respondents who clued me in to the mkhistory solution. Brian --- brian@cimage.com
henry@zoo.toronto.edu (Henry Spencer) (02/21/91)
In article <1991Feb20.195229.10997@cimage.com> brian@dgsi.UUCP (Brian Kelley) writes: >Though when I ran mkhistory, I got the following message: > >mkhistory: <swill@trash> found in history.n -- aborting Sounds like you need a more modern C News. :-) Mkhistory now just warns you about this; it sets up legal history lines that call for immediate expiry. (The problem is article-like files that don't seem to be legal articles.) >Thanks to the two very quick respondents who clued me in to the mkhistory >solution. Addmissing is actually a cleaner solution, although by the sounds of it your C News may be too old to have it. Mkhistory does have the wart of losing the history lines for expired articles. -- "Read the OSI protocol specifications? | Henry Spencer @ U of Toronto Zoology I can't even *lift* them!" | henry@zoo.toronto.edu utzoo!henry