[news.software.b] Problems with expire in CNews

schwager@m.cs.uiuc.edu (Michael Schwager) (04/24/91)

Hi,
Today is April 23.  I have some files like these:

57837   39 -rw-rw-r--  1 news     notes       39906 Apr 15 09:34 ./1179
57675   30 -rw-rw-r--  1 news     notes       30310 Apr 15 09:35 ./1180
57678   24 -rw-rw-r--  1 news     notes       24299 Apr 15 09:35 ./1181
57681    1 -rw-rw-r--  1 news     notes         465 Apr 15 09:40 ./1182
57691    1 -rw-rw-r--  1 news     notes         468 Apr 15 09:40 ./1183
57762   30 -rw-rw-r--  1 news     notes       30314 Apr 15 09:41 ./1184
57838   30 -rw-rw-r--  1 news     notes       30308 Apr 15 09:41 ./1185
57839   30 -rw-rw-r--  1 news     notes       30315 Apr 15 09:41 ./1186
57840   30 -rw-rw-r--  1 news     notes       30309 Apr 15 09:42 ./1187

(I even have some that are older!)
They are in a local newsgroup called uiuc.cs.maint.nfs .  Looking at the
explist, I show:

uiuc.cs.maint.nfs               x       4-7     -

Now, from what I know, this means that the articles will be removed from
our disk in 4 days, but as you can see it's not happening.  Can anyone
explain why?
-Mike
P.S. here is a sample header from a file, specifically file 1183 listed
above:

From: root@cs.uiuc.edu
Date: 14 Apr 91 23:43 CDT
Newsgroups: uiuc.cs.maint.nfs
Subject: Outbound a.cs.uiuc.edu:/usr/dcs/sun
Message-ID: <78007042@m.cs.uiuc.edu>
Path: m.cs.uiuc.edu!m.cs.uiuc.edu!seefromline
Nf-ID: #N:m.cs.uiuc.edu:78007042:000:167
Nf-From: cs.uiuc.edu!root    Apr 14 23:43:00 1991

(text here)

-Mike Schwager                             | Machine: Amiga 500, 3 MB RAM/30 HD
INTERNET:schwager@cs.uiuc.edu              | Bike: '83 Kawasaki KZ750 LTD
UUCP:{uunet|convex|pur-ee}!uiucdcs!schwager| Band: Poi Dog Pondering      //
BITNET:schwager@mike.cs.uiuc.edu           | Hero: Robert Bly         \\ //
University of Illinois, Dept. of Comp. Sci.| DoD: #0301                \X/Amiga

henry@zoo.toronto.edu (Henry Spencer) (04/24/91)

In article <1991Apr23.170825.8303@m.cs.uiuc.edu> schwager@m.cs.uiuc.edu (Michael Schwager) writes:
>Now, from what I know, this means that the articles will be removed from
>our disk in 4 days, but as you can see it's not happening.  Can anyone
>explain why?

The first thing you need to check is whether expire is, in fact, being
run.  Is there a history.o file in /usr/lib/news?  How old is it?  (It
should date to the last time expire was run.)  Is anyone reading the
mail to "usenet" (or whoever you specified as the destination for trouble
reports)?
-- 
And the bean-counter replied,           | Henry Spencer @ U of Toronto Zoology
"beans are more important".             |  henry@zoo.toronto.edu  utzoo!henry

schwager@m.cs.uiuc.edu (Michael Schwager) (04/24/91)

henry@zoo.toronto.edu (Henry Spencer) writes:

>In article <1991Apr23.170825.8303@m.cs.uiuc.edu> schwager@m.cs.uiuc.edu (Michael Schwager) writes:
>>Now, from what I know, this means that the articles will be removed from
>>our disk in 4 days, but as you can see it's not happening.  Can anyone
>>explain why?

>The first thing you need to check is whether expire is, in fact, being
>run.  Is there a history.o file in /usr/lib/news?
	Yes
>How old is it?  (It
>should date to the last time expire was run.)

It's dated as of around 2 am.  I start the expire at 1am.  I have it send
me mail every night when expire is run; I've been getting it regularly.  Of
course, this only means that cron has fired it up.

>Is anyone reading the
>mail to "usenet" (or whoever you specified as the destination for trouble
>reports)?

Yup, I am.  What should I look for?
-Mike

henry@zoo.toronto.edu (Henry Spencer) (04/25/91)

In article <1991Apr24.160705.29971@m.cs.uiuc.edu> schwager@m.cs.uiuc.edu (Michael Schwager) writes:
>>...Is there a history.o file in /usr/lib/news?
>	Yes	[and the date is right]

Okay, sounds like expire is running.  Also, you're not getting any bizarre
complaints in mail.  So the next question is:  are those persistent articles
in your history file?  The newshist command can tell you that.

Actually, another question is, are you *sure* that the explist line for that
newsgroup is in fact the one that's being applied?  An earlier line which
also covers that group will cause its line to be ignored.  A look at your
whole explist, not just that line, is in order.
-- 
And the bean-counter replied,           | Henry Spencer @ U of Toronto Zoology
"beans are more important".             |  henry@zoo.toronto.edu  utzoo!henry

schwager@m.cs.uiuc.edu (Michael Schwager) (04/25/91)

henry@zoo.toronto.edu (Henry Spencer) writes:

>In article <1991Apr24.160705.29971@m.cs.uiuc.edu> schwager@m.cs.uiuc.edu (Michael Schwager) writes:
>>>...Is there a history.o file in /usr/lib/news?
>>	Yes	[and the date is right]

>Okay, sounds like expire is running.  Also, you're not getting any bizarre
>complaints in mail.  So the next question is:  are those persistent articles
>in your history file?  The newshist command can tell you that.

>Actually, another question is, are you *sure* that the explist line for that
>newsgroup is in fact the one that's being applied?  An earlier line which
>also covers that group will cause its line to be ignored.  A look at your
>whole explist, not just that line, is in order.

Well, unfortunately I went and cleaned up a files throughout my directory,
so I can't do a newshist.  But if the problem persists, I'm sure it will
turn up again.

I went and cleaned up my explist file considerably.  I'm somewhat
embarrassed to admit that I didn't catch the part about the line ordering.
So this could very well take care of it.
-Mike

molenda@s1.msi.umn.edu (Jason Molenda) (04/25/91)

henry@zoo.toronto.edu (Henry Spencer) writes:

> An earlier line which
>also covers that group will cause its line to be ignored.  

I've known about this for a long time and I know it is documented and
everything, but am I the only person who thinks it's a huge hole inviting
mistakes by new and old sysadmins alike?  I don't know much (read: nothing)
about the internals of expire, so I have no idea how hard it would be
to make the ordering of the lines non-important, but it sure would be
nice.  Am I missing something obvious here, or shouldn't I be able to
say

local          5 x  -
local.test     1 x  -

I mean, I know this won't work right with the current expire, but would
it be that hard to make it work the way I expect it to?
-- 
Jason Molenda, Tech Support, Iris & News Admin, Minnesota Supercomputer Inst
molenda@msi.umn.edu || "You can tune a piano but you can't tuna fish."

schwager@m.cs.uiuc.edu (Michael Schwager) (04/25/91)

molenda@s1.msi.umn.edu (Jason Molenda) writes:

>henry@zoo.toronto.edu (Henry Spencer) writes:

>> An earlier line which
>>also covers that group will cause its line to be ignored.  

>I've known about this for a long time and I know it is documented and
>everything, but am I the only person who thinks it's a huge hole inviting
>mistakes by new and old sysadmins alike?

Indeed, if I may make a comment regarding News (we ran notes up until
December of last year):

Seems like there are all kinds of pitfalls, troubles, and caveats one can
enter when bringing News up.  For example, one must make more inodes than
the default on our News partitions, because News stores every blasted
article in a seperate file.  I'm guessing that that's the way it is
elsewhere, too.  News is so big and cantankerous, that it's so dang
confusing to figure out problems.  It's taken me a few months to get it
down, and even now (as my previous problem will attest), I'm still
confused.  Like, what all does the history file do?  Which programs use it?
(I know, relaynews and expire use it... any others?  Don't answer that.
It's rhetorical.)

I had a huge problem one time, where I was sending some files over to our
machine from another machine and for some reason, they didn't get put on
disk, but they did get entered into our history file (mea culpa- I didn't
document the whole sad scene; I was too busy being frustrated).  Anyway, I
have a good idea of how the whole mass fits together now, but it's taken
awhile, and I'm always nervous that another gremlin is gonna jump out of
the closet...

In a nutshell: News is big, bad, and ugly.  I would not wish upon my worst
enemy that they be forced to bring it up from scratch.
-Mike Schwager                             | Machine: Amiga 500, 3 MB RAM/30 HD
INTERNET:schwager@cs.uiuc.edu              | Bike: '83 Kawasaki KZ750 LTD
UUCP:{uunet|convex|pur-ee}!uiucdcs!schwager| Band: Poi Dog Pondering      //
BITNET:schwager@mike.cs.uiuc.edu           | Hero: Robert Bly         \\ //
University of Illinois, Dept. of Comp. Sci.| DoD: #0301                \X/Amiga

flee@cs.psu.edu (Felix Lee) (04/26/91)

>I have no idea how hard it would be to make the ordering of the lines
>non-important, but it sure would be nice.

It's a matter of using a different "best match" rule.  C News expire
currently uses "first match".  Nearly every alternative is
incompatible with this behavior.

One alternative is "last match".  This is possibly more natural than
first match.  I tend to want "all" near the top, since a general rule
should appear before exceptions to the rule.

Another rule is "earliest expire": pick the matching line that expires
the article soonest.  This rule makes it hard to say something like:
	news.announce	x 30 -
	news		x 5 -
The opposite rule is "latest expire", which has the opposite problem.

A good rule is "strongest match": X is a stronger match than Y if the
set of X is a proper subset of the set of Y.  This will probably do
what everyone expects most of the time.  Unfortunately, it isn't
specific enough.  What if you have:
	all.test	x 3 -
	misc.all	x 5 -
How do you expire misc.test?  You have to defer to another rule.
--
Felix Lee	flee@cs.psu.edu

a3@earth.rivm.nl (Adri Verhoef) (04/26/91)

Henry writes:
>> An earlier line which
>>also covers that group will cause its line to be ignored.  

Maybe it's a good idea to put this text in the 'explist' file,
so that people are warned after they start editing it.
In this simple way you can prevent some of the expire problems
showing up.  Just put in a short notice.  In the meantime,
do it for the sake of people that come to administer the
news system after you will have been doing so.

henry@zoo.toronto.edu (Henry Spencer) (04/26/91)

In article <1991Apr25.001832.19898@s1.msi.umn.edu> molenda@s1.msi.umn.edu (Jason Molenda) writes:
>> An earlier line which
>>also covers that group will cause its line to be ignored.  
>
>I've known about this for a long time and I know it is documented and
>everything, but am I the only person who thinks it's a huge hole inviting
>mistakes by new and old sysadmins alike? ...

I thought about the issue somewhat.  The trouble is that it's useful to be
able to state a general rule and then establish exceptions, and any other
definition of which rule dominates is more complicated, harder to understand,
and more mistake-prone.
-- 
And the bean-counter replied,           | Henry Spencer @ U of Toronto Zoology
"beans are more important".             |  henry@zoo.toronto.edu  utzoo!henry

alexis@panix.uucp (Alexis Rosen) (04/27/91)

schwager@m.cs.uiuc.edu (Michael Schwager) writes:
>Indeed, if I may make a comment regarding News (we ran notes up until
>December of last year):
>
>Seems like there are all kinds of pitfalls, troubles, and caveats one can
>enter when bringing News up. [...]
>[...]  News is so big and cantankerous, that it's so dang
>confusing to figure out problems.  It's taken me a few months to get it [...]
>
>In a nutshell: News is big, bad, and ugly.  I would not wish upon my worst
>enemy that they be forced to bring it up from scratch.

I'd have to disagree with this. When I brought up Cnews, I had never done
any sort of news administration. I had *no* experience writing in C under
Unix (though I had a reading knowledge of C left over from earlier days)
and in fact still haven't had much time to learn about it. I had not much
experience with administering Unix in general, and was working with a fairly
oddball system (A/UX, getting less odd all the time...) which nobody could
offer much help with. That was over a year ago. 

It took me about a day to _thoroughly_ read all the docs and stuff, and to
figure out what the appropriate answers were to the various questions that
the Cnews config script asks you. I made careful notes that day about what
each file in Cnews was and where it went.

The next day I brought up Cnews. Took most of the day to get it to work- I
had to figure out a few missing headers, and that sort of thing. Then I
spent five minutes writing up the explist, sys, batchparams, and so on, and
that was it. Done. Since then it's run for over a year without a single
problem.

I don't attribute this to any particular brilliance on my part. It comes
from knowing that, even though it's running on a Mac, it's not Mac software.
:-)  Seriously, you can't just browse the docs for a few minutes and hope
things will work out, because they won't.

Especially given the unix-variant-independance of Cnews Henry and Geoff have
done a remarkable job of uniformly superb quality. Sure it's big, but it
has an awful lot to do. As for the bad and the ugly, I'd have to lay that
at the door of sysops who don't read documentation and (at least as frequently)
weird variants of Unix that barely live up to their name.

The best thing you can do if you want to bring up Cnews from scratch is forget
what you think you know and learn it right, the first time. Do that and you'll
be very happy with the results. Cnews is at least as good as my professional
work (and it's free!)... Those who know what an egotist I am will know what
high praise that is. :-)

---
Alexis Rosen
Owner/Sysadmin, PANIX Public Access Unix, NY
{cmcl2,apple}!panix!alexis

henry@zoo.toronto.edu (Henry Spencer) (04/28/91)

In article <1991Apr26.124402.4151@rivm.nl> a3@earth.rivm.nl (Adri Verhoef) writes:
>>> An earlier line which
>>>also covers that group will cause its line to be ignored.  
>
>Maybe it's a good idea to put this text in the 'explist' file,
>so that people are warned after they start editing it.

I have limited sympathy for people who start editing without reading the
documentation, and I have a strong feeling that such people will not read
a warning in explist either.

However, I've added a note to remind the careless that the order of lines
is significant.
-- 
And the bean-counter replied,           | Henry Spencer @ U of Toronto Zoology
"beans are more important".             |  henry@zoo.toronto.edu  utzoo!henry

jad@nyama.UUCP (04/29/91)

In <1991Apr25.161229.18843@m.cs.uiuc.edu>, schwager@m.cs.uiuc.edu (Michael Schwager) writes:
>
>In a nutshell: News is big, bad, and ugly.  I would not wish upon my worst
>enemy that they be forced to bring it up from scratch.
Ah, excuse me?  A friend of mine and I got c-news and rn intalled, receiving
and reading news in 3 hours.  And that included a coffee break!

C-news is big, but it's not bad and not ugly.  I find the use of shell
for most of simple use to be an advantage.  What you do need to do is
answer each and every question in conf/build *very carefully* or else.
Chances are the defaults are ok, but if not then don't bother "fixing"
doit.bin and friends: start over, it will make you happier in the end.

This is not meant as a flame, on the contraire.  You have to be carefull
because c-news assumes you know what you're doing:-)

>-Mike Schwager                             | Machine: Amiga 500, 3 MB RAM/30 HD
>INTERNET:schwager@cs.uiuc.edu              | Bike: '83 Kawasaki KZ750 LTD
>UUCP:{uunet|convex|pur-ee}!uiucdcs!schwager| Band: Poi Dog Pondering      //
>BITNET:schwager@mike.cs.uiuc.edu           | Hero: Robert Bly         \\ //
>University of Illinois, Dept. of Comp. Sci.| DoD: #0301                \X/Amiga

-- 
Jose Dias		jad@nyama.UUCP		 Who me? I didn't say anything!
-- 
Jose Dias		jad@nyama.UUCP		 Who me? I didn't say anything!