dcox@ssd.kodak.com (Don Cox (253-7121)) (04/01/91)
Every now and then I get an nntp buildup, such as:
-rw-r--r-- 1 root 17073 Mar 31 01:23 nntp.a01263
-rw-r--r-- 1 root 37712 Mar 31 01:23 nntp.a01634
-rw-r--r-- 1 root 3526 Mar 31 02:08 nntp.a02308
-rw-r--r-- 1 root 10383 Mar 31 02:09 nntp.a02358
-rw-r--r-- 1 root 8617 Mar 31 05:26 nntp.a04349
-rw-r--r-- 1 root 9583 Mar 31 05:25 nntp.a05502
-rw-r--r-- 1 root 7796 Mar 31 09:00 nntp.a08503
-rw-r--r-- 1 root 31650 Mar 31 09:01 nntp.a09344
-rw-r--r-- 1 root 28105 Mar 31 11:26 nntp.a11368
-rw-r--r-- 1 root 11272 Mar 31 16:45 nntp.a17040
-rw-r--r-- 1 root 21426 Mar 31 16:45 nntp.a17577
I eventually end up deleting the files that don't get processed. What
program is suppose to take care of this? Is there something that I can
run by hand, or is this normal? Thanks much.
--
Don Cox
Phone (716) 253-7121 KMX (716) 253-7998
INTERNET dcox@ssd.kodak.comrickert@mp.cs.niu.edu (Neil Rickert) (04/01/91)
In article <1991Mar31.214919.17956@ssd.kodak.com> dcox@ssd.kodak.com (Don Cox (253-7121)) writes: >Every now and then I get an nntp buildup, such as: > >-rw-r--r-- 1 root 17073 Mar 31 01:23 nntp.a01263 >-rw-r--r-- 1 root 37712 Mar 31 01:23 nntp.a01634 >... This is caused when nntpd terminates prematurely, without properly renaming the file. It is sometime caused by short term network outages. Instead of periodically running 'newsrun' from 'crontab', I am running the following script which itself eventually invokes newsrun. It is run as 'news', whereas nntpd runs as root. -------------- Begin ------------------- #! /bin/sh # Process spooled news. # =()<. ${NEWSCONFIG-@<NEWSCONFIG>@}>()= . ${NEWSCONFIG-/usr/lib/news/bin/config} PATH=$NEWSCTL/bin:$NEWSBIN/input:$NEWSBIN/relay:$NEWSBIN:$NEWSPATH ; export PATH umask $NEWSUMASK hold="$1" here="$NEWSARTS/in.coming" cd $here # First, is it worth trying at all? if test -r stop then exit 0 fi # Convert any left-over 'nntp.' files. for file in `ls -rtd . nntp.*` do if [ "$file" = "." ] ; then break ; else true ; fi pid=`expr $file : 'nntp.\([0-9]*\)$'` numprocs=`ps \#$pid | wc -l` state="`/bin/kill -0 $pid`" case "$state" in *"No such process"*) ln $file 100$pid && rm -f $file ;; *) ;; esac done $NEWSBIN/input/newsrun $hold -------------- End ------------------- The script only touches nntp.* files older than the last directory modification date. It tests whether nntpd is still running with the 'kill -0'. You might want to check the message your system provides in this case, for use in the 'case' statement -- =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*= Neil W. Rickert, Computer Science <rickert@cs.niu.edu> Northern Illinois Univ. DeKalb, IL 60115 +1-815-753-6940
overby@cobber.cord.edu (Glen Overby) (04/04/91)
In article <1991Mar31.214919.17956@ssd.kodak.com> dcox@ssd.kodak.com (Don Cox (253-7121)) writes: >Every now and then I get an nntp buildup, such as: [lots of nntp.a<pid?> files] I had a similar problem here, and I found that I also had an nntpd processes to accompany each of the files! Apparently nntpd wasn't exiting when the session ended. I usually noticed this after I'd get a dozen or so NNTPDs hanging around apparently using a little CPU time here and there. The problem magically went away when I recompiled WITHOUT CNEWS Batching (I'm running nntp 1.5.11 of course) so now I'm back to using newsspool for every incoming article. YUK! Any ideas? (I'm on a Solbourne Series 4 with OS/MPX 4.0D (SMP) ~= SunOS 4.0.3 running C-news. My feed comes from a VMS site, if that makes any difference). -- Glen Overby overby@cobber.cord.edu Technical Manager uunet!plains!cobber!overby (UUCP) Concordia College overby@plains (Bitnet) 901 S 8th St, (218) 299-4192 (ofc) 299-4151 (dept)
geoff@world.std.com (Geoff Collyer) (04/05/91)
Glen Overby: >I had a similar problem here, and I found that I also had an nntpd processes >to accompany each of the files! Apparently nntpd wasn't exiting when the >session ended. I usually noticed this after I'd get a dozen or so NNTPDs >hanging around apparently using a little CPU time here and there. nntpd needs to be made smarter about impolite, anti-social (one is tempted say ``obnoxious'') NNTP transmitters that keep an idle connection open just in case an article might arrive eventually (e.g. nntplink). Work is in progress on such an nntpd; no promises about when or how it will be available. In the meantime, you could try asking your NNTP neighbours to drop connections when they have nothing to send (and to not connect just because they have 0001 articles for you), if possible (I believe even nntplink can be configured to be polite). >The problem magically went away when I recompiled WITHOUT CNEWS Batching >(I'm running nntp 1.5.11 of course) so now I'm back to using newsspool for >every incoming article. YUK! You are taking quite a performance hit by having nntpd start a new rnews process (especially on a Sun with its slow forks and execs) for each incoming article. It really would be much better to ask your neighbours to be polite (i.e. say ``nice doggy'' until such time as we can find a rock). -- Geoff Collyer world.std.com!geoff, uunet.uu.net!geoff
alden@shape.mps.ohio-state.edu (Dave Alden) (04/05/91)
In article <1991Apr4.210714.13027@world.std.com> Geoff writes: >nntpd needs to be made smarter about impolite, anti-social (one is >tempted say ``obnoxious'') NNTP transmitters that keep an idle connection >open just in case an article might arrive eventually (e.g. nntplink). I wasn't sure if I really wanted to respond to this, but out of the 10 unsolicited messages I've received so far commenting on this, I belive the following line sums up the whole thing: "Jesus, you'd think you slept with his 16-year-old daughter or something." anti-social: "contrary or hostile to the well-being of society". Hmm - I guess I just don't see how nntplink fits in this category. >In the meantime, you could try asking your NNTP neighbours to drop >connections when they have nothing to send (and to not connect just >because they have 0001 articles for you), if possible (I believe even >nntplink can be configured to be polite). Or you could do what over 200 sites have done so far - pick up my relaynews daemon patches. Granted, Henry and Geoff will tell you to jump in a lake if you run into any problems - but then again I've always enjoyed swimming - and will gladly help you swim too. :-) I do however agree with you that if you don't want your neighbors to run nntplink to you that you should ask them not to. I've had a note in my "standard blurb" that I send to all my news neighbors for over 1.5 years now, ie: [...info about my site...] Incoming Connection Info: We accept connections around the clock. I prefer nntplink over nntpxmit if you run it. Outgoing Connection Info: I run nntplink to your site by default. If you would prefer something else just drop me a note and we'll set you up. :-) [...more info about my site :-)...] So they are forewarned. I'm also going to add a note in the README for nntplink 3.0 stating that you should probably check with your neighbors before switching. Funny thing is that only 1 site has requested that I batch up their articles and only send them once/10 minutes (out of 32 nntp neighbors). And yes, it is very simple to have nntplink act exactly like nntpxmit: nntplink -I0 -S600 <sitename> which makes it close the connection as soon as it finishes sending the current batch and then sleep for 10 minutes before checking for a new one. I wish you wouldn't waste your time finding a "rock" and instead work on "making my patches obsolete" along with "the output part of nntplink obsolete". The more I work with C News the happier I am that I am using it - it truly is better than a lot of the "so-called" professional packages that are out there - I just wish that we didn't have to have the religion along with the package. :-( ...dave -- Dave Alden, System Programmer MaBellNet: (614) 292-4962 Department of Mathematics FaxNet: (614) 292-7174 The Ohio State University Internet: alden@shape.mps.ohio-state.edu 231 W 18th Avenue UUCP: ...!osu-cis!alden Columbus, OH USA 43210 BITNET: alden@ohstpy
geoff@world.std.com (Geoff Collyer) (04/06/91)
Hmmm, well maybe "anti-social" is a bit strong, but it does seem rather pointless to hang there on the line on the off chance that an article might show up. I expect an article transfer session to run like an SMTP session or a telephone call: you say what you have to say and hangup. I suppose I shouldn't be annoyed at nntplink, per se, but at the news administrators who insist on running it without -I0. On the other hand, -I0 seems like a reasonable default. Dave Alden: >I wish you wouldn't waste your time finding a "rock" and instead work >on "making my patches obsolete" along with "the output part of nntplink >obsolete". I did put a smiley after that comment about find a rock, didn't I? The point is that nntpd needs a lot of work. As you know, we've just spent a lot of energy fixing nntplink so it doesn't drop a significant fraction of the articles it is asked to send. I don't recall if it is now as fast reading batch files as reading our modified log file or not, so I'm not sure if we can live with the current nntplink and a stock relaynews, or whether we have more code to write. As for the relaynews daemon, the work to obsolete it is essentially done, there's just some clean up needed. Satisfied? :-) :-) >The more I work with C News the happier I am that I am using it - it >truly is better than a lot of the "so-called" professional packages that >are out there Thank you. >I just wish that we didn't have to have the religion along with the >package. :-( "religion" is not an issue here; try "software engineering", "quality control", and "thoughtfulness" instead. -- Geoff Collyer world.std.com!geoff, uunet.uu.net!geoff
emv@ox.com (Ed Vielmetti) (04/07/91)
In article <1991Apr5.235156.20894@world.std.com> geoff@world.std.com (Geoff Collyer) writes:
Hmmm, well maybe "anti-social" is a bit strong, but it does seem rather
pointless to hang there on the line on the off chance that an article
might show up. I expect an article transfer session to run like an SMTP
session or a telephone call: you say what you have to say and hangup.
I don't think that's quite right, Geoff. The proper analogy might be
to a bucket brigade fighting a fire; if there aren't any pails coming
your way, you don't just sit down, because you know that pretty soon
one will be coming along. It's the same way with news feeds in some
situations, a steady dribble of articles at random but smallish
intervals rather than a big burst at the top of the hour.
Keeping the line up should have a net beneficial effect on network
traffic; it's that many less domain lookups, tcp connection starts and
teardowns, and initial NNTP authorization sessions at the expense of
possibly some TCP keepalive traffic.
This presumes that you care enough and have enough core and CPU to
make the effort to send stuff as soon as it comes in. Naturally, there
are some aspects of it that are more efficient when you batch the
articles and do them all in a burst, but that's a latency vs.
efficiency issue best left to local policy.
--
Msen Edward Vielmetti
/|--- moderator, comp.archives
emv@msen.com
"With all of the attention and publicity focused on gigabit networks,
not much notice has been given to small and largely unfunded research
efforts which are studying innovative approaches for dealing with
technical issues within the constraints of economic science."
RFC 1216