[news.sysadmin] How do I issue a "sendme" control message?

lenny@icus.islp.ny.us (Lenny Tropiano) (07/15/89)

I had the infamous System V i-node demon attack me today... lost all
my i-nodes, and then did a fsck and they came back.  Lost about 40 news
articles, I would like to retrieve them.   How would I go about
issuing a "sendme" control message with those message-id's that are
in my errlog ...

Thanks,
Lenny
-- 
Lenny Tropiano             ICUS Software Systems         [w] +1 (516) 589-7930
lenny@icus.islp.ny.us      Telex; 154232428 ICUS         [h] +1 (516) 968-8576
{ames,talcott,decuac,hombre,pacbell,sbcs}!icus!lenny     attmail!icus!lenny
        ICUS Software Systems -- PO Box 1; Islip Terrace, NY  11752

jack@csccat.UUCP (Jack Hudler) (07/16/89)

In article <749@icus.islp.ny.us> lenny@icus.islp.ny.us (Lenny Tropiano) writes:
>I had the infamous System V i-node demon attack me today... lost all
>my i-nodes, and then did a fsck and they came back.  Lost about 40 news
>articles, I would like to retrieve them.   How would I go about
>issuing a "sendme" control message with those message-id's that are
>in my errlog ...

	I wouldn't worry about it.
	I often wonder how much news I have never read because
	of some upstream glitch. I shudder to think of
	the number of articles that have been lost over the
	years due to various problems at this site.
	Besides your host site would have to be in ihave/sendme mode
	for your site in order to expect an sendme control message,
	then again, you could try it, but don't lose any sleep over it.

dave@dlb.uucp (Dave Buck) (07/18/89)

In article <749@icus.islp.ny.us> lenny@icus.islp.ny.us (Lenny Tropiano) writes:
>I had the infamous System V i-node demon attack me today... lost all
>my i-nodes, and then did a fsck and they came back.  Lost about 40 news
>articles, I would like to retrieve them.   How would I go about
>issuing a "sendme" control message with those message-id's that are
>in my errlog ...

I often suffer from the same set of problems with inodes here.
I got upset over the loss of news articles, as we feed other sites
as well.  I noticed, as you did, that the error log has the article ids
in it, and thought "if I made this into a sendme message, what would
happen?"

I created the attached perl script to scan the error log, get rid of
some of the garbage I don't want to hear about, and looked for the
"no space" etc. messages.  I obtained the article ids, then invoked
"inews" to create an "ihave" message indicating the system having those
articles was one that I expected did have all those articles.
Inews then discards the article ids that we already have, possibly due to
having received the articles in the interim from another site, and turns
the message into a "sendme", and we get the articles.  Works great for me.
You mileage may vary, and I expect to hear lotsa flames from better perlers
about what crap this is, and from others who say "but I don't have perl".
Sorry.  This can all be done by hand ... just make a list of the ids
you missed, one per line and with the <>'s, and give them to "inews"
with parameters
	inews -t "cmsg ihave xxx" -n to.yyy.ctl
where "xxx" is the site that usually feeds you, and "yyy" is your site name.
In the attached script, "dlb" is our site name, and "ames" is our best feed.

By the way, we converted to "C News" recently, and the only foul gripe is
that this type of scenerio is not possible (well, not as they've implemented
rnews).  Their rnews will discard an incoming batch if they think it might
not fit, and the lost article ids are not saved.  I thought about two
approaches to change this behavior ... first thought was to scan the incoming
batch for article ids and save them for rerequesting (nah, too difficult) ...
second thought was to just save the incoming batch in another file system
known to have lotsa space, and let us clean up after the following day.

---- Cut somewhere around here ----
#!/usr/bin/perl -P
eval "exec /usr/bin/perl -P -S $0 $*"
	if $running_under_some_shell;

$who="ames";
if ( $#ARGV >= 0 ) {
	$who = $ARGV[0];
}
$sendme = 0;
$sendcount = 0;
open(errlog,"/usr/lib/news/errlog") || die "Can't read errlog";
open(newlog,">/usr/lib/news/newerrlog");
while (<errlog>) {
	next if /Unparsable/;
	next if /moved to junk/;
	if ( /No space/ ||
             /inews: Write failed/ ||
	     /inews: Cannot open \/usr\/spool\/news/ ||
             /inews: write failed to temp file/ ||
             /inews: Cannot reread/ ) {
		if (! $sendme) {
			open(inews,
				'|/usr/lib/news/inews -t "cmsg ihave ' .
				$who .
				'" -n to.dlb.ctl');
			$sendme = 1;
		}
		($date,$id,$complaint) = split(/\t/);
		print inews $id, "\n";
		$sendcount++;
	} else	{
		print newlog $_;
		print $_;
	};
}
close errlog;
close newlog;
unlink "/usr/lib/news/olderrlog";
rename ("/usr/lib/news/errlog","/usr/lib/news/olderrlog");
rename ("/usr/lib/news/newerrlog","/usr/lib/news/errlog");
if ( $sendme ) {
	close inews;
	printf "\n%d articles re-requested via sendme to %s\n", $sendcount,$who;
}
-- 
Dave Buck	{amdahl,sun,megatest,plx,ardent,ubvax}!dlb!dave
D. L. Buck and Associates, Inc.; San Jose, California 95119; (408)972-2825

henry@utzoo.uucp (Henry Spencer) (07/19/89)

In article <1989Jul18.154058.2638@dlb.uucp> dave@dlb.UUCP (Dave Buck) writes:
>By the way, we converted to "C News" recently, and the only foul gripe is
>that this type of scenerio is not possible (well, not as they've implemented
>rnews).  Their rnews will discard an incoming batch if they think it might
>not fit, and the lost article ids are not saved.  I thought about two
>approaches to change this behavior ... first thought was to scan the incoming
>batch for article ids and save them for rerequesting (nah, too difficult) ...
>second thought was to just save the incoming batch in another file system
>known to have lotsa space, and let us clean up after the following day.

If you've got somewhere you can put a batch when space runs out, or something
else useful you can do with it, great!  Unfortunately, in the general case
"there just ain't no graceful way".  Even writing to log files is risky if
they are on the same filesystem and space is really tight.  We decided that
about all we could do was (a) discard the batch, (b) send mail to $NEWSMASTER
so it wouldn't pass unnoticed (a bit risky if the mailboxes are in the same
filesystem, but silent data loss just didn't seem a good idea), and
(c) write rnews as a shell script so people who've got a better way can
change things easily.
-- 
$10 million equals 18 PM       |     Henry Spencer at U of Toronto Zoology
(Pentagon-Minutes). -Tom Neff  | uunet!attcan!utzoo!henry henry@zoo.toronto.edu

mason@tmsoft.uucp (Dave Mason) (07/19/89)

In article <1989Jul18.154058.2638@dlb.uucp> dave@dlb.UUCP (Dave Buck) writes:
>[....other stuff]
>By the way, we converted to "C News" recently, and the only foul gripe is
>that this type of scenerio is not possible (well, not as they've implemented
>rnews).  Their rnews will discard an incoming batch if they think it might
>not fit, and the lost article ids are not saved.  I thought about two
>approaches to change this behavior ... first thought was to scan the incoming
>batch for article ids and save them for rerequesting (nah, too difficult) ...
>second thought was to just save the incoming batch in another file system
>known to have lotsa space, and let us clean up after the following day.
>[...a perl script...]

We have /usr/lib/news (where we have in.coming & history),
/usr/spool/news and /usr/spool/uucp on three different file systems,
with the various free disk checkers set up appropriately.  Works like
a charm: automatically handles low disk problems without so much as a
wimper.  (Also uuxqt won't run if /usr (the file system where
/usr/lib/news is found) is low on free space, so things just slow
down, but nothing breaks.)

We haven't installed the new release yet because they put in.coming in
the article file system & it seems to be hardcoded too many places to
quickly change it to the lib file system.  It should be a seperate
define, setup up BY DEFAULT however they wish.

	../Dave

john@frog.UUCP (John Woods) (07/20/89)

In article <749@icus.islp.ny.us>, lenny@icus.islp.ny.us (Lenny Tropiano) writes:
> I had the infamous System V i-node demon attack me today... lost all
> my i-nodes, and then did a fsck and they came back.  Lost about 40 news
> articles, I would like to retrieve them.   How would I go about
> issuing a "sendme" control message with those message-id's that are
> in my errlog ...

The following is a shell-script that I keep around just for such emergencies.
It assumes the presense of a SystemV-like date command, such as the one I
posted to alt.sources and comp.unix.something-or-other a few months back,
and it assumes that uname is the proper way to find out your host name,
and that `uname`.UUCP is your news domain name.  Change these last to suit
yourself.  NEWSSITE should be what your news neighbor knows you as, and
HOST is what appears in the From: line and Message-ID: line.  You could
speed it up a little by just looking at one of your own messages and filling
in these items accordingly.

The input is a list of article ids, the output is a news article suitable
for piping to uux (as in "forgesendme lost_list | uux - neighbor!rnews").

The massaging of errlog into Message-id's is left to the student as an
exercise :-).

----snip----------------snip----------------snip---------------------------
#! /bin/sh
# forgesendme -- a shell script which invents a control article asking
# for lost news.  If an argument is given, it is a file containing
# article ids, otherwise article ids are read from standard input.
if [ $# -eq 0 ]
then
	filename=/tmp/idlike.$$
	trap 'rm -f $filename' 0 1 2 3 15
	cat > $filename
	set $filename
fi
LINES=`wc -l <$1`
NEWSSITE=`uname`			# change if inappropriate
HOST=$NEWSSITE.UUCP			# change if inappropriate
cat << @@@
Path: $NEWSSITE!news
From: news@$HOST
Newsgroups: control
Subject: sendme $NEWSSITE
Message-ID: <`date +%d%h%y.%T`@$HOST>
Date: `date "+%d %h 19%y %T`
Control: sendme $NEWSSITE
Organization: Restoring crashed partition
Lines: $LINES

`cat $1`
@@@

-- 
John Woods, Charles River Data Systems, Framingham MA, (508) 626-1101
...!decvax!frog!john, john@frog.UUCP, ...!mit-eddie!jfw, jfw@eddie.mit.edu
    People...How you gonna FIGURE 'em?
    Don't bother, S.L.--Just stand back and enjoy the EVOLUTIONARY PROCESS...