[net.news.group] Program to check news feed accuracy

mkg (05/02/83)

#N:whuxlb:2300001:000:2713
whuxlb!mkg    May  2 00:24:00 1983

Have you ever wondered how reliable your netnews connections are?
We have several netnews feeds now and I was curious as to who is
our most reliable feed.  I have included below a shell/awk script
that will rummage around your netnews log files and report who is
your best/worst netnews feed.

It tells you how many articles were considered, and, for each system
that sends you news, reports how many articles arrived from that
system first and how many never arrived at all (well, not arrived yet)
from that system.

There are a few anomolies though.  It is possible for the total number
of articles to be less than the sum of the number of arrived "first"
articles.  This is because control messages, especially cancel messages,
can arrive before the intended victim article.  When this happens,
the article is considered "received first" and news then throws it on
the floor.  It can arrive again from another source and be "first" again.
Another problem is that this program isn't fair to 'slow' feeds.
The program will reject any article for which the "first"  system
cannot be determined.  This happens when the log entry for the
first system is aged off the log file.  On the other hand, if an article
arrives just before running the program, the article will be counted
against the other systems as never received.  Not exactly fair, but...

Anyway, now you can stop wondering why you always see articles with
titles like "Re: ..." and never see the original article and see how
good (or bad) you really have it.

Enjoy!!!
   Marsh Gosnell  BTL Whippany  (201) 386-7095  whuxlb!mkg



egrep "received article|Duplicate article" /usr/lib/news/log* | \
	sed 's/.* \([^ ]*\)	\([^ ]*\) article \([^ ]*\).*/\3:\2:\1/' | \
	sort -u | \
	awk -F: '
BEGIN { nsystems = 0; narticles = 0; got_received = 0 }

$1 != last {
	if (got_received == 1) {
		for (i = 1; i <= nsystems; i++) {
			if (received[i] == "received")
				first[i]++;
			else if (received[i] != "Duplicate")
				missing[i]++;
		}
		narticles++;
	}
	for (i = 1; i <= nsystems; i++)
		received[i] = "";
	got_received = 0;
	last = $1;
}

$1 == last {
	for (i = 1; i <= nsystems; i++)
		if (system[i] == $3)
			break;
	if (i > nsystems) {
		nsystems++;
		first[nsystems] = 0;
		missing[nsystems] = narticles;
		system[nsystems] = $3;
	}

	received[i] = $2;

	if ($2 == "received")
		got_received = 1;
}

END {
	if (got_received == 1) {
		for (i = 1; i <= nsystems; i++) {
			if (received[i] == "received")
				first[i]++;
			else if (received[i] != "Duplicate")
				missing[i]++;
		}
		narticles++;
	}
	print narticles " articles";
	for (i = 1; i <= nsystems; i++) 
		printf "%9s %6d first, %6d missing\n", system[i]":", first[i], missing[i];
      }' -