[net.news] Program for news feed statistics

mkg@whuxlb.UUCP (06/20/83)

#N:whuxlb:7700006:000:1836
whuxlb!mkg    Jun 20 00:36:00 1983

Here is a nifty program that can be used to check on the reliability
of your netnews feeds.  It will tell you who got you articles first
and who didn't get you some articles.  

For example, whuxlb's latest stats are:
804 articles
         floyd:    429 first,     46 missing
    gummo.UUCP:    375 first,    110 missing

I originally posted this program for 2.9 news but since the format
of the log file changed in 2.10, here is an updated version of
that program.  Oh yes, our ncut program works like cut(1) but gives
you the fields in the order you ask for them (rather than in ascending
order).  If your cut can't do this, you'll have to swap the fields around
with sed.
   Marsh Gosnell  BTL Whippany  (201) 386-7095  whuxlb!mkg


egrep "received|Duplicate" /usr/lib/news/log* | \
	sed -e 's/	/ /g' -e 's/article //'  | \
	ncut -f6,5,4 -d' ' | \
	sort -u | \
	awk '
BEGIN { nsystems = 0; narticles = 0; got_received = 0 }

$1 != last {
	if (got_received == 1) {
		for (i = 1; i <= nsystems; i++) {
			if (received[i] == "received")
				first[i]++;
			else if (received[i] != "Duplicate")
				missing[i]++;
		}
		narticles++;
	}
	for (i = 1; i <= nsystems; i++)
		received[i] = "";
	got_received = 0;
	last = $1;
}

$1 == last {
	for (i = 1; i <= nsystems; i++)
		if (system[i] == $3)
			break;
	if (i > nsystems) {
		nsystems++;
		first[nsystems] = 0;
		missing[nsystems] = narticles;
		system[nsystems] = $3;
	}

	received[i] = $2;

	if ($2 == "received")
		got_received = 1;
}

END {
	if (got_received == 1) {
		for (i = 1; i <= nsystems; i++) {
			if (received[i] == "received")
				first[i]++;
			else if (received[i] != "Duplicate")
				missing[i]++;
		}
		narticles++;
	}
	print narticles " articles";
	for (i = 1; i <= nsystems; i++) 
		printf "%15s %6d first, %6d missing\n", system[i]":", first[i], missing[i];
      }' -

john@genrad.UUCP (John Nelson) (06/20/83)

Nevermind ncut(1), what the heck is cut(1)?  I hate when people submit
shell scripts that use utilities that are not on out system.  Could
some kind person out there PLEASE tell me what cut(1) is SUPPOSED to
do?

chris@grkermit.UUCP (Chris Hibbert) (06/20/83)

ncut -f6,5,4 -d' '
	can be simply mimiced by 
awk "{ print \$6 , \$5 , \$4 }"
	in the reliability script.  

I've never seen that program before either, but I would probably have
used awk even if I had it.  

I assume the "-d' '" set the field delimiter character to be a space.

mkg@whuxlb.UUCP (06/21/83)

#R:whuxlb:7700006:whuxlb:7700007:000:565
whuxlb!mkg    Jun 21 00:02:00 1983

For those of you who don't have cut or ncut, here is a revised sed
script that performs the same function.  Although it reduces the
pipeline by one process, it takes slightly longer to run--probably
because of the complexity of the script.  Or the suggestion to
replace ncut with awk '{ print $6,$5,$4 }' will also work.
   Marsh Gosnell  BTL Whippany  (201) 386-7095  whuxlb!mkg


egrep "received|Duplicate" /usr/lib/news/log* | \
	sed -e 's/	/ /g' -e 's/article //' \
	 -e 's/[^ ]* [^ ]* [^ ]* \([^ ]*\) \([^ ]*\) \([^ ]*\) .*/\3 \2 \1/' | \
	sort -u | \
	etc...