[news.sysadmin] question on bad newsgroups

fr@icdi10.UUCP (Fred Rump from home) (05/28/89)

My news leaves me a daily message about all the bad newsgroups that have come 
in.  How can I check into the validity of this?  Is someting being scrambled
or what?  This is a daily occurrence.

I also get the message that newsgroups are sys but not in active. I can't find 
those suckers for trying. Can I get it to tell me which ones are the culprits?

 *****


From root Sat May 27 00:33:30 1989
To:  news
Date:  Sat May 27 00:33:28 1989
Status:  RO

starting expire of junk,control at Sat May 27 00:25:00 EDT 1989
starting main expire at Sat May 27 00:27:09 EDT 1989
expire:  Unparsable date "31 Dec 69 23:59:59 GMT"
starting cdi expire at Sat May 27 00:30:02 EDT 1989
History expiration time < article expiration time.  Default used.
May 26 01:15	cdin-1	Newsgroups in active, but not sys
May 26 01:25	cdin-1	Newsgroups in active, but not sys
May 26 01:25	cdin-1	Newsgroups in active, but not sys
May 26 01:25	cdin-1	Newsgroups in active, but not sys
May 26 05:11	cdin-1	Newsgroups in active, but not sys
May 27 00:29	local	Unparsable date "31 Dec 69 23:59:59 GMT"

System	Accept	Reject	Junked	Xmit to	Control	% total	% rejct
cdin-1	786	0	30	7	1	99%	0%
icdi20	0	0	0	138	0	0%	0%
[list sites I feed]

Control	Invocations
cancel 1

Received Article Length Problems
System Zero Short Smax Savg Long Lmax Lavg Total % Tot
cdin-1 0 0 0 0 17 4 1 17 2%

Netnews Categories Received
comp 255
rec 241
talk 136
soc 113
alt 85
news 46
misc 27      *
ca 18        *
cdi 12
sci 5        *
ingr 1       *
ba 1         * = not in even in my sys file nor my sending site

Bad Newsgroups Received
a.politic 18               This is the stuff I'm talking about.
a.single 1
alk.politics.gun 5
lt.bb 1
lt.conspirac 1
lt.drug 2
lt.flam 2
lt.rock-n-rol 5
lt.sex.bondag 8
ngr.genera 1
oc.culture.chin 1
oc.culture.japa 1
oc.culture.turkis 2
omp.mail.mus 1
omp.periphs.printer 2
omp.sys.celerit 1
omp.sys.supe 1
omp.theor 1
omp.unix.au 11
omp.unix.cra 1
omp.unix.ultri 12
starting trimlib at Sat May 27 00:33:26 EDT 1989
news pack done at Sat May 27 00:33:27 EDT 1989


#Anybody give me a hint as to what is happening here? seems like front and back
#characters are cut off.

*************************************************
Cron:  The previous message is the standard output
and standard error of one of your cron commands.


Thanks
Fred Rump
(a slow learner, but willing to)
icdi10!fr@cdin-1.uu.net


-- 
This is my house. My castle will get started right after I finish with news. 
26 Warren St.                          ...{dsinc bpa uunet}!cdin-1!icdi10!fr
Beverly, NJ 08010 or INTERNET:  fred@cdin-1.uu.net or icdi10!fr@icdi10.uu.net
609-386-6846          "Freude... Alle Menschen werden Brueder..."  -  Schiller

scott@dtscp1.UUCP (Scott Barman) (05/28/89)

In article <185@icdi10.UUCP> fr@icdi10.UUCP (Fred Rump from home) writes:
>My news leaves me a daily message about all the bad newsgroups that have come 
>in.  How can I check into the validity of this?  Is someting being scrambled
>or what?  This is a daily occurrence.

I'm not going to talk about this, but something related that I noticed
in your article and lately in my mail messages.

>May 27 00:29	local	Unparsable date "31 Dec 69 23:59:59 GMT"

Now for nearly a month, I have been getting mail that says this.  Now
please do not tell me it is the same articles hanging around since
we have a 3-day expire on comp (7 days on sources), and 2 days on
everything else--that is if expire does not get the picture that
we are nearly 20 years past this date :-)

Where do these come from?  Does the SysAdmin for the site producing
these know about them?  How does one find out who, what, where, etc.
concerning articles about these dates?

Inquiring minds... well... are just curious!	:-)

-- 
scott barman
{gatech, emory}!dtscp1!scott

jjb@wsu-cs.uucp (J. Brewster) (05/29/89)

In article <730@dtscp1.UUCP> scott@dtscp1.UUCP (Scott Barman) writes:
->May 27 00:29	local	Unparsable date "31 Dec 69 23:59:59 GMT"
-
-Now for nearly a month, I have been getting mail that says this.  Now
-please do not tell me it is the same articles hanging around since
-we have a 3-day expire on comp (7 days on sources), and 2 days on
-everything else--that is if expire does not get the picture that
-we are nearly 20 years past this date :-)
-
-Where do these come from?  Does the SysAdmin for the site producing
-these know about them?  How does one find out who, what, where, etc.
-concerning articles about these dates?
-
-Inquiring minds... well... are just curious!	:-)
-
I've seen this too, and don't know exactly where it comes from, but
the date and time correspond to -1 sec...
-- 
J. Brewster               | This is *not* a test:
jjb@cs.wayne.edu          | this is a real attitude.
...!mailrus!wsu-cs!jjb    | 

news@ivucsb.sba.ca.us (Todd Day) (05/30/89)

In article <185@icdi10.uucp> fr@icdi10.UUCP (Fred Rump from home) writes:
~Bad Newsgroups Received
~a.politic 18               This is the stuff I'm talking about.
~a.single 1
~alk.politics.gun 5
~lt.bb 1
~lt.conspirac 1

I'm getting the same problem here, but I can't seem to trace down the
culprit.  I can't even find the messages that have this problem.  Anybody
have any clues?

-- 

Todd Day | todd@ivucsb.sba.ca.us | ivucsb!todd@anise.acc.com
"All theory, no practice.  That's the story of my life."

news@ivucsb.sba.ca.us (Todd Day) (05/30/89)

In article <730@dtscp1.uucp> scott@dtscp1.UUCP (Scott Barman) writes:
~>May 27 00:29	local	Unparsable date "31 Dec 69 23:59:59 GMT"
~Where do these come from?

I'm not sure, but here's my theory.  Someone's machine (probably a
Xenix system :-) is returning -1 from the time(2) function.  There
is probably no error checking done by inews (who would ever suspect
that time(2) would return with an error?), and it passes this to
ctime(3c).  ctime(3c) does no error checking, either, and faithfully
converts the -1 to one second before the UNIX big-bang.  The funny
thing here is, I'll bet that the date parsing function DOES parse
the bogus date properly, and returns the -1, which is interpreted
as an error by inews!

Actually, since your logfile says "local", maybe it is YOUR machine
that is producing the bogus date!


-- 

Todd Day | todd@ivucsb.sba.ca.us | ivucsb!todd@anise.acc.com
"All theory, no practice.  That's the story of my life."

pmb@swituc.UUCP (Pat Berry) (05/30/89)

In article <804@ivucsb.sba.ca.us>, news@ivucsb.sba.ca.us (Todd Day) writes:
> ~Bad Newsgroups Received
> ~a.politic 18               This is the stuff I'm talking about.
> ~a.single 1
> ~alk.politics.gun 5
> ~lt.bb 1
> ~lt.conspirac 1
We are getting the same thing to the tune of 10 to 15 per day.  It appears
to be a handshaking problem, clipping off the first few chars of the group
names.

fr@icdi10.UUCP (Fred Rump from home) (05/30/89)

In article <804@ivucsb.sba.ca.us> news@ivucsb.UUCP (Todd Day) writes:
>In article <185@icdi10.uucp> fr@icdi10.UUCP (Fred Rump from home) writes:
>~Bad Newsgroups Received
>~a.politic 18               This is the stuff I'm talking about.
>~a.single 1
>~alk.politics.gun 5
>~lt.bb 1
>~lt.conspirac 1
>
>I'm getting the same problem here, but I can't seem to trace down the
>culprit.  I can't even find the messages that have this problem.  Anybody
>have any clues?

I don't know enough about awk to find the problem but I think it is created in 
Eric Fair's awk script originally written in May, 1984 to give some news stats.

I don't want to post the whole script but here is the part that prints:


# USAGE:  awk -f report_awk /usr/lib/news/log
#
# AWK script which eats netnews log files and produces a summary of USENET
# traffic and errors over the period of time that the log was collected.
#
# August 31, 1986
#
# Erik E.  Fair <dual!fair>
# Original Author, May 22, 1984
#
# Brad Eacker <onyx!brad>
# Modified to simplify the record processing and to sort the output.
#
# Erik E.  Fair <dual!fair>
# Modifed to provide information about control messages.
#
# Erik E.  Fair <dual!fair>
# Bug in system name extraction fixed.  It was assumed that the forth field
# (system name) always had a dot.  local is one that doesn't.  Some others
# (including 2.9 sites) don't either.
#
# Earl Wallace <pesnta!earlw>
# The "sent" field was changed from $5 to $6 in 2.10.2 (beta)
# named "newstats" and called with no arguments.
#
# Erik E.  Fair <dual!fair>
# Remove support for 2.10.1, revise for 2.10.2 to provide information
# about junked articles, garbled articles, and bad newsgroups
#
# Erik E.  Fair <ucbvax!fair>
# Minor bug fix to bad newsgroup reporting, also now counting ``old''
# articles as junked, with counter for number that are `old'.
#
# Erik E.  Fair <ucbvax!fair>
# Fix up the domain & local hosts support
#
# Erik E.  Fair <ucbvax!fair>
# Fix up the counting of gatewayed material, add counting of "linecount"
# problems.  Additional cleanup to make things faster.

Is it in here?
Perhaps some kind soul would look at the whole script and fix this problem?

#
# selection sort
			max = 0;
			for( j in newsgcnt ) {
				if (newsgcnt[j] > max) {
					i = j;
					max = newsgcnt[j];
				}
			}
			if (max == 0) break;
			printf(fmt, i, newsgcnt[i]);
			newsgcnt[i] = 0;
		}
	}

	if (badgrp) {
		printf("\nBad Newsgroups Received\n");
		l = 0;
		for( i in badng ) {
			if (l < length(i)) l = length(i);
		}
		fmt = sprintf("%%-%ds %%5d\n", l);
		for( ; ; ) {
# selection sort
			i = invalid;
			for( j in badng ) {
				if (badng[j] > 0 && j < i) i = j;
			}
			if (i == invalid) break;
			printf(fmt, i, badng[i]);
			badng[i] = 0;
		}
	}
}






-- 
This is my house. My castle will get started right after I finish with news. 
26 Warren St.                          ...{dsinc bpa uunet}!cdin-1!icdi10!fr
Beverly, NJ 08010 or INTERNET:  fred@cdin-1.uu.net or icdi10!fr@icdi10.uu.net
609-386-6846          "Freude... Alle Menschen werden Brueder..."  -  Schiller

wedgingt@udenva.cair.du.edu (Will Edgington) (05/30/89)

In article <804@ivucsb.sba.ca.us> news@ivucsb.UUCP (Todd Day) writes:
]In article <185@icdi10.uucp> fr@icdi10.UUCP (Fred Rump from home) writes:
]~Bad Newsgroups Received
]~a.politic 18               This is the stuff I'm talking about.
]~a.single 1
]~alk.politics.gun 5
]~lt.bb 1
]~lt.conspirac 1
]
]I'm getting the same problem here, but I can't seem to trace down the
]culprit.  I can't even find the messages that have this problem.  Anybody
]have any clues?

I missed the original article (way behind on news as usual), but I presume
you're both looking at the output of a variant of Eric Fair's newslog.awk.
Certainly, I had the same problem for a while when using one I got from
UC/Boulder.  The bug is there in the awk script, not the news software
itself.  You see, under 2.10.2 (?) and earlier versions of B news, the
error message about "bad newsgroup" prints the newsgroup within quotes
('ba.politics', for example) while B news 2.11 doesn't (ba.politics).
Newslog.awk, knowing 2.10.2's method, strips the first and last character
of that field off without checking them to be sure they're quotes.  The
fix is easy, of course:  take off any quotes, but nothing else.
-- 
Will Edgington, Computing and Information Resources, University of Denver
System Administrator for udenva (== dueos), dutyche, duorion, dunike, ...
{{hplabs,uunet}!ncar,ucbvax!nbires,boulder,cires}!udenva!wedgingt,
wedgingt@nike.cair.du.edu, wedgingt@udenva.cair.du.edu, ... (Internet)
WEDGINGT@DUCAIR (BITNET), wedgingt@ccndu (CSN/CCN), ...
COMING SOON: wedgingt@du.edu (all nets)|"No two addresses are the same ..."
2020 S. Race BA469, Denver CO 80208    | +1 303 871 2081

fr@icdi10.UUCP (Fred Rump from home) (05/30/89)

In article <805@ivucsb.sba.ca.us> news@ivucsb.UUCP (Todd Day) writes:
>In article <730@dtscp1.uucp> scott@dtscp1.UUCP (Scott Barman) writes:
>~>May 27 00:29	local	Unparsable date "31 Dec 69 23:59:59 GMT"
>~Where do these come from?
>
>I'm not sure, but here's my theory.  Someone's machine (probably a
>Xenix system :-) is returning -1 from the time(2) function.  There
>is probably no error checking done by inews (who would ever suspect
>that time(2) would return with an error?), and it passes this to
>ctime(3c).  ctime(3c) does no error checking, either, and faithfully
>converts the -1 to one second before the UNIX big-bang.  
>Actually, since your logfile says "local", maybe it is YOUR machine
>that is producing the bogus date!

I have no doubt that it is. But it's doing the same thing on a bunch of other 
machines too. So it's not unique to the world and must be something that is 
fixable.

But how? 

And yes this is Xenix (386) 2.3.2

Maybe it'll just go away? It's not been around since I asked about it. Like 
magic I gave away it's secret and it went into blissful sleep somewhere in the 
past before the last second of 1969?
Fred
icdi10!fr@cdin-1.uu.net
-- 
This is my house. My castle will get started right after I finish with news. 
26 Warren St.                          ...{dsinc bpa uunet}!cdin-1!icdi10!fr
Beverly, NJ 08010 or INTERNET:  fred@cdin-1.uu.net or icdi10!fr@icdi10.uu.net
609-386-6846          "Freude... Alle Menschen werden Brueder..."  -  Schiller

jay@banzai.UUCP (Jay Schuster) (05/31/89)

In article <137@swituc.UUCP> pmb@swituc.UUCP (Pat Berry) writes:
>In article <804@ivucsb.sba.ca.us>, news@ivucsb.sba.ca.us (Todd Day) writes:
>> ~Bad Newsgroups Received
>> ~a.politic 18               This is the stuff I'm talking about.
   ba.politics
>> ~a.single 1
   ba.singles
>> ~alk.politics.gun 5
   talk.politics.guns
>> ~lt.bb 1
   alt.bbs
>> ~lt.conspirac 1
   alt.conspiracy
>We are getting the same thing to the tune of 10 to 15 per day.  It appears
>to be a handshaking problem, clipping off the first few chars of the group
>names.

It's due to a bug in your report.awk script that you use to generate that
report.  It processes the lines and drops the first and last characters of
the newsgroup name.  The script comes from Eric E. Fair.  It was reported
as having this bug a year or two ago and there was a fix at that time.

Now, as to why you even get those messages:  If something is crossposted
to a group you do get and a group you don't get, you get one of those
messages for the groups you don't get.
-- 
Jay Schuster			uunet!uvm-gen!banzai!jay, attmail!banzai!jay
The People's Computer Company	`Revolutionary Programming'

lae@ernest.ti.com (Alan Edmonds) (05/31/89)

In article <137@swituc.UUCP> pmb@swituc.UUCP (Pat Berry) writes:
>In article <804@ivucsb.sba.ca.us>, news@ivucsb.sba.ca.us (Todd Day) writes:
>> ~Bad Newsgroups Received
>> ~a.politic 18               This is the stuff I'm talking about.
>> ~a.single 1
>> ~alk.politics.gun 5
>> ~lt.bb 1
>> ~lt.conspirac 1
>We are getting the same thing to the tune of 10 to 15 per day.  It appears
>to be a handshaking problem, clipping off the first few chars of the group
>names.

Actually, there is a bug in the script I got as part of the news sources.
Enclosed is a context-diff of the fix.  I thought everyone had already 
fixed this, so I kept the fix to myself.  I also added a Subject: line
header.

Enjoy.
====================== cut here =================
*** report_awk
--- /usr/lib/news/report_awk
**************
*** 60,66
  #  Fix up the counting of gatewayed material, add counting of "linecount"
  #  problems. Additional cleanup to make things faster.
  #
! BEGIN{
  #
  #	this is the prefix that your site uses in hostnames to identify your
  #	hosts (e.g. ucbarpa, ucbvax, su-score, mit-mc, mit-ai)
--- 37,45 -----
  #  Fix up the counting of gatewayed material, add counting of "linecount"
  #  problems. Additional cleanup to make things faster.
  #
! #  Alan Edmonds (alan@ernest.ti.com)
! #  Add subject line to output message
! #  fix length problem with bad newsgroups section
  #
  BEGIN{
  #
**************
*** 62,67
  #
  BEGIN{
  #
  #	this is the prefix that your site uses in hostnames to identify your
  #	hosts (e.g. ucbarpa, ucbvax, su-score, mit-mc, mit-ai)
  #	You will probably want to change (or add to) the following line
--- 41,52 -----
  #  Add subject line to output message
  #  fix length problem with bad newsgroups section
  #
+ BEGIN{
+ #
+ #	print subject line for mail
+ #
+ 	printf("Subject: USENET Activity Log\n\n");
+ #
  #	this is the prefix that your site uses in hostnames to identify your
  #	hosts (e.g. ucbarpa, ucbvax, su-score, mit-mc, mit-ai)
  #	You will probably want to change (or add to) the following line
**************
*** 219,226
  $6 == "valid"		{ junked[sys]++; next }
  $6 == "too"		{ junked[sys]++; old++; next }
  $5 == "Unknown"		{
! 	x = length($7) - 2;
! 	ng = substr($7, 2, x);
  	badng[ng]++;
  	badgrp++;
  	next;
--- 204,213 -----
  $6 == "valid"		{ junked[sys]++; next }
  $6 == "too"		{ junked[sys]++; old++; next }
  $5 == "Unknown"		{
! #	x = length($7) - 2;
! 	x = length($7);
! #	ng = substr($7, 2, x);
! 	ng = substr($7, 1, x);
  	badng[ng]++;
  	badgrp++;
  	next;
======================== end =================

usenet@statware.UUCP (Usenet News) (05/31/89)

In article <137@swituc.UUCP> pmb@swituc.UUCP (Pat Berry) writes:
>In article <804@ivucsb.sba.ca.us>, news@ivucsb.sba.ca.us (Todd Day) writes:
>> ~Bad Newsgroups Received
>> ~a.politic 18               This is the stuff I'm talking about.
>> ~a.single 1
>> ~alk.politics.gun 5
>> ~lt.bb 1
>> ~lt.conspirac 1
>We are getting the same thing to the tune of 10 to 15 per day.  It appears
>to be a handshaking problem, clipping off the first few chars of the group
>names.

  We also get the same thing.  I agree with Berry, as an earlier message
indicates "short articles received".
  Look for these bad articles in spool/junk.  I'm using the mvjunk script
posted by David Wright (uunet.uu.net!ukc!std.stc.co.uk!dww), with some later
fixes, to move these articles to the correct newsgroup based on the
NEWSGROUP line.


-- 
Mathieu Federspiel                  mcf%statware.uucp@cs.orst.edu
Statware                            {hp-pcd,tektronix}!orstcs!statware!mcf
260 SW Madison Avenue, Suite 109    503-753-5382
Corvallis  OR  97333  USA

hartman@abacab.UUCP (Mark A. Hartman) (06/02/89)

In article <137@swituc.UUCP>, pmb@swituc.UUCP (Pat Berry) writes:
> In article <804@ivucsb.sba.ca.us>, news@ivucsb.sba.ca.us (Todd Day) writes:
> > ~Bad Newsgroups Received
> > ~a.politic 18               This is the stuff I'm talking about.
> > ~a.single 1
> > ~alk.politics.gun 5
> > ~lt.bb 1
> > ~lt.conspirac 1
> We are getting the same thing to the tune of 10 to 15 per day.  It appears
> to be a handshaking problem, clipping off the first few chars of the group
> names.

Try looking in the "log" file in your news directory so see if the full
group names are there.  If so, just make the following two line change
to report.awk to clean up the format a bit:

< 	x = length($7) - 2;
< 	ng = substr($7, 2, x);
---
> 	x = length($7);
> 	ng = substr($7, 1, x);
-- 
Mark Hartman			{att,obdient}!abacab!hartman