[news.software.b] NNTPD hates Message-IDs with TWO '@'s in them.

ronald@robobar.co.uk (Ronald S H Khoo) (06/11/91)

sob@tmc.edu (Stan Barber) writes:

> One might argue that if Cnews throws away articles with bad dates, why doesn't
> it throw away articles with bad Message-ids?

It should.  I'd hazard a guess that the reason it doesn't already do so
is simply because Zmailer doesn't choke on two '@'s in a message-id.
But that's a guess.  If you can show that this particular violation of
the RFC's breaks an existing installed software base, I would urge
the C News maintainers to drop these too.

And while I'm here, I'm going to mount my soapbox.  .nntp readers can
leave here.  Followups are redirected to .software.b.

In this current storm, mostly stirred by Sean "mathew" Murphy @
mantis.co.uk, people seem to have forgotten that all this article
dropping isn't Henry's idea anyway.  In fact, Henry's well on record for
opposing the "guerilla tactics" of deliberately being cruel to force people
to get their software up to scratch.

I have to disagree with that point of view.  The truth of the matter is that
only squeaking wheels get fixed.  Henry knows this, he's just too soft. (:-)

At the end of the day, networking is about getting stuff implemented from
all different directions to talk to each other.  You can't do this without
adhering to standards.  While software should be lenient in accepting
non-conformant rubbish, the other half of that maxim says that it must
not let that stuff escape out.  That's the "conservative in what you generate"
half.  Current C News software does this.  The error reporting may not
be all things to all men, but it's good enough.  I don't really want to
spend lots of cpu time picking up the the rubbish left behind by non-conformant
software -- that would be ridiculous.  Why should I suffer for their
errors ?

You can't network in a vacuum either.  To those who complained about
Henry's warnings being not sufficiently loud, I have only this to say:
If you're maintaining or writing software that interoperates with B News,
you're problems have only just started if you don't subscribe to
news.software.b.  This is (IMHO) the paramount reason for not splitting
off news.software.c.  Maintainers of systems, and implementers of the
software need to know what each other are doing, hence both need
to read news.software.b and news.admin.  Both groups have a reasonably
good signal to noise ratio if you put /@mantis.co.uk/h:j into your
kill file.

All the current furore serves to do is waste the precious time of
Henry Spencer.  Many of us are waiting for time when he can put down
this C News project and go back to the things he's left behind --
like the superfast string library.

I wish I could put that above-mentioned line into Henry's Kill file for him.

Anyway, it's time for bed.  flames > /dev/null please.
-- 
Ronald Khoo <ronald@robobar.co.uk> +44 81 991 1142 (O) +44 71 229 7741 (H)

aipdc@castle.ed.ac.uk (Paul Crowley) (06/11/91)

In article <1991Jun10.202510.6405@robobar.co.uk> ronald@robobar.co.uk (Ronald S H Khoo) writes:
>Anyway, it's time for bed.  flames > /dev/null please.

Ah, yes, the magic incantation.  You're going to reiterate all the stale
old arguments that we've already heard, flame Mathew several times, talk
various kinds of garbage about error reporting, and generally stir the
shit without adding anything constructive, but you don't want to be
flamed for it.

Of course we won't flame you, you poor dear, you'd probably faint.
                                         ____
\/ o\ Paul Crowley aipdc@castle.ed.ac.uk \  /
/\__/ Part straight. Part gay. All queer. \/
"I say we kill him and eat his brain."
"That's not the solution to _every_ problem, you know!" -- Rudy Rucker

eggert@twinsun.com (Paul Eggert) (06/16/91)

sob@tmc.edu (Stan Barber) writes:

>As I read RFC1036, it follows RFC822/1123 for all headers defined in
>those documents and sets standards for those headers not defined in
>RFC822/1123. To me, that means that a Date field that does not conform to
>RFC823/1123, it is illegal. You and Geoff seem to agree with that. Now,
>we have Messages IDs that don't conform and you say that that is ok
>despite what RFC822/1123 say.

C News needn't discard _incoming_ articles with nonconforming
Message-IDs; it is allowed to, but it doesn't have to.  However, it
must _generate_ articles with conforming Message-IDs.

Unfortunately, C News's policy of passing through headers unaltered
means that, if it wants to conform to the RFCs, it must check that
incoming headers strictly conform to the RFCs before passing them
through to the next news host.  Instead of ``be generous in what you
accept and strict in what you produce'', C News must be strict in what
it accepts because it mustn't pass through bad headers.

In practice, C News isn't this strict, accepts and passes through
mistakes like Message-IDs with two `@'s, and therefore doesn't conform
strictly with the RFCs.  I suspect there are several reasons for this:

	The RFCs are too complicated.

	Making C News strict would cause even more of an uproar than
	the recently added loose header checking.

	Being strict would make C News a little less efficient.

	Which is more important, obeying the RFCs precisely, or
	transporting news effectively?

The lessons for NNTPD should be clear (:-).

sob@tmc.edu (Stan Barber) (06/17/91)

In article <1991Jun15.212751.1558@twinsun.com> eggert@twinsun.com (Paul Eggert) writes:
>C News needn't discard _incoming_ articles with nonconforming
>Message-IDs; it is allowed to, but it doesn't have to.  However, it
>must _generate_ articles with conforming Message-IDs.
This is inconsistant. If CNEWS is discarding articles with bad dates
(and those articles are being discarded during RELAY, not when first 
posted), then it should do that for all headers.

It has already been stated by others that CNEWS will gladly generate bad
Messages-IDs if improperly installed. I think most any news software
(transports or posters) can do this. There must be some software that
checks to be sure that the Message-ID is valid. Some have suggested that
inews is the right place (when the article is first posted). I agree with this.
However, not every news transport (anu-news, various MS-DOS systems,
ATARI systems, etc) may be totally compliant (witness the messages involving
"Give CNEWS a *HUG*"/"CNEWS MUST DIE"). This means more checking is 
necessary.





-- 
Stan           internet: sob@bcm.tmc.edu         Director, Networking 
Olan           uucp: rutgers!bcm!sob             and Systems Support
Barber         Opinions expressed are only mine. Baylor College of Medicine

eggert@twinsun.com (Paul Eggert) (06/17/91)

sob@tmc.edu (Stan Barber) writes:

>This is inconsistant. If CNEWS is discarding articles with bad dates ...
>then it should do that for all headers.

Actually, C News is consistent in not checking for complete RFC conformance.
It doesn't check _dates_ completely either; e.g. it accepts nonconforming dates
like `Sonntag, 16 Jun 91 22:47:05 GMT'.

It would be tricky and a little expensive for C News to check for complete
conformance to the RFCs, because the rules are so convoluted.  C News does the
checking that's cheap and easy, and doesn't bother with the rest.  Alas,
this means C News can output a non-RFC-conforming article that was originally
posted from another host.

henry@zoo.toronto.edu (Henry Spencer) (06/18/91)

In article <6020@gazette.bcm.tmc.edu> sob@tmc.edu (Stan Barber) writes:
>>C News needn't discard _incoming_ articles with nonconforming
>>Message-IDs; it is allowed to, but it doesn't have to.  However, it
>>must _generate_ articles with conforming Message-IDs.
>This is inconsistant. If CNEWS is discarding articles with bad dates
>(and those articles are being discarded during RELAY, not when first 
>posted), then it should do that for all headers.

C News doesn't do 100% RFC822/1036 header enforcement because that is
complex and time-consuming and unnecessary.  We generally enforce only
those restrictions that appear to be necessary for some specific reason.
(In the case of dates, it is essential to have a parsable date to avoid
recirculation of stale news.)  We are somewhat reluctant to tighten up
checking just because somebody else's software is breaking -- we tend
to feel that this is the breakee's problem -- although we have done so
on occasion.
-- 
"We're thinking about upgrading from    | Henry Spencer @ U of Toronto Zoology
SunOS 4.1.1 to SunOS 3.5."              |  henry@zoo.toronto.edu  utzoo!henry

sob@tmc.edu (Stan Barber) (06/18/91)

In article <1991Jun17.191642.27639@zoo.toronto.edu> henry@zoo.toronto.edu (Henry Spencer) writes:
>C News doesn't do 100% RFC822/1036 header enforcement because that is
>complex and time-consuming and unnecessary.  We generally enforce only
>those restrictions that appear to be necessary for some specific reason.
>(In the case of dates, it is essential to have a parsable date to avoid
>recirculation of stale news.)  We are somewhat reluctant to tighten up
>checking just because somebody else's software is breaking -- we tend
>to feel that this is the breakee's problem -- although we have done so
>on occasion.

I see.

I guess this means that badly installed CNEWS sites are no cause for 
concern.

It is interesting to hear that a CNEWS author considers to be irrelevant
if implementing the standard is too hard or too complex. 

However, I do believe we now have seen a case where it is necessary. 
There is news posting software out there that generates bad message-ids.
This should be stopped by the news transport software.

Otherwise, USENET will eventually stop transporting some news....Ummm.
Actually, I guess this has already happened.


-- 
Stan           internet: sob@bcm.tmc.edu         Director, Networking 
Olan           uucp: rutgers!bcm!sob             and Systems Support
Barber         Opinions expressed are only mine. Baylor College of Medicine

henry@zoo.toronto.edu (Henry Spencer) (06/19/91)

In article <6038@gazette.bcm.tmc.edu> sob@tmc.edu (Stan Barber) writes:
>>C News doesn't do 100% RFC822/1036 header enforcement because that is
>>complex and time-consuming and unnecessary.  We generally enforce only
>>those restrictions that appear to be necessary for some specific reason.
>...
>I guess this means that badly installed CNEWS sites are no cause for 
>concern.

I'd be interested to know how you reached that conclusion from what I said.

I plan to tighten up build's checking on site names and such, and fussier
checking of message IDs will at least be considered.

>It is interesting to hear that a CNEWS author considers to be irrelevant
>if implementing the standard is too hard or too complex. 

If you have a fast, small, easy-to-use parser for full RFC822, Geoff would
love to hear from you.  We do not consider the standard "irrelevant", we
consider it "expensive", and prefer to avoid implementing features that
contribute nothing to the functioning of a news system except slowness.
This does mean that we occasionally have cause to revise our opinion of
some feature, when it turns out that its lack causes problems.

>However, I do believe we now have seen a case where it is necessary. 
>There is news posting software out there that generates bad message-ids.
>This should be stopped by the news transport software.

I am inclined to agree, although I would like to see a concise description
of *exactly* what restrictions should be enforced, since full 822 parsing
is ridiculously complex and expensive and appears to be overkill.  If it's
just the absence of multiple @s, that ought to be feasible.
-- 
"We're thinking about upgrading from    | Henry Spencer @ U of Toronto Zoology
SunOS 4.1.1 to SunOS 3.5."              |  henry@zoo.toronto.edu  utzoo!henry

sob@lib.tmc.edu (Stan Barber) (06/19/91)

In article <1991Jun18.191531.10098@zoo.toronto.edu> henry@zoo.toronto.edu (Henry Spencer) writes:
>I plan to tighten up build's checking on site names and such, and fussier
>checking of message IDs will at least be considered.
Excellent. In the meantime, I have asked a group of NNTP managers to test
a fix for NNTP to cope with the problem. I hope that means that everyone
wins.
>This does mean that we occasionally have cause to revise our opinion of
>some feature, when it turns out that its lack causes problems.
Excellent. I am happy to see that you are willing to reexamine your opinions.
I frequently came away with a belief that you were unwilling to consider
the possibility of being incorrect on some issue.
>I would like to see a concise description of *exactly* what restrictions
>should be enforced, since full 822 parsing is ridiculously complex and
>expensive and appears to be overkill.  If it's just the absence of multiple
>@s, that ought to be feasible.
Perhaps this should be the subject of a revision of RFC1036 you proposed.
At the very least, only one @ should be allowed. How would you like to
proceed?

henry@zoo.toronto.edu (Henry Spencer) (06/19/91)

In article <5128@lib.tmc.edu> sob@lib.tmc.edu (Stan Barber) writes:
>>This does mean that we occasionally have cause to revise our opinion of
>>some feature, when it turns out that its lack causes problems.
>Excellent. I am happy to see that you are willing to reexamine your opinions.
>I frequently came away with a belief that you were unwilling to consider
>the possibility of being incorrect on some issue.

We do have strongly held opinions on a lot of things, but we do change them
when the evidence clearly points the other way.  We still don't like having
to parse dates, for example, but it appears to be necessary -- our original
belief that longer histories would suffice to deal with recirculation of
stale news was incorrect.  (It's still the technically-right solution,
actually, but the length of history needed is impractical.)

>>I would like to see a concise description of *exactly* what restrictions
>>should be enforced, since full 822 parsing is ridiculously complex and
>>expensive and appears to be overkill...
>Perhaps this should be the subject of a revision of RFC1036 you proposed.
>At the very least, only one @ should be allowed. How would you like to
>proceed?

By taking a long vacation. :-)  Getting the RFC revised is likely to be a
fairly lengthy process, although it is becoming clear that we need to tackle
it soon.  I'm going to pursue that when I get back from an imminent short
vacation.

Meanwhile, I *think* limitation to a single @ is the only urgent item,
and this is being looked into.
-- 
"We're thinking about upgrading from    | Henry Spencer @ U of Toronto Zoology
SunOS 4.1.1 to SunOS 3.5."              |  henry@zoo.toronto.edu  utzoo!henry

lear@turbo.bio.net (Eliot) (06/20/91)

Henry,

It was my intent to take on updating of the News article description
as the next task for our working group.  I don't forsee that happening
now until September or October.
-- 
Eliot Lear
[lear@turbo.bio.net]

henry@zoo.toronto.edu (Henry Spencer) (06/25/91)

In article <Jun.19.17.06.43.1991.1890@turbo.bio.net> lear@turbo.bio.net (Eliot) writes:
>It was my intent to take on updating of the News article description
>as the next task for our working group.  I don't forsee that happening
>now until September or October.

That sounds reasonable to me.  Heaven knows I'm in no hurry to take it on;
I just wanted to make sure it got done.  I'll probably be able to supply
a revised draft as a starting point, but I'm just as happy if somebody else
handles the IETF interfacing.
-- 
"We're thinking about upgrading from    | Henry Spencer @ U of Toronto Zoology
SunOS 4.1.1 to SunOS 3.5."              |  henry@zoo.toronto.edu  utzoo!henry