[news.software.b] Articles rejected by C news at ukma

henry@zoo.toronto.edu (Henry Spencer) (06/26/91)

In article <1991Jun26.024210.9578@ccu.umanitoba.ca> mills@ccu.umanitoba.ca (Gary Mills) writes:
>>They do.  We've done a lot of things for standards conformance that we
>>don't particularly like, actually.
>
>Whatever happened to: ``be liberal in what you accept ...''?  I don't want
>to stir the fire, but would it be sufficent if Cnews only _generated_
>conformant articles? ...

Unfortunately, no.  We did more or less that for quite a while.  It caused
grief for a lot of people, because nonconforming articles we passed along
wreaked havoc with other (arguably poorly written) software.

It is important to remember that the Internet robustness principle reads,
precisely (from section 1.2.2 of RFC1122):

	Be liberal in what you accept, and
	conservative in what you send.

Not "generate".  "Send".  When you relay an article, you are sending it.
The site next door sees little difference between articles you originate
and articles you pass on.  To properly implement the robustness principle,
when you relay traffic you *must* make sure it meets the specs, either
by repairing deviations or by rejecting nonconforming articles.

We chose rejection because of repeated bad experience with software that
attempts repairs.  Repair works fine when the deviations are the expected
ones.  Unexpected deviations, however, all too often get "repaired" into
legal but erroneous trash.  When dealing with the sort of volume we see
in news traffic, this can be disastrous for you, your neighbors, or even
the entire net.
-- 
"We're thinking about upgrading from    | Henry Spencer @ U of Toronto Zoology
SunOS 4.1.1 to SunOS 3.5."              |  henry@zoo.toronto.edu  utzoo!henry

kyle@uunet.uu.net (Kyle Jones) (06/27/91)

henry@zoo.toronto.edu (Henry Spencer) writes:
 > To properly implement the robustness principle, when you relay
 > traffic you *must* make sure it meets the specs, either by
 > repairing deviations or by rejecting nonconforming articles.
 > 
 > We chose rejection because of repeated bad experience with software that
 > attempts repairs.  Repair works fine when the deviations are the expected
 > ones.  Unexpected deviations, however, all too often get "repaired" into
 > legal but erroneous trash.  When dealing with the sort of volume we see
 > in news traffic, this can be disastrous for you, your neighbors, or even
 > the entire net.

The flaw in this logic is that you are discarding data that you
cannot reproduce.  This is not robustness.  Robustness is coping
intelligently with bad input, not discarding it.  A local agent
like postnews can afford to be finicky and refuse to send
articles, since the local user can try again.  A relay many hops
removed from the original sender doesn't have that luxury.  If
relaynews can't figure out how to repair an article, then the
very _least_ it should do is put the article into a place where a
smarter program (or a human) can have a crack at it.

There's already a newsgroup for wayward articles: "junk".  Why, you could
even partition it into subgroups, e.g. junk.not-in-sys, junk.not-in-active,
junk.bad-date, junk.no-subject, junk.no-newsgroup, junk.no-message-id,
junk.bad-message-id, junk.fubar ... :-)

henry@zoo.toronto.edu (Henry Spencer) (06/27/91)

In article <1991Jun26.220203.17522@uunet.uu.net> kyle@uunet.uu.net (Kyle Jones) writes:
>henry@zoo.toronto.edu (Henry Spencer) writes:
> > To properly implement the robustness principle, when you relay
> > traffic you *must* make sure it meets the specs, either by
> > repairing deviations or by rejecting nonconforming articles.
> > We chose rejection ...
>
>The flaw in this logic is that you are discarding data that you
>cannot reproduce.  This is not robustness.  Robustness is coping
>intelligently with bad input, not discarding it...

This is a curious definition of robustness, and one that is not supported
by existing practice.  If you read RFC1122, for example, you will find a
number of cases where it is not merely permitted but *demanded* that you
silently discard bad input, because experience has shown that any attempt
to deal intelligently with those cases can easily make things much worse.
That is precisely the problem here.  There are things that are worse than
lost articles, and while it may be desirable to improve the situation,
it is *imperative* not to worsen it.

>If relaynews can't figure out how to repair an article, then the
>very _least_ it should do is put the article into a place where a
>smarter program (or a human) can have a crack at it.

We decided long ago that relaynews will not attempt to repair articles.
Nothing in this dreary and repetitive debate has changed our minds on
that.  We do, however, plan to do something about preserving the evidence
for human investigation.

>There's already a newsgroup for wayward articles: "junk"...

Not quite right, unfortunately, since articles in "junk" still get sent
on to other sites.  (There are good reasons for this, by the way; the
junk-handling policy took *A LOT* of thought and experimenting to get
right.)  Having another pseudogroup for discarded articles is an obvious
possibility, although something has to be done to bound its size and be
selective about what gets included.
-- 
"We're thinking about upgrading from    | Henry Spencer @ U of Toronto Zoology
SunOS 4.1.1 to SunOS 3.5."              |  henry@zoo.toronto.edu  utzoo!henry

mathew@mantis.co.uk (Giving C News a *HUG*) (06/28/91)

henry@zoo.toronto.edu (Henry Spencer) writes:
> It is important to remember that the Internet robustness principle reads,
> precisely (from section 1.2.2 of RFC1122):
> 
> 	Be liberal in what you accept, and
> 	conservative in what you send.
> 
> Not "generate".  "Send".  When you relay an article, you are sending it.
> The site next door sees little difference between articles you originate
> and articles you pass on.  To properly implement the robustness principle,
> when you relay traffic you *must* make sure it meets the specs, either
> by repairing deviations or by rejecting nonconforming articles.

Right.  But "rejecting nonconforming articles" is not being "liberal in what
you accept".

So C News does not obey the robustness principle of RFC1122 as written.

You may or may not be right to disobey RFC1122, but I wish you'd just come
clean and say that that is what you are doing, rather than quoting the RFC in
your responses as if you were actually following it.


mathew

 

chip@chinacat.unicom.com (Chip Rosenthal) (06/28/91)

In article <1991Jun26.230017.21259@zoo.toronto.edu>
	henry@zoo.toronto.edu (Henry Spencer) writes:
>Having another pseudogroup for discarded articles is an obvious
>possibility, although something has to be done to bound its size and be
>selective about what gets included.

Why?  The stuff that is getting chucked was localized by B news.  Note
I'm not claiming that because B news did it, C news should.  Instead,
I'm saying that we've been able to afford the disk space for these
messages in the past, and I think we still can.  I'm very much in
favor of the type of checking C news does, and I have zero qualms
about non-conforming articles being chucked.  I just wish the location
they were chucked into wasn't the bit-bucket.

-- 
Chip Rosenthal  512-482-8260  |  Closed user interfaces for open systems?
Unicom Systems Development    |  No, thank you.
<chip@chinacat.Unicom.COM>    |  Boycott Lotus Development Corp.

rickert@mp.cs.niu.edu (Neil Rickert) (06/28/91)

In article <1991Jun27.221243.778@chinacat.unicom.com> chip@chinacat.unicom.com (Chip Rosenthal) writes:
>In article <1991Jun26.230017.21259@zoo.toronto.edu>
>	henry@zoo.toronto.edu (Henry Spencer) writes:
>>Having another pseudogroup for discarded articles is an obvious
>>possibility, although something has to be done to bound its size and be
>>selective about what gets included.
>
>Why?  The stuff that is getting chucked was localized by B news.  Note
>I'm not claiming that because B news did it, C news should.  Instead,
>I'm saying that we've been able to afford the disk space for these
>messages in the past, and I think we still can.  I'm very much in

  I believe you missed an essential point.  These articles are
INVALID.  Therefore they should not be entered into the history database
since that would prevent a VALID version of the same article.  You might
therefore receive a copy from each of your feeds.  In the past you never
had more than one copy.


-- 
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
  Neil W. Rickert, Computer Science               <rickert@cs.niu.edu>
  Northern Illinois Univ.
  DeKalb, IL 60115                                   +1-815-753-6940

cudep@warwick.ac.uk (Ian Dickinson) (06/28/91)

In article <1991Jun26.220203.17522@uunet.uu.net> kyle@uunet.uu.net (Kyle Jones) writes:
>There's already a newsgroup for wayward articles: "junk".  Why, you could
>even partition it into subgroups, e.g. junk.not-in-sys, junk.not-in-active,
>junk.bad-date, junk.no-subject, junk.no-newsgroup, junk.no-message-id,
>junk.bad-message-id, junk.fubar ... :-)

Hey I like this idea!

I certainly want to see the texts of articles which are rejected.

It would require some special casing though:

junk is currently forwarded (by most sites) onto your feeds.

This still needs to continue for junk.not-in-active,
but not for junk.really-crappy-articles.

And for articles which have frogged message-id, and therefore
wouldn't be in history, some special expiry would have to take place.

Further thoughts?
-- 
\/ato                                                               /'\  /`\
Ian Dickinson                 TED KALDIS FOR PRESIDENT!            /^^^\/^^^\
vato@warwick.ac.uk                                                /TWIN/TEATS\
@c=GB@o=University of Warwick@ou=Computing Services@cn=Ian Dickinson  /       \

rickert@mp.cs.niu.edu (Neil Rickert) (06/28/91)

In article <Xio8412w164w@mantis.co.uk> mathew@mantis.co.uk (Giving C News a *HUG*) writes:
>
>Right.  But "rejecting nonconforming articles" is not being "liberal in what
>you accept".

  C news is very liberal in what it accepts.  All the way to the bit bucket.


-- 
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
  Neil W. Rickert, Computer Science               <rickert@cs.niu.edu>
  Northern Illinois Univ.
  DeKalb, IL 60115                                   +1-815-753-6940

henry@zoo.toronto.edu (Henry Spencer) (06/28/91)

In article <Xio8412w164w@mantis.co.uk> mathew@mantis.co.uk (Giving C News a *HUG*) writes:
>> ... To properly implement the robustness principle,
>> when you relay traffic you *must* make sure it meets the specs, either
>> by repairing deviations or by rejecting nonconforming articles.
>
>Right.  But "rejecting nonconforming articles" is not being "liberal in what
>you accept".
>
>So C News does not obey the robustness principle of RFC1122 as written.

Perfection is not possible.  We are as liberal as we can be about accepting,
given the constraint on sending, the need for speed, and the grave dangers
of repair attempts.  Rejecting sufficiently-nonconforming input does not
contravene 1122; 1122 *demands* that in some cases.
-- 
Lightweight protocols?  TCP/IP *is*     | Henry Spencer @ U of Toronto Zoology
lightweight already; just look at OSI.  |  henry@zoo.toronto.edu  utzoo!henry

ske@pkmab.se (Kristoffer Eriksson) (06/29/91)

In article <1991Jun26.155257.5692@zoo.toronto.edu> henry@zoo.toronto.edu (Henry Spencer) writes:
>	Be liberal in what you accept, and	(1)
>	conservative in what you send.		(2)

>when you relay traffic you *must* make sure it meets the specs, either
>by repairing deviations or by rejecting nonconforming articles.

I don't see how unconditionally rejecting nonconforming articles meets (1)
above. Care to explain? What provisions have you included to achieve (1)?
I've received the impression that C-news concentrates solely on (2). Is
there anything you can point to that shows that you have thought about
(1), too?
-- 
Kristoffer Eriksson, Peridot Konsult AB, Hagagatan 6, S-703 40 Oerebro, Sweden
Phone: +46 19-13 03 60  !  e-mail: ske@pkmab.se
Fax:   +46 19-11 51 03  !  or ...!{uunet,mcsun}!sunic.sunet.se!kullmar!pkmab!ske

xanthian@zorch.SF-Bay.ORG (Kent Paul Dolan) (06/29/91)

 henry@zoo.toronto.edu (Henry Spencer) writes:
> mathew@mantis.co.uk (Giving C News a *HUG*) writes:
>> henry@zoo.toronto.edu (Henry Spencer) writes:

>>> ... To properly implement the robustness
>>> principle, when you relay traffic you *must*
>>> make sure it meets the specs, either by
>>> repairing deviations or by rejecting
>>> nonconforming articles.

>> Right. But "rejecting nonconforming articles" is
>> not being "liberal in what you accept".

>> So C News does not obey the robustness principle
>> of RFC1122 as written.

> Perfection is not possible. We are as liberal as
> we can be about accepting, given the constraint on
> sending, the need for speed, and the grave dangers
> of repair attempts. Rejecting
> sufficiently-nonconforming input does not
> contravene 1122; 1122 *demands* that in some
> cases.

Ah, yes, you've said that several times, each time
being careful to avoid mentioning _which_ cases. So
the question of interest, of course, is: does RFC
1122 demand that you reject _those articles under
controversy_, or is this merely another clever
distraction tactic you're using to try still longer
to weasel out of the apology and admission of guilt
for yet another Cnews design blunder (YACnDB(tm))
that should have been forthcoming long ago?

I know you can do it: you apologized for flooding
the net with old news. I can wait. Patience is one
of my long suits. It comes with the grey hair.

Kent, the man from xanth.
<xanthian@Zorch.SF-Bay.ORG> <xanthian@well.sf.ca.us>
--
Convener, rec.arts.sf.* grand reorganization.