[news.software.nntp] Duplicate article reception with NNTP and C News

sanders@cs.ubc.ca (Michael Sanderson) (11/03/90)

I am curious about what people have done regarding the "simultaneous"
acceptance of a particular message from multiple NNTP feeds in a C News
environment.  By "simultaneous" I mean receiving IHAVEs for the same message
in different instances of a nntpd process before processing the batches.

For example,

    nntpd A receives an IHAVE <123@foo.bar>, sends an OK, gets the
        article and continues receiving more articles (doesn't process
        the batch yet).

    nntpd B (from a different site feed) receives an IHAVE <123@foo.bar>,
        sends an OK since <123@foo.bar> isn't found in the history files),
        and get the article as well.

    nntpd A starts up relaynews which processes article <123@foo.bar>

    nntpd B starts up relaynews which junks article <123@foo.bar> since it is
        now in the history file.

I realize that, in the end, there won't be multiple copies of the message in
the news spool directory since relaynews junks duplicates, so this isn't all
that big a deal.  However, if you have multiple NNTP feeds which are
well connected, a significant portion of the articles received may be junked.
NNTP helps to avoid this when running B News, but we seem to have lost that
advantage under C News.  I don't have numbers for you, since I haven't switched
to C News on our main host (yet), partly due to this concern.

I am wondering if anyone has suggestions on how to prevent the reception of the
article by nntpd.  If I recall correctly, this topic came up when C News first
appeared, but was swept into the bit bucket when no one came up with a clean
solution.  Any takers now?
--

Michael Sanderson                   sanders@cs.ubc.ca
                        {alberta, uunet, uw-beaver}!ubc-cs!sanders

geoff@athena.mit.edu (Geoff Collyer) (11/07/90)

I've been testing a new nntpd on several machines at U of Toronto for the
last few months.  It almost completely eliminates transmission of duplicates
and gets articles into the news system quickly.  I want to tune it a little
more before sending it to Stan Barber for inclusion in the official NNTP
release.