[net.news] How to Reduce the incidence of Duplicate Rejected with Redundant Feeds

usenet@ucbvax.ARPA (USENET News Administration) (10/05/85)

Simply stated: the number of articles rejected as duplicates will
DECREASE with increasing frequency of contact with ALL your netnews
feeds.

Or, the more frequently you contact ALL of your feeds, the fewer
duplicate rejects you'll get.

Imagine sites A, B, and C with the following set up:

	A - B - C

where B gets netnews from both A & C. At first glance, this should lead
to a 50% rejection rate. B will get article `x' from both A & C, and
reject it the second time it arrives (from whichever system).

The reality is different. In the case above, B gets article `x' from A,
logs it, and queues it for C. B can avoid getting a duplicate copy of
article `x' from C by sending it off to C as quickly as possible.
Assuming that B gets article `x' to system C before any other system,
C will never send a copy of `x' to B, because it got `x' from B in
the first place.

The window is between the time that B gets article `x' from A, and the
time that C receives it. In that window, C can get the article from
elsewhere, and queue it to go to B in the next communication, despite
the fact that B already has received article `x' from A.

However, C doesn't know that B has article `x' already, because B
hasn't sent it to them yet. So B queues its copy for C, C queues its
copy for B, the two copies of article `x' cross each other in transit
during the next contact, and both systems reject the article. Consider
also, that in that particular communication, if `x' was the only article
transferred, 100% of the bandwidth was wasted.

The way to prevent this is frequent contact, and minimum latency in
forwarding of netnews. The longer an article lingers in queue on your
system, the greater the probability that one of your feeds will get the
article from elsewhere, and queue it for you.

So, phone bill high? Rejection rate high? Don't cut back! Increase
your frequency of contact, and attempt to minimize the time that an
article spends in queue on your system. The overall amount of time
spent on the phone transferring netnews should decrease.

	Erik E. Fair	ucbvax!fair	fair@ucbarpa.BERKELEY.EDU