[news.software.nntp] excessive rewriting of queue file

jerry@olivey.ATC.Olivetti.Com (Jerry Aguirre) (03/27/91)

The requeueing strategy used by nntpxmit (and nntplink) can result in
excessive overhead when the target host is blocking incomming news.  It
can result in nntpxmit needlessly copying an increasingly large queue
file while the target is running expire or low on disk space.  If the
target nntpd just refuses the connection then this overhead is
eliminated but some hosts (bbn) only check when an article is offered
rather than at connection time, presumably to allow reading.

A typical senerio is that nntpxmit will open a connection, offer a
single article, and receive a "400 sorry expire running" message.  This
could also happen for other conditions such as being low in disk
space.  Nntpxmit will then copy the host.nntp file to a temporary file,
append the one article it attempted to offer, and then recopy it back
to the queue file.  As only one article was offered and the 400 message
had nothing to do with that particular article, modifying the queue
file is wasted overhead.

Given that nntpsend will continue to append the queue file to the nntp
file this can result in nntpxmit copying larger and larger amounts of
data to a temp file and back.  Depending on how frequently one polls
the remote site this could add up to a substantial amount of overhead.

My answer to the problem was to add a flag (doneany) that is cleared
when the queue file is opened and set whever there is any response to
an "ihave" other than 400.  Then, in cleanup, if doneany is true then
rewrite is called as before, otherwise the queue file is just closed and
left as is.

					Jerry Aguirre