[news.software.b] Faster batching in B-news

fitz@wang.com (Tom Fitzgerald) (03/15/90)

mike@cimcor.MN.ORG (Michael Grenier) writes:

> Could somebody gives this try?
> The following diff to batch.c forces it to build each batch
> (actually, the entire sendbatch run) with the articles
> sorted by newsgroup.

I tried this (using a script with 'sort', outside news) a long time ago and
found no measurable difference.  I was pretty depressed, it sounded like a
great idea to me too.  I rationalized that the disk was so fragmented that
there was almost no locality of reference; and that the working set of data
was so large that things in the working set were being paged out during
batching anyway.  (This was all running on a fairly feeble machine, maybe
people with more memory might have better luck).

Besides not helping any, this greatly increases the chance of reordering
articles so that a followup to an article precedes the article itself,
which is confusing to readers on the receiving end.

The only thing I found that consistently increased batching performance
was increasing the size of individual batch files.  Boosting batch size
from 50K to 100K (compressed) boosted throughput by like 30%-40%, because
compress and uux are forked half as many times, and the compression ratio
is noticeably better.

---
Tom Fitzgerald      Wang Labs           fitz@wang.com
1-508-967-5278      Lowell MA, USA      ...!uunet!wang!fitz