stephen@comp.lancs.ac.uk (Stephen J. Muir) (04/16/87)
I find that inews spends a vast amount of *CPU* time copying each article of a batch to a temporary file -- something like several minutes to copy about 50K bytes of a single article (this takes about 15 minutes real time for a single article at the moment with the load at 5.0). I simply cannot believe it can be so inefficient! It was a *lot* faster when it forked "unbatch". We run 2.11, with patches up to #7 applied, on 4.2 BSD on a VAX 11/750. -- EMAIL: stephen@comp.lancs.ac.uk | Post: University of Lancaster, UUCP: ...!mcvax!ukc!dcl-cs!stephen | Department of Computing, Phone: +44 524 65201 Ext. 4120 | Bailrigg, Lancaster, UK. Project:Alvey ECLIPSE Distribution | LA1 4YR
jerry@oliveb.UUCP (Jerry F Aguirre) (04/21/87)
In article <330@dcl-csvax.comp.lancs.ac.uk> stephen@comp.lancs.ac.uk (Stephen J. Muir) writes: >I find that inews spends a vast amount of *CPU* time copying each article of a >batch to a temporary file -- something like several minutes to copy about 50K >bytes of a single article (this takes about 15 minutes real time for a single >article at the moment with the load at 5.0). I simply cannot believe it can be >so inefficient! It was a *lot* faster when it forked "unbatch". Before submitting the code in question I ran a performance comparison with the following results: unbatch 9.6u 5.2s 0:23 63% 41+69k 80+160io 62pf+0w new 5.5u 3.6s 0:12 71% 68+115k 44+126io 6pf+0w This was for a uncompressed batch consisting of 6 articles all of which were rejected as duplicates. (This was the only way I could get a repeatable test.) This is significantly faster than the "unbatch" code. The code was beset with a number of compatability problems related to buffering and reposition of the input. In the infamous patch number 6 someone added a "setbuf(infp, NULL") to the begining of the checkbatch routine. A shotgun fix to a subtle problem. As far as I can figure out this results in the copy being fread a byte at a time. (The usual idiocy of using buffered IO routines (getc) to do unbuffered IO!) If the batch was in '#! cunbatch' format then this problem will not occur because the buffering will be restored. (Most of my news is in this format.) I will work on a fix. Actually, I kind of like the setbuf...NULL as I wanted to use read instead of fread anyway. Always nice to get back to basics. Jerry Aguirre (Never install an even numbered patch!)