johnl@iecc.cambridge.ma.us (John R. Levine) (01/28/91)
I am running ISC 2.2 Unix on an Intel 302, a pretty much vanilla cached 25
MHz 386 with 16MB, 8MB on the motherboard, and 8MB on an expansion card.
The kernel has TCP/IP, X, and the Norton unerase, of which only the latter
two are turned on.
I have an internal Telebit with a 16550 UART, and uucp connections generally
run between 800 and 1100 cps, depending on who's on the other end. I noticed
that my system is fairly disk bound, and that since I'm not running NFS I
don't fill up all my RAM. It seemed sensible to increase my disk buffers, so
I built a new kernel with 4000 rather than 2000 buffers. It works OK, except
for one thing -- uucp throughput stinks. It was down around 400 cps. I
rebooted the old kernel and throughput is back up. As far as I can tell, the
only difference between the old and new kernels is NBUF.
What is going on? The most likely thing I can think of is that somewhere in
the buffer management code there is an N^2 algorithm that runs with
interrupts masked, and with 4000 buffers it takes so long that even with a
16550 I lose interrupts. I note that in the mtune file, 2000 is the maximum
number of buffers you're normally supposed to configure, but losing
performance when you add more buffers is pretty pitiful. As I recall, a
16550 has a 16 character silo, so to lose characters you'd have to stay
masked for upwards of 15ms at a time.
Regards,
John Levine, johnl@iecc.cambridge.ma.us, {spdcc|ima|world}!iecc!johnlgemini@geminix.in-berlin.de (Uwe Doering) (01/28/91)
johnl@iecc.cambridge.ma.us (John R. Levine) writes: >I have an internal Telebit with a 16550 UART, and uucp connections generally >run between 800 and 1100 cps, depending on who's on the other end. I noticed >that my system is fairly disk bound, and that since I'm not running NFS I >don't fill up all my RAM. It seemed sensible to increase my disk buffers, so >I built a new kernel with 4000 rather than 2000 buffers. It works OK, except >for one thing -- uucp throughput stinks. It was down around 400 cps. I >rebooted the old kernel and throughput is back up. As far as I can tell, the >only difference between the old and new kernels is NBUF. > >What is going on? The most likely thing I can think of is that somewhere in >the buffer management code there is an N^2 algorithm that runs with >interrupts masked, and with 4000 buffers it takes so long that even with a >16550 I lose interrupts. I note that in the mtune file, 2000 is the maximum >number of buffers you're normally supposed to configure, but losing >performance when you add more buffers is pretty pitiful. As I recall, a >16550 has a 16 character silo, so to lose characters you'd have to stay >masked for upwards of 15ms at a time. I noticed this, too, on ISC 2.0.2. On my system even 2000 buffers are too much. Yes, there seems to be some incredible nasty code in the kernel that disables interrupts for long enough that even an NS16550A chip loses characters, at least at 38400 bps. I mention this problem in the README file of FAS 2.08 (to be released in the next few days), so at least those who read this file will know the consequences of increasing the NBUF parameter. I wonder whether AT&T has changed this code in SysVr4 to be less reckless. We'll see. Uwe -- Uwe Doering | INET : gemini@geminix.in-berlin.de Berlin |---------------------------------------------------------------- Germany | UUCP : ...!unido!fub!geminix.in-berlin.de!gemini
gsteckel@vergil.East.Sun.COM (Geoff Steckel - Sun BOS Hardware CONTRACTOR) (01/30/91)
johnl@iecc.cambridge.ma.us (John R. Levine) writes: >I have an internal Telebit with a 16550 UART, and uucp connections generally >run between 800 and 1100 cps, depending on who's on the other end...... >I built a new kernel with 4000 rather than 2000 buffers. It works OK, except >for one thing -- uucp throughput stinks. It was down around 400 cps..... > >What is going on? The most likely thing I can think of is that somewhere in >the buffer management code there is an N^2 algorithm that runs with >interrupts masked, and with 4000 buffers it takes so long that even with a >16550 I lose interrupts. I note that in the mtune file, 2000 is the maximum I don't know if this has survived into missed'em V.3, BUT... There were at least two places where truly silly things happen w.r.t. the buffer cache in V7 & friends. 1) disksort - inserting a buffer into a driver's to-be-written queue used a quadratic (or worse) algorithm at spl6(). 2) sync() - when a sync occurs, the entire cache was scanned, and one by one, each modified block was placed in the driver's write queue, using disksort.... (can you say n^3!). 3) every read() and write() call on the serial device caused the inode for the serial line to be updated with the current time (access/modification), which searches the buffer pool (at spl6()) - hundreds of times a second... I think there were more, but I forget lots. Losing even one character can cause a truly awful degradation of UUCP transmission rate. I hope this helps someone. geoff steckel (gwes@wjh12.harvard.EDU) (...!husc6!wjh12!omnivore!gws) Disclaimer: I am not affiliated with Sun Microsystems, despite the From: line. This posting is entirely the author's responsibility.