[comp.unix.i386] Tuning information for ISC 386/ix

madd@bu-cs.BU.EDU (Jim Frost) (09/20/89)

I recently installed Interactive Systems Corp's 386/ix version 2.0.2
on my 386 machine.  Unfortunately my machine has only 4mb and I'm
trying to run X.

I expected the thing to run slowly, and it does.  What I didn't expect
is that after a few hours of disuse it begins to hang terribly (not
swap, HANG).  Worse, even unloading the system (shutting down X and
logging all users out) doesn't restore performance to the system; this
morning I got it into a state where there were no users logged in but
a login from the console would time out.  This was distressing.

I think that this is the result of improper tuning (what in hell could
it be *doing* while it's hanging?), but I'm from the BSD world and
have no idea how to tune a SysV kernel.

If some kind soul could send me an mtune file that works reasonably
well (considering) for a system with 4mb memory, ethernet, and x11 (a
heavy STREAMS user), I would be very grateful.  Additionally, I will
be upgrading to 8mb and putting NFS on the thing, so any tuning
information for such a system would also be useful.

Please respond via email to madd@std.com (uunet!skuld!madd).  Thanks.

jim frost
software tool & die
madd@std.com

trb@haddock.ima.isc.com (Andrew Tannenbaum) (09/21/89)

I am speaking as a hacker employed by ISC, not as ISC tech support or
as an official spokesman of ISC.  I thought this might be of general
use so I am posting it instead of mailing directly to madd@std.com.
No guarantees expressed or implied.

In article <38451@bu-cs.BU.EDU> madd@std.com writes:
> If some kind soul could send me an mtune file that works reasonably
> well (considering) for a system with 4mb memory, ethernet, and x11 (a
> heavy STREAMS user), I would be very grateful.  Additionally, I will
> be upgrading to 8mb and putting NFS on the thing, so any tuning
> information for such a system would also be useful.

I believe that you are right in guessing that the tunable parameters
need to be fiddled.  Until these data structures are properly tuned,
your system will fail in strange ways as you've described.

The mtune file is a master file, which you shouldn't touch.  The one
that you change is the stune file (these files are in /etc/conf/cf.d
in 386/ix 2.0).  The way to do this is to run netstat -m, and you'll
get some output like this:

These numbers are off my machine; I arrived at the alloc values by
trial and error.  If a number is 1024, it probably means that I tried
it with 512 and it maxed out and failed.  I run X11 and ethernet but
no NFS on this 9Mb 386.

$ netstat -m
                 alloc   inuse     total     max    fail
streams:           128      52       840      58       0
queues:            512     290      5016     326       0
mblocks:          2840     251    384394     316       0
dblocks:          2272     251    304876     306       0
dblock class:
    0 (   4)        32       0     25655       3       0
    1 (  16)       256      62     42923      73       0
    2 (  64)       512      27    219359      44       0
    3 ( 128)      1024     162      9671     189       0
    4 ( 256)       256       0      5800      15       0
    5 ( 512)        64       0       746       2       0
    6 (1024)        64       0       389       2       0
    7 (2048)        32       0       306       9       0
    8 (4096)        32       0        27       2       0

The numbers in the alloc column are out of your stune file.  If there
are any non-zero numbers in the fail column, recompile your kernel
after fiddling the stune numbers by running idtune, or by running
kconfig (which runs idtune) using the menu options 1) CONFIGURE KERNEL
and 6) ADD TUNABLE PARAMETERS.  If you want to change the 128 byte
dblocks, the parameter is NBLK128, and so on.

Use kconfig to do this, it's easy to break if you do it by hand.
The kconfig process copies files from cf.d to kconfig.d, and it will
happily overwrite a file you've edited by hand if you play in the
wrong place.  Experiment with kconfig and understand it before
you try to subvert it.

The method for modifying tunable kernel parameters is well-described in
the 386/ix Operations/System Adminstration Guide, Chapter 5,
"Customizing your Computer" - I have a perfect-bound 6x9 copy with ISC
covers, I think this is an AT&T document, so other versions may exist.

Here's my stune file, in case it might help:

NINODE        	300
NS5INODE      	300
NFILE         	300
NBUF          	250
NPROC         	100
MAXUP         	60
NCLIST        	120
NSTREAM       	128
NQUEUE        	512
NBLK4096      	32
NBLK2048      	32
NBLK1024      	64
NBLK512       	64
NBLK256       	256
NBLK128       	1024
NBLK64        	512
NBLK16        	256
NBLK4         	32
SHLBMAX       	8
NOFILES       	64

	Andrew Tannenbaum   Interactive   Cambridge, MA   +1 617 661 7474

larry@nstar.UUCP (Larry Snyder) (09/22/89)

In article <38451@bu-cs.BU.EDU>, madd@bu-cs.BU.EDU (Jim Frost) writes:
> I recently installed Interactive Systems Corp's 386/ix version 2.0.2
> on my 386 machine.  Unfortunately my machine has only 4mb and I'm
> trying to run X.
> 
> I think that this is the result of improper tuning (what in hell could
> it be *doing* while it's hanging?), but I'm from the BSD world and
> have no idea how to tune a SysV kernel.

Jim, did you trying re-configuring the kernel using the default tuning
parameters for a system with 4 megabytes of memory?


-- 
                                                               Larry Snyder
                                      uucp: iuvax!ndcheg!ndmath!nstar!larry 
                                 The Northern Star Usenet Distribution Site 
                                                    Notre Dame, Indiana USA

madd@bu-cs.BU.EDU (Jim Frost) (09/23/89)

In article <14663@haddock.ima.isc.com> trb@haddock.ima.isc.com (Andrew Tannenbaum) writes:
|Here's my stune file, in case it might help:
[...]

The things to tune are NBUF, NSTREAM, NQUEUE, and the NBLK parameters.
I took the default tuning after installing the thing and changed
NSTREAM to 96 (64 will work but it's close with X11, the default
number (24?) is far too low), NQUEUE to 400, and NBLK128 to 256 (it
failed at the default of 128, causing things to hang strangely).

Performance improved dramatically with these changes.  Thanks to all
who helped.

jim frost
software tool & die
madd@std.com

trb@haddock.ima.isc.com (Andrew Tannenbaum) (09/23/89)

In article <14663@haddock.ima.isc.com> I responded to madd@std.com about
reconfiguring his kernel to clean up his dblock failure problems.  One
of my co-workers made some helpful comments on my posting that I'd
like to share.

I think my advice on which programs to run to change the params was
OK.  My own system was probably configured a little on the liberal
side, though.  I just kept doubling stuff when the max values for
dblocks got close to the alloc values, or when I got failures.  Since
non-zero numbers in the fail column (of a netstat -m listing) seemed to
me to coincide with strange hangs and crashes, I figured that as
superstitious practices go, this was a pretty good one.

I still don't know exactly which parts of the kernel use which size of
dblocks.  I am told by reliable sources, however, that failures on the
4096 byte (NBLK4096 - dblock class 8) blocks is no big deal, apparently
386/ix will happily use 2048 byte blocks instead or something, so 4096
failures are not a problem.  Perhaps the rest of my estimates are also
a bit liberal, but when I try to cut back on them, I get failures in
netstat -m, so these are things you can play with (by recompiling and
reinstalling new kernels, these guys are not dynamic, else we wouldn't
have to go through this stuff).

In the tuning arena, my colleague also mentioned kernel buffers and
RAMdisk.  Note that in the following paragraphs the defaults I talk
about are in 2.0.1, so they might be different on your system.

I see that there are files called /etc/conf/kconfig.d/param.*MB which
have something to do with configuration.  (I don't know how they get
used.)  These param files have stune kind of param numbers, but not in
the stune format.  Apparently, there is one file for 2MB, and second
for 4, 8, and 16MB.  Since configuration has to do with more than how
much memory you have, this is excusable.  The point is that you can
play with these params in stune.  Many of the params are nicely
described in the ops/sysadm guide.

Anyway, I have a 9meg machine where I jacked up the NBUF/NHBUFs from the
default 250/64 to 1000/512, and I ran my pathalias build under each.
(I had to fix the mtune file to allow NHBUF to get that big.  Note
that NHBUF should be a power of 2 according to the docs.  386/ix uses
NHBUF to find a prime for a hash table size.)

The timings for pathalias went from over 14 minutes to under 5 minutes
real time.  This was on a multi-user system both times, but the
difference is still quite significant:

(I just ran it again, and the real time was less than 4 minutes, but I
don't feel like changing all the numbers in the note.)

	250 bufs
	real    14m17.79s
	user    1m0.37s
	sys     1m17.48s

	1000 bufs
	real    4m20.88s
	user    0m59.98s
	sys     0m36.41s

If you want to check out your buffer stats, you can run a command like
sar -b 10 50 (system activity report on buffers, every 10 seconds, 50
iterations).  Check the sar(1) man page to figure out what the stats
mean.  Anyway, I found when I jacked up the buffers, I got a whole lot
more buffers processed per second, and it seems the system time on my
pathalias went down from 77 secs to 36 secs, which must been related to
buffer munging.  (Note that this was a pathalias, sort, and makedb on
16600 entries, I don't want you all thinking that the machine is slow.
The actual pathalias run time went from 112 secs real to 60 secs real,
and makedb went from 12 minutes real to less than 3.)

As for RAMdisk for /tmp, I have no experience with it, but if you have
memory locations that have never been referenced since you bought your
system, you might want to try sticking /tmp in a meg or two.  I am not
saying this is bad, I'm saying I have no experience with it.  Some
people swear by it, I don't want to argue the virtues of /tmp on
RAMdisk, but it's something for you tuning heads to play with.
Frankly I don't have any complaints with the speed of the 386/ix file
system, perhaps someone could post some comparative stats on RAMdisk
stuff.

In summary, you can goof with your kernel by playing with the tune
parameters, (and maybe by running with files like /tmp on RAMdisk) and
you can check them with sar and with the special purpose stat programs
like netstat, and by timing your favorite applications.  I hope this
proved helpful if not inspirational.  If anyone knows about any
386/ix bottlenecks that can be fixed by diddling a parameter, by all
means speak up!

	Andrew Tannenbaum   Interactive   Cambridge, MA   +1 617 661 7474

madd@bu-cs.BU.EDU (Jim Frost) (09/26/89)

In article <14711@haddock.ima.isc.com> trb@haddock.ima.isc.com (Andrew Tannenbaum) writes:
|Frankly I don't have any complaints with the speed of the 386/ix file
|system,

I'm kind of curious about the 386/ix "FFS".  What I'd like to know is
if the free list management has been improved to help minimize
fragmentation (eg allocation by cylinder cluster).  This is pretty
trivial to do -- even without changes to the on-disk filesystem.
Comments from the crowd at ISC?

jim frost
software tool & die
madd@std.com

johnl@esegue.segue.boston.ma.us (John R. Levine) (09/27/89)

In article <38839@bu-cs.BU.EDU> madd@cs.bu.edu (Jim Frost) writes:
>I'm kind of curious about the 386/ix "FFS".  What I'd like to know is
>if the free list management has been improved ...

When FFS mounts a filesystem, it reads in the free list and keeps an
in-core bitmap of free space that it uses to allocate space.  When it
unmounts a filesystem it rewrites an updated free list based on the
bit map.  (In between it writes an empty free list so that in case of
a crash fsck will notice all the lost blocks and recreate the free list.)
It uses the bit map to do sensible file allocation.  It also does things
like noticing when it can do a large read for a contiguous bunch of
blocks.

A nice thing about this scheme is that the format of an unmounted disk
remains exactly the same as AT&T's SysV/386, so you can carry file systems
back and forth.
-- 
John R. Levine, Segue Software, POB 349, Cambridge MA 02238, +1 617 492 3869
johnl@esegue.segue.boston.ma.us, {ima|lotus}!esegue!johnl, Levine@YALE.edu
Massachusetts has 64 licensed drivers who are over 100 years old.  -The Globe

rcd@ico.ISC.COM (Dick Dunn) (09/27/89)

johnl@esegue.segue.boston.ma.us (John R. Levine) writes about the 386/ix
Fast File System, among other things that:

> A nice thing about this scheme is that the format of an unmounted disk
> remains exactly the same as AT&T's SysV/386, so you can carry file systems
> back and forth.

In fact, you can take a disk from an AT&T system to a 386/ix system and
mount it, then unmount it to clean up the free list into more usable
order!  (Of course, it would make more sense just to run 386/ix.:-)
-- 
+---------+     Dick Dunn    rcd@ico.isc.com    ico!rcd      (303)449-2870
| In this |         4th annual MadHatterDay [10/6/89]:
|  style  |            A Thousand Points of Madness
|__10/6___|