edwin@ruuinf.cs.ruu.nl (Edwin Kremer) (07/05/89)
Ok, there has been some discussion on large diskless clusters. Something that relates to this: how should I configure the kernel on the server to get the best performance ??? Let's describe the situation: 1 HP9000/370 server, 16 Mb RAM 20-30 HP9000/340 diskless clients, 4 Mb RAM There are no user processes on the server, only some additional daemons like "sendmail" and "nnmaster" Most of the clients run X11 I've just increased some kernel parameters and I'm anxious to find out what will happen tomorrow morning when time is there for all of them to hack whatever they dreamed about last night into their keyboard :-) Well, if anyone has experimented with tunable kernel like "dskless_*", "nbuf" and "netmem*" parameters in an environment like the one described above or has any suggestion into improving the server's performance, I'd really like to know... I'll let you know if any of my changes turns out to be one of the "I had better not changed that" choices. Thanks in advance, --[ Edwin ]-- -- Edwin Kremer, Department of Computer Science, University of Utrecht Padualaan 14, P.O. Box 80.089, 3508 TB Utrecht, The Netherlands Phone : +31 - 30 - 534104 | Telefax: +31 - 30 - 521105 E-Mail: edwin@cs.ruu.nl | UUCP to: ...!hp4nl!ruuinf!edwin
rocky@hpfcmgw.HP.COM (Rocky Craig) (07/07/89)
> Well, if anyone has experimented with tunable kernel like "dskless_*", "nbuf" > and "netmem*" parameters in an environment like the one described above or I'll do the easy ones first :-) The netmem* parameters are for IP connections (and other things) in our ARPA product. The diskless message protocol does not use those buffers, so changing them will not affect DISKLESS performance (file system and remote swap). Your overall system performance could change depending on the number of "normal" network connections you establish (Telnet, rlogin, etc.) HP's diskless has a "dual-cache" scheme for file system access. There is a buffer pool on each diskless node. When a buffer needs to be flushed, it will go to a (shared) buffer pool on the server. The server will then manage the asynchronous disk access, deciding when to actually write the data to disk. Various synchronization checks exist to guarantee expected UNIX* operation (mulitple writers, distributed FIFOs, etc.); HP-UX diskless passes the SVVS! The network buffers used for remote filesystem access are taken as a subset of the overall file system cache. To improve file system performance, you want to maximize the size of the shared cache on the server. The size of the cache is controlled by nbuf. On your 16 megabyte system, this number will default to 268 pages (one page = 4k bytes), or a little over one megabyte. If you are using the server in a dedicated fashion and can spare the extra memory, bump nbuf up to its maximum of 768 (three megabytes). Be careful about changing nbuf on diskless nodes. You might try it, but it will definitely take some experimentation to find the right value. The other "diskless" parameters you mentioned are all calculated (by default) based mainly on the parameter "num_cnodes". This is not a limiting parameter, but a guestimate as to the average number of cnodes this server should be expected to support. It defines other parameters and the size of various tables and buffer pools. In your case, I would suggest setting it to 25 (or maybe even 30). The only "cost" is server memory (of which you have plenty on your dedicated server). It's doubtful that changing the individual parameters (instead of relying on their value obtained from num_cnodes) will get you anything extra. The main performance bottleneck in diskless clusters occurs when diskless nodes start paging (remotely). Do everything to maximize available memory on those nodes: remove unneeded drivers from the kernel (fpa, nfs if you don't need it, etc.), don't run superfluous processes (netlogstart, vtdaemon, rwhod are good examples). Your server will still have quite a bit of available memory; if it makes sense, you might want to run some of the X11 clients on the server. Naturally, in the benchmarking/performance tuning world, the final answer is always "That's too vague to answer definitively", or the simpler, "That depends" :-) Your mileage will vary, but this should be a good start. Most of this information is contained in the System Administrator manual. Some pieces are in the networking manuals. Good Luck! Rocky Craig Hewlett-Packard Workstation Group, Marketing Event Technical Support 3404 E. Harmony Road, Ft. Collins, Colorado 80525-9599 Internet: rocky%hpfcmr@hplabs.hp.com UUCP: ...!{hplabs | hpu*}!hpfcmr!rocky This article does not represent the official position of the Hewlett-Packard Company. The above data is provided for informational purposes only. It is supplied without warranty of any kind. * UNIX is a trademark of AT&T in the U.S. and other countries