estes@iris.ucdavis.edu (Robert Estes) (09/12/90)
We have an 835 running version 7.0 of hpux (Release A.B7.00 B) that seriously bogs down when we run more than a couple of jobs simultaneously. The `boblem seems to revolve around memory usage...we have 16M of memory and do a lot of image processing (programs are memory hogs). Typically, if no one is on the console running X, we're fine. But when someone is running X, the load average may shoot up 8, or more, due to the initiation of one of our programs (or any other memory hog), which can slow the system to an unusable pace -> interactive jobs getting serviced once/minute, etc. It gas suggested that the problem has to do with the swapping strategy - the operating system has to swap in the entire working enfironment for a job, before giving it its time slice. Since our jobs use a lot of memory, most of the processors time is spent swapping the contents of memory. 1. Is that how the swapping is done? 2. Other than purchasing more memory, is there anything we can do to alter/alleviate this behavior? Any help would be appreciated. E-mail is fine. Thanks, Robert
bobd@nisca.ircc.ohio-state.edu (Bob Debula) (09/18/90)
In article <117@bwilab3.UUCP> brad@bwilab3.UUCP (Bradley Ward) writes: >I hope the answer is not simply "buy more memory". I can easily drop >8 Meg more into my PC's and Mac's, cause I can do it for $1000 or so. >$10,000 + for HP 800 memory makes that a much more painful >alternative. Too right! Actually, with a little judicious shopping, you can get Mac 1 Mb SIMMs for around $50 apiece (~$400 for 8Mb) and PC SIMMs for not a whole lot more. We have an HP9000/840 which will probably stay at 16Mb in perpetuity. The price we were quoted for upgrading to 32Mb ran somewhere around $1000/Mb if I remember correctly (including HPs largish university discount). We took the attitude "just let it swap" until we can get a larger system(s) in place. IMHO HP's memory pricing (for the HP9000/8xx series) is SCANDALOUS. -- ========================================================================== Bob DeBula | Internet: bobd@hpuxa.ircc.ohio-state.edu The Ohio State University | Disclaimer: These are my views, not the U's Davros sez: When my Daleks compute they use X-TER-MI-NALS!
mck@hpcuhc.HP.COM (Doug McKenzie) (09/21/90)
> when snail mode is entered, I notice the following: > > 1 - Free memory goes from its normal amount of 1 mb or so to 6 mb free > out of 16mb; corresponding to this, user memory utilization goes > from its normal level of 75% or so to 30%. > > 2 - Idle CPU time goes to 80% > > 3 - 1 minute load average shoots up to 14 or 15, instead of its > normal state of < 2. > > 4 - Swap space utilization remains around 25% > > On the "V" screen of monitor (Virtual memory status): > > On each screen update, the "processes swapped in" and "processes > swapped out" BOTH show 10 or 15. Normally they show 0. > > Background information on our system: > HP-835 with 16 Mb, 129 Mb of swap, running HP-UX 7.0 All the above suggest swapping in/out is causing the performance slowdown. "1" shows that some memory resident processes have been swapped out. "2" shows that the system is busy swapping stuff to disk, "3" means that whatever you're doing is generating a need for more memory (15 processes worth), and the V screen confirms swapping is going on. Utilization of swap space ("4") is not really relevant, since swap is allocated on exec, not based on a working set, or other dynamic parameter. (Of course if a process allocates more memory, it'll show up in swap allocation.) Swapping is basically a last resort effort to keep the system running, when either free memory is very low, or I/O bandwidth (disk reads/writes) are excessive. For a properly sized and configured system, the "pageout" process should take care of memory needs during typical usage. With "monitor" you can see this happening, and when pageout can't keep up with demands. Pageout will run when 1/4 or less of usable memory is free, and speed up as needed, until it maxes out at 200 pages scanned per second (if *my* memory serves.) If that's not enough, process(es) are swapped out based on complex, pretty intelligent choices, until things are OK again. All, or virtually all pages of a process are swapped out when the process is swapped out. A minimal number of pages are swapped back in when the proc is brought in again. Robert mentioned that an entire process's "environment" has to be swapped in. I'm not sure what was meant, but neither all the code, nor all the data pages are typically swapped in. The user area is swapped in/out with the proc's pages, but is only about 2K bytes. > I hope the answer is not simply "buy more memory". It may be that disk I/O is causing the swapping, if many processes are reading/writing simultaneously. I'd guess more likely your memory is causing the swaps, or both I/O and memory. X is certainly a hog. My (diskless) 330 was a slug with 4 megs, OK with 8, running X. So your other processes are limited to the remaining 8 MB. I'd guess inadequate memory is causing swapping. A load average of 15? Yikes, this is a lot of runnable processes. How big are they? Why are they all trying to run at once? Does the X startup start the application processes also? Could some of these be delayed or "niced", so they weren't all trying to go at once? This should help a *transient* snail-mode, if it's the simultaneity (sp?) of startup (rather than the "cruising" needs or the needed processes) that's the problem. If the process(es) make significant startup efforts and then sleep, this would bog down other processes's interactive response unnecessarily, during the startup. So I'd suggest examining how your X startup scene behaves, using monitor, and tweak it to be a little nicer to system resources. Stick some "sleep 300"'s in X's startup, and trade-off getting windows up quick, for prevention of swapping. Hope my meandering helps. > $10,000 + for HP 800 memory makes that a much more painful > alternative. Because of this, we tend to expand by other workstations > instead! Ouch. Doug McKenzie S800 HP-UX Support mck@hpda.hp.com OR ...hplabs!hpda!mck
michel@es.ele.tue.nl (Michel Berkelaar) (09/24/90)
The fact that system performance of a hp9000/835 decreases more than
dramatically when forced to swap has been observed in our group as well.
Our configuration: 2 x HP 9000/835 with 16 MB.
Typical load: about 5 users, 1 minute load average between 0.5 and 1.5 each
When we were experimenting with swapon to increase our swap space, I wrote the
following little program, just to see how much memory I could get:
main()
{
int i;
for(i = 1; calloc((unsigned)(1<<20), (unsigned)1); i++)
printf("megabytes: %d\n", i);
perror("test");
printf ("Come to the end\n");
sleep(10000);
}
Running this on the HP showed a slow increase in allocated memory during
the first 10 MB or so, but after that each new MB took longer to calloc, and
the system load started growing, and growing. When I reached 23 MB after 10
minutes or so (!!!), the load had gone up to 22. I then decided to stop the
process, but even that took minutes. And, even more strange, it took the HP
more than 10 minutes to recover from this event, with the load only very
gradually coming down to the normal valu of about 1. No need to tell you that
some of the other users were furious by that time. Btw, none of the other
users were doing something that could have accounted for the load.
So: what is going on inside the 835 when it starts swapping? Why is it so
overly sensitive to this situation? Do other people see the same behaviour?
By the way, running the same test on our Alliant FX8 (with 120 MB of memory)
also showed a dramatic increase in system load when the process started
swapping, but not as bad as the HP. The same test on my humble Apollo DN2500
with 16MB runs without problems, the load staying around 1, and the megabytes
being calloc-ed faster then on the HP. It continues on the 2500 untill the
disk is full, and then stops gracefully because calloc returns 0.
It would be interesting if some people could try the above code on different
hardware platforms, and report the observed behaviour.
--
-------------------------------------------------------------------------------
Michel Berkelaar | Email: michel@ele.tue.nl
Eindhoven University of Technology |
Dept. of Electrical Engineering |
Design Automation Section |
P.O. Box 513 | Phone: ... - 31 - 40 - 473345
NL-5600 MB Eindhoven | Fax: ... - 31 - 40 - 448375
The Netherlands |
-------------------------------------------------------------------------------