McGuire_Ed@GRINNELL.MAILNET (07/22/86)
Here's a summary of the information people sent me that I didn't know about tuning page and swap files along with my own observations and some things I found in Internals & Data Structures. And also here's my page/swapfile procedure, by popular demand. ---------- >Date: Wed, 9 Jul 86 10:34:12 PDT >From: carl@CitHex.Caltech.Edu (Carl J Lydick) > >Given that you can determine the process's initial quota, you can >use the $GETJPI system service to get the remaining page file quota and >compute the usage. The item of interest is "PAGFILCNT". Turns out that PAGFILCNT is a number I've wanted to know for awhile. It tells you the number of virtual pages associated with a process, just as PGFLQUOTA is really the virtual page limit for a process. Unfortunately, it doesn't tell how many pages are actually in the page file. ---------- >Date: Tue, 8 Jul 86 10:39:42 EDT >From: sasaki@harvard.HARVARD.EDU (Marty Sasaki) > >Processes don't swap until the working sets have been trimmed to WSMIN. This note made me feel much more optimistic about cutting the size of my swapfile, until I read in I&DS that processes in FPG, COLPG, MWAIT, PFW, and COM states get swapped without trimming (the explanation is that it would be inappropriate to trim processes which are waiting for a page or are likely to fault all the trimmed pages right back when they are inswapped. Upshot is, PPGCNT + GPGCNT pages are allocated in the swapfile for each process, rounded up to the next multiple of parameter MPG_WRTCLUSTER, just in case the process will be swapped without trimming. >One thing you might think about is using the dump file as a swapping >file. The only trick is that if you want to look at your dump, you >need to save it somewhere before you enable it. Neat idea! I can use this for 750 systems, where I have the page file on the system disk. On 8600s, I want to put the page file on a separate spindle. Is it possible to relocate the dump file to a different disk? ---------- >Date: Wed, 9 Jul 86 17:14:01 pdt >From: hplabs!oliveb!mipos3!mark@ucbvax.Berkeley.EDU (Mark Fineman) > >I'm not sure what you are trying to accomplish. Also, I >don't know how you decided that 1/2 and 1/4 free are the >critical numbers. My goals are to provide sufficient page and swap file space, without using up an RA81 common system disk, and to balance disk activity by spreading different cluster members' page files on different spindles (Heaven help me if I have swap files influencing disk activity :-)). DEC recommends in the Guide to Performance that page files be kept 1/2 free, and swap files be kept 1/4 free. Speaking from experience, VMS comes to a GRINDING HALT when free swap file drops much below that limit. VMS simply can't find sufficient swap space for growing processes, and they hang until other processes shrink to make room. I have excess page file right now, so I haven't seen the effect of dropping below 1/2. >We have had problems with processes going into wait >states for swap space when the swap file was 1.5 times real >memory. I have found that the swap file needs to be 2x >real memory (this is not worst case, but works for our loads) >Not that this space is never really used for swapping >on our systems, but the system insists on allocating the >contiguous space in the swap file. Right now, my swapfiles on the two 11/750s are at about 1.25X physical memory and that seems to be enough for our workload. We aren't running the 8600s at full capacity yet, so I don't know what will happen with them. The 11/750s are paging on the system disk, and the 8600s each on different disks. I put the secondary swapfiles for each 8600 with the pagefiles, for convenience and because I had sufficient disk space to do it. >We get more memory, so we do less paging, never did swapping >in the first place, and have to make the swap file bigger. This is a pretty inane situation, I agree. Shame to have to waste so much disk space when you've tuned your system to never swap. If the swapper would always trim to WSQUOTA before swapping, we could set our WSQUOTA low and WSEXTENT high so that the swap space would be smaller. Of course, this could have other unpleasant side effects. I&DS says that a process's swap space can grow up to WSQUOTA pages, but this doesn't appear to be true on V4.2, anyway. I've totalled the working sets (rounded up to MPW_WRTCLUSTER) on my system and they appear to add up exactly to swap file in use, and some of the working sets have grown beyond WSQUOTA. ---------- *** Use your favorite editor to global search/replace <TAB> with ASCII 9. *** In EDT, it's command SUB/<TAB>/^I/WHOLE/NOTYPE $!++ $! TITLE:<TAB>SWAPFILE<TAB>Monitor system page/swapfile usage $! $! VERSION:<TAB>1-006 $! $! FACILITY:<TAB>System Management Procedure Library $! $! ABSTRACT:<TAB>Reports usage/capacity of system swapfiles, pagefiles, and $!<TAB><TAB>dumpfile. $! $! ENVIRONMENT:<TAB>DCL, privileged. $! $! AUTHOR:<TAB>Edward K. McGuire, CREATION DATE: 18-JUL-1986 $! $! MODIFIED BY: $!<TAB>1-006 EKM 18-JUL-1986<TAB>Remove system pagefile computation. $!<TAB>1-005 EKM 18-JUL-1986<TAB>Base swapfile usage on PPGCNT+GPGPNT instead $!<TAB><TAB><TAB><TAB>of on WSSIZE (Internals Manual apparently in $!<TAB><TAB><TAB><TAB>error on page 297). Fix documentation error. $!<TAB><TAB><TAB><TAB>Add code to compute system process pagefile $!<TAB><TAB><TAB><TAB>usage. $!<TAB>1-004 EKM 08-JUL-1986<TAB>Fix projection; use consistent variable names. $!<TAB>1-003 EKM 07-JUL-1986<TAB>Exclude system processes from computation $!<TAB><TAB><TAB><TAB>of file capacity projection. $!<TAB>1-002 EKM 07-JUL-1986<TAB>Refer to "processes" instead of "users". $!<TAB>1-001 EKM 03-JUL-1986<TAB>Original. $! $! FUNCTIONAL DESCRIPTION: $! $!<TAB>PROCESS REPORT: $!<TAB><TAB>Number of processes is reported, broken down by process type. $! $!<TAB>PAGE/SWAP REPORT: $!<TAB><TAB>Size of files and remaining space is reported. Additionally, $!<TAB><TAB>process capacity of the pagefiles and swapfiles is calculated. $! $!<TAB><TAB>This estimate is made by projecting from the current process $!<TAB><TAB>count and free file pages, assuming that swapfile capacity is $!<TAB><TAB>3/4 of swapfile size, and pagefile capacity is 1/2 of pagefile $!<TAB><TAB>size. $! $!<TAB><TAB>Certain processes are recognized as fixed system overhead by $!<TAB><TAB>process name, and are excluded from the estimate, improving $!<TAB><TAB>its accuracy since it is projected only from the actual workload $!<TAB><TAB>on the system. $!<TAB><TAB>-<TAB>When computing pagefile process capacity, the total $!<TAB><TAB><TAB>pagefile usage of the system processes is subtracted $!<TAB><TAB><TAB>from the pagefile capacity. This total is currently $!<TAB><TAB><TAB>always 0, because I don't know how to compute it. $!<TAB><TAB>-<TAB>When computing swapfile process capacity, the total $!<TAB><TAB><TAB>swapfile usage of the system processes is subtracted $!<TAB><TAB><TAB>from the swapfile capacity. This total is the working $!<TAB><TAB><TAB>set for each process raised to the next multiple of $!<TAB><TAB><TAB>MPW_WRTCLUSTER. $! $! CALLING SEQUENCE:<TAB>@SWAPFILE $!-- $!+ $! Get privileges. $!- $<TAB>PRIVS_NEEDED = "OPER,WORLD,BYPASS" $<TAB>OLDPRV = F$SETPRV (PRIVS_NEEDED) $<TAB>IF F$PRIVILEGE (PRIVS_NEEDED) THEN $ GOTO GOT_PRIVS $<TAB>WRITE SYS$OUTPUT PRIVS_NEEDED, " or SETPRV privileges required." $<TAB>EXIT $GOT_PRIVS: $!+ $! Get my PID for temporary files. $!- $<TAB>MYPID = F$GETJPI ("", "PID") $!+ $! Get some useful system parameters. $!- $<TAB>MAX_PRCCNT = F$GETSYI ("MAXPROCESSCNT") $<TAB>MPW_WRTCLUSTER = F$GETSYI ("MPW_WRTCLUSTER") $<TAB>PF_SIZE = F$GETSYI ("PAGEFILE_PAGE") $<TAB>PF_FREE = F$GETSYI ("PAGEFILE_FREE") $<TAB>SF_SIZE = F$GETSYI ("SWAPFILE_PAGE") $<TAB>SF_FREE = F$GETSYI ("SWAPFILE_FREE") $!+ $! Initialize process count variables. $!- $<TAB>PRCCNT = 0 $<TAB>PRCCNT_INTERACTIVE = 0 $<TAB>PRCCNT_BATCH = 0 $<TAB>PRCCNT_NETWORK = 0 $<TAB>PRCCNT_OTHER = 0 $<TAB>SYS_PRCCNT = 0 $<TAB>SYS_PF_INUSE = 0 $<TAB>SYS_SF_INUSE = 0 $!+ $! Get info on each process. $!- $PIDLOOP: $<TAB> PID = F$PID (CONTEXT) $<TAB> IF PID .EQS. "" THEN GOTO ENDPIDLOOP $<TAB> PRCCNT = PRCCNT + 1 $<TAB> MODE = F$GETJPI (PID, "MODE") $<TAB> PRCCNT_'MODE' = PRCCNT_'MODE' + 1 $<TAB> PRCNAM = F$GETJPI (PID, "PRCNAM") $<TAB> IF PRCNAM .EQS. "NULL" .OR. PRCNAM .EQS. "SWAPPER" .OR. PRCNAM .EQS. - <TAB> "ERRFMT" .OR. PRCNAM .EQS. "CACHE_SERVER" .OR. PRCNAM .EQS. - <TAB> "CLUSTER_SERVER" .OR. PRCNAM .EQS. "OPCOM" .OR. PRCNAM .EQS. - <TAB> "JOB_CONTROL" .OR. PRCNAM .EQS. "CONFIGURE" .OR. F$EXTRACT (0, 3, - <TAB> PRCNAM) .EQS. "W11" .OR. F$EXTRACT (0, 8, PRCNAM) .EQS. "SYMBIONT" - <TAB> THEN $ GOTO SYSPROC $<TAB> IF PRCNAM .EQS. "NETACP" .OR. PRCNAM .EQS. "EVL" .OR. PRCNAM .EQS. - <TAB> "REMACP" THEN $ GOTO SYSPROC $<TAB> GOTO PIDLOOP $SYSPROC: $<TAB> SYS_PRCCNT = SYS_PRCCNT + 1 $<TAB> SYS_SF_INUSE = SYS_SF_INUSE + - <TAB> ((F$GETJPI (PID, "PPGCNT") + F$GETJPI (PID, "GPGCNT")) - <TAB> / MPW_WRTCLUSTER + 1) * MPW_WRTCLUSTER $<TAB> GOTO PIDLOOP $ENDPIDLOOP: $!+ $! Get login interactive limit. $!- $<TAB>DEFINE/USER_MODE SYS$OUTPUT SWAPFILE'MYPID'.TMP $<TAB>DEFINE/USER_MODE SYS$ERROR _NL: $<TAB>SET LOGINS/INTERACTIVE $<TAB>OPEN/READ TEMP SWAPFILE'MYPID'.TMP $<TAB>READ TEMP LINE $<TAB>CLOSE TEMP $<TAB>DELETE SWAPFILE'MYPID'.TMP; $<TAB>LINE = F$EXTRACT (F$LOCATE ("= ", LINE) + 2, 255, LINE) $<TAB>MAX_PRCCNT_INTERACTIVE = 'F$EXTRACT (0, F$LOCATE (",", LINE), LINE)' $!+ $! Figure page/swapfile percent free. $!- $<TAB>PF_FREE_PCT = PF_FREE * 100 / PF_SIZE $<TAB>SF_FREE_PCT = SF_FREE * 100 / SF_SIZE $!+ $! Figure projected maximum users from free space in page/swapfile using the $! following computations. Individual steps are shown for clarity; actual $! computation is done in as few steps as possible to minimize integer division $! roundoff error. $! $! 1. Processes belonging to workload. $! $!<TAB>REAL_PRCCNT = PRCCNT - SYS_PRCCNT $! $! 2. Page/swapfile usage by workload processes. $! $!<TAB>REAL_PF_INUSE = PF_SIZE - PF_FREE - SYS_PF_INUSE $! $!<TAB>REAL_SF_INUSE = SF_SIZE - SF_FREE - SYS_SF_INUSE $! $! 4. Mean pages per workload process. $! $!<TAB><TAB><TAB> REAL_PF_INUSE $!<TAB>REAL_PRC_PF_MEAN = ------------- $!<TAB><TAB><TAB> REAL_PRCCNT $! $!<TAB><TAB><TAB> REAL_SF_INUSE $!<TAB>REAL_PRC_SF_MEAN = ------------- $!<TAB><TAB><TAB> REAL_PRCCNT $! $! 3. Page/swapfile workload capacity. $! $!<TAB><TAB> 1 $!<TAB>REAL_PF_CAP = - PF_SIZE - SYS_PF_INUSE $!<TAB><TAB> 2 $! $!<TAB><TAB> 3 $!<TAB>REAL_SF_CAP = - SF_SIZE - SYS_SF_INUSE $!<TAB><TAB> 4 $! $! 4. Page/swapfile workload process capacity. $! $!<TAB><TAB><TAB> REAL_PF_CAP $!<TAB>REAL_PF_PRC_CAP = ---------------- $!<TAB><TAB><TAB> REAL_PRC_PF_MEAN $! $!<TAB><TAB><TAB> REAL_SF_CAP $!<TAB>REAL_SF_PRC_CAP = ---------------- $!<TAB><TAB><TAB> REAL_PRC_SF_MEAN $! $! 5. Page/swapfile process capacity. $! $!<TAB>PF_PRC_CAP = REAL_PF_PRC_CAP + SYS_PRCCNT $! $!<TAB>SF_PRC_CAP = REAL_SF_PRC_CAP + SYS_PRCCNT $!- $<TAB>REAL_PRCCNT = PRCCNT - SYS_PRCCNT $<TAB>REAL_PRC_PF_MEAN = (PF_SIZE - PF_FREE - SYS_PF_INUSE) / REAL_PRCCNT $<TAB>REAL_PRC_SF_MEAN = (SF_SIZE - SF_FREE - SYS_SF_INUSE) / REAL_PRCCNT $<TAB>PF_PRC_CAP = (PF_SIZE / 2 - SYS_PF_INUSE) / REAL_PRC_PF_MEAN - <TAB> + SYS_PRCCNT $<TAB>SF_PRC_CAP = (3 * SF_SIZE / 4 - SYS_SF_INUSE) / REAL_PRC_SF_MEAN - <TAB> + SYS_PRCCNT $!+ $! Print report. $!- $<TAB>WRITE SYS$OUTPUT F$FAO ("Processes: !UL/!UL (Inter: !UL/!UL, " - <TAB> + "Batch: !UL, Network: !UL, Other: !UL)", PRCCNT, MAX_PRCCNT, - <TAB> PRCCNT_INTERACTIVE, MAX_PRCCNT_INTERACTIVE, PRCCNT_BATCH, - <TAB> PRCCNT_NETWORK, PRCCNT_OTHER) $<TAB>WRITE SYS$OUTPUT F$FAO ("System Processes: !3UL Pagefile: " - <TAB> + "!4UL Swapfile: !4UL", SYS_PRCCNT, SYS_PF_INUSE, SYS_SF_INUSE) $<TAB>WRITE SYS$OUTPUT F$FAO ("Workload Processes: !3UL Mean Pagefile: " - <TAB> + "!4UL Mean Swapfile: !4UL", REAL_PRCCNT, REAL_PRC_PF_MEAN, - <TAB> REAL_PRC_SF_MEAN) $<TAB>WRITE SYS$OUTPUT F$FAO ("Free Page File: !6UL/!6UL (!2UL%) Projected" - <TAB> + " capacity !3UL processes", PF_FREE, PF_SIZE, PF_FREE_PCT, - <TAB> PF_PRC_CAP) $<TAB>WRITE SYS$OUTPUT F$FAO ("Free Swap file: !6UL/!6UL (!2UL%) Projected" - <TAB> + " capacity !3UL processes", SF_FREE, SF_SIZE, SF_FREE_PCT, - <TAB> SF_PRC_CAP) $!+ $! Finished. $!- $<TAB>TEMP = F$SETPRV (OLDPRV) $<TAB>EXIT