BEAR@S34.Prime.COM (08/23/89)
I believe what you're looking for is a working set scheduler. In a nutshell, this type of scheduler pages a job against itself rather than against the system (i.e. the pool of available pages belongs to the process (rather than the system) and is of a size set by the system when then process is created). Unfortunately, I can't think of an available OS that uses a WS scheduler off the top of my head (that doesn't mean they don't exist!). Good luck. -- Bob Beckwith Prime Computer, Inc. Internet: bear@s34.prime.com
gmb@iclswe.UUCP (Grahame Budd) (08/25/89)
In article <261500008@S34.Prime.COM> BEAR@S34.Prime.COM writes: > > I believe what you're looking for is a working set scheduler. In a >nutshell, this type of scheduler pages a job against itself rather than >against the system (i.e. the pool of available pages belongs to the process >(rather than the system) and is of a size set by the system when then process >is created). Unfortunately, I can't think of an available OS that uses a WS >scheduler off the top of my head (that doesn't mean they don't exist!). Good >luck. > >Bob Beckwith The VME operating system from ICL(UK) uses this technique and extends it by maintaining working set 'quota's for both jobs and classes of jobs. For example all interactive development work has a store 'quota' which is the sum of the quotas of the active, (i.e. paged in, ready to run), work in that class. The total of all quotas is normally less than the total available real store, the rest is then shared between needy processes by priority. When a process is created it has a start quota which can be modified, (within limits), as the work runs to reflect changing demands. Processes work well when the actual working set is less than the working set quota. The technique is refined by the concept of 'policy'. A policy is a set of scheduling values which define processor priority, store priority, timeslice values, comms priority and quotas. Each class of work is assigned to a policy which then defines how work of a given class competes against other classes of work. Work which exceeds the values set in its policy is demoted so that, for example, an interactive user who starts doing long compilations in foreground will have his process demoted s o that it runs in a batch-like rather than interactive-like way. The actual system is more detailed than this, for example a process which holds a resource required by a higher priority process is given the higher priority until the resource is released. However for a system which makes decisions at the kernel level on a milisecond by milisecond basis, and as such cannot use toomuch system resource, it works very well. Together with higher level schedulers, which adjust policy values, assign work to classes and adjust quotas, the system as a whole gives very good mixed workload performance.
jonasn@ttds.UUCP (Jonas Nygren) (08/25/89)
In article <261500008@S34.Prime.COM> BEAR@S34.Prime.COM writes: > > I believe what you're looking for is a working set scheduler. In a >nutshell, this type of scheduler pages a job against itself rather than >against the system (i.e. the pool of available pages belongs to the process >(rather than the system) and is of a size set by the system when then process >is created). Unfortunately, I can't think of an available OS that uses a WS >scheduler off the top of my head (that doesn't mean they don't exist!). Good >luck. > >-- > >Bob Beckwith >Prime Computer, Inc. > >Internet: bear@s34.prime.com Hasn't RT/AIX some similar feature to working-set scheduling. I have a vague remembrance that this was stated in something I read. Can anybody confirm? (This is a secret poll to estimate the number of RTs out there :-) /jonas
hascall@atanasoff.cs.iastate.edu (John Hascall) (08/25/89)
} I believe what you're looking for is a working set scheduler. In a }nutshell, this type of scheduler pages a job against itself rather than }against the system (i.e. the pool of available pages belongs to the process }(rather than the system) and is of a size set by the system when then process }is created). Unfortunately, I can't think of an available OS that uses a WS }scheduler off the top of my head (that doesn't mean they don't exist!). Good }luck. Well, gee, how about VMS? John Hascall ISU Comp Center This .signature left intentionally blank. I F N O E D W D S E R
mcdonald@uxe.cso.uiuc.edu (08/26/89)
One solution to the thrashing problem (in the particular case in point, one or a few processes which are much too big for memory) is to break down and recode the program so that it does explicit IO to get the necessary pages. This can be particularly helpful if just ONE process is too big for memory. There are people in our department who diagonalize 1,000,000x1,000,000 sparse matrices (with maybe 50,000,000 non-zero elements) on machines with 16 megabytes of memory, and nary a page fault ever occurs. Sometimes they can get full CPU utrilization with just one process. What this DOES require, of course, is the ability to do disk reads and writes concurrently with program execution. Some operating systems don't allow this, at least not efficiently. Doug McDonald
gary@dgcad.SV.DG.COM (Gary Bridgewater) (08/27/89)
In article <261500008@S34.Prime.COM> BEAR@S34.Prime.COM writes: >... Unfortunately, I can't think of an available OS that uses a WS >scheduler off the top of my head (that doesn't mean they don't exist!). Good >luck. How about DG's AOS/VS. No luck required. Quite available. Another poster points out that VMS has one also. All to available :->. I'm moderately surprised that Unix doesn't have such a thing. It does have the concept of limiting other things. In its simplest form it can be accomplished by setting a maximum working set size on the process. When the process needs another page and there is no free one and it has its limit - page it against itself using whatever method is popular at the moment. Note: don't drag out that tired old 'memory is cheap and getting cheaper so just go real' cliche. Data requirements can outstrip memory pricing without even breathing hard. One job at a time? Grow up. -- Gary Bridgewater, Data General Corp., Sunnyvale Ca. gary@sv4.ceo.sv.dg.com or {amdahl,aeras,amdcad,mas1,matra3}!dgcad.SV.DG.COM!gary No good deed goes unpunished.
callen@inmet (08/28/89)
This looks like an area where IBM's MVS operating system (bless its black little heart) is ahead of the pack. MVS has something called the "System Resource Manager" (SRM) which constantly watches three major areas of contention: CPU, main storage and IO. Each batch job or interactive user in the system belongs to a "performance group" with specific performance "objectives." SRM's job is to dole out those three major resources in a manner that will fulfill each performance group's objectives. If, for instance, the system's paging rate is "excessive" (a slippery term, of course), SRM will try to find the worst offender and swap that process out. "Swap out" means use bulk paging to dump the most heavily referenced pages to disk in one fast operation, then let the remaining pages be "stolen" as necessary by other processes. SRM has gobs of parameters to be tuned by, unfortunately, a human, but the complexity can pay off big time if the workload can reasonably be categorized into distinct kinds of work. And the example I gave above was horribly simplified; I'm not up to keying in the hundred or so pages that describe the SRM in all its gory detail. -- Jerry "I can' believe I'm defending MVS" Callen ...!uunet!inmet!callen
jesup@cbmvax.UUCP (Randell Jesup) (08/29/89)
In article <1072@svx.SV.DG.COM> gary@svx.SV.DG.COM () writes: > In article <261500008@S34.Prime.COM> BEAR@S34.Prime.COM writes: > >... Unfortunately, I can't think of an available OS that uses a WS > >scheduler off the top of my head (that doesn't mean they don't exist!). Good > >luck. > >How about DG's AOS/VS. No luck required. Quite available. >Another poster points out that VMS has one also. All to available :->. Stratus VOS also using working sets. -- Randell Jesup, Keeper of AmigaDos, Commodore Engineering. {uunet|rutgers}!cbmvax!jesup, jesup@cbmvax.cbm.commodore.com BIX: rjesup Common phrase heard at Amiga Devcon '89: "It's in there!"
dswartz@bbn.com (Dan Swartzendruber) (08/29/89)
In article <7764@cbmvax.UUCP> jesup@cbmvax.UUCP (Randell Jesup) writes: >In article <1072@svx.SV.DG.COM> gary@svx.SV.DG.COM () writes: >> In article <261500008@S34.Prime.COM> BEAR@S34.Prime.COM writes: >> >... Unfortunately, I can't think of an available OS that uses a WS >> >scheduler off the top of my head (that doesn't mean they don't exist!). Good >> >luck. >> >>How about DG's AOS/VS. No luck required. Quite available. >>Another poster points out that VMS has one also. All to available :->. > > Stratus VOS also using working sets. > Sorry, the VOS scheduler hasn't used working sets for about 2 years...
jesup@cbmvax.UUCP (Randell Jesup) (08/30/89)
In article <44914@bbn.COM> dswartz@BBN.COM (Dan Swartzendruber) writes: >In article <7764@cbmvax.UUCP> jesup@cbmvax.UUCP (Randell Jesup) writes: >> Stratus VOS is also using working sets. >> >Sorry, the VOS scheduler hasn't used working sets for about 2 years... Sorry, I haven't had access to VOS in the last 3 years. -- Randell Jesup, Keeper of AmigaDos, Commodore Engineering. {uunet|rutgers}!cbmvax!jesup, jesup@cbmvax.cbm.commodore.com BIX: rjesup Common phrase heard at Amiga Devcon '89: "It's in there!"
asg@pyuxf.UUCP (alan geller) (08/31/89)
In article <7764@cbmvax.UUCP>, jesup@cbmvax.UUCP (Randell Jesup) writes: > > In article <261500008@S34.Prime.COM> BEAR@S34.Prime.COM writes: > > >... Unfortunately, I can't think of an available OS that uses a WS > > >scheduler off the top of my head (that doesn't mean they don't exist!). Good > > >luck. > Stratus VOS also using working sets. Well, it used to, but the last few releases (i.e., for about two years now), it hasn't. Of course, there are still references to working sets in the documentation, and in the system meters, and so forth, but since about 5.0, VOS has used a system page pool. Alan Geller Bellcore (I use Stratus/(IBM System88) computers on the job, but I don't work for them, nor own stock, etc.). (Finally, a comp.arch discussion I can take meaningful part in!!)