jjp@necis.UUCP (Jeff Phillips) (05/23/91)
A friend of mine is writing a paper on balanced system approach. In it he makes the assertion that "...(the UNIX operating system is) too large to fit in system RAM all at once, therefore pieces of the operating system are swapped between system RAM and disk, thereby generating even more disk I/O requests." Seven years ago I learned that one of the definitions for kernel is "memory resident software". Is the kernel always memory resident, or do parts of it get paged (or swapped) out to the swap partition? If it does go to the swapper, what algorithm is used to determine which kernel subsystem (i/o, file, process, or dev.drv's) gets swapped out? @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@ Jeffrey J. Phillips UUCP: jjp@necis.nec.com @@ @@ NEC Technologies, Inc. PHONE: (508)635-6077 @@ @@ "UNIX isn't a philosophy, it's a way of life" - anon @@ @@ CASUAL DISCLAIMER: Opinions are mine - definitely not corporate policy @@ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
ken@dali.cc.gatech.edu (Ken Seefried iii) (05/23/91)
In article <1423@necis.UUCP> jjp@necis.UUCP (Jeff Phillips) writes: >A friend of mine is writing a paper on balanced system approach. In it he >makes the assertion that "...(the UNIX operating system is) too large to fit >in system RAM all at once, therefore pieces of the operating system are swapped >between system RAM and disk, thereby generating even more disk I/O requests." Your friend has absolutely no idea what he's talking about. In any traditional Unix system (V6, V7, Sys[35], BSD), the kernel is never swapped. If you don't have enough physical memory to hold the kernel, you can't boot. Seems to me maybe he should have actually done some research before writing this "paper". Good places to start would be Bach's "Design of the Unix Operating System", Leffler, et.al. "The Design and Implimentation of the Unix Operating System" and Quarterman, et.al., "4.2BSD and 4.3BSD as Examples of the Unix Operating System" in ACM Computing Surveys, December, 1985. -- ken seefried iii "I'll have what the gentleman ken@dali.cc.gatech.edu on the floor is having..."
dvk@sei.cmu.edu (Daniel Klein) (05/23/91)
In article <1423@necis.UUCP>, jjp@necis.UUCP (Jeff Phillips) writes: |> A friend of mine is writing a paper on balanced system approach. In it he |> makes the assertion that "...(the UNIX operating system is) too large to fit |> in system RAM all at once, therefore pieces of the operating system are swapped |> between system RAM and disk, thereby generating even more disk I/O requests." I believe that your friend is sadly mistaken. Unless things have changed radically in the past year or two, the kernel is wholly memory resident. Some so-called "system" processes are swap/pageable in the same way that "user" processes are. Some examples are the FTP daemon, the RLOGIN daemon, and so forth). Even these are not true "system" processes. "System" processes (like the swapper and page daemon) are not swapped or paged (for obvious reasons). -- ============ -- =========== -- =========== -- =========== -- =========== -- "The only thing that separates us from the animals is superstition and mindless rituals" (Latke) Daniel Klein CMU-SEI +1 412/268-7791 dvk@sei.cmu.edu
mouse@thunder.mcrcim.mcgill.edu (der Mouse) (05/24/91)
In article <1423@necis.UUCP>, jjp@necis.UUCP (Jeff Phillips) writes: > A friend of mine is writing a paper on balanced system approach. In > it he makes the assertion that "...(the UNIX operating system is) too > large to fit in system RAM all at once, [...]" This is true only if the term is interpreted very broadly. All UNIXish systems I know of keep the kernel permanently resident; you have to take "the system" to include things like the shells and compilers and other utilities for the statement to be true. Someone else pointed out that u-areas and kernel stacks are swapped and/or paged in some UNIX(ish?) systems. This is probably true but is sort of irrelevant, since they would *fit*, it's just that the system believes it can make better use of the space - and the statement was that things wouldn't fit. One can also consider those as part of the process, and nobody argues that processes have to be swapped. As an aside, I've used systems (well, a system - a Sun-4/something) which had as much RAM as swap space. Sure made for snappy response. der Mouse old: mcgill-vision!mouse new: mouse@larry.mcrcim.mcgill.edu
jfh@rpp386.cactus.org (John F Haugh II) (05/24/91)
In article <1991May24.065316.27331@thunder.mcrcim.mcgill.edu> mouse@thunder.mcrcim.mcgill.edu (der Mouse) writes: >In article <1423@necis.UUCP>, jjp@necis.UUCP (Jeff Phillips) writes: >> A friend of mine is writing a paper on balanced system approach. In >> it he makes the assertion that "...(the UNIX operating system is) too >> large to fit in system RAM all at once, [...]" > >This is true only if the term is interpreted very broadly. All UNIXish >systems I know of keep the kernel permanently resident; you have to >take "the system" to include things like the shells and compilers and >other utilities for the statement to be true. The AIX v3 kernel is pagable. It is divided into three "segments", as it were. The first is code that is only required for initialization. The second is "pinned" code that must always be present (like the page fault handler ;-) and the third is pagable code [ by "code" I mean object code - that is, text and data ] The various system tables are defined with huge sizes, and the system just page faults in the new pages for the kernel tables as it needs them, thus ending the dilema about creating new proc table or whateve entries by rebuilding the system. On a typical system the size of the kernel is somewheres near 20MB, much of which is never even used - it just remains off in virtual la-la land waiting to be referenced ... -- John F. Haugh II | Distribution to | UUCP: ...!cs.utexas.edu!rpp386!jfh Ma Bell: (512) 255-8251 | GEnie PROHIBITED :-) | Domain: jfh@rpp386.cactus.org "If liberals interpreted the 2nd Amendment the same way they interpret the rest of the Constitution, gun ownership would be mandatory."
peter@ficc.ferranti.com (Peter da Silva) (05/24/91)
In article <29696@hydra.gatech.EDU> ken@dali.cc.gatech.edu (Ken Seefried iii) writes: > Your friend has absolutely no idea what he's talking about. In > any traditional Unix system (V6, V7, Sys[35], BSD), the kernel > is never swapped. Weren't there some varieties of 2BSD that had an overlayed kernel on the PDP-11? But that was pretty late in the life of 2BSD, when they were putting all sorts of 4BSD stuff in it. I think it was 2.11 that started doing this. (when was the last time you ran into an overlayed program?) -- Peter da Silva; Ferranti International Controls Corporation; +1 713 274 5180; Sugar Land, TX 77487-5012; `-_-' "Have you hugged your wolf, today?"
mal1@pyuxf.UUCP (maureen lecuona) (05/24/91)
The kernel image is not swapped. It exists in ALL process images' memory maps. Since the kernel in reentrant, it would be foolish to page or swap it. Maureen Lecuona Integrated Business Solutions, Inc. 4 Spring Lane Long Valley, N.J. 07853 (908) 850-0174
greg@travis.cica.indiana.edu (Gregory TRAVIS) (05/25/91)
In <48JBS77@xds13.ferranti.com> peter@ficc.ferranti.com (Peter da Silva) writes: >In article <29696@hydra.gatech.EDU> ken@dali.cc.gatech.edu (Ken Seefried iii) writes: >> Your friend has absolutely no idea what he's talking about. In >> any traditional Unix system (V6, V7, Sys[35], BSD), the kernel >> is never swapped. >Weren't there some varieties of 2BSD that had an overlayed kernel on the >PDP-11? But that was pretty late in the life of 2BSD, when they were putting >all sorts of 4BSD stuff in it. I think it was 2.11 that started doing this. >(when was the last time you ran into an overlayed program?) Not overlayed in the traditional sense. From 2.8BSD and beyond, the kernel text (program) size had bloated beyond the 64K barrier. The solution was to modify the linker so that multiple "overlays" of the kernel were linked to begin at the same spot. What you ended up with was a microkernel whos size was a multiple of 8K. Then you had a whole bunch of chunks of the rest of the kernel laying around that were 64K - <size of microkernel> into which you put the various other parts of the kernel. The PDP-11 segment registers allowed the 64K program space to be divided up into eight 8K segments. With the above scheme, the kernel could be 300K of code - the crt0.o code and the code tacked onto the beginning of every function took care of modifying the segment registers on the fly to map in the appropriate code for whatever syscall was going on. For performance reasons you usually wanted to make sure that stuff like machdep.c, the trap code, the tty driver, the disk drivers, and the scheduler all lived in the microkernel. There was no function call overhead when calling routines in the microkernel. You also wanted to put routines that were likely to call each other in the same "overlay." The highest overhead was encountered when a routine in overlay A called a routine in overlay B. Swapping out parts of the kernel was never part of the plan. All of the code for the kernel, even if it was 300K, was locked into core from system startup to shutdown. All in continguous memory. Only the hardware segment registers were modified to bring the appropriate pieces of this code into the 64K "execution window." This overlay scheme was also used at the application level. Big Emacs would have been impossible without it! And it was almost completely transparent to the programmer. greg -- Gregory R. Travis Indiana University, Bloomington IN 47405 greg@cica.indiana.edu Center for Innovative Computer Applications
guy@auspex.auspex.com (Guy Harris) (05/27/91)
>The kernel image is not swapped. It exists in ALL process images' memory >maps. Since the kernel in reentrant, it would be foolish to page or swap it. Umm, what do you mean by "reentrant" here?
richard@locus.com (Richard M. Mathews) (05/30/91)
jfh@rpp386.cactus.org (John F Haugh II) writes: >The AIX v3 kernel is pagable. And AIX V3 isn't the only pagable Unix kernel, and I suspect this is going to get common soon. This is wandering off the subject a bit, but I thought I'd mention some of the fun involved in making Unix pagable since it sounds like most of the "wizards" out there haven't played with this at all. There are many, many places where Unix kernel code assumes that there are only 3 ways that something can magically change: an interrupt can change things; hardware registers can change; and anything can change if we allow a context switch. If you know that the first 2 don't happen or are irrelevant, then all you have to do is avoid calling swtch() (or something that calls it, most obviously sleep()); and then you know that you have complete control. If, however, you can take page faults in the kernel, then you can get context switched on any line of code. You better hunt out each place where such assumptions are made, and either temporarily pin down the pages you need or get a lock for the appropiate resource. And then you better think about deadlock and performance implications of keeping things pinned or locked. It's a lot of fun! Note, by the way, that the work to make a kernel pagable is similar to the work needed to make it work with multiple processors. Richard M. Mathews Freedom for Lithuania richard@locus.com Laisve! lcc!richard@seas.ucla.edu ...!{uunet|ucla-se|turnkey}!lcc!richard
wes@harem.clydeunix.com (Barnacle Wes) (06/01/91)
In article <19332@rpp386.cactus.org>, jfh@rpp386.cactus.org (John F Haugh II) writes:
% The AIX v3 kernel is pagable. It is divided into three "segments", as
% it were. The first is code that is only required for initialization.
% The second is "pinned" code that must always be present (like the page
% fault handler ;-) and the third is pagable code [ by "code" I mean
% object code - that is, text and data ]
%
% The various system tables are defined with huge sizes, and the system
% just page faults in the new pages for the kernel tables as it needs
% them, thus ending the dilema about creating new proc table or whateve
% entries by rebuilding the system. On a typical system the size of the
% kernel is somewheres near 20MB, much of which is never even used - it
% just remains off in virtual la-la land waiting to be referenced ...
But the tables still require page space, right? V.4's dynamically-sized
tables sound like a better idea.
Wes Peters
--
#include <std/disclaimer.h> The worst day sailing
My opinions, your screen. is much better than
Raxco had nothing to do with this! the best day at work.
Wes Peters: wes@harem.clydeunix.com ...!sun!unislc!harem!wes
wes@harem.clydeunix.com (Barnacle Wes) (06/01/91)
In article <greg.675107364@travis>, greg@travis.cica.indiana.edu (Gregory TRAVIS) writes: > Swapping out parts of the kernel was never part of the plan. All of the > code for the kernel, even if it was 300K, was locked into core from system > startup to shutdown. All in continguous memory. Only the hardware > segment registers were modified to bring the appropriate pieces of this > code into the 64K "execution window." This scheme should sound familiar to everyone reading this thread in comp.unix.sysv286; it is similar to the way a 286 kernel is managed. My current (V/AT 2.3) kernel has 3 text segments. The 286 doesn't have, or need, the microkernel concept discussed earlier in Greg's post, but the segment loading is very similar in concept. > This overlay scheme was also used at the application level. Big Emacs would > have been impossible without it! And it was almost completely transparent > to the programmer. Ditto for V/AT. Just compile with -Ml to get multiple-segment programs. Wes Peters -- #include <std/disclaimer.h> The worst day sailing My opinions, your screen. is much better than Raxco had nothing to do with this! the best day at work. Wes Peters: wes@harem.clydeunix.com ...!sun!unislc!harem!wes
jfh@rpp386.cactus.org (John F Haugh II) (06/02/91)
In article <311@harem.clydeunix.com> wes@harem.clydeunix.com (Barnacle Wes) writes: >But the tables still require page space, right? V.4's dynamically-sized >tables sound like a better idea. Only when they are used, in which case so do SVR4's - you just see it in a different place, namely the user process. If the kernel is taking up the memory the user isn't free to do so as well, so it must page. So when SVR4 grows a table (or whatever, I've not seen the kernel sources yet), it deprives a user process of available physical memory which must then be backed with paging store. When AIX grows a table the same thing happens - except it is the kernel table that gets backed (except when it is paged in, and then all bets are off ;-). -- John F. Haugh II | Distribution to | UUCP: ...!cs.utexas.edu!rpp386!jfh Ma Bell: (512) 255-8251 | GEnie PROHIBITED :-) | Domain: jfh@rpp386.cactus.org "If liberals interpreted the 2nd Amendment the same way they interpret the rest of the Constitution, gun ownership would be mandatory."