jjp@necis.UUCP (Jeff Phillips) (05/23/91)
A friend of mine is writing a paper on balanced system approach. In it he makes the assertion that "...(the UNIX operating system is) too large to fit in system RAM all at once, therefore pieces of the operating system are swapped between system RAM and disk, thereby generating even more disk I/O requests." Seven years ago I learned that one of the definitions for kernel is "memory resident software". Is the kernel always memory resident, or do parts of it get paged (or swapped) out to the swap partition? If it does go to the swapper, what algorithm is used to determine which kernel subsystem (i/o, file, process, or dev.drv's) gets swapped out? @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@ Jeffrey J. Phillips UUCP: jjp@necis.nec.com @@ @@ NEC Technologies, Inc. PHONE: (508)635-6077 @@ @@ "UNIX isn't a philosophy, it's a way of life" - anon @@ @@ CASUAL DISCLAIMER: Opinions are mine - definitely not corporate policy @@ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
ken@dali.cc.gatech.edu (Ken Seefried iii) (05/23/91)
In article <1423@necis.UUCP> jjp@necis.UUCP (Jeff Phillips) writes: >A friend of mine is writing a paper on balanced system approach. In it he >makes the assertion that "...(the UNIX operating system is) too large to fit >in system RAM all at once, therefore pieces of the operating system are swapped >between system RAM and disk, thereby generating even more disk I/O requests." Your friend has absolutely no idea what he's talking about. In any traditional Unix system (V6, V7, Sys[35], BSD), the kernel is never swapped. If you don't have enough physical memory to hold the kernel, you can't boot. Seems to me maybe he should have actually done some research before writing this "paper". Good places to start would be Bach's "Design of the Unix Operating System", Leffler, et.al. "The Design and Implimentation of the Unix Operating System" and Quarterman, et.al., "4.2BSD and 4.3BSD as Examples of the Unix Operating System" in ACM Computing Surveys, December, 1985. -- ken seefried iii "I'll have what the gentleman ken@dali.cc.gatech.edu on the floor is having..."
dvk@sei.cmu.edu (Daniel Klein) (05/23/91)
In article <1423@necis.UUCP>, jjp@necis.UUCP (Jeff Phillips) writes: |> A friend of mine is writing a paper on balanced system approach. In it he |> makes the assertion that "...(the UNIX operating system is) too large to fit |> in system RAM all at once, therefore pieces of the operating system are swapped |> between system RAM and disk, thereby generating even more disk I/O requests." I believe that your friend is sadly mistaken. Unless things have changed radically in the past year or two, the kernel is wholly memory resident. Some so-called "system" processes are swap/pageable in the same way that "user" processes are. Some examples are the FTP daemon, the RLOGIN daemon, and so forth). Even these are not true "system" processes. "System" processes (like the swapper and page daemon) are not swapped or paged (for obvious reasons). -- ============ -- =========== -- =========== -- =========== -- =========== -- "The only thing that separates us from the animals is superstition and mindless rituals" (Latke) Daniel Klein CMU-SEI +1 412/268-7791 dvk@sei.cmu.edu
vandys@sequent.com (Andrew Valencia) (05/24/91)
jjp@necis.UUCP (Jeff Phillips) writes: >A friend of mine is writing a paper on balanced system approach. In it he >makes the assertion that "...(the UNIX operating system is) too large to fit >in system RAM all at once, therefore pieces of the operating system are swapped >between system RAM and disk, thereby generating even more disk I/O requests." Many people will tell you that the kernel is not paged or swapped to disk. This is right so far as it goes, however there are certainly kernel data structures which go out to disk and come back, and these definitely add overhead. In most UNIX implementations, the "u" area and the kernel stack are swapped out along with the rest of a user process which is being swapped out. I believe the PTEs and DBDs from a Regions-based kernel are also swapped out when the last in-core process referencing the particular region is swapped out. Similarly in BSD, the PTEs from a process are swapped out with the process; the PTEs for a mapped file are swapped out when the last in-core process is swapped which references that mapped file. I agree that the UNIX OS is probably getting too large to fit in RAM all at once. I am not convinced that pageable kernels is the answer. Different organizations--especially microkernels--will probably become increasingly important in the next 5 years. "kernel" memory use is only one of many issues which microkernels seem to address better than current UNIX technology. But enough of the soap box.... I've pared the newsgroups line down to unix.wizards; I think that'll provide more than adequate coverage for this topic. Andy Valencia vandys@sequent.com
mouse@thunder.mcrcim.mcgill.edu (der Mouse) (05/24/91)
In article <1423@necis.UUCP>, jjp@necis.UUCP (Jeff Phillips) writes: > A friend of mine is writing a paper on balanced system approach. In > it he makes the assertion that "...(the UNIX operating system is) too > large to fit in system RAM all at once, [...]" This is true only if the term is interpreted very broadly. All UNIXish systems I know of keep the kernel permanently resident; you have to take "the system" to include things like the shells and compilers and other utilities for the statement to be true. Someone else pointed out that u-areas and kernel stacks are swapped and/or paged in some UNIX(ish?) systems. This is probably true but is sort of irrelevant, since they would *fit*, it's just that the system believes it can make better use of the space - and the statement was that things wouldn't fit. One can also consider those as part of the process, and nobody argues that processes have to be swapped. As an aside, I've used systems (well, a system - a Sun-4/something) which had as much RAM as swap space. Sure made for snappy response. der Mouse old: mcgill-vision!mouse new: mouse@larry.mcrcim.mcgill.edu
jfh@rpp386.cactus.org (John F Haugh II) (05/24/91)
In article <1991May24.065316.27331@thunder.mcrcim.mcgill.edu> mouse@thunder.mcrcim.mcgill.edu (der Mouse) writes: >In article <1423@necis.UUCP>, jjp@necis.UUCP (Jeff Phillips) writes: >> A friend of mine is writing a paper on balanced system approach. In >> it he makes the assertion that "...(the UNIX operating system is) too >> large to fit in system RAM all at once, [...]" > >This is true only if the term is interpreted very broadly. All UNIXish >systems I know of keep the kernel permanently resident; you have to >take "the system" to include things like the shells and compilers and >other utilities for the statement to be true. The AIX v3 kernel is pagable. It is divided into three "segments", as it were. The first is code that is only required for initialization. The second is "pinned" code that must always be present (like the page fault handler ;-) and the third is pagable code [ by "code" I mean object code - that is, text and data ] The various system tables are defined with huge sizes, and the system just page faults in the new pages for the kernel tables as it needs them, thus ending the dilema about creating new proc table or whateve entries by rebuilding the system. On a typical system the size of the kernel is somewheres near 20MB, much of which is never even used - it just remains off in virtual la-la land waiting to be referenced ... -- John F. Haugh II | Distribution to | UUCP: ...!cs.utexas.edu!rpp386!jfh Ma Bell: (512) 255-8251 | GEnie PROHIBITED :-) | Domain: jfh@rpp386.cactus.org "If liberals interpreted the 2nd Amendment the same way they interpret the rest of the Constitution, gun ownership would be mandatory."
mal1@pyuxf.UUCP (maureen lecuona) (05/24/91)
The kernel image is not swapped. It exists in ALL process images' memory maps. Since the kernel in reentrant, it would be foolish to page or swap it. Maureen Lecuona Integrated Business Solutions, Inc. 4 Spring Lane Long Valley, N.J. 07853 (908) 850-0174
richard@aiai.ed.ac.uk (Richard Tobin) (05/27/91)
In article <1991May23.170918.11612@sequent.com> vandys@sequent.com (Andrew Valencia) writes: >I agree that the UNIX OS is probably getting too large to fit in RAM all >at once. Several unix kernels require 1-2 Mbytes. I haven't seen more than this. Thus, the kernel fits into about $50 worth of memory. -- Richard -- Richard Tobin, JANET: R.Tobin@uk.ac.ed AI Applications Institute, ARPA: R.Tobin%uk.ac.ed@nsfnet-relay.ac.uk Edinburgh University. UUCP: ...!ukc!ed.ac.uk!R.Tobin
richard@locus.com (Richard M. Mathews) (05/30/91)
jfh@rpp386.cactus.org (John F Haugh II) writes: >The AIX v3 kernel is pagable. And AIX V3 isn't the only pagable Unix kernel, and I suspect this is going to get common soon. This is wandering off the subject a bit, but I thought I'd mention some of the fun involved in making Unix pagable since it sounds like most of the "wizards" out there haven't played with this at all. There are many, many places where Unix kernel code assumes that there are only 3 ways that something can magically change: an interrupt can change things; hardware registers can change; and anything can change if we allow a context switch. If you know that the first 2 don't happen or are irrelevant, then all you have to do is avoid calling swtch() (or something that calls it, most obviously sleep()); and then you know that you have complete control. If, however, you can take page faults in the kernel, then you can get context switched on any line of code. You better hunt out each place where such assumptions are made, and either temporarily pin down the pages you need or get a lock for the appropiate resource. And then you better think about deadlock and performance implications of keeping things pinned or locked. It's a lot of fun! Note, by the way, that the work to make a kernel pagable is similar to the work needed to make it work with multiple processors. Richard M. Mathews Freedom for Lithuania richard@locus.com Laisve! lcc!richard@seas.ucla.edu ...!{uunet|ucla-se|turnkey}!lcc!richard
michael@xzaphod.uucp (Michael R. Miller) (05/30/91)
In article <4814@skye.ed.ac.uk> richard@aiai.UUCP (Richard Tobin) writes: >In article <1991May23.170918.11612@sequent.com> vandys@sequent.com (Andrew Valencia) writes: >>I agree that the UNIX OS is probably getting too large to fit in RAM all >>at once. > >Several unix kernels require 1-2 Mbytes. I haven't seen more than this. >Thus, the kernel fits into about $50 worth of memory. You've not seen the UNIX 3.2 from SCO. 2.2 Meg is one machine's kernel size. > >-- Richard >-- >Richard Tobin, JANET: R.Tobin@uk.ac.ed >AI Applications Institute, ARPA: R.Tobin%uk.ac.ed@nsfnet-relay.ac.uk >Edinburgh University. UUCP: ...!ukc!ed.ac.uk!R.Tobin Michael R. Miller uunet!xzaphod!michael
wes@harem.clydeunix.com (Barnacle Wes) (06/01/91)
In article <19332@rpp386.cactus.org>, jfh@rpp386.cactus.org (John F Haugh II) writes:
% The AIX v3 kernel is pagable. It is divided into three "segments", as
% it were. The first is code that is only required for initialization.
% The second is "pinned" code that must always be present (like the page
% fault handler ;-) and the third is pagable code [ by "code" I mean
% object code - that is, text and data ]
%
% The various system tables are defined with huge sizes, and the system
% just page faults in the new pages for the kernel tables as it needs
% them, thus ending the dilema about creating new proc table or whateve
% entries by rebuilding the system. On a typical system the size of the
% kernel is somewheres near 20MB, much of which is never even used - it
% just remains off in virtual la-la land waiting to be referenced ...
But the tables still require page space, right? V.4's dynamically-sized
tables sound like a better idea.
Wes Peters
--
#include <std/disclaimer.h> The worst day sailing
My opinions, your screen. is much better than
Raxco had nothing to do with this! the best day at work.
Wes Peters: wes@harem.clydeunix.com ...!sun!unislc!harem!wesscl@unislc.uucp (Sean Landis) (06/05/91)
richard@aiai.ed.ac.uk (Richard Tobin) writes: >Several unix kernels require 1-2 Mbytes. I haven't seen more than this. >Thus, the kernel fits into about $50 worth of memory. >-- Richard >-- >Richard Tobin, JANET: R.Tobin@uk.ac.ed >AI Applications Institute, ARPA: R.Tobin%uk.ac.ed@nsfnet-relay.ac.uk >Edinburgh University. UUCP: ...!ukc!ed.ac.uk!R.Tobin Do you mean: size /unix? I rarely see a unix *image* less than 2.5 Mb. Of course, it all depends what you're running and how you tune. For example, one the system I use every day, the buffer cache alone is 2.5 Mb. But my process load is extremely memory intensive: gnu emacs, X window, LaTeX, etc. I find that 3-4 Mb is typical. As an aside, I find that it's not very easy to determine the exact image of unix. Anyone have any good ideas on that? Sean -- Sean C. Landis | {hpda, sun, uplherc}!unislc!scl Unisys Open Systems Group | unislc!scl@cs.utah.edu 320 North 2200 West B2D01 | (801) 594-3988 Salt Lake City, Utah 84116 | (801) 594-3827 Fax
richard@aiai.ed.ac.uk (Richard Tobin) (06/06/91)
In article <1991Jun5.162705.3325@unislc.uucp> scl@unislc.uucp (Sean Landis) writes: >Do you mean: size /unix? I rarely see a unix *image* less than 2.5 Mb. Of >course, it all depends what you're running and how you tune. For example, >one the system I use every day, the buffer cache alone is 2.5 Mb. People keep saying to me "what about the buffer cache", but we were discussing whether the kernel was so large that it should be paged. Paging the buffer cache is a rather strange idea, to say the least. -- Richard -- Richard Tobin, JANET: R.Tobin@uk.ac.ed AI Applications Institute, ARPA: R.Tobin%uk.ac.ed@nsfnet-relay.ac.uk Edinburgh University. UUCP: ...!ukc!ed.ac.uk!R.Tobin