kevin@cfctech.cfc.com (Kevin Darcy) (06/15/91)
In article <1991Jun13.192444.7500@ucunix.san.uc.edu> adams@ucunix.san.uc.edu (J. Adams - SunOS Wizard) writes: >In article <1991Jun13.122803.25362@ims.alaska.edu> floyd@ims.alaska.edu >(Floyd Davidson) writes: >>[...] >>In article <1991Jun13.065207.10089@ucunix.san.uc.edu> >>adams@ucunix.san.uc.edu (J. Adams - SunOS Wizard) writes: > >System V rel. 2.1 and later and 4.1BSD and later adopted different >philosophies regarding the implementation of demand-paged virtual >memory. The BSD systems, being research-oriented and likely to be used >by programmers with a more intimate knowledge of the inner workings of >the O/S, chose simple, less elegant approaches to memory management with >provisions, such as the vfork() call, for the programmer to tune the >system to the application. In 4.xBSD, swap space is allocated at the >birth of a process, with enough space being allocated to contain the >entire virtual image of the process in its initial state >(text+data+BSS), excepting of course malloc'd (heap) space obtained >via sbrk() or stack space pushdown expansion, and the user structure and >page tables which are RAM-resident for an active (non-swapped) process. > >System V took the approach of being as elegant as possible (e.g., using >copy-on-write and dynamic allocation of all tables except the actual >physical memory map) and hiding the inner workings of the system from >programmer intervention. In the System V scheme, swap space is >allocated as needed by the page stealer to swap out pages or processes >as needed. Thus, allocation of swap space is not made until it is >actually used. Since process sharable text and data may be >demand-paged in from the filesystem, virtual pages may be mapped to one >of three places: physical RAM, swap space, or the executable program >file. Although I'm no kernel hacker, I will point out, from a sysadmin's point-of- view (with advice from the higher levels of AT&T support/development people) that later SysV implementations still retain vestiges of the old virtual memory scheme you describe: on an AT&T 3B2/600 running r3, for instance, the kernel always attempts to map two -entire- sets of -contiguous- pages in the swap area for every process upon startup, whenever sbrk() is called for more memory, and so on. One of these sets is for paging, and the other, for swapping. If, for some reason, there is not enough contiguous space available to allocate one of these sets, the kernel spits out a "getcpages: cannot allocate X contiguous pages" console message (X being some-number-or-other). If there is insufficient contiguous space to map EITHER page, new processes will die on startup, or ungrowable processes will (usually) seg violate, with all of this mayhem accompanied by delightful kernel messages such as "growreg: unable to allocate ...", or "dupreg: unable to allocate..." or "uballoc: <something-or-other>" spewing on the console. The algorithm for mapping this swap space is unknown to me, but it appears to be imperfect - from time to time, one of our 3B2's or another will suddenly experience these symptoms for no particular reason, and plenty of available swap space. Once it starts, even programs with relatively small run-time images (e.g. /bin/cat) will then die on startup, with the kernel complaining that it cannot allocate swap-area space. Obviously, the only cure at that point is to reboot... Perhaps AT&T has been slow in -implementing- the VM "elegance" of which you speak? I see very little of it on their own products. >-- > Jim Adams Department of Physiology and Biophysics >adams@ucunix.san.uc.edu University of Cincinnati College of Medicine > "I like the symbol visually, plus it will confuse people." > ... Jim Morrison ------------------------------------------------------------------------------ kevin@cfctech.cfc.com | Kevin Darcy, Unix Systems Administrator ...sharkey!cfctech!kevin | Technical Services (CFC) Voice: (313) 759-7140 | Chrysler Corporation Fax: (313) 758-8173 | 25999 Lawrence Ave, Center Line, MI 48015 ------------------------------------------------------------------------------
adam@cs.UAlberta.CA (Michel Adam; Gov't of NWT) (06/16/91)
In article <1991Jun14.225216.23361@cfctech.cfc.com> kevin@cfctech.cfc.com (Kevin Darcy) writes: >In article <1991Jun13.192444.7500@ucunix.san.uc.edu> adams@ucunix.san.uc.edu (J. Adams - SunOS Wizard) writes: >>In article <1991Jun13.122803.25362@ims.alaska.edu> floyd@ims.alaska.edu >>(Floyd Davidson) writes: >>>[...] >>>In article <1991Jun13.065207.10089@ucunix.san.uc.edu> >>>adams@ucunix.san.uc.edu (J. Adams - SunOS Wizard) writes: >> >>System V rel. 2.1 and later and 4.1BSD and later adopted different >>philosophies regarding the implementation of demand-paged virtual >>memory. The BSD systems, being research-oriented and likely to be used >>by programmers with a more intimate knowledge of the inner workings of >>the O/S, chose simple, less elegant approaches to memory management with >>provisions, such as the vfork() call, for the programmer to tune the >>system to the application. In 4.xBSD, swap space is allocated at the >>birth of a process, with enough space being allocated to contain the >>entire virtual image of the process in its initial state >>(text+data+BSS), excepting of course malloc'd (heap) space obtained [ ... ] > >Although I'm no kernel hacker, I will point out, from a sysadmin's point-of- >view (with advice from the higher levels of AT&T support/development people) >that later SysV implementations still retain vestiges of the old virtual >memory scheme you describe: on an AT&T 3B2/600 running r3, for instance, the >kernel always attempts to map two -entire- sets of -contiguous- pages in the >swap area for every process upon startup, whenever sbrk() is called for more >memory, and so on. One of these sets is for paging, and the other, for >swapping. If, for some reason, there is not enough contiguous space available >to allocate one of these sets, the kernel spits out a "getcpages: cannot >allocate X contiguous pages" console message (X being some-number-or-other). >If there is insufficient contiguous space to map EITHER page, new processes >will die on startup, or ungrowable processes will (usually) seg violate, with >all of this mayhem accompanied by delightful kernel messages such as "growreg: >unable to allocate ...", or "dupreg: unable to allocate..." or "uballoc: ><something-or-other>" spewing on the console. > >The algorithm for mapping this swap space is unknown to me, but it appears to >be imperfect - from time to time, one of our 3B2's or another will suddenly >experience these symptoms for no particular reason, and plenty of available >swap space. Once it starts, even programs with relatively small run-time >images (e.g. /bin/cat) will then die on startup, with the kernel complaining >that it cannot allocate swap-area space. Obviously, the only cure at that >point is to reboot... > >Perhaps AT&T has been slow in -implementing- the VM "elegance" of which you >speak? I see very little of it on their own products. > I seem to remember a discussion on this subject, 6 to 12 months ago, on the problems with insufficient RAM available on the AT&T 386 machines, (must have been a SYS V Ver. 3.something). One comment that struck my mind at that time was that for some reason, there was a recommendation to have as much real RAM as the swap partition's size ( I may be mistaken on that ...), to avoid some kind of trashing. There was an explicit reference to the implementation by Convergent, for the 7300, being somewhat different and managing to avoid this problem. There where numerous references to the Bach book, but I don't remember if the difference between the CT version for the 3B1 and the other versions running on 386 machine was described. Can someone who archived these articles look up the conclusion? What was it that CT did differently in their implementation? (Can someone with access to the kernel source find out?) >>-- >> Jim Adams Department of Physiology and Biophysics >>adams@ucunix.san.uc.edu University of Cincinnati College of Medicine >> "I like the symbol visually, plus it will confuse people." >> ... Jim Morrison > >------------------------------------------------------------------------------ >kevin@cfctech.cfc.com | Kevin Darcy, Unix Systems Administrator >...sharkey!cfctech!kevin | Technical Services (CFC) >Voice: (313) 759-7140 | Chrysler Corporation >Fax: (313) 758-8173 | 25999 Lawrence Ave, Center Line, MI 48015 >------------------------------------------------------------------------------ Michel Adam adam@cs.ualberta.ca (...!alberta!adam) or adam@iceman.UUCP (...!alberta!iceman!adam)
rhealey@kas.helios.mn.org (Rob Healey) (06/16/91)
In article <1991Jun15.195720.25820@cs.UAlberta.CA> adam@cs.UAlberta.CA (Michel Adam; Gov't of NWT) writes:
=I seem to remember a discussion on this subject, 6 to 12 months ago, on the
=problems with insufficient RAM available on the AT&T 386 machines, (must have
=been a SYS V Ver. 3.something). One comment that struck my mind at that time
=was that for some reason, there was a recommendation to have as much real RAM
=as the swap partition's size ( I may be mistaken on that ...), to avoid some
=kind of trashing. There was an explicit reference to the implementation
=by Convergent, for the 7300, being somewhat different and managing to avoid
=this problem. There where numerous references to the Bach book, but I don't
=remember if the difference between the CT version for the 3B1 and the other
=versions running on 386 machine was described.
=
=Can someone who archived these articles look up the conclusion? What was it
=that CT did differently in their implementation?
=
They used a better processor family... B^). They also grafted 4.1BSD
VM onto a System V R2 kernel. As such, you HAVE to have enough
swap to page in the whole program to swap and then memory. I.e. I
ran into troubles on a UNIX PC where I had 700k free memory but
only 100k of swap, the system REFUSED to load a 400k program; sucked
rocks. This just goes to show that both methods have their "evil twin"
side. SVR3.x chews up too much swap, Early BSD needs enough swap for the
whole initial program even if there is enough main memory but not
enough swap/paging.
I believe SVR4 and 4.3 BSD are better at paging and are more
reasonable with it all. If nothing else you can add paging files
under both to solve the running out of swap problem, you just take
performance hits when the paging file blocks are scattered all
over a disk.
-Robadam@cs.UAlberta.CA (Michel Adam; Gov't of NWT) (06/22/91)
In article <224@kas.helios.mn.org> rhealey@kas.helios.mn.org (Rob Healey) writes: >In article <1991Jun15.195720.25820@cs.UAlberta.CA> adam@cs.UAlberta.CA (Michel Adam; Gov't of NWT) writes: >= >=Can someone who archived these articles look up the conclusion? What was it >=that CT did differently in their implementation? >= > They used a better processor family... B^). They also grafted 4.1BSD > VM onto a System V R2 kernel. As such, you HAVE to have enough I tought that demand paged virtual memory only came with 4.2 or 4.3 ...? > swap to page in the whole program to swap and then memory. I.e. I > ran into troubles on a UNIX PC where I had 700k free memory but > only 100k of swap, the system REFUSED to load a 400k program; sucked Disk space is cheap, relative to RAM anyway ... It seems a fairly minor compromise, if it avoid the trashing that seemed to have been a common problem with the other implementation of SYS V R. 2 or 3. Of course it may not be a 'purist' approach, but I'm in business, not academia ... Real world tends to intrude ... > rocks. This just goes to show that both methods have their "evil twin" > side. SVR3.x chews up too much swap, Early BSD needs enough swap for the > whole initial program even if there is enough main memory but not > enough swap/paging. I was under the impression that the actual 'a.out' file was used for paging in the code (read 'text'), and that was decreasing the need for disk space in the swap partition. I once tried to delete a program and got an error message about the file being in use. Did I miss something here? What about the shared library? Isn't it supposed to be locked when in use? Could someone send me a list of recommended books on the design of the BSD system, particularly the version that was used as a 'donnor' for the UNIX-PC kernel. A list of all the BSDism in this kernel would also be interesting. I'll summarize if there is interest. > > I believe SVR4 and 4.3 BSD are better at paging and are more > reasonable with it all. If nothing else you can add paging files > under both to solve the running out of swap problem, you just take > performance hits when the paging file blocks are scattered all > over a disk. > > -Rob Michel Adam Yellowknife, N.W.T. adam@cs.ualberta.ca (...!alberta!adam)