gillies@p.cs.uiuc.edu (02/09/90)
Re: Is mapping files into memory a big win? Here's a cult-favorite paper that addresses this question, "Hints for Computer System Designers", by Butler Lampson I can't remember the source (probably ACM SOSP), but it came out in 1985, and Lampson (formerly at PARC) argued that simple is sometimes better. The Pilot OS designers copied the Multics file mapping trick, and later lived to regret it. The kernel code eventually ate up about 11,000 lines, and the applications programmers never recouped this huge kernel investment.
pcg@rupert.cs.aber.ac.uk (Piercarlo Grandi) (02/12/90)
In article <76700129@p.cs.uiuc.edu> gillies@p.cs.uiuc.edu writes:
Re: Is mapping files into memory a big win?
Here's a cult-favorite paper that addresses this question,
"Hints for Computer System Designers", by Butler Lampson
I can't remember the source (probably ACM SOSP), but it came out in
1985, and Lampson (formerly at PARC) argued that simple is sometimes
better. The Pilot OS designers copied the Multics file mapping trick,
and later lived to regret it. The kernel code eventually ate up about
11,000 lines, and the applications programmers never recouped this
huge kernel investment.
But the Pilot paper makes it very clear that the implemented the
most complicated possible form of single level storage. It can be
said that it was a colossal mistake. From this deducing that
single level storage is a bad idea is a huge leap.
I would also like to point out that many current Unix
implementations of memory mapped files, most notably the SUN one,
are hopelessly misdesigned from a simplicity and performance
point of view.
For two of the most revolting aspects of the SUN design, let me
point out the 8Kb page size (when virtually all Unix files, and
most obviously directories, are far smaller than that), and the
inability to properly advise the pager (there is a minimal
facility for this, but then programs like 'cp' do not use it)
about non wroking set like reference patterns for regions of
address space, e.g. of the desirability to have read-ahead and
free-behind on sequential accesses, etc... These two problems
may well account for 90% of the increased memory requirements of
SunOS 4, in the form of uselessly bloated working sets and vastly
increased internal fragementation.
Not to speak of paging strategies, cold starts, dynamic linking
overheads, and other bogosities.
The original Unix design was carefully tuned for PDP-11s, and
extracted very good performance out of very limited hardware and
architecture. A complete overhaul is needed now, but we are not
even up to the level of Multics, not to mention progress after
that.
--
Piercarlo "Peter" Grandi | ARPA: pcg%cs.aber.ac.uk@nsfnet-relay.ac.uk
Dept of CS, UCW Aberystwyth | UUCP: ...!mcvax!ukc!aber-cs!pcg
Penglais, Aberystwyth SY23 3BZ, UK | INET: pcg@cs.aber.ac.ukvinoski@apollo.HP.COM (Stephen Vinoski) (02/14/90)
In article <PCG.90Feb11210841@rupert.cs.aber.ac.uk> pcg@rupert.cs.aber.ac.uk (Piercarlo Grandi) writes: >For two of the most revolting aspects of the SUN design, let me >point out the 8Kb page size (when virtually all Unix files, and >most obviously directories, are far smaller than that), and the >inability to properly advise the pager (there is a minimal >facility for this, but then programs like 'cp' do not use it) >about non wroking set like reference patterns for regions of >address space, e.g. of the desirability to have read-ahead and >free-behind on sequential accesses, etc... These two problems >may well account for 90% of the increased memory requirements of >SunOS 4, in the form of uselessly bloated working sets and vastly >increased internal fragementation. Many UNIX bigots think Sun invented the concept of memory mapping. If what Piercarlo says above is true, it must be that those arguing against the concept use Sun machines! Apollo did it right. For instance, our Domain/OS file mapping functions allow the user to specify expected access patterns. I believe there was a paper written in 1983 which describes the Apollo single-level store, and I'm pretty sure most of what is described there is still being used. If anyone is interested, send me email and I'll try to dig up the reference for you. -steve Newsgroups: comp.arch Subject: Re: the Multics from the black lago Summary: Expires: References: <90@<KHB> <76700129@p.cs.uiuc.edu> <PCG.90Feb11210841@rupert.cs.aber.ac.uk> Sender: Reply-To: vinoski@thor.UUCP (Stephen Vinoski) Followup-To: Distribution: Organization: Apollo Computer, Chelmsford, MA Keywords: In article <PCG.90Feb11210841@rupert.cs.aber.ac.uk> pcg@rupert.cs.aber.ac.uk (Piercarlo Grandi) writes: >In article <76700129@p.cs.uiuc.edu> gillies@p.cs.uiuc.edu writes: > > Re: Is mapping files into memory a big win? > > Here's a cult-favorite paper that addresses this question, > > "Hints for Computer System Designers", by Butler Lampson > > I can't remember the source (probably ACM SOSP), but it came out in > 1985, and Lampson (formerly at PARC) argued that simple is sometimes > better. The Pilot OS designers copied the Multics file mapping trick, > and later lived to regret it. The kernel code eventually ate up about > 11,000 lines, and the applications programmers never recouped this > huge kernel investment. > >But the Pilot paper makes it very clear that the implemented the >most complicated possible form of single level storage. It can be >said that it was a colossal mistake. From this deducing that >single level storage is a bad idea is a huge leap. > >I would also like to point out that many current Unix >implementations of memory mapped files, most notably the SUN one, >are hopelessly misdesigned from a simplicity and performance >point of view. > >For two of the most revolting aspects of the SUN design, let me >point out the 8Kb page size (when virtually all Unix files, and >most obviously directories, are far smaller than that), and the >inability to properly advise the pager (there is a minimal >facility for this, but then programs like 'cp' do not use it) >about non wroking set like reference patterns for regions of >address space, e.g. of the desirability to have read-ahead and >free-behind on sequential accesses, etc... These two problems >may well account for 90% of the increased memory requirements of >SunOS 4, in the form of uselessly bloated working sets and vastly >increased internal fragementation. > >Not to speak of paging strategies, cold starts, dynamic linking >overheads, and other bogosities. > >The original Unix design was carefully tuned for PDP-11s, and >extracted very good performance out of very limited hardware and >architecture. A complete overhaul is needed now, but we are not >even up to the level of Multics, not to mention progress after >that. >-- >Piercarlo "Peter" Grandi | ARPA: pcg%cs.aber.ac.uk@nsfnet-relay.ac.uk >Dept of CS, UCW Aberystwyth | UUCP: ...!mcvax!ukc!aber-cs!pcg >Penglais, Aberystwyth SY23 3BZ, UK | INET: pcg@cs.aber.ac.uk Newsgroups: comp.arch Subject: Re: the Multics from the black lago Summary: Expires: References: <90@<KHB> <76700129@p.cs.uiuc.edu> <PCG.90Feb11210841@rupert.cs.aber.ac.uk> Sender: Reply-To: vinoski@thor.UUCP (Stephen Vinoski) Followup-To: Distribution: Organization: Apollo Computer, Chelmsford, MA Keywords: In article <PCG.90Feb11210841@rupert.cs.aber.ac.uk> pcg@rupert.cs.aber.ac.uk (Piercarlo Grandi) writes: >For two of the most revolting aspects of the SUN design, let me >point out the 8Kb page size (when virtually all Unix files, and >most obviously directories, are far smaller than that), and the >inability to properly advise the pager (there is a minimal >facility for this, but then programs like 'cp' do not use it) >about non wroking set like reference patterns for regions of >address space, e.g. of the desirability to have read-ahead and >free-behind on sequential accesses, etc... These two problems >may well account for 90% of the increased memory requirements of >SunOS 4, in the form of uselessly bloated working sets and vastly >increased internal fragementation. Many UNIX bigots think Sun invented the concept of memory mapping. If what Piercarlo says above is true, it must be that those arguing against the concept use Sun machines! Apollo did it right. For instance, our Domain/OS file mapping functions allow the user to specify expected access patterns. I believe there was a paper written in 1983 which describes the Apollo single-level store, and I'm pretty sure most of what is described there is still being used. If anyone is interested, send me email and I'll try to dig up the reference for you. -steve | Steve Vinoski (508)256-6600 x5904 | Internet: vinoski@apollo.com | | Testability and Diagnostics | UUCP: ...mit-eddie!apollo!vinoski| | HP Apollo Division, Chelmsford, MA 01824 | ...uw-beaver!apollo!vinoski| | "This is a dangerous place." -King Crimson, "Thela Hun Ginjeet" |