roy@phri.UUCP (Roy Smith) (09/22/87)
In article <2067@sfsup.UUCP> shap@sfsup.UUCP (J.S.Shapiro) gives a pretty
good introductory rundown on shared libraries.
One thing that has always bothered me about shared libraries (or
dynamic loading, if you prefer) is the problem of backward compatability.
It's not too uncommon that a new version of a program comes along, but some
people still want to use the old version, for whatever reason. Let's say a
new version of scanf comes along which follows some standard a bit closer
than the old one. This is arguably a good thing, but what if I just happen
to have a program which depends on the old (albeit wrong) behavior of the
scanf library function.
On a compile-time library loading system, all I would have to do
would be to keep around a copy of the executable loaded with the old
library and I'd be happy. We have more than one 3.0 binary running on our
SunOS-3.2 systems for exactly this reason. What do I do on a shared
library system when scanf changes out from under me and breaks my program?
--
Roy Smith, {allegra,cmcl2,philabs}!phri!roy
System Administrator, Public Health Research Institute
455 First Avenue, New York, NY 10016jesup@mizar.steinmetz (Randell Jesup) (09/23/87)
In article <2903@phri.UUCP> roy@phri.UUCP (Roy Smith) writes: > One thing that has always bothered me about shared libraries (or >dynamic loading, if you prefer) is the problem of backward compatability. ... >SunOS-3.2 systems for exactly this reason. What do I do on a shared >library system when scanf changes out from under me and breaks my program? >Roy Smith, {allegra,cmcl2,philabs}!phri!roy Take a page from the Amiga Exec (which uses shared libraries for almost everything. When opening the shared library, tell it what version of the library you want. (And maybe whether you'll accept a newer version). This also solves the problem of running a binary that want's version 33 of a library on a system that is only up to 32, for example. Randell Jesup (Please use one of these paths for mail) sungod!jesup@steinmetz.UUCP (uunet!steinmetz!{sungod|crd}!jesup) jesup@ge-crd.ARPA
gwyn@brl-smoke.ARPA (Doug Gwyn ) (09/23/87)
In article <2903@phri.UUCP> roy@phri.UUCP (Roy Smith) writes: >What do I do on a shared >library system when scanf changes out from under me and breaks my program? Obviously, you fix your program. You need to do that anyway the next time you recompile it (e.g. during routine maintenance). This use of shared libraries depends on there being a stable interface definition for the shared routines. If your code depends on anything not guaranteed by the interface definition, then you were asking for trouble all along.
jss@hector.UUCP (Jerry Schwarz) (09/23/87)
In article <2903@phri.UUCP> roy@phri.UUCP (Roy Smith) writes: > > One thing that has always bothered me about shared libraries (or >dynamic loading, if you prefer) is the problem of backward compatability. >It's not too uncommon that a new version of a program comes along, but some >people still want to use the old version, for whatever reason. You have to relink with the unshared version of the old library. My understanding is that the tools that build a shared library on System V build an unshared version at the same time. In most circumstances I would expect the unshared version to be distributed with the shared version. Jerry Schwarz
guy%gorodish@Sun.COM (Guy Harris) (09/24/87)
> >It's not too uncommon that a new version of a program comes along, but some > >people still want to use the old version, for whatever reason. > > You have to relink with the unshared version of the old library. You do in some, but not all, implementations. An earlier article indicated that the Amiga implementation permits you to specify a version number when the library is opened (does this mean programs have to explicitly open shared libraries, i.e., they have to know that they're using a shared library?). See "Shared Libraries in SunOS" from the last USENIX proceedings for another scheme for handling this problem. The scheme described therein permits multiple versions of the library to be resident; when the interface changes in a way that is expected to break old programs, the major version number is increased, so that programs depending on an earlier major version of the library won't bind to a later version. Of course, this requires you to keep old versions of the library around, but you can't have everything; if this causes a disk-space problem, you might have to link with an unshared version of the library. Guy Harris {ihnp4, decvax, seismo, decwrl, ...}!sun!guy guy@sun.com
roy@phri.UUCP (09/24/87)
In <2903@phri.UUCP> I asked: > What do I do on a shared library system when scanf changes out > from under me and breaks my program? In <6461@brl-smoke.ARPA> gwyn@brl.arpa (Doug Gwyn (VLD/VMB) <gwyn>) replied: > Obviously, you fix your program. [...] If your code depends on anything > not guaranteed by the interface definition, then you were asking for > trouble all along. I respect Doug Gwyn. He seems to know what he's talking about more than most people, so I don't flame him lightly, but that's the kind of pompus, if-you-can't-do-it-right-don't-do-it-at-all answer that gives computer science a bad name. In the same category as "if it doesn't do X, then it's not C". Yes, it's nice to follow the rules and write code according to the interface definition. But where does that leave the poor schmuck who didn't follow the rules two years ago when he was writing his very grungey but very useful and productive program? In the dirt where he belongs? Sorry, it doesn't work that way in the real world. Maybe we don't have the source any more, or never did. Maybe it's just more trouble than it's worth to fix the code. I don't run a computer science research center, I run a production machine. My users expect to get their work done. If it runs and gives them the right answers, that's more important than following the rules. Many of the people who write production programs here are biologists, not professional programmers. If one of them has a program which has been working for N years and all of a sudden it breaks because I put up a new version of Unix, he doesn't want to know that the new version is better and the reason his program won't run any more is because he didn't follow the interface definition. As far as he's concerned, I broke his program by upgrading the OS and he's pissed at me because he can't get his work done. Guess what? I agree with him. Let me give you an example. One of our big users has a 7000 line Fortran program. The code is 15 years old but it still works, and for what it does, there still isn't anything better. The classic dusty deck. He made a few minor modifications to it involving calling system(3F). Everything worked fine for a year or so running under SunOS-3.0, but then we upgraded to 3.2. Guess what, in 3.2 (and, I understand, in 3.4 as well), the Fortran system() routine is broken. Now it's not even a case of not following the rules, but simply that the new library routine has a bug which the old one didn't. Until we tracked the bug down to the system() call and Sun was able to supply the workaround, our solution was (you guessed it) to continue program development on a 3.0 machine and run the binaries on the 3.2 machines (which had better floating point hardware). It just so happens that Sun was able to supply a workaround which allowed us to link using the new (broken) library, but imagine what would have happened if Sun was using a dynamic library loading scheme and there was no workaround other than linking with the old library. I could have shaved my head and marched up and down the halls chanting "follow the interface definition" till I was blue in the face and it wouldn't have done a damn bit of good. I supposed I could have uninstalled the 3.2 upgrade and refused to pay for it, but that's not what we wanted to do. My background is engineering. In engineering school they teach you to follow the rules and design things according to sound theoretical principles. They also teach you that when push comes to shove, it's sometimes better to break the rules and put a bag on side rather than not have it work at all. Sure, design it so it works like it's supposed to, but just in case, leave the hooks in so you have someplace to hang that bag from if you ever have to. -- Roy Smith, {allegra,cmcl2,philabs}!phri!roy System Administrator, Public Health Research Institute 455 First Avenue, New York, NY 10016
des@jplpro.JPL.NASA.GOV (David Smyth) (09/24/87)
In article <28957@sun.uucp> guy%gorodish@Sun.COM (Guy Harris) writes: >> >It's not too uncommon that a new version of a program comes along, but some >> >people still want to use the old version, for whatever reason. >> >> You have to relink with the unshared version of the old library. > >You do in some, but not all, implementations. An earlier article indicated >See "Shared Libraries in SunOS" from the last USENIX proceedings for another >Of course, this requires you to keep old versions of the library around, but >you can't have everything; if this causes a disk-space problem, you might have ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >to link with an unshared version of the library. > Guy Harris How is COPYING the old shared libraries into executables which need them ANY savings in disk usage? It seems it will be a DEAD LOSS: core (bigger executable images); virtual memory (it gets used up even if paged out); AND disk space (the executable file gets bigger for EVERY program which needs the unshared library). Why EVER have unsharable libraries??? Why EVER have libraries specifically linked to an executable??? a) If it is an application which makes repeated calls to a library, the FIRST invocation may be slower, but all following invocations can be VERY CLOSE to the same speed [Message/Object Programming, Brad J. Cox, see table 1]. b) Speed Critical Applications probably want to be vectorized, and I would think reducing the competition for core via shared libraries would be a BIG win if swapping is reduced even a little bit (I don't know much about vectorized algorithms, I only work on these archaic Suns, Vaxen, and such Von Nueman rubbish :^) ).
daveh@cbmvax.UUCP (Dave Haynie) (09/24/87)
in article <28957@sun.uucp>, guy%gorodish@Sun.COM (Guy Harris) says: > Xref: cbmvax comp.arch:2266 comp.unix.wizards:4354 comp.os.misc:224 > >> >It's not too uncommon that a new version of a program comes along, but some >> >people still want to use the old version, for whatever reason. >> >> You have to relink with the unshared version of the old library. > > You do in some, but not all, implementations. An earlier article indicated > that the Amiga implementation permits you to specify a version number when the > library is opened (does this mean programs have to explicitly open shared > libraries, i.e., they have to know that they're using a shared library?). Well, somewhere in each executable you'll find an explicit request to open and subsequently close any shared libraries. The is very important for Amiga with small amounts of memory, since it allows the OS to load on demand any disk-based libraries, and to dump any that are no longer used. Currenly all Amiga system functions are available only as shared libraries, and the programmer normally includes a single OS call to open each, followed by one to close each. Of course, if unshared libraries were available, a linker could link in NOP open/close requests and maybe as much as 300K-400K of library code into your executable. Now, when you're compiling in C using UNIX compatibility calls or some such, there's UNIX style start-up code that opens everything you'd need, so in that case the programmer has probably no idea what's shared and what isn't. > See "Shared Libraries in SunOS" from the last USENIX proceedings for another > scheme for handling this problem. The scheme described therein permits > multiple versions of the library to be resident; when the interface changes in > a way that is expected to break old programs, the major version number is > increased, so that programs depending on an earlier major version of the > library won't bind to a later version. As far as I know no one's done this kind of thing on the Amiga yet, but I think a small modification to the OpenLibrary() function could allow for multiple generations to be kept around, if necessary. I think what's done now is that if a function is changed enough to cause most things to break, it would appear as a new function, while the old one is kept in place for backward compatibility. > Of course, this requires you to keep old versions of the library around, but > you can't have everything; if this causes a disk-space problem, you might have > to link with an unshared version of the library. Or old functions around in new libraries. > Guy Harris > {ihnp4, decvax, seismo, decwrl, ...}!sun!guy > guy@sun.com -- Dave Haynie Commodore-Amiga Usenet: {ihnp4|caip|rutgers}!cbmvax!daveh "The A2000 Guy" PLINK : D-DAVE H BIX : hazy "God, I wish I was sailing again" -Jimmy Buffett
henry@utzoo.UUCP (Henry Spencer) (09/24/87)
> ... What do I do on a shared > library system when scanf changes out from under me and breaks my program? Whisper the magic phrase: "search paths". There is no *inherent* reason why you can't keep multiple versions around and have some way to say "run this program with the old library, please". One can even envision doing it automatically with some sort of date-stamp system: the loader finds the newest library version that predates the program, or failing that, the oldest version remaining. -- "There's a lot more to do in space | Henry Spencer @ U of Toronto Zoology than sending people to Mars." --Bova | {allegra,ihnp4,decvax,utai}!utzoo!henry
guy%gorodish@Sun.COM (Guy Harris) (09/25/87)
> How is COPYING the old shared libraries into executables which need > them ANY savings in disk usage? If the sum of the disk size differences between the executables in question as linked with the shared library and the executables in question as linked with the unshared library is less than the size of the old shared library, it's obviously going to save you disk space not to keep the old library around. I will not rule out this possibility out of hand. > It seems it will be a DEAD LOSS: core (bigger executable images); What do you mean by "core" here? Do you mean "physical memory"? Yes, shared libraries will share physical memory better, but that has nothing to do with the size of the executable image *per se*. The smaller size of the image is, in part, misleading; in order for the image to *run* it still needs to have the shared libraries added in. > virtual memory (it gets used up even if paged out); Umm, images built without shared libraries will usually require *less* virtual memory than ones built with shared libraries; the one built without shared libraries doesn't include a whole pile of routines that it doesn't use. That stuff may be paged out, but you said "virtual memory", not "physical memory" here. > Why EVER have unsharable libraries??? So that you can have a program with which to to back up to an old version of a shared library if a new minor version has a bug in it (or if the file containing the shared library got trashed, or if it got unlinked, or....)? Guy Harris {ihnp4, decvax, seismo, decwrl, ...}!sun!guy guy@sun.com
tim@ism780c.UUCP (09/25/87)
In article <2903@phri.UUCP> roy@phri.UUCP (Roy Smith) writes:
< SunOS-3.2 systems for exactly this reason. What do I do on a shared
< library system when scanf changes out from under me and breaks my program?
On System V shared libraries, the path names of the shared libraries are
in the header of the a.out file. It shouldn't be too hard to provide
a tool that allows you to change these.
Then, when you find out that a library update breaks your program, you
could obtain a copy of the old library, and fix your a.out to use
this.
--
Tim Smith, Knowledgian {sdcrdcf,uunet}!ism780c!tim
tim@ism780c.isc.com
"Oh I wish I were Matthew Wiener, That is who I truly want to be,
'Cause if I were Matthew Wiener, Tim Maroney would send flames to me"gew@dnlunx.UUCP (Weijers G.A.H.) (09/25/87)
In article <28957@sun.uucp>, guy%gorodish@Sun.COM (Guy Harris) writes: > Of course, this requires you to keep old versions of the library around, but > you can't have everything; if this causes a disk-space problem, you might have > to link with an unshared version of the library. How can this solve the disk space problem? You'll have *at*least* one copy of the old version around, linked to the unconverted application, and possibly many copies. -- | Ge' Weijers | Disclaimer: the views expressed | | PTT Dr. Neher Laboratories | are usually entirely my own, | | Leidschendam, the Netherlands | not my employers'. I don't own | | uucp: ...!mcvax!dnlunx!gew | a cat to share them with. |
gwyn@brl-smoke.ARPA (Doug Gwyn ) (09/25/87)
In article <2906@phri.UUCP> roy@phri.UUCP (Roy Smith) writes: >My background is engineering. In engineering school they teach you >to follow the rules and design things according to sound theoretical >principles. They also teach you that when push comes to shove, it's >sometimes better to break the rules and put a bag on side rather than not >have it work at all. Sure, design it so it works like it's supposed to, >but just in case, leave the hooks in so you have someplace to hang that bag >from if you ever have to. I hope they also taught you how to migrate forward with progressing technology. I, too, come from a "production shop" background, but we knew enough to keep our sources around and (in a non-shared library environment) would recompile everything at regular intervals, simply to ensure that our applications matched the current state of the system software. Naturally, we had test procedures.. I don't see that the situation with shared libraries is appreciably different from that without them, other than that your application will breaker sooner (when the new library is installed) rather than later (when you have to rebuild the application for some reason, such as making a small change). The causes for breakage are the same either way. I stress the library interface specifications because that is the "treaty point" on which the application programmer and the system implementor must agree. As to the people with dusty decks not being in a position to maintain their own code, I can't sympathize very much. Most code like that that I have seen has given the appearance of "working" but in fact was producing various degrees of garbage. I don't know what the solution to this is, other than to have more qualified professional programmers available to help others exploit computers more robustly and effectively. It's a problem here (BRL), too.
crc@abvax.icd.ab.com (Clive R. Charlwood) (09/25/87)
in article <2903@phri.UUCP>, roy@phri.UUCP (Roy Smith) says: > Xref: abvax comp.arch:1448 comp.unix.wizards:2522 comp.os.misc:138 > dynamic loading, if you prefer) is the problem of backward compatability. I would hope that any shared library system would give some ability for each executable to link into specific revisions of each library routine. The author of the code will probably have to go and update the code to make it work with the latest and greatest sometime. BUT you can't get into the situation where whenever a library changes all installed software has to be retested immediatly. -- Clive R. Charlwood @ Allen-Bradley Company Industrial Computer Division 747 Alpha Drive, Highland Heights, OH 44143 ...!{decvax,masscomp,pyramid,cwruecmp}!abic!crc
earl@mips.UUCP (Earl Killian) (09/26/87)
In article <2906@phri.UUCP>, roy@phri.UUCP (Roy Smith) writes:
[lots of words about programming in the real world]
I agree with what Roy Smith wrote with respect to the real world.
But, shared libraries aren't bad for his world. I think they make
life simpler. At least if done right (I don't think System V shared
libraries qualify, but I don't know). On Multics, it works the way
Henry Spencer expects; to fix a broken function, you simply put a
replacement in the search path, and the problem is fixed. You don't
need to relink everything that's been worked in since the bug first
appeared (finding all the programs linked with the bad system() in
your example might be a real chore, depending on when the bug is
discovered).
Bug fixes, workarounds, and new features are major uses of Multics
dynamic linking. Consider something trivial on Multics and impossible
on vanilla Unix. I login to a system over the arpanet from far away.
It prints out time in the wrong time zone. Suppose there's no option
to specify a timezone. I just put a new binary to ascii time
conversion function in my search path, and I'm happy. On Unix I'd
have to relink all the appropriate programs in /usr (not easy if I
don't have the source).peter@sugar.UUCP (Peter da Silva) (09/26/87)
In article <28957@sun.uucp>, guy%gorodish@Sun.COM (Guy Harris) writes: > that the Amiga implementation permits you to specify a version number when the > library is opened (does this mean programs have to explicitly open shared > libraries, i.e., they have to know that they're using a shared library?). Yes, but this can be included in the startup code (and in fact is, for the most common shared library "dos.library"). Shared library calls are made by loading the base address of the library jump table into A6 and doing an indirect call off it. -- -- Peter da Silva `-_-' ...!hoptoad!academ!uhnix1!sugar!peter -- 'U` Have you hugged your wolf today? -- Disclaimer: These aren't mere opinions... these are *values*.
ka@uw-june.UUCP (09/27/87)
> What do I do on a shared library system when scanf changes out from > under me and breaks my program? You recompile your program to use the old version of the shared library instead of the current one. Sure, it would be nice not to have to recompile these programs, but in the case of nonshared libraries you have to recompile also. With nonshared libraries, you would have to recompile all the programs which were broken due to the bug in scanf. With shared libraries, on the other hand, you have to recompile the programs that depend upon the bug in scanf. However, these programs should really be fixed, which will force them to be recompiled anyway. Kenneth Almquist uw-june!ka (New address!)
stachour@umn-cs.UUCP (09/29/87)
In article <8650@utzoo.UUCP>, henry@utzoo.UUCP (Henry Spencer) writes: > > ... What do I do on a shared > > library system when scanf changes out from under me and breaks my program? > > Whisper the magic phrase: "search paths". There is no *inherent* reason > why you can't keep multiple versions around and have some way to say "run > this program with the old library, please". That's why Multics has had search-paths since day-1. That's also why it has the phrase "-referencing_dir" in its rules, so that programs pointed at an old version of a dynamically-linked shared library thereby getting the first of a set of routines from that particular version of the library will get the rest of the set that match. (Multics also has done up-referencing and down-referenceing of data, argument-interogation, compatible-incompatible replacements, system_library_obsolete, and other ways to handle multiple versions of a program, but that's another 20-year old story that few read, because it works so well no-one needs to read or change the code.) Paul Stachour Why don't computer people seem to know Stachour@HI-Multics.arpa Computer History? It isn't like there stachour at umn-cs.equ is 1000 or even 50 years of it!
elg@usl.UUCP (09/29/87)
in article <28957@sun.uucp>, guy%gorodish@Sun.COM (Guy Harris) says: >> >It's not too uncommon that a new version of a program comes along, but some >> >people still want to use the old version, for whatever reason. >> You have to relink with the unshared version of the old library. > You do in some, but not all, implementations. An earlier article indicated > that the Amiga implementation permits you to specify a version number when the > library is opened (does this mean programs have to explicitly open shared > libraries, i.e., they have to know that they're using a shared library?). The Amiga operating system consists of a tiny multi-tasking message-passing kernel, various system processes to handle I/O, the mouse, etc., and a whole slew of libraries, all of which are shared, all of which are loaded upon demand (i.e. 'opened'). So yes, programs have to explicitly open shared libraries. However, Amiga "C" compilers handle that for stdio and other "vanilla" stuff, it's just when you're worrying about windowing and other down'n'dirty machine-dependent stuff that you need to explicitly worry about opening and closing libraries. Needless to say, the Amiga OS is not Unix, for one thing, Unix isn't going to run on a 512K machine with no MMU and two floppies. So they don't have to worry about backward compatibility with Version 7 & earlier programs written 15 years ago like Sun does :-). -- Eric Green elg@usl.CSNET from BEYOND nowhere: {ihnp4,cbosgd}!killer!elg, P.O. Box 92191, Lafayette, LA 70509 {akgua,killer}!usl!elg "there's someone in my head, but it's not me..."
guy%gorodish@Sun.COM (Guy Harris) (09/30/87)
> > if this causes a disk-space problem, you might have to link with an > > unshared version of the library. > > How can this solve the disk space problem? You'll have *at*least* one > copy of the old version around, linked to the unconverted application, > and possibly many copies. No, you have only a copy of the routines from the old version that the unconverted application *actually uses*. If the difference between the increase in size of the application due to having a copy of those routines bound in, and the size of the old library, is large enough, this *will* save disk space. Guy Harris {ihnp4, decvax, seismo, decwrl, ...}!sun!guy guy@sun.com
daveb@geac.UUCP (Dave Collier-Brown) (10/04/87)
In article <2208@umn-cs.UUCP> stachour@umn-cs.UUCP writes: | That's why Multics has had search-paths since day-1. That's also why it | has the phrase "-referencing_dir" in its rules, so that programs pointed | at an old version of a dynamically-linked shared library thereby | getting the first of a set of routines from that particular version | of the library will get the rest of the set that match. | | (Multics also has done up-referencing and down-referenceing of data, | argument-interogation, compatible-incompatible replacements, | system_library_obsolete, and other ways to handle multiple versions | of a program, but that's another 20-year old story that few read, | because it works so well no-one needs to read or change the code.) I suspect, from working with a programmer (Andrew Forber) who put up an almost-virtual-memory system in place of an overlay loader, that a proper, elegant and even reasonably efficent memory manager of the Multics sort could be put into Minix without seriously impacting either its performance or size. Yes, I'm suggesting that a page-and-segment manager could be smaller than a swapper and some bits of the file system associated with it. --dave (ever notice that sV and 4.3 are *bigger* than Multics) c-b -- David Collier-Brown. {mnetor|yetti|utgpu}!geac!daveb Geac Computers International Inc., | Computer Science loses its 350 Steelcase Road,Markham, Ontario, | memory (if not its mind) CANADA, L3R 1B3 (416) 475-0525 x3279 | every 6 months.
guy%gorodish@Sun.COM (Guy Harris) (10/04/87)
> Needless to say, the Amiga OS is not Unix, for one thing, Unix isn't > going to run on a 512K machine with no MMU and two floppies. So they > don't have to worry about backward compatibility with Version 7 & > earlier programs written 15 years ago like Sun does :-). Well, that may have been *part* of the reason the scheme described in the Sun paper doesn't require libraries to be explicitly opened, but it's not all of it. Since you tell the linker which libraries you're linking with, it can leave information about which shared libraries are needed around for the startup code to use, so that code can do the opening and attaching for you. This scheme could *permit* you to open libraries at any time, or close them, if it wanted, but it doesn't *require* you to do so, regardless of which library you're using. Such a scheme follows the dictum "simple things should be simple" (i.e., if you want to call a routine, you just write a call in your code and tell the linker which libraries to look in) and "complicated things should be possible" (while the scheme described in that paper doesn't include a programmatic interface to the run-time loader, it doesn't prevent such an interface from being provided, and lists this as a future direction). (Multics, one of the original systems that provided dynamic linking to shared libraries, didn't require you to explicitly attach these libraries either.) Guy Harris {ihnp4, decvax, seismo, decwrl, ...}!sun!guy guy@sun.com
guy%gorodish@Sun.COM (Guy Harris) (10/05/87)
> --dave (ever notice that sV and 4.3 are *bigger* than Multics) c-b
OK:
1) What are the definitions of "sV", "4.3", and "Multics" here? Do you
mean the kernel (in the case of the UNIX systems) and the hardcore
supervisor (in the case of Multics), or do you mean the entire
system in all cases?
If you mean the former, is the Multics TCP/IP code part of the
hardcore supervisor? If so, are you comparing the sizes of 4.3 with
TCP/IP and Multics with TCP/IP, both without TCP/IP, or are you
comparing apples and oranges?
2) With the aforementioned questions answered, what *are* the sizes in
question?
Guy Harris
{ihnp4, decvax, seismo, decwrl, ...}!sun!guy
guy@sun.comwesommer@athena.mit.edu (William E. Sommerfeld) (10/05/87)
In article <29944@sun.uucp> guy%gorodish@Sun.COM (Guy Harris) writes: > .... is the Multics TCP/IP code part of the > hardcore supervisor? No, it's not. There is some device driver support (I'm not familiar with the details) in hardcore, but the bulk of the code runs in the user ring (or at least outside of hardcore) in a daemon process. Because of this, it's possible to completely stop TCP, install a new version, and restart it while the system is still running. It's also very unlikely that a failure in the TCP/IP code will cause the entire system to crash. As for sizes.. Looking in >lib>net>e on MIT-Multics, I see about 50 records (of 1024 36 bit words each) of object segments which corespond to the BSD /sys/netinet kernel code. That's about 230K 8-bit bytes.. about 1/2 of a 4.3BSD+NFS VAX kernel (including the symbol table). - Bill
guy%gorodish@Sun.COM (Guy Harris) (10/05/87)
> As for sizes.. Looking in >lib>net>e on MIT-Multics, I see about 50 > records (of 1024 36 bit words each) of object segments which corespond > to the BSD /sys/netinet kernel code. That's about 230K 8-bit bytes.. > about 1/2 of a 4.3BSD+NFS VAX kernel (including the symbol table). Good, somebody has access to real and current data.... 1) How much of the 50 pages are code+data as opposed to symbol table? (I haven't used a Multics system in ages, but I presume it has some way of finding out how much of the segment code and data, right?) 2) How does the size of the hardcore supervisor plus the TCP/IP code compare to the size of a 4.3BSD kernel (with NFS if the Multics in question includes a similar sort of distributed file system, without NFS if it doesn't)? I presume the VAX figure is for the entire kernel, not just the "/sys/netinet" code. Guy Harris {ihnp4, decvax, seismo, decwrl, ...}!sun!guy guy@sun.com
henry@utzoo.UUCP (Henry Spencer) (10/06/87)
> Needless to say, the Amiga OS is not Unix, for one thing, Unix isn't > going to run on a 512K machine with no MMU and two floppies... You might be interested to know that early versions of Unix ran on machines on which 512KB would have been considered an impossible dream, with no MMU, and with hard disks not much bigger or faster than modern floppies. It was a bit primitive by modern standards, but it did work. Minix is said to run fine on a 512K ST. -- PS/2: Yesterday's hardware today. | Henry Spencer @ U of Toronto Zoology OS/2: Yesterday's software tomorrow. | {allegra,ihnp4,decvax,utai}!utzoo!henry
wesommer@athena.mit.edu (William Sommerfeld) (10/08/87)
In article <30034@sun.uucp> guy%gorodish@Sun.COM (Guy Harris) writes: >Good, somebody has access to real and current data.... [regarding the size of Multics] The Multics hardcore supervisor (roughly equivalent to the UNIX kernel less its network & socket code) is about 160,000 lines of PL/1 and 70,000 lines of "non-PL/1" (assembler, mostly). By comparison, the equivalent part of the VAX BSD+NFS kernel (/sys/h, /sys/ufs, /sys/vax, /sys/vaxuba, /sys/vaxmba, and /sys/sys less /sys/sys/uipc_*) is about 110,000 lines, of which about half are device drivers; of the device drivers, maybe 1/4 are in use on an average system. So, it's about 230,000 lines for Multics vs. about 65,000 for an equivalent section of BSD unix. For the *total* Multics system as of 1985, someone counted about 2 millon lines of PL/1 source, 224 thousand lines of assembler source, and 165 thousand lines of "other source", compiled into 10 million 36 bit words of object code (that's 45 MB). - Bill
elg@usl (Eric Lee Green) (10/11/87)
in article <8714@utzoo.UUCP>, henry@utzoo.UUCP (Henry Spencer) says: > Xref: usl-pc comp.arch:776 comp.unix.wizards:1359 comp.os.misc:161 >> Needless to say, the Amiga OS is not Unix, for one thing, Unix isn't >> going to run on a 512K machine with no MMU and two floppies... > You might be interested to know that early versions of Unix ran on machines > on which 512KB would have been considered an impossible dream, with no MMU, > and with hard disks not much bigger or faster than modern floppies. It was > a bit primitive by modern standards, but it did work. I assume you are talking about the PDP-11. The PDP-11 does have a MMU, which does address relocation (i.e. you have a virtual address space of 64K, located in a larger physical address space of 256K to 4 megabytes depend on 18, 20, or 22 bit models). I've never used a PDP-11 myself, but I have the architecture manual for the critter sitting somewhere in my closet... what the world needs is a 32 bit PDP-11 :-). I am not certain as to how Minix is implemented on a 68000-based machine. Probably they use the old pc-relative-relocatable-code trick of OS9-68K, or use an intelligent relocating loader like the AmigaDOS loader. Although how you would handle forks is beyond me (all your pointers in stuff on the heap would be pointing back to the old copy of the heap, not to the current one that you just generated!). However, suffice it to say that Minix is not "full" Unix, not even v7 (no swapping, for one thing) and that 68000 machine code is nowhere near as terse and compact as PDP-11 or even eighty-eighty-sux (8086). -- Eric Green elg@usl.CSNET from BEYOND nowhere: {ihnp4,cbosgd}!killer!elg, P.O. Box 92191, Lafayette, LA 70509 {ut-sally,killer}!usl!elg "there's someone in my head, but it's not me..."
mahler@usl-pc.UUCP (Stephen J. Mahler) (10/11/87)
In article <286@usl> elg@usl (Eric Lee Green) writes: >in article <8714@utzoo.UUCP>, henry@utzoo.UUCP (Henry Spencer) says: >> Xref: usl-pc comp.arch:776 comp.unix.wizards:1359 comp.os.misc:161 >>> Needless to say, the Amiga OS is not Unix, for one thing, Unix isn't >>> going to run on a 512K machine with no MMU and two floppies... >> You might be interested to know that early versions of Unix ran on machines >> on which 512KB would have been considered an impossible dream, with no MMU, >> and with hard disks not much bigger or faster than modern floppies. It was >> a bit primitive by modern standards, but it did work. > >I assume you are talking about the PDP-11. The PDP-11 does have a MMU, >which does address relocation (i.e. you have a virtual address space >of 64K, located in a larger physical address space of 256K to >4 megabytes depend on 18, 20, or 22 bit models). I've never used a >PDP-11 :-). > DEC machines of PDP design included versions that did not include MMUs. The two that jump to mind are the pdp-11/20-15-10 and the LSI-11/03 series. UNIX ran on these machines. The straight forward approach was called MINI-UNIX. It had features between V6 and V7 and the only real stripping of the OS was the removal of account "groups" (/etc/group, newgrp, chgrp, etc.). It did run on systems that had only two floppies (like the AED 8 inch floppy system with about 1 meg per drive). It ran well, even from the portable generator on the corn combine !! ......... Steve
jpdres10@usl-pc.UUCP (Green Eric Lee) (10/13/87)
Distribution: Keywords: Summary: Expires: Sender: Followup-To: In message <73@usl-pc.UUCP>, mahler@usl-pc.UUCP (Stephen J. Mahler) says: >> Needless to say, the Amiga OS is not Unix, for one thing, Unix isn't >> going to run on a 512K machine with no MMU and two floppies... >DEC machines of PDP design included versions that did not include >MMUs. The two that jump to mind are the pdp-11/20-15-10 and the >LSI-11/03 series. UNIX ran on these machines. The straight forward >approach was called MINI-UNIX. It had features between V6 and V7 >and the only real stripping of the OS was the removal of account "groups" >(/etc/group, newgrp, chgrp, etc.). It did run on systems that had only >two floppies (like the AED 8 inch floppy system with about 1 meg per >drive). It ran well, even from the portable generator on the >corn combine !! ......... Steve I suspect that this "MINI-UNIX" was the version that swapped the entire used address space whenever there was a context switch. I seriously doubt whether it was as powerful as the current crop of 68000/no MMU OS's such as OS-9 and TRIPOS (AmigaDOS), which give up the "fork" call in favor of a "spawn" call in order to put more than one process in a single address space (with fork, you'd have to re-adjust every pointer on the heap when you duplicated it for the new process, or else keep tags hanging around telling you the data type -- anathema for "C"). In any event, this whole ball of hair started with "Well, why doesn't" <insert machine name here> "run Unix instead of some odd-ball operating system that I've never heard of?" The answer, once again, is because <insert machine name> was designed to cost around $1,000, and thus left out hardware (MMU & hefty hard drive) necessary to run a MODERN Unix (Sys V.2 or BSD -- forget all these v6's and such, nobody would WANT to use something like that nowadays, although I'm sure it was revolutionary enough originally). -- Eric Green elg@usl.CSNET from BEYOND nowhere: {ihnp4,cbosgd}!killer!elg, P.O. Box 92191, Lafayette, LA 70509 {ut-sally,killer}!usl!elg "there's someone in my head, but it's not me..."
steve@edm.UUCP (Stephen Samuel) (10/13/87)
In article <8714@utzoo.UUCP>, henry@utzoo.UUCP (Henry Spencer) writes: > > Needless to say, the Amiga OS is not Unix, for one thing, Unix isn't > > going to run on a 512K machine with no MMU and two floppies... I have every reason to believe it COULD do so: The first Radio Shack Model 16's were 512K machines and were distributed on 512K floppies. The neat thing about it was that the first floppy was actually a XENIX system disk. With a little fooling around I was able to get a minimal system to boot on a singld-sided (500k) floppy disk -- The big problem was that I needed to set aside ~200k for swap space for FSCK which left little room for more than a minimal subset of /bin. I was, however able to get a relatively nice working set (incl VI and CC) onto 2 double-sided floppies at the request of the alberta research council. Other than the fact that the M16 had an MMU, this almost proves the point by example. -- ------------- Stephen Samuel Disclaimer: You betcha! {ihnp4,ubc-vision,seismo!mnetor,vax135}!alberta!edm!steve BITNET: USERZXCV@UQV-MTS
jfh@killer.UUCP (The Beach Bum) (10/15/87)
In article <192@edm.UUCP>, steve@edm.UUCP (Stephen Samuel) writes: > In article <8714@utzoo.UUCP>, henry@utzoo.UUCP (Henry Spencer) writes: > > > Needless to say, the Amiga OS is not Unix, for one thing, Unix isn't > > > going to run on a 512K machine with no MMU and two floppies... > I have every reason to believe it COULD do so: The first Radio Shack Model > 16's were 512K machines and were distributed on 512K floppies. What about LSX? (Unix for LSI-11's)? I seem to recall that there was a thing that ran unprotected on 11/03's in 56K on RX01's. > The neat thing about it was that the first floppy was actually a XENIX system > disk. > With a little fooling around I was able to get a minimal system to boot on > a singld-sided (500k) floppy disk -- The big problem was that I needed to set > aside ~200k for swap space for FSCK which left little room for more than a > minimal subset of /bin. I too did that. The hard part was building the first double sided root disk. I also had the FSCK crash bug. Seems you couldn't run FSCK standalone off of the floppy disk, so if you had real bad corruption, you might have to scrag the disk and start from backups. > I was, however able to get a relatively nice working set (incl VI and CC) > onto 2 double-sided floppies at the request of the alberta research council. I have gone one step wierder. I have a number of dual-partition floppies (details on request) that are partitioned into binaries and sources. Each partition is 608K which is enough for gov'ment work. One set is a liberal collection of kernel sources I have come up with (mostly wierd device drivers and a re-write of mdep/malloc.c with best-fit allocation). That's the environment I use for my stranger kernel exploits. > Other than the fact that the M16 had an MMU, this almost proves the point > by example. > -- > Stephen Samuel Disclaimer: You betcha! You call what the 16 had an MMU? It has a pair of base and length registers. The VA of one set is 0, the VA for the other is 800000. Separate I&D loads the text at 800000, which is kind of strange. The MMU limits the addressable memory to 1MB as the length is 8 bits with a 4K click. Not much of an MMU. - John. -- John F. Haugh II HECI Exploration Co. Inc. UUCP: ...!ihnp4!killer!jfh 11910 Greenville Ave, Suite 600 "Don't Have an Oil Well?" Dallas, TX. 75243 " ... Then Buy One!" (214) 231-0993
lindsay@cheviot.newcastle.ac.uk (Lindsay F. Marshall) (10/15/87)
In article <286@usl> elg@usl (Eric Lee Green) writes: >... what the world needs is a 32 bit PDP-11 :-). It's called a VAX.......... Why does everyone talk about the 11 as though there werent thousands of them running away happily all over the world? (Come to that what about all the 8's there must be!) Lindsay -- Lindsay F. Marshall JANET: lindsay@uk.ac.newcastle.cheviot ARPA: lindsay%cheviot.newcastle@ucl-cs PHONE: +44-91-2329233 UUCP: <UK>!ukc!cheviot!lindsay "How can you be in two places at once when you're not anywhere at all?"
peter@sugar.UUCP (Peter da Silva) (10/16/87)
> DEC machines of PDP design included versions that did not include > MMUs. ... UNIX ran on these machines. ... It had features between V6 > and V7 and the only real stripping of the OS was the removal of account > "groups" (/etc/group, newgrp, chgrp, etc.). So tell me... how did they implement fork() without an MMU? Did they do addres fixups on running code, or did they require all code be relocatable (not an unreasonable request on the PDP-11, by the way, and one place where the 11 has the 68000 beat hollow). -- -- Peter da Silva `-_-' ...!hoptoad!academ!uhnix1!sugar!peter -- Disclaimer: These U aren't mere opinions... these are *values*.
rbj@icst-cmr.arpa (Root Boy Jim) (10/19/87)
> Other than the fact that the M16 had an MMU, this almost proves the point > by example. > -- > Stephen Samuel Disclaimer: You betcha! You call what the 16 had an MMU? It has a pair of base and length registers. The VA of one set is 0, the VA for the other is 800000. Separate I&D loads the text at 800000, which is kind of strange. The MMU limits the addressable memory to 1MB as the length is 8 bits with a 4K click. Not much of an MMU. Um, that is really all you need. And one step further than the 8086, which only has base registers (no length regs). The Univac 1108 had exactly this kind of MMU, and had a 256K address space. Of course those addresses were 36 bit words, so that is pretty much 1 Meg. John F. Haugh II HECI Exploration Co. Inc. UUCP: ...!ihnp4!killer!jfh 11910 Greenville Ave, Suite 600 "Don't Have an Oil Well?" Dallas, TX. 75243 " ... Then Buy One!" (214) 231-0993 (Root Boy) Jim Cottrell <rbj@icst-cmr.arpa> National Bureau of Standards Flamer's Hotline: (301) 975-5688
mahler@usl-pc.UUCP (Stephen J. Mahler) (10/21/87)
In article <881@sugar.UUCP> peter@sugar.UUCP (Peter da Silva) writes: >> DEC machines of PDP design included versions that did not include >> MMUs. ... UNIX ran on these machines. ... It had features between V6 >> and V7 and the only real stripping of the OS was the removal of account >> "groups" (/etc/group, newgrp, chgrp, etc.). > >So tell me... how did they implement fork() without an MMU? Did they do >addres fixups on running code, or did they require all code be relocatable >(not an unreasonable request on the PDP-11, by the way, and one place where >the 11 has the 68000 beat hollow). >-- Only one proc was in memory at a time. When swtch() was called the entire proc moved to swap. Considering the ~ 20K (word) proc space (small by todays standards ... but reasonable back then) and the speed of the CPU it made a good match with floppies that were DMA. On the other hand DECs RX01 floppies were PIO, now that was slow (but it to was also working at some locations)! .... Steve KF5VH
tainter@ihlpg.ATT.COM (Tainter) (10/26/87)
In article <2455@cheviot.newcastle.ac.uk>, lindsay@cheviot.newcastle.ac.uk (Lindsay F. Marshall) writes: > In article <286@usl> elg@usl (Eric Lee Green) writes: > >... what the world needs is a 32 bit PDP-11 :-). > It's called a VAX.......... Despite the fact that it has a PDP 11 submode the VAX is very definately not a 32 bit PDP 11. The beauty of the PDP 11 the low percentage of special case nonsense. The VAX is almost all special case goop. (Admittedly, it isn't anywhere near as bad as the intel 80* family of satanic torture equipment.) > Lindsay F. Marshall
james@bigtex.uucp (James Van Artsdalen) (09/10/88)
In article <67440@sun.uucp>, guy@gorodish.Sun.COM (Guy Harris) wrote: > S5R3 does it in the kernel. SunOS 4.0 does it in user mode. [...] > It also means it's easier to make it more powerful; the SunOS shared > libraries are not tied to specific locations, and relocation is done > when the library is mapped in. What are the performance trade-offs here? All of Sun's hardware may support position independent code, but I wonder what the expense would be of actually relocating the library code on each and every exec() on a processor where all branches are absolutely addressed. How do the two schemes fair under existing hardware? Does it make a significant difference in the real world? -- James R. Van Artsdalen ...!uunet!utastro!bigtex!james "Live Free or Die" Home: 512-346-2444 Work: 328-0282; 110 Wild Basin Rd. Ste #230, Austin TX 78746
hedrick@athos.rutgers.edu (Charles Hedrick) (09/10/88)
james@bigtex.UUCP (James Van Artsdalen) asks about the overhead of the position-independent code used in supporting Sun's shared library scheme. I don't think that's likely to be an issue. There are several ways of doing position-independent code. One is to use PC relative addressing where you would have used absolute before. That is, suppose you've got load r1,foo Normally you'd expect the loader to relocate foo to an absolute address. If you've got PC-relative addressing, you can instead have it take the difference between foo's location and the location of the instruction itself (which is always the same, no matter where the sharable library happens to be) and use a PC-relative mode. If I read the 68000 instruction book correctly, PC-relative addressing is just as fast as absolute. I think the Intel chips tend to use PC-relative a lot more, and so I'd think there would be no overhead there either. If the machine can't do that, then you can make everything indexed by a register (the old IBM/360 scheme). For Intel chips you could arrange to load segment registers with the address of the code. This involves slight overhead, since when you call a routine in the shared library, you have to load a register with the address of the library, but presumably that just adds an instruction or so to those calls. Finally, if all else fails, you can use a run-time loader to resolve symbols. Sun has such a thing. By various clever techniques they avoid having to do very much run-time relocation, but when all else fails, they can do fixups that are based on the address where you have mapped in the sharable library. The words that need to be fixed up are put into a contiguous area, so that the fixups leave the majority of the code pure. I'd be willing to bet that on any of the common architectures a combination of these techniques can reduce the overhead to the point where it isn't noticable.
idall@augean.OZ (Ian Dall) (11/15/89)
Share libraries seem to me a really neat idea. The implimentation,
however, also seems non trivial. I was wondering how they could be
implimented and had a look at the SunOs 4 man page for ld(1). They
mention that the shared libraries are linked at run time by crt0.o
"invoking" ld.so. I am wondering how this really works.
Is ld.so itself a shared object? If so how does crt0.o get to "invoke"
it before ld.so has been executed? i.e. How do we bootstrap the
process?
Secondly, does the runtime linking only involve relocation or is the
actual code copied from the .so file to the processes address space as
well?
The latter scheme would have the disadvantage that the process text
space would not really be shared (there would be multiple copies of
the library code in core or swap space) and start up might be slow. It
would seem to be better if the library was mapped into the process
virtual address space and demand paged out of the .so file. This might
be difficult to arrange without using up a lot of virtual address
space. Mapping the whole library (whether or not it is all used)
clearly uses up a lot of virtual address space and mapping each
function referenced would waste fragments of a page (average 1 per
function) since the functions do not start and finish on a page
boundary, although is would be possible to align the start of the
functions in the .so file to page boundaries reducing the average size
of the unused page fragments to 0.5 pages per funtion.
The only other scheme I can think of off the top of my head, would
involve the functions being replaced by stubs which requested the
kernel to remap a region of the virtual address space. This implies a
system call per funtion call which might not be too bad for a lot of
libc funtions which do system calls anyway, but would be a significant
overhead for (say) maths libraries.
Anyway, can anyone elaborate on a) how SunOs does shared libraries,
b) how SysV does shared libraries and c) any better schemes.
--
Ian Dall life (n). A sexually transmitted disease which afflicts
some people more severely than others.
idall@augean.ozdeastman@pilot.njin.net (Don Eastman) (11/18/89)
In article <656@augean.OZ>, idall@augean.OZ (Ian Dall) writes: > > Share libraries seem to me a really neat idea. The implimentation, > however, also seems non trivial. I was wondering how they could be > implimented and had a look at the SunOs 4 man page for ld(1). They I might be able to offer a modicum of insight into how SunOS is handling shared libraries. Much of my understanding comes from reading between the widely-spaced lines of various man pages, which includes acres of unexplained details. So, if anyone can point out the cracks in the ice beneath my feet, I'd appreciate it. At start-up, crt0.o checks __DYNAMIC symbol to determine if execution- time linking is required. If __DYNAMIC is defined, crt0.o mmap()'s /usr/lib/ld.so into the program's address space and begins executing at the first text address in ld.so. The first step is to execute a small bootstrap routine which performs the relocations need for ld.so. It then proceeds to mmap() and mark copy-on-write all required objects (libraries) named by the __DYNAMIC struct. (This was established by ld; see man pages for a.out & link.) It is then in a position to relocate any remaining symbols. (If a .sa file exists, those initialized externs are statically linked by ld.) Unresolved functions are resolved during first invocation, at which time, ld.so intervenes and binds references to absolute addresses and updates the linkage table for future calls. > Ian Dall life (n). A sexually transmitted disease which afflicts > some people more severely than others. language (n). A virus from outer space. William Burroughs Don Eastman deastman@pilot.njin.net or ...!rutgers!pilot.njin.net!deastman "If you lick it and it tastes sweet, perhaps it's candy."
guy@auspex.auspex.com (Guy Harris) (11/21/89)
>Is ld.so itself a shared object? Yes. >If so how does crt0.o get to "invoke" it before ld.so has been >executed? It knows where "ld.so" lives, and how to map it in, etc.. It may assume that "ld.so" is position-independent code, so that it doesn't have to relocate it (it is position-independent code, but I don't remember any more whether the "crt0" code could relocate it if it weren't); "ld.so" contains enough code of its own so that it doesn't depend on "libc" or anything like that. >Secondly, does the runtime linking only involve relocation or is the >actual code copied from the .so file to the processes address space as >well? It involves only relocation, and if you've build the ".so" file as position-independent code ("-pic" or "-PIC" for C code, other options may be available for other compilers, do it yourself for assembler code), it requires little if any of that. >The latter scheme would have the disadvantage that the process text >space would not really be shared (there would be multiple copies of >the library code in core or swap space) and start up might be slow. It >would seem to be better if the library was mapped into the process >virtual address space and demand paged out of the .so file. That is exactly what happens; the run-time loader uses "mmap" to map it into the process' address space. It maps it copy-on-write, so that if it *does* have to relocate any code, copies are made of the relocated pages, and so that if you set a breakpoint in shared library code you don't stomp on anybody else using the shared library. (The same holds true of the main executable image; you're not disallowed from debugging something just because somebody else is running it.) >This might be difficult to arrange without using up a lot of virtual address >space. Mapping the whole library (whether or not it is all used) >clearly uses up a lot of virtual address space and mapping each >function referenced would waste fragments of a page (average 1 per >function) since the functions do not start and finish on a page >boundary, although is would be possible to align the start of the >functions in the .so file to page boundaries reducing the average size >of the unused page fragments to 0.5 pages per funtion. Script started on Mon Nov 20 13:20:36 1989 auspex% cat /etc/motd SunOS Release 4.0.3 (AUSPEX) #1: Wed Oct 4 19:00:32 PDT 1989 auspex% arch sun4 auspex% size /usr/lib/lib*.so* text data bss dec hex 319488 16384 0 335872 52000 /usr/lib/libc.so.1.3 294912 57344 632 352888 56278 /usr/lib/libcore.so.0.1 16384 8192 0 24576 6000 /usr/lib/libkvm.so.0.3 180224 8192 0 188416 2e000 /usr/lib/libpixrect.so.2.4 655360 32768 0 688128 a8000 /usr/lib/libsuntool.so.0.40 196608 16384 0 212992 34000 /usr/lib/libsunwindow.so.0.40 which isn't *that* much virtual address space; even Sun-2s support 16MB of virtual address space per process. The entire library is mapped in. >Anyway, can anyone elaborate on a) how SunOs does shared libraries, See above. >b) how SysV does shared libraries For System V Release 4, see above, with one change; instead of "crt0" mapping in "ld.so", the ELF executable image header specifies that the code to be run when the image is started is "ld.so", not the executable image itself, and "ld.so" is handed a file descriptor that refers to the executable image. The intent behind this is to remove the requirement that the executable image itself make any system calls directly, rather than going through a shared library to do so, so that you can change the way those calls are implemented and still have executable images work (with modified "ld.so" and other shared libraries supplied with the system). For System V Release 3, libraries are assigned fixed locations in the address space, so they aren't built with position-independent code and no relocation is done; the entire library is mapped in (done by the "exec" code at start-up time, not with "mmap", since there is no "mmap" in S5R3 from AT&T).
guy@auspex.auspex.com (Guy Harris) (11/22/89)
I went back and checked the paper "Shared Libraries in SunOS" in the Summer 1987 USENIX proceedings, which gives some details on how the SunOS 4.x shared library mechanism works. It's worth reading for anybody interested in SunOS 4.x or System V Release 4 dynamic linking. In particular, it answers a question about "ld.so" left unanswered by my previous posting: >It knows where "ld.so" lives, and how to map it in, etc.. It may assume >that "ld.so" is position-independent code, so that it doesn't have to >relocate it (it is position-independent code, but I don't remember any >more whether the "crt0" code could relocate it if it weren't); "ld.so" >contains enough code of its own so that it doesn't depend on "libc" or >anything like that. In fact, "ld.so" may require relocation, but only "very simple forms of relocation" that can be handled by a bootstrap routine in "ld.so" itself. The paper discusses this briefly in section 6.2 in which it describes how "ld.so" itself is built.
mjh@cs.vu.nl (Maarten J Huisjes) (03/19/91)
Is there a way to convert executables that are linked with shared
libraries (to be loaded by ld.so when executing) into executables that
do not use shared libraries (eg. with all library references resolved and
the used lib functions loaded in the executable) ?
If so, does there exists a program that performs this task ?
--
Maarten Huisjes. mjh@cs.vu.nl (192.31.231.42)
(..!uunet.uu.net!cs.vu.nl!mjh)d89peter@odalix.ida.liu.se (Peter Eriksson) (04/16/91)
jik@athena.mit.edu (Jonathan I. Kamens) writes: >In article <97@titccy.cc.titech.ac.jp>, mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: (some discussion about windows vs text terminals deleted) >|> Shared library is NO solution. It only moves complexity, unstability >|> and ineffeciency of X to UNIX. >This, too, strikes me as an unnecessarily broad statement with little basis in >fact. >X isn't the only thing that uses libraries, and X isn't the only thing that >benefits from shared libraries. It is my impression that the Unix industry >has, in general, come to agreement on the idea that shared libraries are a >good thing, simply because they make more memory available to the user while >having little or no negative side-effects. I would even go as far as saying that they mostly have positive side-effects. That way bug-fixes in the libraries can be introduced and automatically be in effect in all the programs that use those libraries. No need to recompile all zillions small and big programs. >-- >Jonathan Kamens USnail: >MIT Project Athena 11 Ashford Terrace >jik@Athena.MIT.EDU Allston, MA 02134 >Office: 617-253-8085 Home: 617-782-0710 -- Peter Eriksson pen@lysator.liu.se Lysator Computer Club ...!uunet!lysator.liu.se!pen University of Linkoping, Sweden Support the LPF!
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (04/16/91)
In article <1991Apr15.224311.3950@ida.liu.se> d89peter@odalix.ida.liu.se (Peter Eriksson) writes: >(some discussion about windows vs text terminals deleted) That's fine. I only want to show an alternative. I don't want to impose it to everyone else. >>|> Shared library is NO solution. It only moves complexity, unstability >>|> and ineffeciency of X to UNIX. >I would even go as far as saying that they mostly have positive side-effects. >That way bug-fixes in the libraries can be introduced and automatically be >in effect in all the programs that use those libraries. If you have a rationally small environment, there will be much fewer bugs than X-aged-several-years. >No need to recompile >all zillions small and big programs. If you have a rationally small environment, you don't have zillions of small and big programs. I suspect shared libraries may introduce new types of security holes. Masataka Ohta
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (04/17/91)
In article <1991Apr17.020930.25198@NCoast.ORG> allbery@ncoast.ORG (Brandon S. Allbery KB8JRR/AA) writes: >| Shared library is NO solution. It only moves complexity, unstability >| and ineffeciency of X to UNIX. >With that argument, I bet you'd love to still be using v6 Unix.... If something is useful and implemented relatively cleanly, like networking of 4.2BSD, it is OK to expand UNIX. >Shared libraries *do* have a place in standard Unix. And not just for >gigabyte GUIs... the standard C library is a shared library in V.3, which cuts >the size of the system down by quite a bit. I assume this has been continued >into V.4. Then, let's face the reality. On a old 4.2BSD+NFS3.2 workstation (SONY NWS830, 68020) having 180MB of default disk drive capacity, % du -s /bin /usr/bin /usr/ucb /etc /usr/etc 1773 /bin 2601 /usr/bin 6804 /usr/ucb 1998 /etc 2279 /usr/etc The amount of disk space consumed by commands is only 15455 KB. On a 4.3BSD+NFS4.0 workstation (SONY NWS3860, R3000) having 600MB of default disk drive capacity, % du -s /bin /usr/ucb /etc /usr/etc /usr/bin /usr/bin/X11 1 /bin 5265 /usr/ucb 1098 /etc 10648 /usr/etc 44193 /usr/bin 36321 /usr/bin/X11 The amount of disk space consuumed by commands excluding X11 is still only 24884 KB. You may claim shared libraries reduces the amount of required memory. But, even if you run all the command at the same time, total code space consumed is only 24884KB (including non-library code). On the other hand, if you run single X11 application, with shared library, mere /lib/X11.a consumes 3.3MB of virtual (and often real, as libraries tends to be scattered) memory. So, just throw away huge window systems and forget about shared libraries. Masataka Ohta
lerman@stpstn.UUCP (Ken Lerman) (04/17/91)
In article <1991Apr15.224311.3950@ida.liu.se> d89peter@odalix.ida.liu.se (Peter Eriksson) writes: > >I would even go as far as saying that they mostly have positive side-effects. >That way bug-fixes in the libraries can be introduced and automatically be >in effect in all the programs that use those libraries. No need to recompile >all zillions small and big programs. > >-- >Peter Eriksson pen@lysator.liu.se >Lysator Computer Club ...!uunet!lysator.liu.se!pen >University of Linkoping, Sweden Support the LPF! Way to go... So I spend weeks trying to track down some bug in one of your libraries. Then, since I don't have the source code, I come up with some ghastly workaround so I can ship my product. Two weeks after I ship the release to two thousand customers, you release a new version of your library. And my code stops working. A bug fix IS a change in functionality. Ken
jdarcy@seqp4.ORG (Jeff d'Arcy) (04/18/91)
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: >You may claim shared libraries reduces the amount of required memory. >But, even if you run all the command at the same time, total code >space consumed is only 24884KB (including non-library code). On the >other hand, if you run single X11 application, with shared library, >mere /lib/X11.a consumes 3.3MB of virtual (and often real, as libraries >tends to be scattered) memory. Wizardly note: du isn't very useful for determining a program's resident size. This may work for the *workstation* case, but X clients usually run on a shared host. On the machine I'm using right now, the various X libraries include a total of about 525KB of text (data and bss don't count for this comparison since they wouldn't be shared anyway). Without shared libraries, we would need 10.5MB to run 20 different programs linked with all of these libraries. With shared libraries we'd only need 525K, which is an approximately 10MB savings. In point of fact, 10MB is not a lot on real machines, especially since it's pagable. This means that shared libraries are not a really *big* win for X, which uses ridiculous amounts of private data space, but they work really well for something like libc.a. I guess having libc shared would free up space for all those X applications, though. 8] In short, your implication that shared libraries are useless is unfounded. I don't think I'm the only one who'd like to see more *facts* in your posts to back up your abundant opinions. -- Jeff d'Arcy, Generic MTS, Sequoia Systems Inc. <jdarcy%seqp4@m2c.org> Yonder nor sorghum stenches shut ladle gulls stopper torque wet strainers
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (04/18/91)
In article <112@titccy.cc.titech.ac.jp> mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: >You may claim shared libraries reduces the amount of required memory. >But, even if you run all the command at the same time, total code >space consumed is only 24884KB (including non-library code). On the >other hand, if you run single X11 application, with shared library, >mere /lib/X11.a consumes 3.3MB of virtual (and often real, as libraries >tends to be scattered) memory. Let me add one more fact. On SONY NWS3860, the largest X application under /usr/bin/X11 is mwm. If you run mwm, xdm and as many xterms as you want, text data bss dec hex 1007616 192512 27168 1227296 12ba20 /usr/bin/X11/mwm 413696 86016 27792 527504 80c90 /usr/bin/X11/xdm 585728 139264 39376 764368 ba9d0 /usr/bin/X11/xterm still, total text size is below 2MB. So, shared library is NO solution even to reduce memory consumption of X11. Masataka Ohta
bellman@lysator.liu.se (Thomas Bellman) (04/19/91)
In article <6783@stpstn.UUCP> Ken Lerman (lerman@stpstn.UUCP) writes: > So I spend weeks trying to track down some bug in one of your > libraries. Then, since I don't have the source code, I come up with > some ghastly workaround so I can ship my product. > Two weeks after I ship the release to two thousand customers, you > release a new version of your library. > And my code stops working. > A bug fix IS a change in functionality. Well, don't you know that you should not exploit undocumented features? :-) Actually, if you do a workaround for the library bug, then, IMHO, that is a bug in *your* code if the workaround doesn't work with the bug fixed. What happens if I release a new version of the OS with a bug in a syscall fixed? -- Thomas Bellman, Lysator Computer Club ! "Make Love - Nicht Wahr" Linkoping University, Sweden ! "Too much of a good thing is e-mail: Bellman@Lysator.LiU.Se ! WONDERFUL." -- Mae West
rmtodd@servalan.uucp (Richard Todd) (04/19/91)
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: > mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: >>You may claim shared libraries reduces the amount of required memory. >>But, even if you run all the command at the same time, total code >>space consumed is only 24884KB (including non-library code). On the >>other hand, if you run single X11 application, with shared library, >>mere /lib/X11.a consumes 3.3MB of virtual (and often real, as libraries >>tends to be scattered) memory. 3.3 MEG??? That seems to me to be *way* out of line with what I see. Here's what the shared library module for libX11 is like on my system (Mac IIx, 68030 based machine, X11R4 compiled with gcc and Coolidge's shared library patches.) libX11_s is the actual shared library object file that's mapped into the address space of any X client that wants to use the shared lib: % ll /usr/lib/X11/shlib/libX11_s 27930 488 -rwxr-xr-x 1 root root 244027 Mar 25 20:19 /usr/lib/X11/shlib/libX11_s* And that's an unstripped binary; the actual size is: % size -d /usr/lib/X11/shlib/libX11_s Size of /usr/lib/X11/shlib/libX11_s: 141280 Section Size Physical Address Virtual Address .text 107656 1240465408 1240465408 .data 28980 1240989696 1240989696 .bss 4644 1241018676 1241018676 That means that worst case, Xlib will take up ~120K of memory. That's *120K*, not 3.3M. You're off by a factor of 30. (Even if we throw in libXt, libXmu, and libXaw, that's still a total of only 483K, still a far cry from 3.3M.) If standard MIT Xlib really compiles out to 3.3M on your SONY NEWS workstation, remind me not to buy one to replace my Mac.... >If you run mwm, xdm and as many xterms as you want, > text data bss dec hex > 1007616 192512 27168 1227296 12ba20 /usr/bin/X11/mwm > 413696 86016 27792 527504 80c90 /usr/bin/X11/xdm > 585728 139264 39376 764368 ba9d0 /usr/bin/X11/xterm >still, total text size is below 2MB. Well, I don't have mwm (not being willing to pay OSF one grand to make my windows look really ugly :-), but let's look at the memory sizes on my Mac: text data bss total xterm: 146776 55804 32156 234736 xdm: 111004 27568 25272 163844 tvtwm: 150468 29884 30964 211316 So if we've got one of each client going (or more, considering that in any decent Unix implementation the text is shared, and the data&bss usage for these programs is considerably smaller than the text space), we've got ~600K for the clients + ~480K for the shared libs, and add maybe 1M for the server, and we've got a total memory usage for X of ~2M. Not exactly slim, but better than the 3.3M you were claiming for just the libraries alone. >So, shared library is NO solution even to reduce memory consumption of X11. Seems to be doing good on my computer. And even leaving aside memory considerations, compiling the entire X11R4 core client distribution with shared libraries saved over 4M in disk space, just in not requiring every client to contain a copy of the libraries. (There are a *lot* of clients in the std. X11R4 distribution...) -- Richard Todd rmtodd@uokmax.ecn.uoknor.edu rmtodd@chinet.chi.il.us rmtodd@servalan.uucp "Elvis has left Bettendorf!"
fmayhar@hermes.ladc.bull.com (Frank Mayhar) (04/20/91)
In article <109@titccy.cc.titech.ac.jp>, mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: -> In article <1991Apr15.224311.3950@ida.liu.se> -> d89peter@odalix.ida.liu.se (Peter Eriksson) writes: -> >(some discussion about windows vs text terminals deleted) -> That's fine. I only want to show an alternative. I don't want to impose it -> to everyone else. Most of the people here use text-only terminals. That doesn't mean that they wouldn't be more productive using windowing workstations, it just means that they are productive enough using the technology they're familiar with. Note, though, that several people use more than one terminal at a time. How is this different than having several xterms running? -> If you have a rationally small environment, there will be much fewer bugs -> than X-aged-several-years. What is a "rationally small environment?" Many times you have to deal with very large environments. I also don't understand how this relates to the utility of shared libraries. Shared libraries have been around on mainframes for years; they are quite useful, in that they reduce memory and disk space needs, and they enhance efficiency. -> >No need to recompile -> >all zillions small and big programs. -> If you have a rationally small environment, you don't have zillions of small -> and big programs. But _any_ system used by more than a handful of people _is_ going to have "zillions" of small and big programs. Or at least dozens. And that's where shared libraries really shine, imho. (One of the characteristics of Unix is that it _does_ have "zillions" of little programs that all do a very specific thing. But they all (or most of them) use, say, the standard library, right? If that's a shared library, it means that those routines are only found on one place on disk and in memory, and that they are most likely already paged in when they are needed.) -> I suspect shared libraries may introduce new types of security holes. Of course they do. But these holes can be plugged, with a little effort. And in most cases, the added utility of shared libraries more than makes up for that effort. -- Frank Mayhar fmayhar@hermes.ladc.bull.com (..!{uunet,hacgate}!ladcgw!fmayhar) Bull HN Information Systems Inc. Los Angeles Development Center 5250 W. Century Blvd., LA, CA 90045 Phone: (213) 216-6241
cks@hawkwind.utcs.toronto.edu (Chris Siebenmann) (04/20/91)
jik@athena.mit.edu (Jonathan I. Kamens) writes (or at least I think he does, in a quoted article: | It is my impression that the Unix industry has, in general, come to | agreement on the idea that shared libraries are a good thing, simply | because they make more memory available to the user while having little | or no negative side-effects. Don't forget "if implemented carefully and well". Unfortunately, it seems dangerous to rely on this happening -- compare the startup times of the same application linked statically and with shared libraries on SunOS for a drastic illustration (SunOS also seems to have problems fork()ing dynamically-linked programs, but I don't have a handy test program to check this again). -- "This will be dynamically handled, possibly correctly, in 4.1." - Dan Davison on streams configuration in SunOS 4.0 cks@hawkwind.utcs.toronto.edu ...!{utgpu,utzoo,watmath}!utgpu!cks
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (04/23/91)
In article <1991Apr19.005739.2321@servalan.uucp> rmtodd@servalan.uucp (Richard Todd) writes: >>>mere /lib/X11.a consumes 3.3MB of virtual (and often real, as libraries >>>tends to be scattered) memory. > 3.3 MEG??? That seems to me to be *way* out of line with what I see. Sorry, it was my mistake to have overestimated memory consumption of shared libraries. >That means that worst case, Xlib will take up ~120K of memory. That's >*120K*, not 3.3M. You're off by a factor of 30. On NWS3860, total code size of libX11.a is about 160K. about 1/20 of its file size. But, this means, shared library is not so good, because statically linked binary is small enough (1/20). >And even leaving aside memory >considerations, compiling the entire X11R4 core client distribution with >shared libraries saved over 4M in disk space, 4M is small compared to the size of while X11R4 core. Moreover, binaries can be shared through network. Masataka Ohta
sef@kithrup.COM (Sean Eric Fagan) (04/26/91)
In article <135@titccy.cc.titech.ac.jp> mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: >4M is small compared to the size of while X11R4 core. Moreover, binaries can >be shared through network. Using shared libraries dropped SCO's distribution from 60 or so floppies to about 25 or 30 (they're compressed, as well; the actual floppy count is 13 or 15, I believe; I doubled for that previous estimate). Want to tell me again that shared libraries are bad? -- Sean Eric Fagan | "I made the universe, but please don't blame me for it; sef@kithrup.COM | I had a bellyache at the time." -----------------+ -- The Turtle (Stephen King, _It_) Any opinions expressed are my own, and generally unpopular with others.
cgy@cs.brown.edu (Curtis Yarvin) (04/28/91)
In article <BZS.91Apr28000804@world.std.com> bzs@world.std.com (Barry Shein) writes: > >From: sef@kithrup.COM (Sean Eric Fagan) >>Using shared libraries dropped SCO's distribution from 60 or so floppies to >>about 25 or 30 (they're compressed, as well; the actual floppy count is 13 >>or 15, I believe; I doubled for that previous estimate). >> >>Want to tell me again that shared libraries are bad? > >Well, there's more to life than saving disk space. Shared libraries >also incur a performance hit which varies on different systems (and >applications on the same system, for example.) > >Try running some job using shared and statically linked libraries on >various systems. I've seen from 5-10% to 75% performance degradations >(granted the last was odd enough to look into, but it was there.) This is true, but 99.97% bogus. Statically linked shared libraries should incur little or no performance hit. Dynamically linked shared libraries (a la Sun) will incur a performance hit, but it is completely misleading to talk about the hit in percentage terms. The linking occurs either at initial runtime or when the linked function is called. In the former case, the cost is dependent on the number of libraries linked with; in the latter (I'm not sure which systems use this - maybe none), it is dependent on the number of library functions called. Either way, the cost is CONSTANT relative to the execution time of the program. It will be on the same order for a 10-hour rendering job and main() { printf("Hello, world.\n"); }. Thus it is utterly misleading to talk about the performance cost of shared libraries in percentage terms. Barry Shein is a very intelligent person and no doubt knows this. Curtis
bzs@world.std.com (Barry Shein) (04/28/91)
From: sef@kithrup.COM (Sean Eric Fagan) >Using shared libraries dropped SCO's distribution from 60 or so floppies to >about 25 or 30 (they're compressed, as well; the actual floppy count is 13 >or 15, I believe; I doubled for that previous estimate). > >Want to tell me again that shared libraries are bad? Well, there's more to life than saving disk space. Shared libraries also incur a performance hit which varies on different systems (and applications on the same system, for example.) Try running some job using shared and statically linked libraries on various systems. I've seen from 5-10% to 75% performance degradations (granted the last was odd enough to look into, but it was there.) Something that would be nice would be a way to unshare an application from a vendor's binary distribution. Then no one would have any complaints. -- -Barry Shein Software Tool & Die | bzs@world.std.com | uunet!world!bzs Purveyors to the Trade | Voice: 617-739-0202 | Login: 617-739-WRLD
sef@kithrup.COM (Sean Eric Fagan) (04/29/91)
In article <73761@brunix.UUCP> cgy@cs.brown.edu (Curtis Yarvin) writes: >In article <BZS.91Apr28000804@world.std.com> bzs@world.std.com (Barry Shein) writes: >>Try running some job using shared and statically linked libraries on >>various systems. I've seen from 5-10% to 75% performance degradations >>(granted the last was odd enough to look into, but it was there.) >Statically linked shared libraries should incur little or no performance hit. On a '386 running SCO UNIX (or xenix, for that matter), a call to a shared library function one jump in addition to the normal call. Thus, there is a *very* slight performance hit. I don't necessarily agree that most of libc should be shared. libX, on the other hand, as well as the toolkit libraries, would be a *very* good thing to share, for many reasons; the increase in execution time is offset far more than enough by the savings in paging, physical memory, and disk space used. -- Sean Eric Fagan | "I made the universe, but please don't blame me for it; sef@kithrup.COM | I had a bellyache at the time." -----------------+ -- The Turtle (Stephen King, _It_) Any opinions expressed are my own, and generally unpopular with others.
sef@kithrup.COM (Sean Eric Fagan) (04/29/91)
In article <1991Apr29.025800.3108@kithrup.COM> sef@kithrup.COM (Sean Eric Fagan) writes: >On a '386 running SCO UNIX (or xenix, for that matter), a call to a shared >library function one jump in addition to the normal call. Thus, there is a >*very* slight performance hit. Oops. Forgot something, which explained my next paragraph. In addition to the call overhead, any shared data goes through an extra level of indirection, adding to execution time, and any function within the same shared library will also go through an extra layer of indirection. While this is not necessarily worth it for things like printf, I do believe that it is worth it for things like X. -- Sean Eric Fagan | "I made the universe, but please don't blame me for it; sef@kithrup.COM | I had a bellyache at the time." -----------------+ -- The Turtle (Stephen King, _It_) Any opinions expressed are my own, and generally unpopular with others.
rcd@ico.isc.com (Dick Dunn) (04/30/91)
cgy@cs.brown.edu (Curtis Yarvin) writes: > bzs@world.std.com (Barry Shein) writes: > >...Shared libraries > >also incur a performance hit which varies on different systems (and > >applications on the same system, for example.) ... > >...I've seen from 5-10% to 75% performance degradations > >(granted the last was odd enough to look into, but it was there.) > This is true, but 99.97% bogus. > > Statically linked shared libraries should incur little or no performance hit. Depends on how calls to library routines work. A simplistic implementation would just slap in the address in the shared region--but that's a main- tenance headache. Either you have to coreograph some elaborate song-and- dance with padding so addresses don't move, or applications have to be bound to a particular instance of a shared library. (Remember, this is under the static-link assumption.) If you decouple addressing into the shared library with transfer vectors (and/or linkage vectors for data) the indirection will cost real execution time, and the cost occurs per call (or reference). [Curtis then discusses dynamic-linking costs as incurred at the time the linking is done] > Either way, the cost is CONSTANT relative to the execution time of the > program... Only if references into shared-library code have the same form as local/ private references. It is common (tho by no means universal) that they do not. > Thus it is utterly misleading to talk about the performance cost of shared > libraries in percentage terms. > > Barry Shein is a very intelligent person and no doubt knows this. Perhaps Barry is intelligent enough to know not only what you said, but a little bit more, about cases where there is an execution-time cost? -- Dick Dunn rcd@ico.isc.com -or- ico!rcd Boulder, CO (303)449-2870 ...If you plant ice, you're gonna harvest wind.
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (04/30/91)
In article <1991Apr29.025800.3108@kithrup.COM> sef@kithrup.COM (Sean Eric Fagan) writes: >On a '386 running SCO UNIX (or xenix, for that matter), a call to a shared >library function one jump in addition to the normal call. Thus, there is a >*very* slight performance hit. Branch is the most ineffecient instruction on today's highly pipelined architecture. >I don't necessarily agree that most of libc should be shared. Didn't you claimed sharing of libc is good because of UUCP and MMDF? >libX, on the >other hand, as well as the toolkit libraries, would be a *very* good thing >to share, for many reasons; Well, the following equation holds: uni_X + X_window = multi_X >the increase in execution time is offset far >more than enough by the savings in paging, physical memory, and disk space >used. The best solution is not to have a complex window system. Masataka Ohta
fmayhar@hermes.ladc.bull.com (Frank Mayhar) (05/02/91)
In article <154@titccy.cc.titech.ac.jp>, mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes:
-> The best solution is not to have a complex window system.
Jeez, you keep saying that. But I LIKE being able to do more than one thing
at a time. It increases my efficiency a lot. And since we don't have the
budget for a nicer window system, and since Sunview is a hog, X it is.
Not all of us can stand being limited to 24 lines by 80 columns. Sorry.
--
Frank Mayhar fmayhar@hermes.ladc.bull.com (..!{uunet,hacgate}!ladcgw!fmayhar)
Bull HN Information Systems Inc. Los Angeles Development Center
5250 W. Century Blvd., LA, CA 90045 Phone: (213) 216-6241mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (05/02/91)
In article <1991May1.205923.7669@ladc.bull.com> fmayhar@hermes.ladc.bull.com writes: >-> The best solution is not to have a complex window system. > >Jeez, you keep saying that. But I LIKE being able to do more than one thing >at a time. It increases my efficiency a lot. For me, to do so, job control is better than X. You might disagree, but >And since we don't have the >budget for a nicer window system, and since Sunview is a hog, X it is. if you don't have budget, X is not it. >Not all of us can stand being limited to 24 lines by 80 columns. Sorry. My console is 42 lines by 80 columns. Masataka Ohta
bzs@world.std.com (Barry Shein) (05/05/91)
From: cgy@cs.brown.edu (Curtis Yarvin) >>Try running some job using shared and statically linked libraries on >>various systems. I've seen from 5-10% to 75% performance degradations >>(granted the last was odd enough to look into, but it was there.) > >This is true, but 99.97% bogus. > >Statically linked shared libraries should incur little or no performance hit. > >Dynamically linked shared libraries (a la Sun) will incur a performance hit, >but it is completely misleading to talk about the hit in percentage terms. >The linking occurs either at initial runtime or when the linked function is >called. In the former case, the cost is dependent on the number of >libraries linked with; in the latter (I'm not sure which systems use this - >maybe none), it is dependent on the number of library functions called. Have you actually made any attempt to measure this effect? Or are you just drawing conclusions from a few facts you have lying around? I have been doing some measuring. For an easy laboratory, try compiling dhrystone 2.1 static and dynamic on various systems and see how many dhrystones you get on each. Vary the number of runs through the program also, should give some feeling as to whether or not there is an asymptote. There are other effects here than sheer code count, most noticeably instruction cache flushing due to either branches or even self-modifying code (I haven't even thought about code locality as I'm purposely looking at small programs which shouldn't page.) But the lesson may be: Two things which are different are rarely the same. And they may be different in more ways than you first think. As I said, it depends a lot on the implementation (both software and hardware), finding systems which vary little can not be generalized. Some systems will show little difference, some may show 50% or more performance drops. The latter are probably correctable, although unless you have source or even access to the cache designs this may be difficult. >Barry Shein is a very intelligent person and no doubt knows this. Indeed, that's a good reason to be much more careful when criticizing :-) -- -Barry Shein Software Tool & Die | bzs@world.std.com | uunet!world!bzs Purveyors to the Trade | Voice: 617-739-0202 | Login: 617-739-WRLD
terryl@sail.LABS.TEK.COM (05/07/91)
In article <155@titccy.cc.titech.ac.jp> mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: +In article <1991May1.205923.7669@ladc.bull.com> + fmayhar@hermes.ladc.bull.com writes: + +>-> The best solution is not to have a complex window system. +> +>Jeez, you keep saying that. But I LIKE being able to do more than one thing +>at a time. It increases my efficiency a lot. + +For me, to do so, job control is better than X. +You might disagree, but +>And since we don't have the +>budget for a nicer window system, and since Sunview is a hog, X it is. +if you don't have budget, X is not it. + +>Not all of us can stand being limited to 24 lines by 80 columns. Sorry. + +My console is 42 lines by 80 columns. And how easily can you change that (if at all)??? What, you mean you'll never want to change it??? What if you wanted to view a file that someone had entered on a > 80 column terminal, and you'd like to see each line on ONE physical line, instead of multiple physical lines??? Thanx, but I'll keep my X, bloated as it may be.... __________________________________________________________ Terry Laskodi "There's a permanent crease of in your right and wrong." Tektronix Sly and the Family Stone, "Stand!" __________________________________________________________
lm@slovax.Eng.Sun.COM (Larry McVoy) (05/08/91)
bzs@world.std.com (Barry Shein) writes: > >Dynamically linked shared libraries (a la Sun) will incur a performance hit, > >but it is completely misleading to talk about the hit in percentage terms. > > Have you actually made any attempt to measure this effect? Or are you > just drawing conclusions from a few facts you have lying around? I have (made an attempt to measure it). I measured: main() { exit(0); } statically and dynamically on an SS1 (20 MHZ sparc). I found that dynamically linked version cost about 20milliseconds more than statically linked version. I do not know (not having measured it and not really groking how dynamic linking works) if each libc reference costs 20milliseconds on first reference. I hope not. --- Larry McVoy, Sun Microsystems (415) 336-7627 ...!sun!lm or lm@sun.com
mjr@hussar.dco.dec.com (Marcus J. Ranum) (05/09/91)
lm@slovax.Eng.Sun.COM (Larry McVoy) writes: >I do not know (not having measured it and not really groking how >dynamic linking works) if each libc reference costs 20milliseconds on >first reference. I hope not. I was talking this over with someone the other day who claimed that SunOs' more recent versions have "often used" utilities like /bin/ls statically linked because there were complaints of performance hits - can anyone confirm or deny this? I haven't has access to a machine running a SunOS more recent than 4.0. [I realize that "often used" and critical utilities might be statically linked to prevent the whole system being rendered useless by a hit on the library. Is "ldconfig" or whatever it's called still a dynamically linked file?] mjr.
lm@slovax.Eng.Sun.COM (Larry McVoy) (05/10/91)
mjr@hussar.dco.dec.com (Marcus J. Ranum) writes: > I was talking this over with someone the other day who claimed > that SunOs' more recent versions have "often used" utilities like /bin/ls > statically linked because there were complaints of performance hits - > can anyone confirm or deny this? $ ldd /bin/ls -lc.1 => /usr/lib/libc.so.1.5 $ cat /etc/motd SunOS Release 4.1.1 (SLOVAX) #20: Thu May 9 17:19:32 PDT 1991 I build the entire system now and again and I know of no program that is statically linked other than those that must be (during boot; it's a bummer if init is dynamically linked :-) This doesn't mean that some things won't benefit from static linking. The amount of overhead due to shared libraries strikes me as almost all start up overhead. The only programs that I'd think worth it are things like /bin/echo; almost all of its running time is start up. > Is "ldconfig" or whatever it's called still a dynamically linked file?] $ ldd /usr/etc/ldconfig -lc.1 => /usr/lib/libc.so.1.5 --- Larry McVoy, Sun Microsystems (415) 336-7627 ...!sun!lm or lm@sun.com
mayer@gmdzi.gmd.de (Hans Mayer) (05/10/91)
From article <1991May09.032802.27213@decuac.dec.com>, by mjr@hussar.dco.dec.com (Marcus J. Ranum): > > I was talking this over with someone the other day who claimed > that SunOs' more recent versions have "often used" utilities like /bin/ls > statically linked because there were complaints of performance hits - > can anyone confirm or deny this? jupiter% file /bin/ls /bin/ls: mc68020 pure dynamically linked executable jupiter% strings /vmunix | grep SunOS SunOS Release 4.1.1 (GENERIC) #1: Thu Apr 25 16:42:43 MET DST 1991 jupiter% -- Hans J. Mayer, mayer@gmdzi.gmd.de (or hmayer@venus.darmstadt.gmd.de) German National Research Center for Computer Science (GMD)
mjr@hussar.dco.dec.com (Marcus J. Ranum) (05/10/91)
>>mjr@hussar.dco.dec.com (Marcus J. Ranum): >> >> I was talking this over with someone the other day who claimed >> that SunOs' more recent versions have "often used" utilities like /bin/ls >> statically linked [...] Thanks to everyone who set me straight on this. The source of the disinformation was, surprisingly, an instructor in an internals course - maybe those who don't know do, in fact, teach. I don't have access to Suns around here, for some reason or another... :) mjr.
rickert@mp.cs.niu.edu (Neil Rickert) (05/10/91)
In article <1991May10.133720.12358@decuac.dec.com> mjr@hussar.dco.dec.com (Marcus J. Ranum) writes: >>>mjr@hussar.dco.dec.com (Marcus J. Ranum): >>> I was talking this over with someone the other day who claimed >>> that SunOs' more recent versions have "often used" utilities like /bin/ls >>> statically linked [...] > > Thanks to everyone who set me straight on this. The source of the Actually it is a pity that Sun's /bin/ls is not statically linked. If it were it would make life much easier for those who want to set up an anonymous ftp account. -- =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*= Neil W. Rickert, Computer Science <rickert@cs.niu.edu> Northern Illinois Univ. DeKalb, IL 60115 +1-815-753-6940
fmayhar@hermes.ladc.bull.com (Frank Mayhar) (05/11/91)
I was out sick a few days, and apparently lost article <155@titccy.cc.titech.ac.jp>, but I'll followup terry's post. In article <9471@sail.LABS.TEK.COM>, terryl@sail.LABS.TEK.COM writes: -> In article <155@titccy.cc.titech.ac.jp> mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: -> +In article <1991May1.205923.7669@ladc.bull.com> I write: -> +>Masataka Ohta writes: -> +>-> The best solution is not to have a complex window system. -> +> -> +>Jeez, you keep saying that. But I LIKE being able to do more than one thing -> +>at a time. It increases my efficiency a lot. -> + -> +For me, to do so, job control is better than X. -> +You might disagree, but Well, yeah, I disagree. Job control is not the same thing has being able to run several processes, and have their (copious) output make sense. With multiple windows, this is easy. And what about several simultaneous logins to different systems? I can have a window running a shell on my local system (for local editing or something), a window with a telnet to the site mainframe, a window with an rlogin to the mail router, a window rlogin'ed to the big NFS server (_not_ the same as the mainframe), _and_ another window running a continuous tcpdump. This is very convenient, since I have things to do on all of them, and usually have to go from one to another at a moment's notice. And having separate windows, all appropriately labelled, keeps every- thing straight. Plus I can have an xnetload and a clock running, and a pretty girl in a swimsuit in my root window. Do _that_ with your 42x80 terminal! Oh, yeah, and my mainframe windows (and I can have more than one, if I want) are usually 60x80. (This lets me edit a file on the mainframe in one window, and examine other, related, files in other windows. Simultaneously.) -> +>And since we don't have the -> +>budget for a nicer window system, and since Sunview is a hog, X it is. -> +if you don't have budget, X is not it. OK, name me another windowing system that runs on a Sun 386i and that is free. Besides (yuck) Sunview (free for the price of the 386i, anyway). (Apparently MGR is floating around, but I haven't seen it yet.) -> +>Not all of us can stand being limited to 24 lines by 80 columns. Sorry. -> +My console is 42 lines by 80 columns. -> And how easily can you change that (if at all)??? What, you mean you'll -> never want to change it??? What if you wanted to view a file that someone had -> entered on a > 80 column terminal, and you'd like to see each line on ONE -> physical line, instead of multiple physical lines??? What he said! :-) -> Thanx, but I'll keep my X, bloated as it may be.... Well, if there were a viable alternative, I would use it. But since there doesn't seem to be one at the moment, I guess I'm stuck. Hmm. Maybe the best solution is not to use a _complex_ window system. But are there any _simple_ window systems out there? Particularly that have lots of free software for them? Thought not. -- Frank Mayhar fmayhar@hermes.ladc.bull.com (..!{uunet,hacgate}!ladcgw!fmayhar) Bull HN Information Systems Inc. Los Angeles Development Center 5250 W. Century Blvd., LA, CA 90045 Phone: (213) 216-6241
schwartz@karl.cs.psu.edu (Scott Schwartz) (05/11/91)
rickert@mp.cs.niu.edu (Neil Rickert) writes: | Actually it is a pity that Sun's /bin/ls is not statically linked. If it | were it would make life much easier for those who want to set up an | anonymous ftp account. Maybe Sun will reinvent the multics binder so you can fix that yourself.
guy@auspex.auspex.com (Guy Harris) (05/13/91)
> I was talking this over with someone the other day who claimed >that SunOs' more recent versions have "often used" utilities like /bin/ls >statically linked because there were complaints of performance hits - >can anyone confirm or deny this? Unless they're thinking of some SunOS release that hasn't come out yet, yes, I can completely deny it: bootme% cat /etc/motd SunOS Release 4.1.1 (MCDLCD60) #9: Wed Mar 27 18:23:07 PST 1991 bootme% file /usr/bin/ls /usr/bin/ls: sparc pure dynamically linked executable bootme% cd /usr/bin bootme% file * | egrep -v \ 'dynamically linked|symbolic link|shell script|Permission denied' bar: sparc demand paged executable hostname: sparc pure executable ln: sparc pure executable mv: sparc pure executable rfuadmin: commands text sh: sparc demand paged executable sunview1: directory tar: sparc demand paged executable bootme% cd /usr/ucb bootme% file * | egrep -v \ 'dynamically linked|symbolic link|shell script|Permission denied' rcp: sparc demand paged set-uid executable ("/bin" is, of course, a symbolic link to "/usr/bin", so there's no point in looking at it). So the set of regular utilities that are statically linked includes: bar - beats me why it's statically linked, unless it's by analogy to "tar" hostname - it's run as part of the very early boot sequence, before "/usr" is mounted, i.e. before any shared libraries are available sh - see "hostname" tar, rcp - statically linked so they can be used to snarf shared libraries from somewhere else if you trash them (definitely true of "rcp", I think that was the reason for "tar" as well but I'm not sure) ln, mv - maybe the same as for "tar" and "rcp" >Is "ldconfig" or whatever it's called still a dynamically linked file?] bootme% file /usr/etc/ldconfig /usr/etc/ldconfig: sparc pure dynamically linked executable bootme% cd /usr/etc bootme% file * | egrep -v \ 'dynamically linked|symbolic link|shell script|Permission denied' fsck: sparc demand paged executable ifconfig: sparc demand paged executable init: sparc demand paged executable intr: sparc demand paged executable mount: sparc demand paged executable praudit: sparc pure executable restore: sparc demand paged set-uid executable update: sparc executable yp: directory
guy@auspex.auspex.com (Guy Harris) (05/14/91)
> Actually it is a pity that Sun's /bin/ls is not statically linked. If it >were it would make life much easier for those who want to set up an >anonymous ftp account. Only for those who want to set up an anonymous FTP account and can't be bothered reading FTPD(8), or are too lazy to do what that manual page tells you to do....
guy@auspex.auspex.com (Guy Harris) (05/14/91)
>The only programs that I'd think worth it are things like /bin/echo; >almost all of its running time is start up. Of course, a lot of shell scripts shouldn't be using "/bin/echo" anyway; they should be using the "echo" built into {sh,csh,ksh,...}.