andrew@orca.UUCP (Andrew Klossner) (08/11/83)
Consider a program written for a 16-bit machine which uses
printf("%X",(long)a) to print a long. This program might live into an
era in which printf is redesigned so that the correct arguments become
printf("%lx",(long)a), because "%X" has come to mean that upper case
letters are to be used in the hexadecimal numeral.
If this program had been constructed to use shared libraries,
dynamically loaded at runtime, then it would seem to "break", despite
the lack of changes to it.
This is not a case of exploiting a library bug. What has happened is
that the library has evolved and the program has remained static.
-- Andrew Klossner (decvax!tektronix!tekecs!andrew) [UUCP]
(andrew.tektronix@rand-relay) [ARPA]tim@unc.UUCP (08/11/83)
All the user-visible problems that have recently been mentioned
with respect to shared libraries, for instance programs breaking when
the library changes, are equally true of any library when the program
is relinked, which can reasonably be expected of any program in
general use or of medium or large size. Given this, it is hard to
take these objections seriously.
A much more serious objection is that the addition of a new
segment to each UNIX process, which seems to be the only clean way to
do this, involves far-reaching and non-trivial changes in the kernel.
I have yet to see anyone suggest solutions at this level; it's a
little early to start talking about flag bits in page tables, when we
don't even have an overall design for the thing. This sort of
implementation-first thinking is just the thing that could cause the
kernel to be munged beyond hope, or more likely keep the project from
even getting off the ground.
___________
Tim Maroney
duke!unc!tim (USENET)
tim.unc@udel-relay (ARPA)
The University of North Carolina at Chapel Hilllouie@cvl.UUCP (Louis A. Mamakos) (08/11/83)
Yes, it is true that if the shared libraries change, then existing programs
will operate differently ('break'). From my experience, this has not been
a problem. As a part of my work, I do systems hacking on a UNIVAC 1100 series
systems which has shared libraries, and the thing that keeps everything
working and (most) everyone happy is that the function of a specific shared
library is \clearly/ and \completely/ defined. When working in a production
environment, you don't just change the defintion of a commonly used library
at a whim. The advantages of being able to fix a bug, and have ALL programs
which use it get the new and updated copy greatly outweigh any of the other
problems involved. Of course, this means that you MUST think about the
functions that your library will perform before you rush ahead and implement
it.
Ready to repel flames,
Louis Mamakos
Internet: louie@cvl
Usenet: ..rlgvax!cvl!louiechris@umcp-cs.UUCP (08/11/83)
Everyone's got a point: shared libraries, if changed, will cause
programs to break. So (says the pro-shared) don't change them; do
them right the first time.
Doing it right the first time is certainly commendable, but it's
not always possible.
One of the best things about Unix is that it isn't fixed. 4.2
doesn't look that much like V6 anymore. SysV seems to be even less
like V6.
As new hardware and software techniques are discovered, things will
keep changing; flexibility is required.
Something tells me that this is why any commercial system is never
quite as good as the "hacks" like Unix. (And maybe Unix is going
to fall by the wayside, once everyone starts using it and it's
forced to become static.)
Just a thought.
- Chris
PS How 'bout *optional* shared libraries? Seems to me this gets
around most everyone's arguments. The old programs don't get them
since you aren't recompiling them; the new ones do unless you say
not to. With retrofit libraries, that keeps the workload down. (I
know, I'm not the first one to propose this.)
--
In-Real-Life: Chris Torek, Univ of MD Comp Sci
UUCP: {seismo,allegra,brl-bmd}!umcp-cs!chris
CSNet: chris@umcp-cs ARPA: chris.umcp-cs@UDel-Relaygwyn@brl-vld@sri-unix.UUCP (08/12/83)
From: Doug Gwyn (VLD/VMB) <gwyn@brl-vld> The first time, on your future shared-library UNIX system, that someone updates the shared library and introduces a bug that stops exec() from working, you may change your mind about it being hard to take such objections seriously.
dan@bbncd@sri-unix.UUCP (08/12/83)
From: Dan Franklin <dan@bbncd> What people seem to forget is that shared libraries would be easier to change in EVERY way. With respect to changes that might introduce bugs, shared libraries are actually superior to regular libraries, especially if they are done with the flexibility of Multics-style dynamic linking. When you made a change to the library, you would first make the change in a private version of the library that only your own process tree would use. Now every UNIX command (that you run) will get tested with the new library--a fairly complete test! There would be no possibility of releasing a version of the library in which exec did not work; on the contrary, library changes would be tested far more thoroughly than they are now. Once you decided to release the library, you would again find any remaining bugs much faster than with the current system, because EVERYTHING would be using it. And as soon as users found a sufficiently bad bug, they would just switch to the older version of the library, which you have of course retained in a well-known place. Meanwhile, you go and fix the bug, and now again EVERYTHING is using the fixed version. Someone else brought up the matter of incompatible changes, e.g. to the printf format characters. This is not an argument against shared libraries, but an argument against shared anything, including kernels. You can't make an incompatible change to the kernel without providing some provision for older programs; you would have to be just as careful with shared libraries. And you should already be taking that care with libraries, so that a program doesn't break merely because it's reloaded. (And thus make you think that it was the change you made before you reloaded that caused the problem!) Incompatible changes can almost always be avoided (printf CAN interpret both character sequences), but when they must happen, you can just keep the old version around for awhile. If a program were sufficiently ancient, you would have to have the older version of the library resuscitated from backup tapes, just as you would have to do anyway as soon as someone tried to recompile it. If you haven't made any incompatible changes to the kernel which require the new library, it will work just fine. If you have, well, the shared libraries don't matter; the program would still be broken anyway. In short, shared libraries (properly implemented) would make changing the library routines an easier, safer task than they are now. Dan Franklin
jbray@bbn-unix@sri-unix.UUCP (08/12/83)
From: James Bray <jbray@bbn-unix> Most certainly, the way to do it is to add multiple-segment referencing, and it is just as certainly non-trivial. Processes should be able to reference some reasonably large number of symbolically-named objects, and the kernel should take care of all details of finding them, loading them if necessary, and controlling the type (read-write-execute) of access. Or if one doesn't feel up to hacking dynamic linking into the kernel, the references could be interpreted and established at load time, --this also makes it optional, so that tasks which are paranoid about referencing potentially changeable run-time-libraries could instead use old-style libraries--, and the kernel would then have to simply verify at exec-time that every object which may be referenced by a process is in fact present, and then establish the correct binding in the process' address-space. The process would for example contain in its header information telling the kernel "I expect object <libc_version_ 12345> to be available, I intend to execute it, and I expect it to begin at 0400000 in my address space". It is not worth trying to do it on a machine that doesn't have reasonably sophisticated memory-management. This sort of scheme also gives one sharable memory, and one can probably also think of other nice things to do with it. Full-scale dynamic linking is the way to go, but this is the last stop along the road before you get there. Granted, the implementation would take some doing... By the way, I will ask again: has anyone out there actually seen the way system V actually does what it calls shared memory? I have yet to get my copy, and am dying to know what it looks like. Did they in fact completely redesign major chunks of the kernel, or is it some bizarre and ugly hack involving disk files or worse... --Jim Bray
gwyn@brl-vld@sri-unix.UUCP (08/12/83)
From: Doug Gwyn (VLD/VMB) <gwyn@brl-vld> UNIX System V shared memory sets up page table entries for the processes involved; what else did you expect? I wouldn't say they completely redesigned major chunks of the kernel, but exec, exit, etc. do have hooks into the shared-memory module. This shared memory is really shared memory. I believe it was added to UNIX by the Real-Time folks, who need this kind of thing. Sockets are not the universal answer to everybody's IPC needs.
drforsey@watcgl.UUCP (Dave Forsey) (08/13/83)
The arguments against the use of shared libraries seems to rest on the fact that the structural rather than the functional aspect of the library will change causing a large amount of managerial headaches. This problem goes away if the libraries are not affected by the structure of the data it is handed. Absurd? Perhaps. But reflect on the way that Smalltalk-80 works. Everything in the system is part of a shared library. Handing a floating point number instead of a SmallInteger to a method causes no problems because it still behaves as a "Number". While this can't handle everything it goes a lot further than any other system. This is, by the way, one of the reasons that "Object-Oriented Programming" is different than typical procedural languages and styles. This of course is opening up an old can of worms but...... Dave Forsey Computer Graphics Laboratory University of Waterloo. Waterloo Canada
zrm@mit-eddie.UUCP (Zigurd R. Mednieks) (08/13/83)
In Multics, optional, experimental, outdated, unsupported, and private shareable libraries (segments, really -- nothing distinguishes them from any other sort of program module) are implemented by using search paths for resolving references to outside objects. Cheers, Zig
zrm@mit-eddie.UUCP (Zigurd R. Mednieks) (08/13/83)
On machines with real two level memory management, like the 16000, you don't hve to worry about adding one more segment. In fact, the way to do it right is to add a segment per module of shared routines. So a program that uses both termcap and stdio would have at least three text segments. Simlarly, you could eliminate explicit data i/o by mapping data segments directly into a process's address spacve. But then, it's been done, read all about it in Organic's book "The Multics System." Cheers, Zig
ka@spanky.UUCP (08/15/83)
Adding shared memory to System V involved neither a complete redesign of major chunks of the kernel, nor did it involve a bizarre and ugly hack. UNIX has had the ability to share read only text segments between processes for a long time. Adding shared data segments required *adding* code to various pieces of the kernel to support the shared segments; in particular the code which sets up the page tables for a process now has to include entries to map the segements into the process's address space. But little if any code had to be *changed*. Kenneth Almquist
bob@ucla-locus@sri-unix.UUCP (08/16/83)
From: Bob English <bob@ucla-locus> How about providing a loader option that would "compile in" a specified shared library, thus insulating a program from all future shared library changes? If you kept the old versions of libraries around, you would be able to handle most (if not all) of the retrofit problems. Another possible solution is to keep old versions of library routines around forever, so that old programs find the routines they expect, and new programs get the new versions. This might apply only to genuine revisions, and not to bug fixes. --bob--
bob@ucla-locus@sri-unix.UUCP (08/16/83)
From: Bob English <bob@ucla-locus> I don't think that your argument is universally appropriate. On vax's, for instance, I think it would be moderately simple to add a second shared/read-only text segment. The only real implementation problems would be in modifying the loader to take advantage of it, and that certainly doesn't sound overwhelming. --bob--
silver@csu-cs.UUCP (08/19/83)
We use them on the HP9000 Series 500. The most common calls went into a special library that's loaded (once) by the OS during initialization. Almost ALL object files immediately "lost weight", a significant amount. Ergo, they load faster and eat less memory. We have had NONE of the problems discussed here (about breaking existing code, etc.), though the product is young yet. (We'll have to watch out for them, that's all.)