[comp.unix.wizards] Unix with dynamic loading

kent@xanth.UUCP (01/01/70)

In article <376e654f.1201@apollo.uucp> labo@apollo.UUCP (Dale Labossiere) writes:
>In article <2067@sfsup.UUCP> shap@sfsup.UUCP (J.S.Shapiro) writes:
>>In article <2501@xanth.UUCP>, kyle@xanth.UUCP (Kyle Jones) writes:
>>> In <14888@topaz.rutgers.edu>, hedrick@topaz.rutgers.edu (Charles Hedrick)
>>> sez:
>>> Please explain more about shared libraries.
[good stuff from Kyle about shared libraries omitted]
>>This scheme is referred to as "dynamic loading."
>This "dynamic loading" scheme is in fact what Apollo systems use.  The
>compilers generate position independent code, and the libraries can be
>mapped anywhere in the process's address space.
>
>Rather than have a monolithic jump table for a shared library, shared
>libraries "register" their global addresses in a "Known Global Table"
>(KGT). Programs which invoke shared library functions do so via an
>indirect address linkage variable in their data space (the slight
>difference being that there are only linkage variables for those
>shared functions that are invoked, not a complete table for all of the
>libraries functions).
>
>Object files are tagged and when executed, the system loader (not UNIX
>ld(1)) resolves the undefined global's addresses using the KGT and
>patches up the program's linkage variables.

Well, ring my chimes!  If this is possible in Unix, it would sure help
the commercial success of Unix if it became common practice.

(Sample-of-one argument follows - add salt!)  I worked at a heavy
industry manufacturing company with 6.5 million source code lines of
production COBOL and FORTRAN running in a three shift, seven day per
week operation.  (To give you an idea, twice a week they unloaded a
_semitrailer_ full of laser printer paper into the computer center.)
When I became aware (and enamored) of Unix, I asked "Why aren't we
using this _great_ operating system here for our 200 programmers?  It
sure would boost productivity."  The answer was: "Unix doesn't do
dynamic loading; we could never maintain our software base if we had
to relink every routine each time a subroutine was recompiled for
changes to meet our changing manufacturing requirements."

What these folks did was test a modified routine on a representative
sample of the programs that called it, then add it into the production
dynamic load library, without ever recompiling or relinking the
calling routines.  You can see that, with hundreds of programmers and
thousands or tens of thousands of code modules, dynamic loading
becomes a big, big configuration management win.

Now, were these folks unique, or is there really an opportunity here
to expand the customer base of Unix by adding dynamic loading to more
Unix versions?  

Kent, the man from xanth.  "His expression lit up.  'Hey, you wouldn't
be a dope smuggler, would you?'  Rail looked confused.  'Why would
anyone wish to smuggle stupidity when there is so much of it readily
available?'" -- Alan Dean Foster, GLORY LANE

allbery@ncoast.UUCP (09/30/87)

[Ncoast has neither comp.arch nor comp.os.misc, so I have restricted this
 article to comp.unix.wizards.  Since I'm talking UNIX specifics, that is
 probably appropriate anyway.  ++bsa]

As quoted from <2550@xanth.UUCP> by kent@xanth.UUCP (Kent Paul Dolan):
+---------------
| Well, ring my chimes!  If this is possible in Unix, it would sure help
| the commercial success of Unix if it became common practice.
+---------------

System V can do something like this; it can't patch addresses, but it can use
the jump-table approach.  What works for shared libraries works just as well
for loading part of the address space from a file, and using a loader ifile
to specify symbol addresses within the dynamically-loaded area.  Of course,
this requires that the read/write data space of a program have execute
permissions in the MMU (does it normally?  I've no idea) and pretty much
rules out sharing the code; without kernel support, you can't page it in from
the dynamic load file, either.  Alternatively, the dynamic library could be
read by a routine which builds a jump table from the library's namelist, and
"stubs" could be written which search the jump table for the actual routine
and jump to it.  (They could then store the jump address in a static variable
for faster access on subsequent calls.)  The library itself would be able to
access its static variables indirectly through the jump table, with little
loss in performance if the dynamic load routine insures that externally
visible static variables always have the same offset in the jump table as
their symbols have in the namelist.  (That was unclear, but I hope at least
someone gets the idea I'm trying to get across.)  This kind of dynamic loading
might be a performance loss, but it would be a major win for making the
library truly dynamic.

DId I make any sense, or did I come across as a lunatic, or have I got
something horribly wrong (I'm a programmer; when it comes to MMU-related stuff,
though, I'm out of my depth), or ???
-- 
	    Brandon S. Allbery, moderator of comp.sources.misc
  {{harvard,mit-eddie}!necntc,well!hoptoad,sun!mandrill!hal}!ncoast!allbery
ARPA: necntc!ncoast!allbery@harvard.harvard.edu  Fido: 157/502  MCI: BALLBERY
   <<ncoast Public Access UNIX: +1 216 781 6201 24hrs. 300/1200/2400 baud>>
"`You left off the thunderclap and the lightning flash.', I told him.
`Should I try again?'  `Never mind.'"     --Steven Brust, JHEREG