ccplumb@watnot.UUCP (04/08/87)
In comp.sys.amiga, there's a discussion going on about adding MMU's to the machine. One problem is the Amiga's use of (shared) libraries. How are they handled on machines with address translation? Do they just have to be written relocatable, or are assignments to virtual memory slots done at compile time (yuk), or is there some other technique for fitting them into the address spaces of various processes? If it helps, the Amiga uses an `openlibrary' call which returns a pointer to a table of pointers to library routines. Routines are located at known offsets. Thanks for any info. -- -Colin Plumb (watmath!watnot!ccplumb) Silly quote: It's a fiat accompli.
firth@sei.cmu.edu (Robert Firth) (04/13/87)
In article <12823@watnot.UUCP> ccplumb@watnot.UUCP (Colin Plumb) writes: >In comp.sys.amiga, there's a discussion going on about adding MMU's to >the machine. One problem is the Amiga's use of (shared) libraries. > >How are they handled on machines with address translation? Do they just >have to be written relocatable, or are assignments to virtual memory slots >done at compile time (yuk), or is there some other technique for fitting >them into the address spaces of various processes? > >If it helps, the Amiga uses an `openlibrary' call which returns a pointer >to a table of pointers to library routines. Routines are located at known >offsets. Far and away the simplest way is for the compiler to generate Position Independent Code. The libraries can then reside at any place in the process' virtual address space. They need not be at the same place in all processes. Your tame compiler writer should understand the principles of PIC and the ways to achieve it. If not, mail me.
davidsen@steinmetz.UUCP (04/21/87)
In article <926@aw.sei.cmu.edu.sei.cmu.edu> firth@bd.sei.cmu.edu.UUCP (Robert Firth) writes: >In article <12823@watnot.UUCP> ccplumb@watnot.UUCP (Colin Plumb) writes: >>... about Amiga shared libraries ... > >Far and away the simplest way is for the compiler to generate >Position Independent Code. The libraries can then reside at >any place in the process' virtual address space. They need not >be at the same place in all processes. Position independent code as I have seen it is really "IC relative" code, performing operations relative to the current instruction pointer rather than using an address. This presents some problems with a shared library, since the location depends on the size of the program and/or library. This can be overcome by reserving a portion of the address space for the system routines, but I'm looking for more information on doing things PIC and allowing linking. Several approaches have been taken. NatSemi has some products with the system services located in a fixed place at the bottom of memory (sorry, it's been awhile) and what is there is a jump table. This allows the program and library to be anywhere, but the jump table is fixed. Some systems do it with a hardware jump table accessed by INT instructions which store the IC and transfer to a known location. The old GE600 series had three ways to create a fault, being MME (master mode entry), DRL (derail), and fault tag operator. The last was an address modifier rather than an opcode, and "just happened" to be the modifier used in an ASCII data descriptor. This allowed the descriptor to be "executed" causing the falut tag before the instruction was decoded, and allowing the o/s to fetch the data pointer and output to the terminal. Yecch! Intel 80x86 chips use this "fault vector" approach, and it makes the code quite compact. Other systems put the library routines in known segments, again Intel hardware among others, and this provides about the same effect as using the pagining in a VAX to simulate putting the library at a know location. One prototype package using SysV shared memory uses 'shmget' to fetch the location of a set of shared libraries and services (a pointer vector). Any system using a pointer vector allows the library to be changed while the system is running, by changing the vector values. I'm not sure how useful this is, just mentioning it. -- bill davidsen sixhub \ ihnp4!seismo!rochester!steinmetz -> crdos1!davidsen chinet / ARPA: davidsen%crdos1.uucp@ge-crd.ARPA (or davidsen@ge-crd.ARPA)
henry@utzoo.UUCP (Henry Spencer) (04/22/87)
> Far and away the simplest way is for the compiler to generate > Position Independent Code. The libraries can then reside at > any place in the process' virtual address space. They need not > be at the same place in all processes. Don't forget the problems of (a) where does static data for the library functions live (they need to know this), and (b) how does the user code find out where the foobar() function is. I think the latter is more what the original question was about. -- "If you want PL/I, you know Henry Spencer @ U of Toronto Zoology where to find it." -- DMR {allegra,ihnp4,decvax,pyramid}!utzoo!henry
bjorn@alberta.UUCP (Bjorn R. Bjornsson) (04/23/87)
In article <7952@utzoo.UUCP>, henry@utzoo.UUCP (Henry Spencer) writes: > Don't forget the problems of (a) where does static data for the library > functions live (they need to know this), I favor having a stub that passes the address of these (eg. 'errno', '_iob' etc.) linked into each and every process image. That way you don't have to touch the linker (not that Unix linkers couldn't stand to be touched up {a,"more than a little"} bit B-). This sort of stuff can actually be done *without* source code if you're a crack hack (it helps to have the object modules and 'adb' though). > and (b) how does the user code > find out where the foobar() function is. I think the latter is more what > the original question was about. This one is easier yet (if you have shared memory or default mapping of system address space) and has many solutions unless the architecture is Unix bound and specifically forbids executing out of a process' data segment (assuming System V compatible shared memory here). I'm not much of a socialist B-), even if I advocate sharing common procedures. OR anyone? Bjorn R. Bjornsson {ubc-vision,ihnp4,mnetor}!alberta!bjorn
firth@sei.cmu.edu.UUCP (04/23/87)
More on PIC, mainly in reply to Henry Spencer
(a) the code of the shared libraries is position independent.
(b) all read-only data needed by eg arctan is kept inline with
the code (alternating code Psects and data RO Psects). Yes,
you probably need a PC-relative data access mode.
(c) any read-write data space is supplied by the calling process,
either as an extra parameter of every relevant call, or by
convention via a specific general register. The former is
cleaner, but for things like RANDOM the latter is usually
preferred. The way I've seen it, the Assembler macro that
generates the library EXTERNAL directives also creates
the necessary chunks of library Dsect. Only works for ONE
library at a time, of course.
(d) you can get at the routines in two ways. If you have static
linking, then the linker decides for each process where in its
virtual address the library will live, and resolves references
to it accordingly. It leaves a cue directive in the process
image, and the loader points the proper piece of virtual address
at the physical address where the library is resident. This
costs nothing at run time, but means you must relink your
process to move the virtual position of the library. Of course,
moving the physical code of the library damages nothing, as
long as you fix the address translation tables of the processes
pointing at it.
(e) If you want to be fully dynamic, then you need a jump table. In
principle, you have one table in each process, containing one entry
for each library routine, and the OS changes the table entries
whenever the library is moved. Using another level of indirection,
the process can have one entry per library, pointing to the start
of a relative jump table in that library, containing one entry per
routine. If you do not have memory management but need to do a
lot of multiprogramming, this is perhaps the least messy way. You
can move the library code around to compact free memory, as long
as you change the pointers in every process.johng@orstcs.UUCP (04/25/87)
Apollo implements shared libraries. Each new window you create you 'inlib'
the library you want. Things are linked at program invokation. I think
the linking of the AEGIS (operating system) calls are also postponed until
then also. This allows for some neat tricks to be played. You can overlay
your own routines over or around the operating system calls. One example,
I wanted to catch and log all mouse movements before the application program
could munch on them, so I made my own version of the gpr_event_wait() which
looked like the real thing to the application program. When it was called,
it called the real gpr_event_wait(), logged the transaction, and returned
the data. I could do this without modifying the application's source, or
even recompiling since it was all done at invocation time.
John Gregor
{tektronix, hp-pcd}!orstcs!johngbenson@alcatraz.ksr.com (Benson Margulies) (04/28/87)
In article <7952@utzoo.UUCP> henry@utzoo.UUCP (Henry Spencer) writes: >> Far and away the simplest way is for the compiler to generate >> Position Independent Code. The libraries can then reside at >> any place in the process' virtual address space. They need not >> be at the same place in all processes. > >Don't forget the problems of (a) where does static data for the library >functions live (they need to know this), and (b) how does the user code >find out where the foobar() function is. I think the latter is more what >the original question was about. >-- Cripes, folks, you have been beating around the bush of dynamic linking for weeks now. In the interest of calling a personal earth moving device a spade, I'll sketch the mechanism. Now that Unix runs on computers more like the power of its original parent, perhaps its time for it to recapture some non-chewing-gum-and-bailing-wire system technology from its own history. one scheme works like this: Each object file is structured into four pieces: text definitions static linkage symbol There has to be as storage allocation mechanism independent of "running" a program. Text references to external quantities are references through pointers in the linkage. The initial copy of the linkage contains pointers that take a fault when you reference through them. (your hardware does have such minimal amenities, right? If not, put a trap instruction in the linkage, and have the text jump there). When the fault handler takes one of these linkage faults, it makes a copy of the entire linkage section in per-process storage (on Unix you may want it to be per-group-leader or something), and fixes the faulting pointer to point to the linked-to library routine, using naming information in the definitions section of both the referencer and the referencee. Once you have that, you are on the air with no further faults. Static lives in the static section. All references to static are addressed as offsets from a pointer stored in a table. When a file is first linked to in a process, the static section is copied, and the pointer initialized. Thus, each process gets its own static. Manifestly, this is easier to do in a segmented environment than a flat environment, but that shouldn't stop you for long. Someone's comments about strcopy in shared memory are a red herring. In systems with "shared memory", the vast bulk of the shared data is pure code. As soon as you go to share something that is modifiable, you are in the synchronization business. You use locks, semaphores, or whatever you like. That's what shared data is all about. When I worked on Multics, that's most of what I did was write shared memory programs, and anyone out there working on Multiprocessor Unix is doing the same thing. Benson I. Margulies Kendall Square Research Corp. harvard!ksr!benson All comments the responsibility ksr!benson@harvard.harvard.edu of the author, if anyone.