[comp.unix.programmer] shared libraries, when to use them

shirley@washington.jsc.nasa.gov (Bill Shirley) (06/11/91)

I was wondering when it's appropriate to use shared libraries? (specifically in SunOS)

Is it ever appropriate for non OS work?

How exactly do you build it? (in SunOS)

When you compile something with a -Bstatic flag, where does it get its code from?
Does it extract the object part needed from the shared library (lib*.so.#[.#])?

What do you know?
What do you think?
What will you tell me?

	-thanks IA

-- 
     ____     ____       ____			Bill Shirley
    / ___|   / ___|     / ___|			bill@gothamcity.jsc.nasa.gov
   |_|      |_|ciences |_|			_______________________________
    _omputer     _      _			Opinions expressed are obtained|
   | |___    ___| |    | |___orporation		by a room full of immortal apes|
    \____|  |____/      \____|			with unbreakable typewriters.  |
  						~~~~~~~~~~~DISCLAIMER~~~~~~~~~~~

jik@cats.ucsc.edu (Jonathan I. Kamens) (06/17/91)

In article <1991Jun11.163544.20234@aio.jsc.nasa.gov>, shirley@washington.jsc.nasa.gov (Bill Shirley) writes:
|> I was wondering when it's appropriate to use shared libraries? (specifically in SunOS)

(Please try to put linefeeds in your postings at least once every 80
characters.  The postings in news.announce.newusers discuss why this is
necessary.)

It seems to me that you are asking two different questions here.  First of
all, when is it appropriate to link an application static instead of linking
it dynamic and letting it used the installed shared library (the C library,
among others)?  Second, when is it appropriate, when building libraries of
your own, to use shared libraries?

There are three reasons you might have to link an application static, thus
bypassing the benefits of shared libraries: (1) The application is going to be
installed on the workstation root, and will need to be able to run even when
the filesystems with the shared libraries in them are not mounted, or are
inaccessible (e.g. you're using a diskless workstation and the network
suddenly goes down); (2) Startup time is very, very important, and you don't
want the applicatiom to suffer the slight delay in startup time that occurs
when the dynamic libraries are being linked; (3) Security concerns -- you're
using a relative link path to link against a library in a non-default
directory, and the program is going to be installed setuid or something.

If none of these are a problem for you, then there really isn't any resaon to
link static rather than dynamic.

Now, as to the question of when to build shared libraries of your own and when
to make them static.... In order to answer that well, you need to understand
the benefits that shared libraries provide; if your library will be used in an
environment that will take advantage of those benefits, then make it shared;
if not, the trouble of making it shared probably isn't worth the effort.  The
benefits include the ability to update the library without linking the
application (although some people consider that a flaw; please, I don't wan to
get into more religious wars about shared libraries here!), the fact that
dynamic binaries take up much less space on disk, and the fact that dynamic
programs take up less space in memory when there are several programs using
the same shared library routines.

|> Is it ever appropriate for non OS work?

What do you mean by "appropriate," and what do you mean by "non OS work?"

|> How exactly do you build it? (in SunOS)

There is extensive documentation in the SunOS manual sets about building
shared libraries.  Basically, you compile your object files with the -pic flag
to the compiler, and then use use ld with the flag "-assert pure-text" to put
all the object files into one file which is your shared library.  It is
installed with a ".so" suffix, and possibly a version suffix after that, in
order of the standard link directories, are accessed with a -L flag to the
linker.  It is not necessary to run ranlib over a shared library.

There are also some complications that might require you to also build a ".sa"
library to go along with your ".so" file, although I don't understand all the
details of all that (I've never had to do it), so you should look it up in the
Sun manuals.  Not doing it won't cause your libraries not to work, but will
cause some performance degradation in some situations.

|> When you compile something with a -Bstatic flag, where does it get its code from?
|> Does it extract the object part needed from the shared library (lib*.so.#[.#])?

I believe it extracts the code from the static libraries that are installed
for most libraries in addition to the shared libraries (e.g. libc.a has a
libc.a.so, which is shared, and a libc.a, which is not).  I'm not certain if
the linker is smart enough to use the position-independent code from the
shared library if a static library isn't installed.

-- 
Jonathan Kamens					jik@CATS.UCSC.EDU

mike@unix.cis.pitt.edu (Mike Elliot) (06/17/91)

In article <17106@darkstar.ucsc.edu> jik@cats.ucsc.edu (Jonathan I. Kamens) writes:
>In article <1991Jun11.163544.20234@aio.jsc.nasa.gov>, shirley@washington.jsc.nasa.gov (Bill Shirley) writes:
>> I was wondering when it's appropriate to use shared libraries? (specifically in SunOS)

[response edited for brevity]

> There are three reasons you might have to link an application static...
> (1) The application is going to be installed on the workstation root...
> (2) Startup time is very, very important...(3) Security concerns...

 4.) If you need to support multiple versions of an OS and don't want
     to have to worry about what version of the shared libraries your
     users have.

-mje
------------

mike@unix.cis.pitt.edu
hplabs!hpnmdla!emperor!elliot

mouse@thunder.mcrcim.mcgill.edu (der Mouse) (06/18/91)

In article <1991Jun11.163544.20234@aio.jsc.nasa.gov>, shirley@washington.jsc.nasa.gov (Bill Shirley) writes:

> I was wondering when it's appropriate to use shared libraries?
> (specifically in SunOS)

You can't, because SunOS doesn't have shared libraries.  (What it does
have is shared object files.  What's the difference?  You can link in
part of a library without linking in the rest, among other things.)

This confusion is understandable, since Sun documentation says "shared
libraries" where they mean shared object files.

Assuming you really meant shared objects....

> Is it ever appropriate for non OS work?

I'm not sure what you mean by "OS work", so it's hard to say.

> How exactly do you build it? (in SunOS)

What *I* do is to type "make"; the Makefile worries about the rest.

Presumably you would like to know what's in the Makefile, then :-)

Here, for example, is the Makefile from one directory in which I have
source that gets compiled and put into a shared "library":

	.c.o:
		$(CC) $(CFLAGS) -pic -c $<
		mv $*.o $*.so
		$(CC) $(CFLAGS) -c $<
	
	install:
		cp *.o o-files/unshared
		cp *.so o-files/shared
		( cd o-files ; make $(MFLAGS) install )

And then o-files/Makefile:

	install:
		( cd unshared ; make $(MFLAGS) install )
		( cd shared ; make $(MFLAGS) install )

and o-files/shared/Makefile (o-files/unshared/Makefile is an
uninteresting exercise in the user of ar):

	install:
		-@ rm -f lib.so
		-@ ( ls -1 *.so | sed -e 's/\(.*\).so$$/mv \1.so \1.o/' ) 2> /dev/null | sh -v
		ld -assert pure-text -o lib.so *.o
		cp lib.so /the/installed/version/of/lib.so.1.5

> When you compile something with a -Bstatic flag, where does it get
> its code from?  Does it extract the object part needed from the
> shared library (lib*.so.#[.#])?

No.  A shared "library" cannot be broken up.  When you use -Bstatic, ld
doesn't look for .so files; it pays attention to .a files only, just
the way it used to before .so files came along.

Yes, this can lead to version skew if the .a and .so files were built
from incompatible versions of the .o files.  That's the library
maintainer's lookout.

					der Mouse

			old: mcgill-vision!mouse
			new: mouse@larry.mcrcim.mcgill.edu

mouse@thunder.mcrcim.mcgill.edu (der Mouse) (06/18/91)

In article <17106@darkstar.ucsc.edu>, jik@cats.ucsc.edu (Jonathan I. Kamens) writes:
>> [stuff about SunOS shared "libraries"]

> There are three reasons you might have to link an application static,
> thus bypassing the benefits of shared libraries:

> (1) The application is going to be installed on the workstation root,
> and will need to be able to run even when the filesystems with the
> shared libraries in them are not mounted, or are inaccessible (e.g.
> you're using a diskless workstation and the network suddenly goes
> down);

Uh, if the network goes down on a diskless you generally can't even get
your executable off disk to run it at all.

> (2) Startup time is very, very important, and you don't want the
> applicatiom to suffer the slight delay in startup time that occurs
> when the dynamic libraries are being linked;

> (3) Security concerns -- you're using a relative link path to link
> against a library in a non-default directory, and the program is
> going to be installed setuid or something.

To which I would like to add

(4) The executable needs to be able to survive being run with
    LD_LIBRARY_PATH set such that ld.so would pick up an incompatible
    version of some library.  (MIT X versus "Open"Windows X has
    produced such a situation in my experience.)

(5) If you wish to use a tool like undump, that depends on useful core
    files, because there's a bug in Sun's dump-core routine that causes
    dynamically linked object files' data areas to be omitted from core
    dumps.

(While I'm here: WHEN WHEN WHEN will Sun provide a tool to turn a
dynamically linked executable into a statically linked one, or (even
better) document the format of the magic ld-generated stuff so we can
write our own?!)

					der Mouse

			old: mcgill-vision!mouse
			new: mouse@larry.mcrcim.mcgill.edu

guy@auspex.auspex.com (Guy Harris) (06/21/91)

>It is installed with a ".so" suffix, and possibly a version suffix after
>that,

I think the linker may *require* major and minor version numbers; it may
not recognize the file as a library - at least not if you use "-l" -
otherwise.  Start at version 1.0; bump the minor version number every
time you make a change to the library that would cause programs linked
against the new library to possibly fail if run against the old library;
bump the major version number every time you make a change to the
library that would cause programs linked against the old version to fail
when run against the new one.

For example, if you add a new routine to the library, or add a new
capability to an existing routine, bump the minor version number.  If
you change the calling sequence to a routine, or the shape of a
structure you pass (either by value or reference) to a routine, bump the
major version number.

>It is not necessary to run ranlib over a shared library.

Especially given that "ranlib" will say "Hey!  That's an a.out!  I can't
ranlib that!" and bail out.

>I believe it extracts the code from the static libraries that are installed
>for most libraries in addition to the shared libraries (e.g. libc.a has a
>libc.a.so, which is shared, and a libc.a, which is not).

It does.  If dynamic linking (-Bdynamic) is in effect, which it is by
default, a "-lXXX" option will first try to link against "libXXX.so.X.Y"
and, if no such file is found, try to link against "libXXX.a".  If
static linking is in effect, it'll ignore any "libXXX.so.X.Y" files it
finds.

>I'm not certain if the linker is smart enough to use the
>position-independent code from the shared library if a static library
>isn't installed.

It's not.  If static linking is in effect, and there's no
"libXXX.so.X.Y", it fails.

guy@auspex.auspex.com (Guy Harris) (06/21/91)

> 4.) If you need to support multiple versions of an OS and don't want
>     to have to worry about what version of the shared libraries your
>     users have.

Depends on the multiple versions in question.  In some cases, linking on
the older system should suffice (modulo bugs, but kernel bugs can trip
up binaries built on other system, as well; in some sense, "/*ix*" can
be thought of as a shared library, and I think it *is* one in AIX 3.x).

E.g., if you have both 4.0.3 and 4.1[.x] systems, build on the 4.0.3
system.  (I'm told one vendor keeps a 4.0 - not 4.0.x, *4.0* - system
around for building its products.)

guy@auspex.auspex.com (Guy Harris) (06/21/91)

>> (1) The application is going to be installed on the workstation root,
>> and will need to be able to run even when the filesystems with the
>> shared libraries in them are not mounted, or are inaccessible (e.g.
>> you're using a diskless workstation and the network suddenly goes
>> down);
>
>Uh, if the network goes down on a diskless you generally can't even get
>your executable off disk to run it at all.

Bad example, but there *are* good examples:

	auspex% file /sbin/*
	/sbin/config:   sparc demand paged dynamically linked executable
	/sbin/hostname: sparc demand paged executable
	/sbin/ifconfig: sparc demand paged executable
	/sbin/init:     sparc demand paged executable
	/sbin/mount:    sparc demand paged executable
	/sbin/sh:       sparc demand paged executable

All but "config" (no, I don't know why "config" is there; yes, it's just
"/usr/etc/config", and no, it's not a symlink) have to be able to run
when the filesystems with shared libraries in them aren't mounted.

guy@auspex.auspex.com (Guy Harris) (06/21/91)

 >You can't, because SunOS doesn't have shared libraries.  (What it does
 >have is shared object files.  What's the difference?  You can link in
 >part of a library without linking in the rest, among other things.)

Just out of curiosity, who *has* implemented shared libraries?
("Multics" is, unless I misremember, not the correct answer.)

fkittred@bbn.com (Fletcher Kittredge) (06/21/91)

In article <8448@auspex.auspex.com> guy@auspex.auspex.com (Guy Harris) writes:
>
> >You can't, because SunOS doesn't have shared libraries.  (What it does
> >have is shared object files.  What's the difference?  You can link in
> >part of a library without linking in the rest, among other things.)
>
>Just out of curiosity, who *has* implemented shared libraries?
>("Multics" is, unless I misremember, not the correct answer.)

In the Proceedings of this year's Winter Usenix, the OSF people presented
a paper called "The OSF/1 Program Loader".  I have been playing with
the same, and it seems to satisfy the requirements for real shared libraries.

In the OSF/1 module, you build libraries to explictly export one or
more "packages". At run time, you load the "packages" you need from the
libraries.

check it out...

regards,
fletcher
Fletcher Kittredge
BBN Software Products
150 CambridgePark Dr,  Cambridge, MA. 02140
617-873-3465  /  fkittred@bbn.com  /  fkittred@das.harvard.edu

thorinn@diku.dk (Lars Henrik Mathiesen) (06/21/91)

guy@auspex.auspex.com (Guy Harris) writes:
> >You can't, because SunOS doesn't have shared libraries.  (What it does
> >have is shared object files.  What's the difference?  You can link in
> >part of a library without linking in the rest, among other things.)

>Just out of curiosity, who *has* implemented shared libraries?

I think Borroughs did on their Bx700 series. If I remember correctly
what I was told, each procedure (and array, ...) resides in its own
segment and is referenced through a descriptor; if it's not in core,
the descriptor ``contains'' a filename and other info.

Whne the Binder on those systems resolve a function name it just
insert the proper descriptor in a global list. When a new library is
installed programs must be rebound, but it is a very quick process;
and since the user code is linked the same way, it is very quick to
make a program use an updated version of a single procedure.

(And if it doesn't work that way, it could've.)

--
Lars Mathiesen, DIKU, U of Copenhagen, Denmark	     [uunet!]mcsun!diku!thorinn
Warning: This article may contain unmarked humour.		thorinn@diku.dk

kaleb@thyme.jpl.nasa.gov (Kaleb Keithley) (06/21/91)

In article <8448@auspex.auspex.com> guy@auspex.auspex.com (Guy Harris) writes:
>
> >You can't, because SunOS doesn't have shared libraries.  (What it does
> >have is shared object files.  What's the difference?  You can link in
> >part of a library without linking in the rest, among other things.)
>
>Just out of curiosity, who *has* implemented shared libraries?
>("Multics" is, unless I misremember, not the correct answer.)

This sounds like a semantics debate.  A library is a set of object files.
A *shared* library is a set of *shared* object files, maybe?

So, if I understand SunOS and its shared libraries (and it's possible
I don't.)  The shared libraries are created with the link editor, and
one monolithic binary "library" is created.  Later, at run-time, if a 
reference to one symbol in a SunOS shared library causes the whole image 
to be loaded into memory (unused sections may be paged out later) then 
perhaps Sun has a poor implementation?

Two questions come to mind, one easy, one hard:

How do SV.[34] shared libraries differ from SunOS shared libraries?

What is the "correct" way to implement X Widget libraries, specifically,
should the class record be in the libXaw.sa.4.2 part of the shared library?

-- 

Kaleb Keithley                               kaleb@thyme.jpl.nasa.gov

No flashy sig. No clever quips. No famous quotes. This space for rent.

terryl@sail.LABS.TEK.COM (06/22/91)

In article <8448@auspex.auspex.com> guy@auspex.auspex.com (Guy Harris) writes:
>
> >You can't, because SunOS doesn't have shared libraries.  (What it does
> >have is shared object files.  What's the difference?  You can link in
> >part of a library without linking in the rest, among other things.)
>
>Just out of curiosity, who *has* implemented shared libraries?
>("Multics" is, unless I misremember, not the correct answer.)


     Um, err, maybe VMS???? (Using the few neurons that are still functioning,
and MANY, MANY moons ago in another life....)


__________________________________________________________
Terry Laskodi		"There's a permanent crease
     of			 in your right and wrong."
Tektronix		Sly and the Family Stone, "Stand!"
__________________________________________________________

guy@auspex.auspex.com (Guy Harris) (06/22/91)

> >You can't, because SunOS doesn't have shared libraries.  (What it does
> >have is shared object files.  What's the difference?  You can link in
> >part of a library without linking in the rest, among other things.)
>
>Just out of curiosity, who *has* implemented shared libraries?
>("Multics" is, unless I misremember, not the correct answer.)

I apologize for being too subtle.  A number of people have sent me other
incorrect answers; yes, I know, there are a number of OSes that have
what der Mouse would call "shared object files", but I don't know of any
that have what he'd call "shared libraries", the fact that others -
including the vendors of those systems! - might call them shared
libraries nonwithstanding.  Apparently, the fact that I indicated that,
as far as I know, Multics wasn't one of those systems was a bit too
subtle a clue....

So who has implemented what der Mouse would call "shared libraries" -
i.e., that let you "link in part of a library without linking in the
rest", and that let multiple clients of the library share it?

I assume, given that he said "SunOS doesn't have shared libraries", that
the fact that some system might do demand paging and let only those
pages of the shared object file containing code or data that's actually
been used (and not had the page frame containing it be snatched for
smething else) doesn't mean that system necessarily has shared
libraries, as SunOS definitely demand-pages shared libraries.

I.e., you have to be able to *link* in part of the library, not just be
able to *page* in only part of the library even though the entire
library (or its entire text) has been mapped into the address space of a
process using the library.  (No, I don't know what that would mean;
that's one of the reasons why I'm asking the question....)

mouse@thunder.mcrcim.mcgill.edu (der Mouse) (06/22/91)

In article <8448@auspex.auspex.com>, guy@auspex.auspex.com (Guy Harris) writes:
[Guy apparently deleted, or failed to insert, an attribution header;
the missing attribution is to me. -dM]
>> You can't, because SunOS doesn't have shared libraries.  (What it
>> does have is shared object files.  What's the difference?  You can
>> link in part of a library without linking in the rest, among other
>> things.)
> Just out of curiosity, who *has* implemented shared libraries?

It's been several years since I used it, but I *think* VMS did.  I
don't remember precisely, though.

Actually, this is a good opportunity: I got mail from someone who
pointed out that one can consider a Sun .so file as a "library", which
is true but uninteresting, because one can actually consider any
collection of code as a library: a directory with .o files in it, a tar
archive of .c and .f files, a shar of .s files....  I was using
"library" as something like a technical term.

Which brings up the next question: "fine, but what's your point?".

The main point is that the semantics are different.  A "library", in
the past, is something from which "modules" are extracted and included
in the program as necessary.  No more than necessary is included from
the library.  But the thing so often termed a "shared library" has
different semantics.  While for the most part the difference is not
visible, it is in one case: when a routine in the user program has the
same name as a routine in the library.  This is generally bad
programming practice, when done deliberately - but when linking with a
"shared library", there's often no notification that there's anything
wrong; the program just does mysterious things, and worse, the symptoms
go away or change when linked -Bstatic.

(There are other differences; for example, the SunOS core-dump routine
has a bug in that the core files it generates don't include anything
from any dynamically linked shared objects.)

SunOS could be made to do shared libraries instead of shared objects,
without all that much work.  All that would be necessary would be to
have ld.so map the .a file instead and do the relevant relocations,
which it has to be prepared to do anyway.  Differences I see: (1) this
would almost surely be slower than the current scheme; (2) there's less
memory sharing at run-time, because more of the mmap()ed memory would
have to be modified by relocation; (3) only one library to maintain
instead of two, and the semantics are the same.

(In passing: anyone know why Sun doesn't document the format of the
ld-generated data structures?  There are things I'd like to do for
which I require that information, and <link.h> doesn't have all of it.
Please don't tell me I have to uncompile ld.so to figure this out....)

					der Mouse

			old: mcgill-vision!mouse
			new: mouse@larry.mcrcim.mcgill.edu

benson@odi.com (Benson I. Margulies) (06/23/91)

In article <8448@auspex.auspex.com> guy@auspex.auspex.com (Guy Harris) writes:
>
> >You can't, because SunOS doesn't have shared libraries.  (What it does
> >have is shared object files.  What's the difference?  You can link in
> >part of a library without linking in the rest, among other things.)
>
>Just out of curiosity, who *has* implemented shared libraries?
>("Multics" is, unless I misremember, not the correct answer.)

What could "linking in part of a library" mean?

If this question is meaningful, Multics is the answer. Even SunOS gets
partial credit, as does AIX, perhaps.

The SunOS dynamic linker, at program startup, runs around and potches
with all of the shared objects referenced in the __DYNAMIC structure
of the executable, and and recursively in the library. It resolves all
the data references, and leaves the function references for later.  So
while mmap's of the entire shared object happen, there is some
spurious paging. Since link resolution is left-to-right, a
further-left shared object can always replace symbols in one to the
right.  The search list (LD_LIBRARY_PATH) is only for libraries, not
for names, so that limits the degree to which SunOS qualifies.

When a Multics process is born, no links are snapped. Individual
function and data items are found and linked in as referenced.  There
is no such thing as a "library" as a fundamental concept.  Reference
foo$bar? Search for an object file named foo in the current search
list with an entrypoint named bar. The object files in question are
the equivalent of .o files on Unix -- no ld need. A "binder" is
available to speed things up by lumping them together.

AIX has the characteristic that a shared library is a .o that may be
contained in a .a! So libc.a on AIX contains one shared object and a
bunch of statically linked .o files. On the other hand, AIX forces
much more of the resolution process to be completed at ld time.
-- 
Benson I. Margulies

uad1077@dircon.co.uk (Ian Kemmish) (06/23/91)

fkittred@bbn.com (Fletcher Kittredge) writes:

>In article <8448@auspex.auspex.com> guy@auspex.auspex.com (Guy Harris) writes:
>>
>> >You can't, because SunOS doesn't have shared libraries.  (What it does
>> >have is shared object files.  What's the difference?  You can link in
>> >part of a library without linking in the rest, among other things.)
>>
>>Just out of curiosity, who *has* implemented shared libraries?
>>("Multics" is, unless I misremember, not the correct answer.)

>In the Proceedings of this year's Winter Usenix, the OSF people presented
>a paper called "The OSF/1 Program Loader".  I have been playing with
>the same, and it seems to satisfy the requirements for real shared libraries.

>In the OSF/1 module, you build libraries to explictly export one or
>more "packages". At run time, you load the "packages" you need from the
>libraries.

>check it out...

>regards,
>fletcher
>Fletcher Kittredge
>BBN Software Products
>150 CambridgePark Dr,  Cambridge, MA. 02140
>617-873-3465  /  fkittred@bbn.com  /  fkittred@das.harvard.edu
:-):-):-):-):-):-):-)

Sorry to all concerned but I just couldn't resist this...

Since a truly shared library presumably takes up more memory than
a shared object (well, for probgrams which use most or all of the
library anyway), this sounds just the ticket for programs which
need to call on Motif....  sounds like OSF is going to be an even
better way to sell disc and memory than APL was in the 70's

:-):-):-):-):-):-):-)

-- 
Ian D. Kemmish                    Tel. +44 767 601 361
18 Durham Close                   uad1077@dircon.UUCP
Biggleswade                       ukc!dircon!uad1077
Beds SG18 8HZ United Kingdom    uad1077@dircon.co.uk

schwartz@groucho.cs.psu.edu (Scott Schwartz) (06/24/91)

mouse@thunder.mcrcim.mcgill.edu (der Mouse) writes:
   (In passing: anyone know why Sun doesn't document the format of the
   ld-generated data structures? ...)

Sun?  Aren't they the ones with "industry standards" and "open
systems"?  Followups to comp.unix.marketing, where it will be noted
that psu has more cattle than Sun does. :-)

guy@auspex.auspex.com (Guy Harris) (06/25/91)

>So, if I understand SunOS and its shared libraries (and it's possible
>I don't.)  The shared libraries are created with the link editor, and
>one monolithic binary "library" is created.

True.

>Later, at run-time, if a reference to one symbol in a SunOS shared
>library causes the whole image to be loaded into memory (unused
>sections may be paged out later)

Unless somebody screwed up in making the library, that does *not*
happen.  The entire image is mapped into the virtual address space of
the process, but the pages of that image are pulled into physical memory
only on demand; unless the run-time loader has to relocate something on
every page (if it does, as indicated, somebody screwed up), not every
page gets pulled in.

>Two questions come to mind, one easy, one hard:
>
>How do SV.[34] shared libraries differ from SunOS shared libraries?

The first question to ask might be "How do SV.3 shared libraries differ
from SV.4 shared libraries?"

SV.3 shared libraries are allocated a fixed location in virtual address
space when they're built, and you have to go through some effort to
modify the source of the library routines to make them work.

SV.4 shared libraries are derived from SunOS 4.x shared libraries; they,
like SunOS 4.x shared libraries, are normally built in
position-independent form and don't always have to be at the same place
in the virtual address space; the changes made to the source of library
routines for SV.3 shared libraries aren't done for SV.4 shared libraries
(you may still want to make some changes for performance reasons).

guy@auspex.auspex.com (Guy Harris) (06/30/91)

>The main point is that the semantics are different.  A "library", in
>the past, is something from which "modules" are extracted and included
>in the program as necessary.  No more than necessary is included from
>the library.  But the thing so often termed a "shared library" has
>different semantics.  While for the most part the difference is not
>visible, it is in one case: when a routine in the user program has the
>same name as a routine in the library.  This is generally bad
>programming practice, when done deliberately - but when linking with a
>"shared library", there's often no notification that there's anything
>wrong; the program just does mysterious things, and worse, the symptoms
>go away or change when linked -Bstatic.

What are some cases where problems of that sort showed up?

The SunOS dynamic linking mechanism is intended to have that work the
same way with dynamically-linked libraries as with statically-linked
libraries (for better or for worse). 

I.e., if your program has its own memory-heap manager with "malloc()",
etc. entry points, when linked dynamically, routines from "libc.so.M.N"
that call "malloc()" should end up calling the ones in your program.

>(There are other differences; for example, the SunOS core-dump routine
>has a bug in that the core files it generates don't include anything
>from any dynamically linked shared objects.)

That's not a difference between "shared libraries" and "shared objects";
if any data or BSS in a "shared library" implementation wasn't in the
"data segment" part of the address space, it would have the same
problems as data and BSS in the current "shared object" implementation.

The problem is that the core-dump code only writes out the core dump
header, data segment, stack segment, and U area ("for those who care",
to quote the comment in the code; the idea is that most of the stuff you
want from the U area can be found in the core dump header instead, and
in a fashion less likely to change from release to release, as it's not
just a raw dump of the U area).

It manages to get some of the dynamic linker's data structures because
they're stuck in the stack segment, near the stack limit to keep them
out of the way of a growing stack.  The stack segment is sparsely
mapped, and the core dump code turns that part of the core dump into a
sparse file; that's why core dumps from dynamically-linked programs are
such large files - they have a large hole in them corresponding to the
hole in the stack segment.

(This assumes, of course, that the core dump wasn't from some
"ackerman(100, 100)" or something such as that. :-))

The proper fix is to beef up the core dump format a little more, to
handle sparse address spaces better, e.g. with a map between file
offsets and address-space offsets in the header.  This might also be
useful for getting rid of the sparse-file hack mentioned above.

The astute reader might note that some object-file formats have maps
such as those; SVR4's ELF is one such format, and ELF executables in
SVR4 should dump core files in ELF format, which should include more
than just data+stack.  (Dunno what exactly gets dumped; it's been a
while since I've seen that code.)  (And yes, at least in the code I saw,
COFF executables dumped core files in an SVR3ish core file format. 
Dunno why; maybe they wanted old debuggers to continue to work on old
binaries.)

>SunOS could be made to do shared libraries instead of shared objects,
>without all that much work.  All that would be necessary would be to
>have ld.so map the .a file instead and do the relevant relocations,
>which it has to be prepared to do anyway.

Yup, it's just a Simple Matter of Programming.  :-)

>Differences I see: (1) this would almost surely be slower than the
>current scheme;

Yup - more relocation to do, and because of 2), more copy-on-write
faults.

>(2) there's less memory sharing at run-time, because more of the
>mmap()ed memory would have to be modified by relocation;

You might end up with *no* sharing at run time - you get dynamic
linking, which is nice, but you don't end up sharing the code, which may
or may not make a difference.

>(3) only one library to maintain instead of two,

I'm told the SVR4 version of dynamic linking doesn't involve ".sa"
files, if that's what you're referring to; it's a SunOS 4.x-derived
"shared object" scheme, though.

>and the semantics are the same.

Which semantics?

>(In passing: anyone know why Sun doesn't document the format of the
>ld-generated data structures?

Documentation is always the stuff done last, and often falls through the
cracks....