[comp.sys.next] Library function redefinition

chet@odin.INS.CWRU.Edu (Chet Ramey) (03/01/91)

I just got a NeXT box for a couple of days, and the first thing I built was
bash (of course ;-).  When trying to use private versions of functions in
libsys_s.a, this error results:

>variables.o definition of _getenv in section (__TEXT,__text)
>/lib/libsys_s.a(getenv.o) definition of absolute _getenv (value 0x50025d0)

Is there a switch to cc or ld that will allow you to use a private version
of a library function?

Chet

P.S.  By the way, what's jsh?  Is it the BRL sh, or Arnold Robbins's mods
      to the original 4.2 BSD /bin/sh?  Or something completely different?
-- 
Chet Ramey				``Now, somehow we've brought our sins
Network Services Group			  back physically -- and they're
Case Western Reserve University		  pissed.''
chet@ins.CWRU.Edu		My opinions are just those, and mine alone.

chet@odin.INS.CWRU.Edu (Chet Ramey) (03/01/91)

I wrote:

>I just got a NeXT box for a couple of days, and the first thing I built was
>bash (of course ;-).  When trying to use private versions of functions in
>libsys_s.a, this error results:
>
>>variables.o definition of _getenv in section (__TEXT,__text)
>>/lib/libsys_s.a(getenv.o) definition of absolute _getenv (value 0x50025d0)
>
>Is there a switch to cc or ld that will allow you to use a private version
>of a library function?

Before I get any more mail telling me to comment out the private getenv
definition, let me explain what I am after.

Bash (the Gnu Bourne-Again SHell) provides a private definition of getenv
that searches the list of exported variables (those variables passed to
children of the shell as the environment).  This is so the libraries that
Bash uses, most notably readline, will get the `intuitive' definition of
environment variables when they call getenv().  TERM, for instance. 

The `traditional' Unix linker satisfies only unresolved external references
from libraries.  This allows you to provide an application with a custom
malloc, for instance, and have all of the libraries that use malloc cooperate.
It seems that the NeXT/Mach ld simply maps the library into my process's
address space, and damn any symbols that I already defined.

What I would like is a switch to cc or ld that restores the traditional
behavior, even in the presence of shared libraries.  If it can't be done,
that's OK, too.

Chet
-- 
Chet Ramey				``Now, somehow we've brought our sins
Network Services Group			  back physically -- and they're
Case Western Reserve University		  pissed.''
chet@ins.CWRU.Edu		My opinions are just those, and mine alone.

scott@erick.gac.edu (Scott Hess) (03/02/91)

In article <1991Mar1.143756.12400@usenet.ins.cwru.edu> chet@odin.INS.CWRU.Edu (Chet Ramey) writes:
   The `traditional' Unix linker satisfies only unresolved external references
   from libraries.  This allows you to provide an application with a custom
   malloc, for instance, and have all of the libraries that use malloc cooperate.
   It seems that the NeXT/Mach ld simply maps the library into my process's
   address space, and damn any symbols that I already defined.

   What I would like is a switch to cc or ld that restores the traditional
   behavior, even in the presence of shared libraries.  If it can't be done,
   that's OK, too.

Hmm.  Something has just occurred to me - how in the world does the
libMallocDebug.a library work?  Presumably, the malloc and malloczone
stuff is in the shared libraries, right?  But, where is it in the
MallocDebug library?  Since it uses different calls, at the very least
the routines are not the regular ones - whether they are shared or
not makes no difference.  So, apparently there's something that can
be done to "fix" the multiply-defined thing, because otherwise we'd
have multiple declaration problems with MallocDebug.

Anybody know what that "fix" was?

Later,
--
scott hess                      scott@gac.edu
Independent NeXT Developer	GAC Undergrad
<I still speak for nobody>
"Tried anarchy, once.  Found it had too many constraints . . ."
"Buy `Sweat 'n wit '2 Live Crew'`, a new weight loss program by
Richard Simmons . . ."

eps@toaster.SFSU.EDU (Eric P. Scott) (03/02/91)

In article <1991Mar1.143756.12400@usenet.ins.cwru.edu> chet@po.CWRU.Edu writes:
>It seems that the NeXT/Mach ld simply maps the library into my process's
>address space, and damn any symbols that I already defined.

Aww, poor baby.  Betcha don't use System V or SunOS 4.1+ either.

>What I would like is a switch to cc or ld that restores the traditional
>behavior, even in the presence of shared libraries.  If it can't be done,
>that's OK, too.

-Dgetenv=my_incompatible_replacement_for_getenv

					-=EPS=-

pfkeb@ebnextk.SLAC.Stanford.EDU (Paul Kunz) (03/03/91)

In article <SCOTT.91Mar1132236@erick.gac.edu> scott@erick.gac.edu (Scott Hess) writes:

   Hmm.  Something has just occurred to me - how in the world does the
   libMallocDebug.a library work?  Presumably, the malloc and malloczone
   stuff is in the shared libraries, right?  But, where is it in the
   MallocDebug library?  Since it uses different calls, at the very least
   the routines are not the regular ones - whether they are shared or
   not makes no difference.  So, apparently there's something that can
   be done to "fix" the multiply-defined thing, because otherwise we'd
   have multiple declaration problems with MallocDebug.

   Anybody know what that "fix" was?

The answer was given at the last BaNG SIG meeting by Kevin who wrote
the linker.  If you want to redefine malloc() you can, BUT you must
also redefine a suite of other closely related functions.  To see that
that list is, it is all the functions in libMallocDebug.a.

I hope I got that right, if not, I'm sure someone will correct me.

P.S.

   Earlier this week I posted that I ordered by upgrade boards in July
and still not received them.   This is my first post on my newly arrived
upgrade board.

dbrenner@sparta.weeg.uiowa.edu (Doug Brenner) (03/03/91)

In article <1991Mar1.143756.12400@usenet.ins.cwru.edu> chet@po.CWRU.Edu writes:
  > It seems that the NeXT/Mach ld simply maps the library into my process's
  > address space, and damn any symbols that I already defined.

and then, from an expected source, a typical piece of wisdom:

In article <1383@toaster.SFSU.EDU> eps@toaster.SFSU.EDU (Mr. Toaster) writes:
  > Aww, poor baby.  Betcha don't use System V or SunOS 4.1+ either.

And I bet you haven't used shared libraries on Prime systems running Primos.

Yes, using that old, proprietery OS, you can actually have the benefits of
shared code WITHOUT having to "fix" 10% of your code just because you had the
bad luck to use a common function name.

Give it a rest.  NeXT may have chosen to create the same shared library
problems as "System V and SunOS 4.1+," but that doesn't make things okay.

chet@odin.INS.CWRU.Edu (Chet Ramey) (03/04/91)

I wrote:
>>It seems that the NeXT/Mach ld simply maps the library into my process's
>>address space, and damn any symbols that I already defined.

And Eric Scott replied:
>Aww, poor baby.  Betcha don't use System V or SunOS 4.1+ either.

Think before you answer next time.  Sun (and AT&T) chose not to break
traditional Unix semantics -- Bash builds just fine on my Sparcs running
both SunOS 4.1 and 4.1.1 without using -Bstatic.  I haven't heard any
complaints from people testing it out on System V release 4, and the two
systems have very similar shared library implementations (built atop
`mmap').

This version of Bash runs on a number of System V.3 systems, with the
original System V shared library implementation, and has no trouble
compiling there either.

>>What I would like is a switch to cc or ld that restores the traditional
>>behavior, even in the presence of shared libraries.  If it can't be done,
>>that's OK, too.
>
>-Dgetenv=my_incompatible_replacement_for_getenv

OK, it can't be done.  Thank you.

Chet
-- 
Chet Ramey				``Now, somehow we've brought our sins
Network Services Group			  back physically -- and they're
Case Western Reserve University		  pissed.''
chet@ins.CWRU.Edu		My opinions are just those, and mine alone.

eps@toaster.SFSU.EDU (Eric P. Scott) (03/04/91)

In article <DBRENNER.91Mar3012022@sparta.weeg.uiowa.edu>
	dbrenner@sparta.weeg.uiowa.edu (Doug Brenner) writes:
>And I bet you haven't used shared libraries on Prime systems running Primos.

We still have a Prime 9955-II running PRIMOS that I still use on
occasion.  Pay up.  (And you wondered how I can afford to give
software away for free.  :-) )

>Give it a rest.  NeXT may have chosen to create the same shared library
>problems as "System V and SunOS 4.1+," but that doesn't make things okay.

It's not a bug.  It's a feature.

Try reading someone else's code that's obfuscated things by
redefining common library procedures.  I welcome the change!

					-=EPS=-

dbrenner@sparta.weeg.uiowa.edu (Doug Brenner) (03/04/91)

In article <1389@toaster.SFSU.EDU> eps@toaster.SFSU.EDU (Eric P. Scott) writes:
> We still have a Prime 9955-II running PRIMOS that I still use on
> occasion.  Pay up.

Given your experience with Prime shared libraries, I find it difficult to
understand (what appears to be) your support for how NeXT has chosen to
implement shared libraries.

Since the release of 2.0, you have lambasted people at least three times when
they have mentioned this shared library problem.  Each time trying to defend
this "feature" (as you now call it) by pointing out that SunOS and System V
have the same problem.  (This is, of course, how I understood your notes. :-)

In my note, I simply wanted to offer a counter example -- an implementation
that doesn't have this shared library problem.

It is my opinion that the current NeXT implementation of shared libraries
leaves much to be desired.  The restrictions are unnecessary; there are
implementations (Primos being one) that show this.

I hope NeXT will choose to consider this a problem; one that they will
address in the future.  (And I don't mean by simply giving us a static
library to bind to instead.)

eps@toaster.SFSU.EDU (Eric P. Scott) (03/04/91)

What do you do about the case where procedures in the shared
library call other procedures in the shared library?  Should
they continue to reference their shared library versions?
Or yours?  If you opt for the former, you can no longer
statically link.  If you opt for the latter, you can break
library routines that you _didn't_ redefine (what works in
one software release might not in the next if the library
changes, etc.).

Refusing to link multiply-defined symbols keeps you out of
trouble.

					-=EPS=-

dbrenner@sparta.weeg.uiowa.edu (Doug Brenner) (03/05/91)

Well, I hope everyone has put this subject into their kill files by now. :-)

In article <1396@toaster.SFSU.EDU> eps@toaster.SFSU.EDU (Eric P. Scott) writes:
> What do you do about the case where procedures in the shared
> library call other procedures in the shared library?  Should
> they continue to reference their shared library versions?
> Or yours?  If you opt for the former, you can no longer
> statically link.  If you opt for the latter, you can break
> library routines that you _didn't_ redefine (what works in
> one software release might not in the next if the library
> changes, etc.).

Strange, but I must again point to Prime shared libraries; they work just
fine WITHOUT the endless problems you envision.  (Certainly you already know
this given your stated use of Prime systems.)

I will not go through your notes, giving you solutions to every "horrific"
problem you see looming should symbol redefinition be allowed.  My point
continues to be that there are valid implementations that address and resolve
your questions.  Prime is one; there is no reason NeXT should not be another.

rpruess@umaxc.weeg.uiowa.edu (Rex Pruess) (03/05/91)

In many articles, eps@toaster.SFSU.EDU (Eric P. Scott) writes:
> It's not a bug.  It's a feature.

Feature?  No way.  I, too, disagree with your opinion on shared
libraries.  Too many people are struggling with NeXT's implementation
of shared libraries to consider it a feature.

> Try reading someone else's code that's obfuscated things by redefining
> common library procedures.  I welcome the change!

And I suppose being forced to rewrite someone else's code because they
happened to have a conflict with a shared library name should be
considered a feature.

> What do you do about the case where procedures in the shared library
> call other procedures in the shared library?  Should they continue to
> reference their shared library versions?  Or yours?

They should continue to reference their shared library versions.

> If you opt for the former, you can no longer statically link.

Correct.  Static linking is not the answer.  Variables in a program
have local scope, followed by global scope.  Apply the same logic to
subroutine names.  Resolve the local subroutine names before bringing
in the shared library routines.

> Refusing to link multiply-defined symbols keeps you out of trouble.
>						-=EPS=-

Keeps me out of trouble?  Tis not that simple.  When NeXT Software
Release 3.0, I'll be forced to sift through code to change routine
names that clash with the new ones NeXT has decided to use.  My crystal
ball is not powerful enough to keep me out of "that" trouble.

Now, can I take my phone out of the water?  :-)
--
Rex Pruess, Weeg Computing Center, Univ of Iowa, Iowa City, IA 52242
rpruess@umaxc.weeg.uiowa.edu (NeXTmail)               (319) 335-5452

bennett@mp.cs.niu.edu (Scott Bennett) (03/05/91)

In article <DBRENNER.91Mar4124044@sparta.weeg.uiowa.edu> dbrenner@sparta.weeg.uiowa.edu (Doug Brenner) writes:
>Well, I hope everyone has put this subject into their kill files by now. :-)

     Not yet, but probably shortly.  ;-)
>
>In article <1396@toaster.SFSU.EDU> eps@toaster.SFSU.EDU (Eric P. Scott) writes:
>> What do you do about the case where procedures in the shared
>> library call other procedures in the shared library?  Should
>> they continue to reference their shared library versions?
>> Or yours?  If you opt for the former, you can no longer
>> statically link.  If you opt for the latter, you can break
>> library routines that you _didn't_ redefine (what works in
>> one software release might not in the next if the library
>> changes, etc.).
>
>Strange, but I must again point to Prime shared libraries; they work just
>fine WITHOUT the endless problems you envision.  (Certainly you already know
>this given your stated use of Prime systems.)
>
>I will not go through your notes, giving you solutions to every "horrific"
>problem you see looming should symbol redefinition be allowed.  My point
>continues to be that there are valid implementations that address and resolve
>your questions.  Prime is one; there is no reason NeXT should not be another.

     (IBM bashers please hold your fire a moment.  Thank you.)  
     What all the discussion in this thread is pointing to is the woeful
inadequacy of the UNIX ld(1) program.  More than 25 years ago IBM had a
much more flexible and powerful linkage editor (in several sizes) for
OS/360 that completely bypassed not only Doug's problems with ld(1), but
quite a few others as well.  The descendent of the OS/360 linkage editor
survives today in MVS with relatively few changes in features because
IBM had given it enough flexibility at the outset that not much else
has been needed.  It would be nice if someone (e.g. FSF) would develop
a replacement for (or at least an alternative to) ld(1) that would get
rid of the need for so many of the contortions I see developers going
through to get around ld(1)'s inadequacies.
     The OS and MVS linkage editors allow such things as replacing
individual control sections in an object module (i.e. replace individual
functions in a C program that were compiled together into one object
module) either implicitly or explicitly, changing the names of external
symbols, explicit ordering of control sections within the load module
being produced (this can be essential for getting good virtual memory
performance in some situations), relinkediting load modules (ever wanted
to run a binary that you didn't have source for through ld(1) to sub-
stitute one routine?), aligning control sections on page boundaries for
improved virtual memory performance, and several other useful things.
     IBM may do a lot of despicable things, but linkage editors are one
thing that they designed a lot better than what UNIX users are stuck 
with.  (Okay, bashers, fire at will.  :-)


                                  Scott Bennett, Comm. ASMELG, CFIAG
                                  Systems Programming
                                  Northern Illinois University
                                  DeKalb, Illinois 60115
**********************************************************************
* Internet:       bennett@cs.niu.edu                                 *
* BITNET:         A01SJB1@NIU                                        *
*--------------------------------------------------------------------*
*  "I, however, place economy amoung the first and most important    *
*  of republican virtues, and public debt as the greatest of the     *
*  dangers to be feared."  --Thomas Jefferson                        *
**********************************************************************

eps@toaster.SFSU.EDU (Eric P. Scott) (03/05/91)

In article <RPRUESS.91Mar4141320@umaxc.weeg.uiowa.edu>
	rpruess@umaxc.weeg.uiowa.edu (Rex Pruess) writes:
>names that clash with the new ones NeXT has decided to use.  My crystal
>ball is not powerful enough to keep me out of "that" trouble.

Mine says "don't use names starting with NX"

					-=EPS=-

gumby@Cygnus.COM (David Vinayak Wallace) (03/05/91)

   Date: 4 Mar 91 23:43:45 GMT
   From: bennett@mp.cs.niu.edu (Scott Bennett)

		     It would be nice if someone (e.g. FSF) would develop
   a replacement for (or at least an alternative to) ld(1) that would get
   rid of the need for so many of the contortions I see developers going
   through to get around ld(1)'s inadequacies.

We have; it's in beta and looks like it will be part of the gcc/g++
2.0 release.  It lets you do most of what you asked already; I sent
your message to the poor guy who wrote it so he can fill in the
gaps...

But don't forget, NeXT doesn't use the gnu linker.

bennett@mp.cs.niu.edu (Scott Bennett) (03/05/91)

In article <GUMBY.91Mar4173900@Cygnus.COM> gumby@Cygnus.COM (David Vinayak Wallace) writes:
>
>   Date: 4 Mar 91 23:43:45 GMT
>   From: bennett@mp.cs.niu.edu (Scott Bennett)
>
>		     It would be nice if someone (e.g. FSF) would develop
>   a replacement for (or at least an alternative to) ld(1) that would get
>   rid of the need for so many of the contortions I see developers going
>   through to get around ld(1)'s inadequacies.
>
>We have; it's in beta and looks like it will be part of the gcc/g++
>2.0 release.  It lets you do most of what you asked already; I sent

     Wow!  Ask and ye shall receive, eh?  That's *great* news!  Thanks
a bunch, guys!

>your message to the poor guy who wrote it so he can fill in the
>gaps...
>
>But don't forget, NeXT doesn't use the gnu linker.

     I know, but maybe we could find a way...  Meanwhile, it would be
great on all the machines I use at work (except the Amdahl and (grrr...)
the Data General machine.)


                                  Scott Bennett, Comm. ASMELG, CFIAG
                                  Systems Programming
                                  Northern Illinois University
                                  DeKalb, Illinois 60115
**********************************************************************
* Internet:       bennett@cs.niu.edu                                 *
* BITNET:         A01SJB1@NIU                                        *
*--------------------------------------------------------------------*
*  "I, however, place economy among the first and most important     *
*  of republican virtues, and public debt as the greatest of the     *
*  dangers to be feared."  --Thomas Jefferson                        *
**********************************************************************