das@eplunix.UUCP (David Steffens) (08/18/90)
Apologies if you've seen this before. Posted the original on 6 Aug. Haven't seen any disscussion since, although I was expecting some. Posting a revised version again with a shortened subject line on the assumption that the original article didn't make it offsite. ---------- In article <1990Jul30.104726.22660@mtcchi.uucp> levy@mtcchi.UUCP (Daniel R. Levy) writes: 1> Now my question is, why does the linker silently resolve 1> [ a ] function reference to [ a ] global variable 1> without even a whisper of a warning? ... 1> Yes I know [ using the same name for a function and global variable ] 1> is a stupid thing to do ... But there are cases where this can occur unintentionally. Not all of these cases are immediately obvious to the programmer, and some of them are not "stupid", at least by my definition of the word. 1> ... "lint" does identify the problem ... Only if the "problem" appears in _source code_ which can be linted. In article: <37909@ucbvax.BERKELEY.EDU> edward@ucbarpa.Berkeley.EDU.UUCP (Edward Wang) writes: 2> The only impact this has on the language is that global 2> variables and functions must share the same name space. Said shared name space extends beyond the confines of _my_ code, however. And that makes it a linker problem, _not_ a compiler problem. 2> Your program has a C error, no different from declaring 2> a variable as an int in one place and as a float somewhere else. 2> True, the compiler should catch it, but the current organization 2> makes it difficult. (Use lint.) 2> ... 2> Anyway, it's not the linker's fault. It's the compiler's fault. Yes, the original program has a C error, and yes, lint will catch that one. Nevertheless, the linker _is_ blameworthy because it will _also_ happily use the address of one of my global variables to resolve a function call embedded in a library routine for which I have no lintable source, e.g. int index; main() { /* lots of code, none of which uses index() */ vendor_library_routine(); /* which, unknown to me, uses index() */ } Even if the vendor provides a lint library, "lint" is no help at all! This actually happened to one of our programmers (a good one, IMHO). Should I call her "stupid" just because she doesn't know that "index" is a _dirty name_ and might be used by some vendor in writing his library? The chances of a name collision of this sort rises exponentially with every new UNIX release. Is every programmer supposed to memorize the name of every libc routine just to avoid picking one by accident? No sir. In my view the linker is seriously deficient here. -- David Allan Steffens | I believe in learning from past mistakes... Eaton-Peabody Laboratory | ...but does a good education require so many? Mass. Eye & Ear Infirmary, 243 Charles Street, Boston, MA 02114 {harvard,mit-eddie,think}!eplunix!das (617) 573-3748 (1400-1900h EST)
linden@adapt.Sun.COM (Peter van der Linden) (08/23/90)
David Steffens points out a serious deficiency in "ld", and in fact it's even more heinous than the example he gives. If you happen to choose a function name that duplicates a library name, then your function is used in preference to the library function both in your code AND in any library routines you call! Most of us know to avoid names like malloc, but everyone calls something index sooner or later. This has tremendous potential for mysterious bugs, and I see it 4 or 5 times a year. The first time a programmer encounters it is an enlightening and "character-building" experience. ld should definitely be enhanced to complain about duplicate references in libraries (possibly with an option "shut up -- I meant to do this"). ---------------- Peter van der Linden linden@eng.sun.com (415) 336-6206 The antlers of the largest Irish Elk in the world are at Yale, and those of the smallest are at Harvard.
edward@ucbarpa.Berkeley.EDU (Edward Wang) (08/23/90)
Our difference is mostly one of semantics. I say compiler, you say linker; I say tom-ay-to, you say tom-ah-to. However, there is a point to make here. You said: >. . . >Nevertheless, the linker _is_ blameworthy because it will _also_ happily >use the address of one of my global variables to resolve a function call >embedded in a library routine for which I have no lintable source, e.g. > >int index; >main() >{ > /* lots of code, none of which uses index() */ > vendor_library_routine(); /* which, unknown to me, uses index() */ >} >. . . >This actually happened to one of our programmers (a good one, IMHO). >Should I call her "stupid" just because she doesn't know that "index" >is a _dirty name_ and might be used by some vendor in writing his library? >. . . Why stop there, why not redefine index() while you're at it? Defining index to be a variable is easy to debug. It'll most likely core dump on the call. Unintentionally redefining index to be a function of exactly the same type, arguments and all, is uncatchable and much harder to debug. The compiler-linker combination can at best give a warning, and I'm against that because it gets in the way of correct programs. It's all part of the same ball of wax of uncaught-by-the-compiler C errors. Why are we complaining about the easy ones? [I also wrote a much longer, and somewhat more serious, reply to the the poster (lost his name). Those who get off on this stuff can ask for that.]
gwyn@smoke.BRL.MIL (Doug Gwyn) (08/23/90)
In article <141151@sun.Eng.Sun.COM> linden@eng.sun.com (Peter van der Linden) writes: >it's even more heinous than the example he gives. If you happen to >choose a function name that duplicates a library name, then your function >is used in preference to the library function both in your code AND in >any library routines you call! This is the primary reason why the C standard severely constrains the external-linkage identifiers that an implementation of the standard library is permitted to use. >Most of us know to avoid names like malloc, but everyone calls something >index sooner or later. This particular name is not a problem in a standard-conforming environment. In general, when you link with nonstandard libraries, name-space collision is a real possibility. Ways for library designers to reduce this risk have been discussed in the C newsgroup before.
karl@haddock.ima.isc.com (Karl Heuer) (08/23/90)
In article <930@eplunix.UUCP> das@eplunix.UUCP (David Steffens) writes: >1> ... "lint" does identify the problem ... >Only if the "problem" appears in _source code_ which can be linted. Anything that isn't visible in lintable source code (e.g. a vendor's library) ought to have a corresponding lint library already built. (Not all do, but then this is the vendor's bug.) >int index; >main() { > vendor_library_routine(); /* which, unknown to me, uses index() */ >} >Even if the vendor provides a lint library, "lint" is no help at all! Oh? Let's try it. (Quick switch to a BSD system...) $ cat bug.c int index; int main() { return 0; } $ lint bug.c bug.c: index value declared inconsistently llib-lc(272) :: bug.c(1) $ Perhaps you mean systems on which `index' is not a documented interface, but where a library happens to have a helper function with that name? In that case, the vendor is at fault for failing to declare it static. >Is every programmer supposed to memorize the name of every libc routine just >to avoid picking one by accident? If you replace `every libc routine' with `every symbol reserved by the standard to which the program attempts to conform', then I'd say Yes, it's a good idea. For those whose memory is less than perfect (or who don't have a copy of the relevant standard), we have tools like lint. >No sir. In my view the linker is seriously deficient here. I'll agree with the point that a type-checking linker would be a good thing. It would probably slow down the linking pass, but I'd still use it if I had it. Karl W. Z. Heuer (karl@kelp.ima.isc.com or ima!kelp!karl), The Walking Lint Followups to comp.lang.c only.
ok@goanna.cs.rmit.oz.au (Richard A. O'Keefe) (08/23/90)
In article <930@eplunix.UUCP>, das@eplunix.UUCP (David Steffens) writes: 1> Now my question is, why does the linker silently resolve 1> [ a ] function reference to [ a ] global variable 1> without even a whisper of a warning? ... To start with, there are operating systems where this kind of thing cannot happen (love that B6700 MCP...). I was *appalled* the first time I had a subroutine call link with a common block. So this is a deliberate choice. The UNIX linkers allow a function reference to be resolved by a global variable because they have no way at all of telling one from another. The title of this thread refers to "text" and "data", but a read-only array may well be in "text" space. Consider the "-R" flag on BSD UNIX compilers and the "const" keyword in ANSI C. (There _is_ symbol table information available in COFF format, but the linker can't use it because it isn't always there. Galling, no?) If you use Simula 67, Ada, Modula-2, or recent versions of C++, their language support systems keep around enough information to ensure that this kind of mistake _is_ detected. cfront 2.whatever-it-is kludges it by frobbing the names. Ugh. But it works. So you might consider switching to C++. > Nevertheless, the linker _is_ blameworthy because it will _also_ happily > use the address of one of my global variables to resolve a function call > embedded in a library routine for which I have no lintable source, e.g. > int index; > main() > { > /* lots of code, none of which uses index() */ > vendor_library_routine(); /* which, unknown to me, uses index() */ > } The ANSI C committee thought about this; that's precisely the "namespace pollution" issue they were concerned about. Unfortunately, all that gives you is assurance that the C runtime library doesn't pollute your namespace; no guaranteees about anything else. Myself, I don't see data/function collisions as being any worse than function/function collisions. There is a certain UNIX variant that I sometimes use which provides a dynamic loading library routine, which swipes an _amazing_ number of useful and obvious names; if I get one of those by accident, or if my routine interferes with it, it doesn't improve matters that at least it was a function I collided with. If anything, it makes the mistake _harder_ to find. > The chances of a name collision of this sort rises exponentially > with every new UNIX release. Get an ANSI-compliant compiler and the chance of accidental collision *with the C run-time library* drops to 0. But use a vendor-supplied function which is in neither ANSI C nor POSIX, and I'm afraid you're right. The ultimate problem is that C assumes a _single_ global "extern" namespace. Just like Fortran and Pascal, in fact. Using COFF format, it wouldn't be too hard to produce a "packaging tool" which took description files import <id>, ... export <id>, ... source <file>, ... and did a "ld -r" to hook the files together into one library with names mangled so that only the imports and exports were left untouched (it would have to know about the local Pascal, C, and Fortran run-time libraries, so that their names were preserved, but providing for that wouldn't be hard). I've thought about doing this, but the trouble is that I keep running into machines which are COFF "with extensions" or modifications. It would also be comparatively straightforward to look at the symbol table information left by the "-g" option, execpt that (a) too many compilers won't give you symbol table *and* optimisation at the same time and (b) the symbol table information is not as portable as it might be either. -- The taxonomy of Pleistocene equids is in a state of confusion.
gwc@root.co.uk (Geoff Clare) (08/23/90)
In <38343@ucbvax.BERKELEY.EDU> edward@ucbarpa.Berkeley.EDU (Edward Wang) writes: >Why stop there, why not redefine index() while you're at it? >Defining index to be a variable is easy to debug. It'll most likely >core dump on the call. Unintentionally redefining index to be a function >of exactly the same type, arguments and all, is uncatchable and much harder >to debug. It's not hard to catch at all. In fact it is impossible to miss: $ cat tmp.c char *index(string, chr) char *string; char chr; { *string = chr; return string; } main() { (void) index("a", 'a'); return 0; } $ lint tmp.c tmp.c: index multiply declared tmp.c(4) :: llib-lc(269) $ ^D [This doesn't belong in unix.wizards any more: I have directed followups only to comp.lang.c] -- Geoff Clare <gwc@root.co.uk> (Dumb American mailers: ...!uunet!root.co.uk!gwc) UniSoft Limited, Hayne Street, London EC1A 9HH, England. Tel: +44-71-315-6600
richard@aiai.ed.ac.uk (Richard Tobin) (08/23/90)
In article <930@eplunix.UUCP> das@eplunix.UUCP (David Steffens) writes: >This actually happened to one of our programmers (a good one, IMHO). >Should I call her "stupid" just because she doesn't know that "index" >is a _dirty name_ and might be used by some vendor in writing his library? A few points: - Having the linker refuse to resolve text references against data definitions (and vice versa) doesn't completely solve this problem. Suppose the programmer had written a procedure called "index"? - It's useful for the linker to resolve these references. Pointers to procedures are often passed around as data, and it's often useful to be able to execute code in the data segment. It might be reasonable for the linker to *warn* about the latter case, or require a flag. - A large part of the problem is that library procedures you call (which you probably know about) often call other ones (which you may not know about). I had to debug a program which had run quite happily until it was moved onto a machine which was a Sun yellow pages client, and suddenly getpwent() was calling select(), which the programmer had used as a variable name. The solution is for library routines to call other library routines under different names (eg ones starting with __). On the other hand, it can be useful to be able to replace a library routine and have other library routines use your version. -- Richard -- Richard Tobin, JANET: R.Tobin@uk.ac.ed AI Applications Institute, ARPA: R.Tobin%uk.ac.ed@nsfnet-relay.ac.uk Edinburgh University. UUCP: ...!ukc!ed.ac.uk!R.Tobin
gwyn@smoke.BRL.MIL (Doug Gwyn) (08/24/90)
In article <3605@goanna.cs.rmit.oz.au> ok@goanna.cs.rmit.oz.au (Richard A. O'Keefe) writes: >The UNIX linkers allow a function reference to be resolved by a global >variable because they have no way at all of telling one from another. >The title of this thread refers to "text" and "data", but a read-only >array may well be in "text" space. Consider the "-R" flag on BSD UNIX >compilers and the "const" keyword in ANSI C. There are UNIX linkers that maintain sufficient object module segment attributes that they are able to detect such abuse. However, the linker shipped with 4.nBSD is not one of these. The simpler linkage scheme has both good and bad points, depending on what your detailed needs are. It is not clear to me that you should rely on a linker to help you debug faulty code or poor system library implementations, though. >Myself, I don't see data/function collisions as being any worse than >function/function collisions. Right. The more general issue is how to control allocation of the external ("global") name space. There is no fully satisfactory method, but through means such as object-oriented interfaces the collisions can be drastically reduced, and one can control one's OWN code by setting up a "package prefix" scheme such as I have previously described. There's not much to be done about multiple third-party provided libraries that conflict in the use of external identifiers such as "get" and "put", other than of course to not purchase them (and to let the vendor know why).
gwyn@smoke.BRL.MIL (Doug Gwyn) (08/24/90)
In article <3274@skye.ed.ac.uk> richard@aiai.UUCP (Richard Tobin) writes: > - It's useful for the linker to resolve these references. Pointers > to procedures are often passed around as data, and it's often > useful to be able to execute code in the data segment. It might > be reasonable for the linker to *warn* about the latter case, or > require a flag. Pointers to functions ARE data objects. On the other hand, executing code from data space in inherently nonportable, quite apart from machine instruction issues. Presumably, code that is going to be ported around would not be designed to execute code from data space. > ... I had to debug a program which had run quite > happily until it was moved onto a machine which was a Sun yellow > pages client, and suddenly getpwent() was calling select(), which > the programmer had used as a variable name. It's also fun when the Sun C compiler preprocessing changes your "sun" variable into a "1" constant. > The solution is for > library routines to call other library routines under different > names (eg ones starting with __). Most ANSI C-conscious UNIX vendors seem to favor a somewhat modified version of this, with special support in the linker to avoid extra overhead when the actual library select() function IS wanted. > On the other hand, it can be > useful to be able to replace a library routine and have other > library routines use your version. I absolutely do not recommend this. Implementations of the system library may well have internal constraints that the application programmer is unaware of, and replacing a library function can result in obscure, sometimes subtle, malfunctions.
jensting@skinfaxe.diku.dk (Jens Tingleff) (08/24/90)
As someone points out, this is *not* a problem in the linker, but a problem with the language. Check, for instance, Modula-2, where the external names are *only* brought in together with the (object) module from which they should be extracted at link time. The ``FROM foo IMPORT blahh'' method solves the problem, since ``foo'' only contains one visible ``blahh'', and the linker doesn't care how many ``blahh''s are floting around out there. [yes, you can redefine the PROCEDUREs used by library modules. If such a need arises, the relevant PROCEDURE could be a procedure variable which is assigned a non-default value by the users code] Back to C... Jens Jens Tingleff MSc EE, Institute of Computer Science, Copenhagen University Snail mail: DIKU Universitetsparken 1 DK2100 KBH O "It never runs around here; it just comes crashing down" apologies to Dire Straits
peter@ficc.ferranti.com (Peter da Silva) (08/24/90)
In article <13657@smoke.BRL.MIL> gwyn@smoke.BRL.MIL (Doug Gwyn) writes: > There's > not much to be done about multiple third-party provided libraries that > conflict in the use of external identifiers such as "get" and "put", > other than of course to not purchase them (and to let the vendor know why). Or go hack the binary of the library up with "adb -w" or its moral equivalent. -- Peter da Silva. `-_-' +1 713 274 5180. 'U` peter@ferranti.com
throopw@sheol.UUCP (Wayne Throop) (08/27/90)
> From: das@eplunix.UUCP (David Steffens) > Said shared name space extends beyond the confines of _my_ code, however. > And that makes it a linker problem, _not_ a compiler problem. I disagree most strenuously. The compiler is the agent that "decided" to name things that should definitely not be resolved together with the same name. The linker advertises what it will do with its namespace, and the compiler decides to name two incompatible things with the same linker name. I think it quite clear that the compiler is at fault. One may object that the compiler is merely naming the entity with the name given it in the source code. But this just isn't (normally) so, even (or maybe especially) in C. In fact, it is common to play games with the link-time names of things. Fortran, for example, twiddles with case, C prepends a superfluous "_". And more recent C++ implementations choose names which reflect the type of the link-time object generated. So, if the compiler would simply tag things with link-time names which reflect the distinctions that should be made, the link will do the right thing, and can even produce intelligible source-level error messages if it is made privy to some backtranslation database. -- Wayne Throop <backbone>!mcnc!rti!sheol!throopw or sheol!throopw@rti.rti.org
edward@ucbarpa.Berkeley.EDU (Edward Wang) (08/27/90)
In article <13658@smoke.BRL.MIL> gwyn@smoke.BRL.MIL (Doug Gwyn) writes: >In article <3274@skye.ed.ac.uk> richard@aiai.UUCP (Richard Tobin) writes: >> On the other hand, it can be useful to be able to replace a library >> routine and have other library routines use your version. > >I absolutely do not recommend this. Implementations of the system >library may well have internal constraints that the application >programmer is unaware of, and replacing a library function can result >in obscure, sometimes subtle, malfunctions. However it is an essential feature of C. Redefining library functions is a time-honored C tradition. Many things depend on it, for example Ben Zorn's memory profiler that was so well received at Usenix a few years ago. I don't recommend doing this lightly either. In Lisp, it's possible to redefine car and cdr, but getting it to work is harder.