matt@bacchus.esa.oz.au (Matt Atterbury) (09/05/90)
Sorry if this is the 'wrong' newsgroup, but ...
I have been thinking about shared libraries, and specifically how
difficult it might be to implement them in a BSD4.3 'standard'
(:-) environment using shared memory - mmap(2) i guess.
The basic idea would be:
take a library compiled with the equivalent of -pic (position
independent code? - i don't actually use SUNs) (1)
work out where we are going to locate the shared code (2)
scan each routine
if it uses static memory
stick the whole routine into the stubs .a
else
'link' the routine into the shared code file so that it
is callable
create a stub for the routine & stick it in the stubs .a (3)
stick all the global data in the stubs .a (4)
(1) needed so that the routine can use global data which is not
yet 'linked' (ie. positioned).
(2) maybe a command line arg - no need to get too sophisticated
(3) the stub would load the register pointing to the base of the
global data then jump to the known address for the routine
when linking with the library, the crt0 routine (before or after
calling _main?) would be hacked to call the initialisation routine
for all linked shared libraries.
when running, the shared library initialization routine(s) would
mmap the shared code file into the fixed address.
anyone got any other steps that would have to be performed?
anyone got any idea what effort would be involved?
anyone got any idea if gcc can/could do this (position independent
code, calling initialization routines)?
anyone got any ideas on the subject?
--
-------------------------------------------------------------------------------
Matt Atterbury [matt@bacchus.esa.oz.au] Expert Solutions Australia, Melbourne
UUCP: ...!uunet!munnari!matt@bacchus.esa.oz.au "klaatu barada nikto"
ARPA: matt%bacchus.esa.oz.AU@uunet.UU.NET "life? don't talk to me about life!"mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (04/23/91)
In article <716@seqp4.UUCP> jdarcy@seqp4.ORG (Jeff d'Arcy) writes: >In short, your implication that shared libraries are useless is unfounded. My original opinion is that X is useless so shared libraries are useless. >I >don't think I'm the only one who'd like to see more *facts* in your posts to >back up your abundant opinions. As shown <112@titccy.cc.titech.ac.jp> in comp.unix.wizards, total size of binaries excluding X11 is comparable to its machines real memory size. This means, real memory consumption is negligible unless we want to execute all binaries at once. Moreover, as swap space is usually several times larger than real memory, disk space consumption by statically linked library is negligible compared to the consumption by swap space. Masataka Ohta
jdarcy@seqp4.ORG (Jeff d'Arcy) (04/24/91)
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: >As shown <112@titccy.cc.titech.ac.jp> in comp.unix.wizards, total size of >binaries excluding X11 is comparable to its machines real memory size. Yippee. As you should know, the size *on disk* does not represent the size *in memory*. Consider the following C program: #define SIZE 4096 char fubar[SIZE][SIZE]; int main() { int i; for (;;) { for (i = 0; i < SIZE; ++i) fubar[i][0] = 0; } } On the machine I'm using, this creates an a.out file 32116 bytes long. Do you think that's all the space it will take up while running? If so, then I need not argue any further since you're obviously deranged. It is irrelevant that most of the space used by this program is for data. In real systems, this is the case for many programs, and total virtual space will typically be quite a bit greater than physical memory. What matters most is how much memory you can use for paging the infrequently referenced data. Pages that are used often will not get paged out, and can pretty much be considered fixed memory. This category includes much of the text in many programs, in particular large parts of libc. If you don't use shared libraries, those portions of libc will be resident all over the place, as parts of the text of many *different* programs (yes, text for the *same* program is shared anyway). With shared libraries, they are resident in only one place, saving a significant fraction of physical memory. Increasing the amount of physical memory available for paging by as little as a few percent can cause a *significant* improvement in overall system performance. Do you really think so many skilled and knowledgeable OS developers would implement shared libraries if they weren't worth it? My apologies to the majority of the readership, who already know all this.
barmar@think.com (Barry Margolin) (04/25/91)
In article <718@seqp4.UUCP> jdarcy@seqp4.ORG (Jeff d'Arcy) writes: > Do you really think so many skilled and knowledgeable >OS developers would implement shared libraries if they weren't worth it? What I want to know is why it has taken so long for them to rediscover shared libraries. Multics, the predecessor to Unix, was using dynamically-linked, shared libraries in the 60's. Furthermore, the SunOS dynamic linking design is still inferior to the Multics design (I don't know details of other Unix dynamic linking designs). Libraries should be linked on demand, not all at the beginning of the program. This way, infrequently-used routines do not have any overhead except when they're used. However, this should be coupled with better OS support for dynamic link failures (it would be a shame for the process to core dump just because LD_LIBRARY_PATH is missing a necessary directory). Also, on-demand linking is harder to implement for data than for code, often requiring hardware support (Multics does all dynamic linking by indirecting through pointers with a special trap bit set -- the trap handler resolves the link, rewrites the pointer, and restarts the interrupted instruction, so procedures and data are equivalent). -- Barry Margolin, Thinking Machines Corp. barmar@think.com {uunet,harvard}!think!barmar
brendan@cs.widener.edu (Brendan Kehoe) (04/25/91)
In <1991Apr24.231048.2987@Think.COM>, barmar@think.com writes: >In article <718@seqp4.UUCP> jdarcy@seqp4.ORG (Jeff d'Arcy) writes: >> Do you really think so many skilled and knowledgeable >>OS developers would implement shared libraries if they weren't worth it? > >What I want to know is why it has taken so long for them to rediscover >shared libraries. Multics, the predecessor to Unix, was using >dynamically-linked, shared libraries in the 60's. Is there a decent reason for the executables on a DecStation under Ultrix 4.1 still exceeding 3-4Mb in some cases? -- Brendan Kehoe - Widener Sun Network Manager - brendan@cs.widener.edu Widener University in Chester, PA A Bloody Sun-Dec War Zone "Does this person look relaxed to you? Well, it's actually an experiment of Contour's new 565-E chair!"
sef@kithrup.COM (Sean Eric Fagan) (04/26/91)
In article <136@titccy.cc.titech.ac.jp> mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: >As shown <112@titccy.cc.titech.ac.jp> in comp.unix.wizards, total size of >binaries excluding X11 is comparable to its machines real memory size. >This means, real memory consumption is negligible unless we want to >execute all binaries at once. What mean we, white man? The sum total size of all the binaries I regularly use (on kithrup, of course) is about four or five times the size of kithrup's real memory (kithrup has 8Mbytes these days). Back before I got that additional 4Mbytes, the only reason kithrup was able to run decently was because all or most of the system binaries were built using shared libraries (uucp being the major exception, and the major cause for system slowness as a result). Not everyone has a system in which 8Mbytes is considered a small cache, you know. -- Sean Eric Fagan | "I made the universe, but please don't blame me for it; sef@kithrup.COM | I had a bellyache at the time." -----------------+ -- The Turtle (Stephen King, _It_) Any opinions expressed are my own, and generally unpopular with others.
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (04/26/91)
In article <1991Apr24.231048.2987@Think.COM> barmar@think.com (Barry Margolin) writes: >In article <718@seqp4.UUCP> jdarcy@seqp4.ORG (Jeff d'Arcy) writes: >> Do you really think so many skilled and knowledgeable >>OS developers would implement shared libraries if they weren't worth it? >What I want to know is why it has taken so long for them to rediscover >shared libraries. Multics, the predecessor to Unix, was using >dynamically-linked, shared libraries in the 60's. UNIX was formed by intentionally removing many unnecessary features from Multics and other huge OSs. >(Multics does all dynamic >linking by indirecting through pointers with a special trap bit set If you want shared libraries, use Multics. Don't change UNIX to Multics. Masataka Ohta
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (04/26/91)
In article <1991Apr25.175547.21534@kithrup.COM> sef@kithrup.COM (Sean Eric Fagan) writes: >>As shown <112@titccy.cc.titech.ac.jp> in comp.unix.wizards, total size of >>binaries excluding X11 is comparable to its machines real memory size. >>This means, real memory consumption is negligible unless we want to >>execute all binaries at once. > >What mean we, white man? I amd rather a yellow monkey. :-) >Back before I got that additional 4Mbytes, the only reason kithrup was able >to run decently was because all or most of the system binaries were built >using shared libraries (uucp being the major exception, and the major cause >for system slowness as a result). Strange. Code segment size of UUCP is small compared to 4MB. >Not everyone has a system in which 8Mbytes is considered a small cache, you >know. So, don't run a big window system, white man. Masataka Ohta
jfh@rpp386.cactus.org (John F Haugh II) (04/27/91)
In article <148@titccy.cc.titech.ac.jp> mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: >UNIX was formed by intentionally removing many unnecessary features from >Multics and other huge OSs. The intention was to be simple and elegant, not spartan. The solution to the shared library dilema is to find a simple and elegant implementation of shared libraries. The problems I've always seen revolve about the data requirements of shared libraries. I've always taken this to mean that the libraries were poorly designed, rather than that the concept of a shared library is flawed. -- John F. Haugh II | Distribution to | UUCP: ...!cs.utexas.edu!rpp386!jfh Ma Bell: (512) 255-8251 | GEnie PROHIBITED :-) | Domain: jfh@rpp386.cactus.org "If liberals interpreted the 2nd Amendment the same way they interpret the rest of the Constitution, gun ownership would be mandatory."
peter@cutmcvax.cs.curtin.edu.au (Peter Wemm) (04/27/91)
barmar@think.com (Barry Margolin) writes: >In article <718@seqp4.UUCP> jdarcy@seqp4.ORG (Jeff d'Arcy) writes: >> Do you really think so many skilled and knowledgeable >>OS developers would implement shared libraries if they weren't worth it? >What I want to know is why it has taken so long for them to rediscover >shared libraries. Multics, the predecessor to Unix, was using >dynamically-linked, shared libraries in the 60's. >Furthermore, the SunOS dynamic linking design is still inferior to the >Multics design (I don't know details of other Unix dynamic linking >designs). Libraries should be linked on demand, not all at the beginning >of the program. This way, infrequently-used routines do not have any >overhead except when they're used. However, this should be coupled with >better OS support for dynamic link failures (it would be a shame for the >process to core dump just because LD_LIBRARY_PATH is missing a necessary >directory). Also, on-demand linking is harder to implement for data than >for code, often requiring hardware support (Multics does all dynamic >linking by indirecting through pointers with a special trap bit set -- the >trap handler resolves the link, rewrites the pointer, and restarts the >interrupted instruction, so procedures and data are equivalent). Even the Amiga uses dynamic libraries... The ENTIRE operating system is present as a set of shared libraries... Right from Day 1! It doesn't do it the same way that Unix or Multics does it, but it is VERY fast! >-- >Barry Margolin, Thinking Machines Corp. >barmar@think.com >{uunet,harvard}!think!barmar -- Peter Wemm ------------------------------------------------------------------------------ peter@cs.curtin.edu.au (Home) +61-9-450-5243 Curtin University of Technology, Perth, Western Australia. Amiga... Because life is too short for boring computers. (Dan Zerkle)
mike@bria.UUCP (Michael Stefanik) (04/28/91)
In an article, mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: >If you want shared libraries, use Multics. Don't change UNIX to Multics. Now, I pride myself on being a stuffy UNIX purist, but I disagree that sharable libraries should be thrown out the door for the sake of a spartan kernel. The reality of today is that disk and core *is* less expensive, so it is not so burdensome to include junk, errr, features that *do* have legitimate value. The advantages of sharable libraries are many, and, although they may not be "needed", they certainly are nice to have. -- Michael Stefanik, MGI Inc, Los Angeles | Opinions stated are never realistic Title of the week: Systems Engineer | UUCP: ...!uunet!bria!mike ------------------------------------------------------------------------------- If MS-DOS didn't exist, who would UNIX programmers have to make fun of?
sef@kithrup.COM (Sean Eric Fagan) (04/28/91)
In article <149@titccy.cc.titech.ac.jp> mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: >In article <1991Apr25.175547.21534@kithrup.COM> > sef@kithrup.COM (Sean Eric Fagan) writes: >>What mean we, white man? >I amd rather a yellow monkey. :-) I know. We've met 8-). >Strange. Code segment size of UUCP is small compared to 4MB. True. But when several uucico's are running at once, as are a uuxqt, and some rmails, and some MMDF delivery set-of-processes (several to dozens to deliver one mail message), 4Mbytes becomes *very* small. >>Not everyone has a system in which 8Mbytes is considered a small cache, you >>know. >So, don't run a big window system, white man. Who said I did? Kithrup doesn't have X installed, nor any other windowing system. -- Sean Eric Fagan | "I made the universe, but please don't blame me for it; sef@kithrup.COM | I had a bellyache at the time." -----------------+ -- The Turtle (Stephen King, _It_) Any opinions expressed are my own, and generally unpopular with others.
mjr@hussar.dco.dec.com (Marcus J. Ranum) (04/29/91)
John F Haugh II writes about UNIX: >The intention was to be simple and elegant, not spartan. I couldn't pass this up. The Spartans, in fact, were successful because they did live a simple, elegant, ummm, spartan existence. It was their success, and the ensuing "fat dumb & happy" way of life that resulted from their success that did 'em in. There's an obvious witty remark about "those that do not study history..." I suppose I could make here. > The solution >to the shared library dilema is to find a simple and elegant implementation >of shared libraries. I somewhat disagree. The trick is to take a step back - forget the bloody shared libraries and develop a system with a decent virtual memory system that allows nice shared memory, memory mapping, and copy on write type semantics for those pesky data areas, and shared libraries are less of a problem. If you look at your watermelon/kernel and ask "how can I add shared libraries" you'll wind up with a kludge. mjr.
fmayhar@hermes.ladc.bull.com (Frank Mayhar) (04/30/91)
In article <148@titccy.cc.titech.ac.jp>, mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: -> In article <1991Apr24.231048.2987@Think.COM> -> barmar@think.com (Barry Margolin) writes: -> >(Multics does all dynamic linking by indirecting through pointers -> >with a special trap bit set -> If you want shared libraries, use Multics. Don't change UNIX to Multics. Unfortunately, Multics is dead, dead, dead. That means that if we want shared libraries, we _have_ to use something else. Since Unix is here, we'll use it. And as I've said before (before I got busy and lost track of this thread), shared libraries _are_ useful. Their usefulness, in fact, outweighs the problems with them. Why do you think shared libraries keep being reinvented? -- Frank Mayhar fmayhar@hermes.ladc.bull.com (..!{uunet,hacgate}!ladcgw!fmayhar) Bull HN Information Systems Inc. Los Angeles Development Center 5250 W. Century Blvd., LA, CA 90045 Phone: (213) 216-6241
rickert@mp.cs.niu.edu (Neil Rickert) (04/30/91)
In article <1991Apr29.213522.29521@ladc.bull.com> fmayhar@hermes.ladc.bull.com writes: > >of this thread), shared libraries _are_ useful. Their usefulness, in fact, >outweighs the problems with them. Why do you think shared libraries keep >being reinvented? Perhaps because it is easier to keep reinventing shared libraries than it is to do it right. -- =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*= Neil W. Rickert, Computer Science <rickert@cs.niu.edu> Northern Illinois Univ. DeKalb, IL 60115 +1-815-753-6940
fmayhar@hermes.ladc.bull.com (Frank Mayhar) (05/02/91)
In article <1991Apr30.004343.27551@mp.cs.niu.edu>, rickert@mp.cs.niu.edu (Neil Rickert) writes:
-> In article <1991Apr29.213522.29521@ladc.bull.com> I write:
-> >of this thread), shared libraries _are_ useful. Their usefulness, in fact,
-> >outweighs the problems with them. Why do you think shared libraries keep
-> >being reinvented?
-> Perhaps because it is easier to keep reinventing shared libraries than it is
-> to do it right.
Huh?? Care to explain that statement? IMHO, the reason they keep being
reinvented (as opposed to simply being reimplemented), is that most folks that
implement them haven't ever done so before. (There are, of course, exceptions
to this generalization.) So they end up being reinvented in a relative vacuum,
without the benefit of the experience that many of us have.
Again, the reason, IMHO, that shared libraries appear in so many different
environments is that their benefits tend to outweigh the problems with them. I
certainly would like to see a "right" implementation of shared libraries, but
first we need to decide just what "right" is, anyway.
--
Frank Mayhar fmayhar@hermes.ladc.bull.com (..!{uunet,hacgate}!ladcgw!fmayhar)
Bull HN Information Systems Inc. Los Angeles Development Center
5250 W. Century Blvd., LA, CA 90045 Phone: (213) 216-6241goykhman_a@apollo.HP.COM (Alex Goykhman) (05/02/91)
In article <19239@rpp386.cactus.org> jfh@rpp386.cactus.org (John F Haugh II) writes: >In article <148@titccy.cc.titech.ac.jp> mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: >>UNIX was formed by intentionally removing many unnecessary features from >>Multics and other huge OSs. > >The intention was to be simple and elegant, not spartan. The solution >to the shared library dilema is to find a simple and elegant implementation >of shared libraries. The problems I've always seen revolve about the >data requirements of shared libraries. I've always taken this to mean that >the libraries were poorly designed, rather than that the concept of a shared >library is flawed. >-- >John F. Haugh II | Distribution to | UUCP: ...!cs.utexas.edu!rpp386!jfh >Ma Bell: (512) 255-8251 | GEnie PROHIBITED :-) | Domain: jfh@rpp386.cactus.org >"If liberals interpreted the 2nd Amendment the same way they interpret the > rest of the Constitution, gun ownership would be mandatory." Come one, guys. There is nothing special about a shared library. Think of it as a part of, or an extention to, the kernel. A s.l. is just file with a bunch of entry points. All you need to do is to map it somewhere in the virtual address space, and trap every call to the s.l. entry points, and substitute it with a call to the corresponding routine. Mapping is trivial, trapping is machine-dependent, and you can't change it anyway, so it must take a lot of imagination to produce an implementation of shared libraries that would not be simple and elegant. | Alex Goykhman goykhman_a@apollo.hp.com | | Chelmsford System Software Lab mit-eddie!apollo!goykhman_a | | Hewlett-Packard, Company I speak for myself | ------------------------------------------------------------------
rickert@mp.cs.niu.edu (Neil Rickert) (05/03/91)
In article <5152f9e6.20b6d@apollo.HP.COM> goykhman_a@apollo.HP.COM (Alex Goykhman) writes: > >Come one, guys. There is nothing special about a shared library. Think >of it as a part of, or an extention to, the kernel. A s.l. is just file >with a bunch of entry points. All you need to do is to map it somewhere >in the virtual address space, and trap every call to the s.l. entry >points, and substitute it with a call to the corresponding routine. Unfortunately it is not that simple. The trouble is that the text segment of the module contains references to the data and bss, which will be different for each use of the shared routines. The proper way to do this is to design the shared library so that all modules in it have only text segments, and zero-length bss and data segments. This requires a substantial change in coding style, although one which is also worthwhile for better handling of multiple processors which also require better reentrancy. Essentially it requires a redesign of C. Not really a redesign of the language, but a redesign of the implementation. -- =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*= Neil W. Rickert, Computer Science <rickert@cs.niu.edu> Northern Illinois Univ. DeKalb, IL 60115 +1-815-753-6940
jim@segue.segue.com (Jim Balter) (05/03/91)
In article <1991Apr29.031351.3912@decuac.dec.com> mjr@hussar.dco.dec.com (Marcus J. Ranum) writes: >John F Haugh II writes about UNIX: > >>The intention was to be simple and elegant, not spartan. > > I couldn't pass this up. The Spartans, in fact, were successful >because they did live a simple, elegant, ummm, spartan existence. It was >their success, and the ensuing "fat dumb & happy" way of life that resulted >from their success that did 'em in. Spartan implies ascetic and harsh, certainly not elegant. The Spartans never became "fat dumb & happy". They had laws against being happy (actually against the arts). After the Theban war, they sort of fell apart, due to bad economics, and because they had depended upon their slaves for all commerce (they also had laws against that) and agriculture. It was putting all their eggs in the war skills basket that did them in. Perhaps a lesson for today. > There's an obvious witty remark about >"those that do not study history..." I suppose I could make here. I doubt it. "Those who do not learn from history are bound to repeat it." -- George Santayana
rang@cs.wisc.edu (Anton Rang) (05/03/91)
In article <1991May2.172740.11422@mp.cs.niu.edu> rickert@mp.cs.niu.edu (Neil Rickert) writes: > Unfortunately it is not that simple. The trouble is that the text segment >of the module contains references to the data and bss, which will be different >for each use of the shared routines. This is true; also, on some architectures, it's difficult or impossible to generate fully position-independent code. (If you map the library into a different spot in your address space, you may need to "patch up" some references it makes, which suddenly makes some pages shared and some pages not, or else forces you to make the entire text segment non-shared.) Hopefully these architectures are few.... Another point which shouldn't be too hard to get right, but does require some thought, is that one shared library may refer to routines from another one. > The proper way to do this is to design the shared library so that >all modules in it have only text segments, and zero-length bss and >data segments. Given an architecture with a shared data and instruction memory, a C compiler which puts 'const' variables into the text segment would make this fairly painless, I think. (Personally, I don't like variables which are initialized at compile time but aren't constants--they offend my sense that everything should be re-entrant. :-) Anton +---------------------------+------------------+-------------+----------------+ | Anton Rang (grad student) | rang@cs.wisc.edu | UW--Madison | "VMS Forever!" | +---------------------------+------------------+-------------+----------------+
amolitor@eagle.wesleyan.edu (05/04/91)
In article <5152f9e6.20b6d@apollo.HP.COM>, goykhman_a@apollo.HP.COM (Alex Goykhman) writes: > > Come one, guys. There is nothing special about a shared library. Think > of it as a part of, or an extention to, the kernel. A s.l. is just file > with a bunch of entry points. All you need to do is to map it somewhere > in the virtual address space, and trap every call to the s.l. entry > points, and substitute it with a call to the corresponding routine. > > Mapping is trivial, trapping is machine-dependent, and you can't change it > anyway, so it must take a lot of imagination to produce an implementation > of shared libraries that would not be simple and elegant. > Well, it depends on what you mean by 'trivial'. As has been pointed out, there is a problem with static data required by library routines, and the solution to this is to use a paging system providing copy-on-write. I would hesitate to refer to load time linking as 'trivial', as well. And yeah, I have written a (simple) load-time linker. Doing it right is hard. I've not noticed anyone mention the VMS implementation. It's my impression that the VMS shared libarary system is Done Right. Based on vague memories of a single dark night years ago spent RTFMing and peering at the output from analysis tools, I believe that VMS a) uses the copy-on-write abilities of it's memory system to deal with static data areas gracefully and b) has proper load-time linking to libraries -- i.e. calls to library routines are exactly the same as any other call. There is, therefore, no performance hit besides the startup (both in linkage, and in copying on write of data areas). This is typically more than made up for by the performance GAIN achieved by not loading 100K of library routines when you execute the image. To be precise, I am happier running code linked to sharable libraries than statically linked. Andrew > | Alex Goykhman
goykhman_a@apollo.HP.COM (Alex Goykhman) (05/04/91)
In article <1991May2.172740.11422@mp.cs.niu.edu> rickert@mp.cs.niu.edu (Neil Rickert) writes: >In article <5152f9e6.20b6d@apollo.HP.COM> goykhman_a@apollo.HP.COM (Alex Goykhman) writes: >> >>Come one, guys. There is nothing special about a shared library. Think >>of it as a part of, or an extention to, the kernel. A s.l. is just file >>with a bunch of entry points. All you need to do is to map it somewhere >>in the virtual address space, and trap every call to the s.l. entry >>points, and substitute it with a call to the corresponding routine. > > Unfortunately it is not that simple. The trouble is that the text segment >of the module contains references to the data and bss, which will be different >for each use of the shared routines. > > The proper way to do this is to design the shared library so that all modules >in it have only text segments, and zero-length bss and data segments. This >requires a substantial change in coding style, although one which is also >worthwhile for better handling of multiple processors which also require >better reentrancy. Essentially it requires a redesign of C. Not really a >redesign of the language, but a redesign of the implementation. I agree, but the transition from one design style to another is not nearly as difficult as you just described. A few years ago I had a task of converting a dozen or so C routines to a shared library format. All I had to do was to replace the static variables with their dynamic equivalents, which I accomplished by moving a few dozen lines of code around using my favorite editor. Our C compiler allocated the dynamic variables on the current process' stack, thus making the routines both re-entrant, and suitable for multi-processing. BTW, I can't think of a single s.l.-specific issue introduced by multi-processing. >-- >=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*= > Neil W. Rickert, Computer Science <rickert@cs.niu.edu> > Northern Illinois Univ. > DeKalb, IL 60115 +1-815-753-6940
sef@kithrup.COM (Sean Eric Fagan) (05/04/91)
In article <5152f9e6.20b6d@apollo.HP.COM> goykhman_a@apollo.HP.COM (Alex Goykhman) writes: [a bunch of stuff showing ignorance of issues involved in shared libraries] >Mapping is trivial, trapping is machine-dependent, and you can't change it >anyway, so it must take a lot of imagination to produce an implementation >of shared libraries that would not be simple and elegant. Really? How do you make sure that a user can substitute in his/her own version of, say, malloc, and make sure that all of the routines in the shared library use it? Where do you put the library's data? -- Sean Eric Fagan | "I made the universe, but please don't blame me for it; sef@kithrup.COM | I had a bellyache at the time." -----------------+ -- The Turtle (Stephen King, _It_) Any opinions expressed are my own, and generally unpopular with others.
kre@cs.mu.oz.au (Robert Elz) (05/04/91)
mjr@hussar.dco.dec.com (Marcus J. Ranum) writes: > I somewhat disagree. The trick is to take a step back - forget >the bloody shared libraries and develop a system with a decent virtual >memory system that allows nice shared memory, memory mapping, and copy >on write type semantics for those pesky data areas, and shared libraries >are less of a problem. This seems to assume that you want shared libraries to same memory (VM), which of all the benefits is the least I care about. More important is to save disc space for the binaries (I only run a few of them at a time, but I want thousands around for when I may want to run them), and even more importantly, the ability to make changes to library routines that affect all processes to use the library after the change, without recompilation or re linking (other than normal shared library runtime linking). The trick is to make that work so shared library routines can call routines defined in the application - replacing a routine in the library, just as they would if the library wasn't shared, and so the shared library can have shared initialised data that can be referenced and modified (obviously copy on write) by the application, without the compiler being aware that the data to be referenced is in a shared area (hence, has no fixed address) when the object file is compiled. And next make it work so that the whole thing doesn't run like a camel... kre
rickert@mp.cs.niu.edu (Neil Rickert) (05/04/91)
In article <kre.673335935@mundamutti.cs.mu.OZ.AU> kre@cs.mu.oz.au (Robert Elz) writes: >and even more importantly, the ability to make changes to library >routines that affect all processes to use the library after the change, >without recompilation or re linking (other than normal shared library >runtime linking). You have just listed as a benefit of shared libraries that feature which can cause perfectly good working programs to suddenly be broken just because someone changed the shared libraries. I consider that to be a major disadvantage of shared libraries. On the other hand, I understand your point. It allows you to take old code using the hosts tables, and have it suddenly use the nameserver, for example. But a module structure which allowed re-editing a binary, and explicitely replacing one module in it, would give a much more useful level of control. -- =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*= Neil W. Rickert, Computer Science <rickert@cs.niu.edu> Northern Illinois Univ. DeKalb, IL 60115 +1-815-753-6940
rang@cs.wisc.edu (Anton Rang) (05/05/91)
In article <1991May3.134905.42465@eagle.wesleyan.edu> amolitor@eagle.wesleyan.edu writes: > I've not noticed anyone mention the VMS implementation. It's my >impression that the VMS shared library system is Done Right. I'd agree, though it's not *quite* as good as you make it out to be. (Oh no, I'm finding myself on the other side! Yay! :-) >I believe that VMS a) uses the copy-on-write abilities of it's memory >system to deal with static data areas gracefully Close; actually, though, it uses copy-on-reference, which isn't as good as copy-on-write would be. (At least, VMS 4.x does--I don't have current documentation on VMS 5.x.) When a (writable) data page is faulted in from the shared library, a private copy is made for the process, whether the access was a read or a write. >and b) has proper load-time linking to libraries -- i.e. calls to >library routines are exactly the same as any other call. There is, >therefore, no performance hit besides the startup (both in linkage, >and in copying on write of data areas). Nope. It's a little questionable what the "correct" way to link to a shareable library is, actually. If you go through the original image and fix up all of the calls, you may fault in the entire image in the process, and you also violate the read-only condition (so that it can no longer be paged from the image on disk, but must be in the swap area or page file). VMS normally takes the approach of making function calls to shared images go through one level of indirection. At link time, a table of functions from shared images which are referenced by the main image is built; calls to functions are replaced by indirect calls through a cell in this table. At load time, this table is filled with the correct addresses. A call within the main image is absolute; a call in the shared library is absolute-indirect. If enough registers are available, a good compiler could get rid of most of this overhead (in loops, at least) by loading the address of the function being called into a register and taking it out of the loop. This would be especially easy on RISC systems. (It is also possible to ask VMS to have every reference, or some, fixed at load time, avoiding the indirection at the cost of increased fixup time. This must be done when static data is referenced from a library. As before, this temporarily violates the read-only constraint on code, so that space in the swap file must be allocated, and the code cannot be shared between processes. Of course, VMS doesn't normally share code of programs among processes only--a BAD design decision, IMHO--so it's not as much of an impact as it would be on UNIX.) >This is typically more than made up for by the performance GAIN >achieved by not loading 100K of library routines when you execute the >image. True, at least under VMS.... A few other things about VMS's shared libraries: * They have version numbers done in a relatively reasonable way. There is a major version number, and a minor version number. This is checked against the version of the library which was used when linking a program; it is possible to specify several constraints on the use of the image (e.g. require an exact match, or require that the major version be exact and the minor version no less). * Each image (the main image, and shareable images) can have an initialization routine. These are automatically called in an appropriate order (topological sort). This lets run-time initialization be done pretty easily (useful for some languages though not really in C). * There is a user-callable function which will search for a given function and map the shareable image containing it into address space (if it isn't already). This dynamic loading is done essentially using the existing shareable library loading code. Anton +---------------------------+------------------+-------------+----------------+ | Anton Rang (grad student) | rang@cs.wisc.edu | UW--Madison | "VMS Forever!" | +---------------------------+------------------+-------------+----------------+
kre@cs.mu.oz.au (Robert Elz) (05/05/91)
rickert@mp.cs.niu.edu (Neil Rickert) writes: >You have just listed as a benefit of shared libraries that feature which >can cause perfectly good working programs to suddenly be broken just because >someone changed the shared libraries. Yes, of course - the power to improve is also the power to destroy. This isn't a disadvantage of the mechanism, its an advantage. Like anything powerful, if misused, it can be dangerous. The same argument could be used with respect to the kernel - its not part of the application, so it can be changed - if changed in an improper way applications will break, so obviously it must be a "major disadvantage" to have a separate kernel, and not have it complete linked into every application. kre
sef@kithrup.COM (Sean Eric Fagan) (05/06/91)
In article <1991May3.134905.42465@eagle.wesleyan.edu> amolitor@eagle.wesleyan.edu writes: > I've not noticed anyone mention the VMS implementation. It's my >impression that the VMS shared libarary system is Done Right. >I believe that VMS a) uses the >copy-on-write abilities of it's memory system to deal with static data >areas gracefully and b) has proper load-time linking to libraries -- i.e. >calls to library routines are exactly the same as any other call. You can do that if you have static addresses for your shared libraries. The SysVr3 shared libraries have that. One question, though: can you replace VMS' equivalent of, say, malloc() and then have all of the routines in the shared library use that? If you do completely static linking (for addresses), you can build a shared library that has printf at, say, 0xa0000000, and malloc() at, oh, 0xb0000000. By setting the addresses when you build the library, and putting those addresses into the equivalent of libc.a, you can do things the way you describe. But, as I said earlier, you cannot then replace malloc() and have printf() use it (which was a requirement in the AT&T shared library design). -- Sean Eric Fagan | "I made the universe, but please don't blame me for it; sef@kithrup.COM | I had a bellyache at the time." -----------------+ -- The Turtle (Stephen King, _It_) Any opinions expressed are my own, and generally unpopular with others.
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (05/07/91)
In article <1991May4.132632.13885@mp.cs.niu.edu> rickert@mp.cs.niu.edu (Neil Rickert) writes: >>and even more importantly, the ability to make changes to library >>routines that affect all processes to use the library after the change, >>without recompilation or re linking (other than normal shared library >>runtime linking). > On the other hand, I understand your point. It allows you to take old code >using the hosts tables, and have it suddenly use the nameserver, for example. The reality is a bit different. With DNS, it is common that a signle hostname have multiple IP addresses. Programs were modified so that they try all possible addresses, because it was common that some of IP addresses are often unreachable because of a routing problem. Not so many specification change can be handled simply by replacing libraries. Masataka Ohta
shore@theory.tn.cornell.edu (Melinda Shore) (05/07/91)
In article <161@titccy.cc.titech.ac.jp> mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: >The reality is a bit different. With DNS, it is common that a signle hostname >have multiple IP addresses. The two things are entirely independent. It is not true that there is any relationship between name service and the number of IP addresses on a single host, and it is absolutely not true that it is common for a host to have multiple IP addresses. Unless, of course, your definition of "common" is radically different. >Programs were modified so that they try all >possible addresses, because it was common that some of IP addresses are >often unreachable because of a routing problem. Again, this is independent of the mechanism used for hostname lookup. In the musty, dusty days before name service I had to fix the routing code in a hyperchannel driver for just this reason. -- Software longa, hardware brevis Melinda Shore - Cornell Information Technologies - shore@theory.tn.cornell.edu
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (05/08/91)
In article <1991May7.145228.423@batcomputer.tn.cornell.edu> shore@theory.tn.cornell.edu (Melinda Shore) writes: >>The reality is a bit different. With DNS, it is common that a signle hostname >>have multiple IP addresses. >The two things are entirely independent. It is not true that there >is any relationship between name service and the number of IP addresses >on a single host, But hosts with muptiple IP addresses become common with the introduction of DNS. >and it is absolutely not true that it is common for >a host to have multiple IP addresses. Unless, of course, your definition >of "common" is radically different. There are many examples of hosts with multiple IP addresses scattered through various RFC. See, for example, RFC1034, Page 14. There, VENERA.ISI.EDU. have two IP addresses: 128.9.0.32 and 10.1.0.52. A file for named, root.cache, also contains examples of hosts with multiple IP addresses. If you think examples in RFC1034 (the RFC which defines DNS) and other RFCs is not "common", YOU must explain what you think "common". >>Programs were modified so that they try all >>possible addresses, because it was common that some of IP addresses are >>often unreachable because of a routing problem. >In the musty, dusty days before name service I had to fix the routing >code in a hyperchannel driver for just this reason. "fix the routing code"? Routing code has nothing to do with the problem. Masataka Ohta
shore@theory.tn.cornell.edu (Melinda Shore) (05/09/91)
In article <163@titccy.cc.titech.ac.jp> mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: >But hosts with muptiple IP addresses become common with the introduction >of DNS. Again, it has *absolutely* *nothing* to do with name service. All that dns provides is a mechanism for looking up hostname/ip address mappings. Because of the growth of the internet there has been an increase in the number of gateways, but I wouldn't say that the ratio of gateways to leaf nodes has changed much. At most you've got an entirely spurious correlation. >>In the musty, dusty days before name service I had to fix the routing >>code in a hyperchannel driver for just this reason. >"fix the routing code"? Routing code has nothing to do with the problem. No, but the routing code in this particular driver used ip addresses as hash keys. Duh. -- Software longa, hardware brevis Melinda Shore - Cornell Information Technologies - shore@theory.tn.cornell.edu
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (05/09/91)
In article <1991May8.173813.27064@batcomputer.tn.cornell.edu> shore@theory.tn.cornell.edu (Melinda Shore) writes: >>But hosts with muptiple IP addresses become common with the introduction >>of DNS. >Again, it has *absolutely* *nothing* to do with name service. It dose have. And shared libraries dose *NOT* help the transition from /etc/hosts to DNS. Let's see more detailed facts. With the introduction of DNS, struct hostent, a structure returned by gethostbyname(), was changed. A new field, h_addr_list, was added and an old field, h_addr, was replaced by a #define. Let's see new /usr/include/netdb.h. : char **h_addr_list; /* list of addresses from name server */ :#define h_addr h_addr_list[0] /* address, for backward compatiblity */ As indicated by the comment, the change was done because a name server returns multiple IP addresses. Moreover, as clearly seen, the change dose require recompilation, even if we want to use old code which dose not support a host with mutiple IP addresses. >>"fix the routing code"? Routing code has nothing to do with the problem. > >No, but the routing code in this particular driver used ip addresses >as hash keys. Duh. Well, perhaps, you are talking about a case of a single network interface with multiple IP addresses, while we are talking about a single host with multiple network interfaces and thus with multiple IP addresses. Masataka Ohta
shore@theory.tn.cornell.edu (Melinda Shore) (05/09/91)
In article <166@titccy.cc.titech.ac.jp> mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: >With the introduction of DNS, struct hostent, a structure returned by >gethostbyname(), was changed. A new field, h_addr_list, was added and >an old field, h_addr, was replaced by a #define. That change was introduced with 4.3BSD, not with dns. Be-de-be-de-be-de that's all, folks. -- Software longa, hardware brevis Melinda Shore - Cornell Information Technologies - shore@theory.tn.cornell.edu
kre@cs.mu.oz.au (Robert Elz) (05/09/91)
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: >And shared libraries dose *NOT* help the transition from /etc/hosts to DNS. It certainly does - you don't have to use the multi-address per host version of netdb.h in order to use the DNS. The effect will be that applications will see only one address for any particular name, and not any alternate addresses - but they will certainly still be using the DNS to look up host names, rather than the host table, which is certainly a help in the transition. If you want to investigate the BIND distributions in some detail in the area of the include files, READ-ME's and the resolver, you will see that this possibility is explicitly allowed. Finally, this certainly DOES work, I have (in the past) done exactly this. kre
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (05/10/91)
In article <1991May9.135637.5408@batcomputer.tn.cornell.edu> shore@theory.tn.cornell.edu (Melinda Shore) writes: >>With the introduction of DNS, struct hostent, a structure returned by >>gethostbyname(), was changed. A new field, h_addr_list, was added and >>an old field, h_addr, was replaced by a #define. >That change was introduced with 4.3BSD, not with dns. The change was introduced with 4.3BSD to meet the requirements of DNS. You should have checked the line of h_addr_list in /usr/include/netdb.h of 4.3BSD. As I quoted in my previous posting, the words "name server" can be seen within the comment of the line. Masataka Ohta
shore@theory.tn.cornell.edu (Melinda Shore) (05/10/91)
In article <172@titccy.cc.titech.ac.jp> mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: >The change [array of host addresses] was introduced with 4.3BSD to meet >the requirements of DNS. Let's try thinking about it, shall we? What functionality is provided by static hostname lookup? What kind of information is provided in HOSTS.TXT by the NIC? What functionality is provided by distributed domain name service? What kind of information does it handle? Very good. Now, here's the hard part - keeping in mind that while correlation often implies causality it doesn't prove it, can you explain *why* the introduction of a distributed mechanism would cause a change in the information being provided? Is there a necessary relationship between delivery mechanism and the content of the thing being delivered? Comments in source code don't count - we're looking for understanding and explanation, not rote repetition. What was it about gateways that changed with the introduction of a distributed name service? And, as some one else pointed out, this particular change would not break existing binaries. While there surely are arguments against shared libraries, this is not one of them. -- Software longa, hardware brevis Melinda Shore - Cornell Information Technologies - shore@theory.tn.cornell.edu
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (05/11/91)
In article <1991May10.134839.23763@batcomputer.tn.cornell.edu> shore@theory.tn.cornell.edu (Melinda Shore) writes: >>The change [array of host addresses] was introduced with 4.3BSD to meet >>the requirements of DNS. >Now, here's the hard part - keeping in mind that while correlation >often implies causality it doesn't prove it, can you explain *why* the >introduction of a distributed mechanism would cause a change in the >information being provided? Read all RFCs or make a question in an appropriate newsgroup. >Is there a necessary relationship between >delivery mechanism and the content of the thing being delivered? >Comments in source code don't count - we're looking for understanding >and explanation, not rote repetition. Comments in source code do count. But if you need more evidence, you should investigate gethostbyname() of 4.3BSD by yourself. When /etc/hosts is searched, it dose not return multiple IP addresses. Thus, the capability to return multiple IP addresses are added because of DNS. >And, as some one else pointed out, this particular change would not >break existing binaries. Existing binaries won't work well without modification and recompilation. For example, if you are running old sendmail, it should try all IP addresses of a particular host. It is often the case that a gateway host shutdown one of its interface for a logn period. Then, if sendmail only tries one IP address of the unfunctioning interface, it can't deliver mail to the host. Masataka Ohta
cks@hawkwind.utcs.toronto.edu (Chris Siebenmann) (05/11/91)
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: | Comments in source code do count. But if you need more evidence, you | should investigate gethostbyname() of 4.3BSD by yourself. When /etc/hosts | is searched, it dose not return multiple IP addresses. | Thus, the capability to return multiple IP addresses are added because of | DNS. And this is a bug in 4.2BSD, rightfully corrected in 4.3 while they were changing it anyways. It just so happens that it doesn't usually matter which IP address a gethostname() returns, so few people cared about the breakage. For specific examples: - things which use hostnames as the permission mechanism, but compare internally against IP numbers; NFS mountds, careful rlogins, etc. If you rlogin to a machine on the "wrong" interface of a gateway machine, you're in trouble. - Things which add permissions based on IPs got from hostname lookups; X11R4 'xhost' for example. If you xhost + a gateway machine and are on the "wrong" interface, it won't work very well. Both problems have bitten us locally; they are quite real. -- "Summary: ftell chases cars and howls at the moon." - Larry Campbell cks@hawkwind.utcs.toronto.edu ...!{utgpu,utzoo,watmath}!utgpu!cks
sef@kithrup.COM (Sean Eric Fagan) (05/12/91)
In article <166@titccy.cc.titech.ac.jp> mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: >And shared libraries dose *NOT* help the transition from /etc/hosts to DNS. Uhm, yes, it does. For the original transition, no, you're right, it doesn't help, as you have to change the code in the application. But afterwards, when the applications will deal with multiple addresses, then consider this: if you're not going to run a nameserver, just get addresses out of /etc/hosts, then you don't *need* the nameserver code in your application(s), do you? It does increase the size, in case you weren't aware of it. If you have shared libraries, you can then just install the "DNS" version of the network libraries, and *poof*, you can then get elsewhere. The alternative is to have both in every program that tries to use networking, and this is a waste of both system memory and disk space. But I guess, as usual, you don't care about that. -- Sean Eric Fagan | "I made the universe, but please don't blame me for it; sef@kithrup.COM | I had a bellyache at the time." -----------------+ -- The Turtle (Stephen King, _It_) Any opinions expressed are my own, and generally unpopular with others.
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (05/13/91)
In article <1991May11.100712.27111@jarvis.csri.toronto.edu> cks@hawkwind.utcs.toronto.edu (Chris Siebenmann) writes: >| Thus, the capability to return multiple IP addresses are added because of >| DNS. > And this is a bug in 4.2BSD, rightfully corrected in 4.3 while they >were changing it anyways. It just so happens that it doesn't usually >matter which IP address a gethostname() returns, so few people cared >about the breakage. It is not a bug. There is trade off. If multiple IP addresses are represented by a single hostname, you can't use a hostname to specify a specific IP address. For example, you can't "ifconfig" with a symbolic hostname. If you insist on using symbolic name, you may use hostname alias, which is unique to each IP address. But, it can not be CNAME. So, it is not graceful. On the other hand, with multiple IP addresses, authentification is simplified, as I already stated. Your two authentification related examples: >- things which use hostnames as the permission mechanism, >- Things which add permissions based on IPs got from hostname lookups; > Both problems have bitten us locally; they are quite real. are not problems. It is avoided simply by registering all of hostnames in the authentification data. Masataka Ohta
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (05/13/91)
In article <1991May12.011253.35@kithrup.COM> sef@kithrup.COM (Sean Eric Fagan) writes: >>And shared libraries dose *NOT* help the transition from /etc/hosts to DNS. >Uhm, yes, it does. For the original transition, no, you're right, it >doesn't help, as you have to change the code in the application. So, my claim is confirmed. >But afterwards, when the applications will deal with multiple addresses, >then consider this: if you're not going to run a nameserver, just get >addresses out of /etc/hosts, then you don't *need* the nameserver code in >your application(s), do you? My claim (and the current subject) is >>And shared libraries dose *NOT* help the transition from /etc/hosts to DNS. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ You are talking about the reverse transition. And still, >The alternative is to have both in every program that tries to use >networking, and this is a waste of both system memory and disk space. I showed in my old posting with measurement that the consumed area is negligibly small. So, it should be noted that the measured system (SONY NEWSOS 4.0R) supports both /etc/hosts, DNS and NIS. >But I guess, as usual, you don't care about that. Based on the measurement, I don't care. Masataka Ohta
cks@hawkwind.utcs.toronto.edu (Chris Siebenmann) (05/16/91)
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: [About /etc/hosts-based IP lookups returning or not returning multiple IP addresses when such are listed for a given host in /etc/hosts:] | It is not a bug. There is trade off. If multiple IP addresses are | represented by a single hostname, you can't use a hostname to specify a | specific IP address. For example, you can't "ifconfig" with a symbolic | hostname. Perhaps I am handicapped by never having run a a real nameserver-only 4.3BSD system, but isn't using symbolic hostnames with a nameserver based ifconfig dangerous or fatal? Presumably your interfaces and IP addresses are not yet set at that point, which makes it hard to querry nameservers. You can always have aliases for specific interfaces; I don't think the need to use nameserver A records instead of CNAMEs is all that much of a hardship (or all that inelegant). | Your two authentification related examples: | >- things which use hostnames as the permission mechanism, | >- Things which add permissions based on IPs got from hostname lookups; | are not problems. It is avoided simply by registering all of hostnames | in the authentification data. Registering all of the hostnames in the authentication data seems to be equivalent (in this case) to requiring the users to name all the interfaces (either by name or by IPs) when adding such data. Perhaps you think this is a reasonable thing to require users to do; I don't. I would much rather our users be completely shielded from how the network looks today. -- "This Vi mode "feels" like Vi to me; it drives me nuts in the ways that I am used to Vi driving me nuts." - Brian Fox cks@hawkwind.utcs.toronto.edu ...!{utgpu,utzoo,watmath}!utgpu!cks
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (05/16/91)
In article <1991May15.222226.22708@jarvis.csri.toronto.edu> cks@hawkwind.utcs.toronto.edu (Chris Siebenmann) writes: My point is that gethostent of 4.2BSD is not so bad, definitely not bug. I am not claiming DNS is bad. OK? >| For example, you can't "ifconfig" with a symbolic >| hostname. >but isn't using symbolic hostnames with a nameserver >based ifconfig dangerous or fatal? Presumably your interfaces and IP >addresses are not yet set at that point, which makes it hard to querry >nameservers. Some machines allow host name look up of both local files and DNS, so, at boot time, symbolic ifconfig works because of local files and later... It is at least annoying and maybe dangerous but I don't think it fatal. >You can always have aliases for specific interfaces; I >don't think the need to use nameserver A records instead of CNAMEs is >all that much of a hardship (or all that inelegant). It is, I think, just as ungraceful as naming all interfaces for authentification. > Registering all of the hostnames in the authentication data seems to >be equivalent (in this case) to requiring the users to name all the >interfaces (either by name or by IPs) when adding such data. Yes, but not for users. >Perhaps >you think this is a reasonable thing to require users to do; I don't. I >would much rather our users be completely shielded from how the network >looks today. In general, there is not so many gateways. Addition of non-gateway hosts is much more frequent than the network topology change. Network topology change often accompanied by hosts addition. So... BTW, I don't think it is reasonable to require normal users to cope with subtle authentification. For normal users, relying on netgroups of NIS, which is maintained by system administrators, is the way to go. Users should not care minor hosts addition nor topology change. Masataka Ohta
cks@hawkwind.utcs.toronto.edu (Chris Siebenmann) (05/22/91)
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: ['| >' inclusion is by me.] | My point is that gethostent of 4.2BSD is not so bad, definitely not bug. | I am not claiming DNS is bad. Since I dispute your first statement and agree with your second, we seem to have a small problem (or not have a small problem as the case may be). | >You can always have aliases for specific interfaces; I don't think | >the need to use nameserver A records instead of CNAMEs is all that | >much of a hardship (or all that inelegant). | It is, I think, just as ungraceful as naming all interfaces for | authentification. Apart from ifconfig purposes, one almost never needs to name specific interfaces if one has well written software (and even less if you have smart software). Certainly I can't think of a case where ordinary users need to, unlike the authentification case with routines that only return one IP address. | > Registering all of the hostnames in the authentication data seems to | >be equivalent (in this case) to requiring the users to name all the | >interfaces (either by name or by IPs) when adding such data. | Yes, but not for users. There are, at least around here, unavoidable cases where users do the authentificiation setup and not the system administrator. As an example I quoted in the beginning, consider X11 with IP-based connection authentification. I as a user's system administrator have no idea what hosts he will want to authentificate, nor which of them are multi-homed. The user types xhost +neat.cs; xhost +bay.csri and tries to start up an X application on both. Depending on where on the network s/he happens to be sitting today, one fails and the other works, both work, or both fail. This because xhost is only finding (with a 4.2BSD gethostent()) the first listed IP address, and the user's X server may wind up receiving connections that originated from the second IP interfaces on both of those (multi-homed) hosts. At the same time, an 'xhost +snow.white' works fine, because snow.white is only a single-homed host ... today. That might change tomorrow. | In general, there is not so many gateways. Addition of non-gateway | hosts is much more frequent than the network topology change. Network | topology change often accompanied by hosts addition. So... [...] | For normal users, relying on netgroups of NIS, which is maintained by | system administrators, is the way to go. Users should not care minor | hosts addition nor topology change. On the other hand, network topology updates are often distributed, not centralized. I don't have any idea whether or not the Department of Statistics is splitting their network today, nor if this will affect any of my users or any of their users who're trying to use my machines. I maintain I shouldn't have to care. [Note that locally we usually automatically distribute an /etc/hosts around to various machines, via ftp and some scripts. Not having to manually update my /etc/hosts every time something changes is quite nice.] | BTW, I don't think it is reasonable to require normal users to cope with | subtle authentification. One can either make the system administrator try and cope with it instead, or give the user simple(r) authentification mechanisms. I think a gethostent() setup that returns multiple IP address for multi-homed hosts is "simpler" than one that returns just one when others exist. -- "This will be dynamically handled, possibly correctly, in 4.1." - Dan Davison on streams configuration in SunOS 4.0 cks@hawkwind.utcs.toronto.edu ...!{utgpu,utzoo,watmath}!utgpu!cks
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (05/22/91)
In article <1991May21.232345.8739@jarvis.csri.toronto.edu> cks@hawkwind.utcs.toronto.edu (Chris Siebenmann) writes: > Apart from ifconfig purposes, one almost never needs to name specific >interfaces if one has well written software (and even less if you have >smart software). Certainly I can't think of a case where ordinary >users need to, unlike the authentification case with routines that >only return one IP address. If some interface of a gateway is down, it is necessary to reach there by explicitely specifying its IP address or its name. Even if software is written properly, it is usually annoying to wait time-out of the first IP address. >Depending on where on >the network s/he happens to be sitting today, one fails and the other >works, both work, or both fail. The user should authentificate all of IP addresses, for simplicity. Anyway, X11 with IP-based connection authentification is not a real authentification. A little better autehtification is by .rhosts. > On the other hand, network topology updates are often distributed, >not centralized. Network topology updates affect routing and often cause network trouble. So, it should be informed to all related administrators. >I don't have any idea whether or not the Department >of Statistics is splitting their network today, nor if this will >affect any of my users or any of their users who're trying to use my >machines. Consider the case where you have your account in Department of Statistics and want to trust all workstations there. As the addition of non-gateway hosts is much more frequent and often distributed, it is convenient for users that authentification is done with netgroups of NIS. Masataka Ohta
cks@hawkwind.utcs.toronto.edu (Chris Siebenmann) (05/25/91)
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes: | If some interface of a gateway is down, it is necessary to reach there | by explicitely specifying its IP address or its name. | Even if software is written properly, it is usually annoying to wait | time-out of the first IP address. Locally, cases when one interface is down but the rest of the machine is up are vanishingly small. As a sysadmin, when I suspect that a particular interface has flaked out, I always use an interface-specific hostname or IP address, instead of relying on telnet fallbacks. I would claim that sufficiently much else breaks when an interface goes down that for ordinary users, a down interface is roughly equivalent to a down machine. | The user should authentificate all of IP addresses, for simplicity. | Anyway, X11 with IP-based connection authentification is not a real | authentification. Without a 4.3BSD style struct hostent, this requires the user to know all of the interfaces on a particular machine, which is not what I consider reasonable. I'm aware of the flaws in the host-based authentification for X, and use better mechanisms when possible, but most groups around here have yet to upgrade to X11R4 for various reasons. | A little better autehtification is by .rhosts. Use of .rhosts is impractical for X11 connections; for one thing, I may not have .rhosts entries although I want an X connection for that particular pair, either for organizational or technical reasons. With 4.2BSD style struct hostents, .rhosts-based authentification can get fouled up by an intelligent, nameserver-aware rlogind that does checking by forward-mapping from hostnames, instead of reverse mapping from IP addresses. You can say it's a bad thing to run such a daemon on non-DNS systems, but we have systems (like Ultrix) where one can configure which lookup mechanisms to use, and in what order, and the /etc/hosts based one only returns one IP address for a particular hostname while the DNS ones return multiple IP addresses. | > On the other hand, network topology updates are often distributed, | >not centralized. | Network topology updates affect routing and often cause network trouble. | So, it should be informed to all related administrators. Perhaps it often causes trouble in your organization; it doesn't in ours. Anything short of a major backbone reorganization tends to confine its effects to the groups or departments involved. I think this is a good thing, and something to strive for. Also, being informed of a change is somewhat less work than having to run around and fix various netgroups (or whatever) files, and making sure these propagate. | >I don't have any idea whether or not the Department | >of Statistics is splitting their network today, nor if this will | >affect any of my users or any of their users who're trying to use my | >machines. | Consider the case where you have your account in Department of Statistics | and want to trust all workstations there. The authentification and authorization mechanisms in things like rlogind need work, agreed. Perhaps something like Kerberos Realms is the right way to go. But it's an sepperate issue. In this specific case, I would re-run a script which generates my .rhosts file; the script includes all the machines it finds in /etc/hosts that are in specific domains. Inside the Dept. of Stats, they probably use /etc/hosts.equiv to let everyone cross-rlogin without needing to know about .rhosts. | As the addition of non-gateway hosts is much more frequent and often | distributed, it is convenient for users that authentification is done | with netgroups of NIS. We don't use NIS locally, considering it a vile swill of bugs, inefficiencies, and security holes; we tend to distribute around necessary information in other, often more efficient ways. Unfortunately, we'd like to use netgroups without running NIS, a feature most vendors have never heard of (NFS permissions is the big thing here; periodically we hack new mountd's up to do netgroups without NIS, a fairly minor change if you have source). -- "If the vendors started doing everything right, we would be out of a job. Let's hear it for OSI and X! With those babies in the wings, we can count on being employed until we drop, or get smart and switch to gardening, paper folding, or something." - C. Philip Wood cks@hawkwind.utcs.toronto.edu ...!{utgpu,utzoo,watmath}!utgpu!cks