chased@rbbb.Eng.Sun.COM (David Chase) (01/26/91)
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: >Now you can talk all you want about reallocating memory (btw, there's no >safe way to use realloc(), but you knew that) to read in as many users >as possible. I'll skip the comments about a quadratic time >requirement... I'll bite -- are you saying that reallocating memory to obtain a larger array and copying old into new always takes ("requires") quadratic time? (Quadratic time in what? I assume number of array elements.) I always double the size when I do this, by the way. 1 + 2 + 4 + 8 + ... + 2^N = 2 * (2^N) - 1, which is linear in 2^N, which is the number of array elements. But I'm sure you knew that, so you must have been talking about something else. What were you talking about? Inquiring minds want to know. David Chase Sun Microsystems
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) (01/29/91)
In article <6662@exodus.Eng.Sun.COM> chased@rbbb.Eng.Sun.COM (David Chase) writes: > I always double the size when I do this, by the way. Then you are using poor programming techniques. No program should take up half the memory of your machine when it could take up a fourth with hardly any effort on the programmer's part. Any single-pass ``users'' has to either use quadratic time or be a rude abuser of resources. Why not use two passes and be done with it? ---Dan
chased@rbbb.Eng.Sun.COM (David Chase) (01/30/91)
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: >Any single-pass ``users'' has to either use quadratic time or be a rude >abuser of resources. Why not use two passes and be done with it? No reason not to do that, and that's not what I was arguing (hard to imagine rudely abusing resources on a machine large enough to support enough users that the users command would rudely abuse resources, but so it goes). Maybe this is a sensible tradeoff on the machines that you use. However, you said that quadratic time was "required". No "or abuse resources" to qualify that statement. I'm interested in knowing how you arrived at this conclusion, since, in fact, it is false. I figure you'll find some way to redefine your way out of this blunder; just wanted to be sure that you did in it in public so that all could see that you aren't the infallible "mathematician" that you claim you are. NB -- doubling yields peak waste of 66% (N bytes in old, 2N in new, copy not yet made, only N useful bytes), not 50%. See? You didn't even get that right. Be more careful when you post, ok? Takes people's time to read these things. Yours, David Chase Sun Microsystems (and member of the Dan Bernstein USENET fan-the-flames club)
jfh@rpp386.cactus.org (John F Haugh II) (01/30/91)
In article <15325:Jan2903:19:4991@kramden.acf.nyu.edu> brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: >Any single-pass ``users'' has to either use quadratic time or be a rude >abuser of resources. Why not use two passes and be done with it? No, it doesn't. It can get it exactly right the first time and be done with it. fd = open ("/etc/utmp", O_RDONLY); fstat (fd, &statb); users = (struct utmp *) malloc (statb.st_size); read (fd, (char *) users, statb.st_size); This has certain problems, all of which Dan will point out, but any non-atomic operation has other "problems". I would argue that this has "less" problems because it is "more" atomic and far less complex. Besides, this is my posting, and quite clearly I am right about everything I post [ ;-)'s for the humor impaired only ] As for doubling realloc, given the large number of demand paged systems these days, doubling realloc often is a big win because the wasted pages are never touched, so they neither take physical memory nor backing store. Doubling realloc avoids the silly practice of gradually incrementing your memory up to the appropriate size. That is a sure performance sucker if ever there was one. -- John F. Haugh II UUCP: ...!cs.utexas.edu!rpp386!jfh Ma Bell: (512) 832-8832 Domain: jfh@rpp386.cactus.org "13 of 17 valedictorians in Boston High Schools last spring were immigrants or children of immigrants" -- US News and World Report, May 15, 1990
rang@cs.wisc.edu (Anton Rang) (01/30/91)
In article <15325:Jan2903:19:4991@kramden.acf.nyu.edu> brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: >Any single-pass ``users'' has to either use quadratic time or be a rude >abuser of resources. Why not use two passes and be done with it? Umm, are linked lists in disfavor in this group, or am I missing something obvious here? Anton +---------------------------+------------------+-------------+ | Anton Rang (grad student) | rang@cs.wisc.edu | UW--Madison | +---------------------------+------------------+-------------+
jef@well.sf.ca.us (Jef Poskanzer) (01/30/91)
In the referenced message, brnstnd@kramden.acf.nyu.edu (Dan Bernstein) wrote: }Any single-pass ``users'' has to either use quadratic time or be a rude }abuser of resources. Why not use two passes and be done with it? Well, no. There's at least one way of figuring out how much space to malloc without having to read utmp an extra time. --- Jef Jef Poskanzer jef@well.sf.ca.us {apple, ucbvax, hplabs}!well!jef "If you don't use Saber to develop your next C program, you're a dork." -- Brian Reid
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) (01/31/91)
In article <19010@rpp386.cactus.org> jfh@rpp386.cactus.org (John F Haugh II) writes: > In article <15325:Jan2903:19:4991@kramden.acf.nyu.edu> brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: > >Any single-pass ``users'' has to either use quadratic time or be a rude > >abuser of resources. Why not use two passes and be done with it? > No, it doesn't. It can get it exactly right the first time and be done > with it. No, your solution insists upon allocating space for empty entries in utmp. If utmp has ten thousand slots and there are only a thousand people logged in, you're using ten times too much space. Other than that it's fine. ---Dan
oz@yunexus.yorku.ca (Ozan Yigit) (01/31/91)
In article <15325:Jan2903:19:4991@kramden.acf.nyu.edu> brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: >Any single-pass ``users'' has to either use quadratic time or be a rude >abuser of resources. I think *unnecessary* multiple passes on disk files could best be described as "a rude abuse of resource". >Why not use two passes and be done with it? To rephrase: ... you are using poor programming techniques. No program should make multiple passes over a disk file when it could accomplish its task with a single pass with hardly any effort on the programmer's part. oz
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) (02/01/91)
In article <21548@yunexus.YorkU.CA> oz@yunexus.yorku.ca (Ozan Yigit) writes: > To rephrase: ... you are using poor programming techniques. No program > should make multiple passes over a disk file when it could accomplish its > task with a single pass with hardly any effort on the programmer's part. If you spent a minute putting your priorities into order you'd realize that this is an entirely ridiculous generalization. If two programs are just as fast, just as small, and take just as little programmer (and user) effort, then it simply doesn't matter whether one of them uses more passes than the other. To rephrase: You don't seem to know what good programming techniques are. Tradeoffs between speed, space, and human effort are important. Tradeoffs between ``multiple passes'' and ``single pass'' are entirely irrelevant when they aren't reflected in speed, space, or human effort. ---Dan
ok@goanna.cs.rmit.oz.au (Richard A. O'Keefe) (02/01/91)
In article <19010@rpp386.cactus.org>, jfh@rpp386.cactus.org (John F Haugh II) writes: > fd = open ("/etc/utmp", O_RDONLY); > fstat (fd, &statb); > users = (struct utmp *) malloc (statb.st_size); > read (fd, (char *) users, statb.st_size); That's what I used. /etc/utmp appears to be a fixed size file anyway, and the number of slots in that file doesn't seem to bear any obvious relation to the number of entries in /etc/ttys (other than being bigger). But I've been wondering whether a super-user editing /etc/ttys and then sending SIGHUP to 'init' might break this and what would be the best way to tell. If it was my own file, I could use locks. -- The Marxists have merely _interpreted_ Marxism in various ways; the point, however, is to _change_ it. -- R. Hochhuth.
dik@cwi.nl (Dik T. Winter) (02/02/91)
In article <5883:Feb102:05:4991@kramden.acf.nyu.edu> brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: > In article <21548@yunexus.YorkU.CA> oz@yunexus.yorku.ca (Ozan Yigit) writes: > > To rephrase: ... you are using poor programming techniques. No program > > should make multiple passes over a disk file when it could accomplish its > > task with a single pass with hardly any effort on the programmer's part. > > If you spent a minute putting your priorities into order you'd realize > that this is an entirely ridiculous generalization. If two programs are > just as fast, just as small, and take just as little programmer (and > user) effort, then it simply doesn't matter whether one of them uses > more passes than the other. > Ah, but that is the problem. Just as fast comes in flavors. You have CPU time, real time, IO time and some more. If one of the programs uses more passes it is not equally fast IO time wise (unless you are using SSD on a Cray of course). (Moreover, a program doing IO might well block another program that wants to do IO.) -- dik t. winter, cwi, amsterdam, nederland dik@cwi.nl
geoff@zoo.toronto.edu (Geoffrey Collyer) (02/02/91)
Dan Bernstein, in response to Ozan Yigit: > To rephrase: You don't seem to know what good programming techniques are. From pty/master.c: (void) lseek(fdsess,(long) 0,0); (void) read(fdsess,(char *) &newuid,sizeof(int)); (void) sprintf(newsuid,"%d",newuid); (void) chdir(".."); if (chdir(newsuid) == -1) { (void) mkdir(newsuid,0700); (void) chdir(newsuid); } (void) sprintf(foo,"../%d/%s",uid,fnsess); (void) rename(foo,fnsess); (void) sprintf(foo,"../%d/%s",uid,fnre); (void) rename(foo,fnre); /* in case we're already disconnected */ -- Geoff Collyer utzoo!geoff, zoo.toronto.edu!geoff
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) (02/02/91)
In article <1991Feb2.045119.22199@zoo.toronto.edu> geoff@zoo.toronto.edu (Geoffrey Collyer) writes: > Dan Bernstein, in response to Ozan Yigit: > > To rephrase: You don't seem to know what good programming techniques are. > From pty/master.c: Yes, what's your point? I have several hundred notes on pieces of pty that could be improved. In the list I find ``check for phys I/O errors on reading fdsess''; that's the only problem with the section of code you quoted. Are you worried about all the ignored return codes? I'll go through them one by one. Sure, I could've programmed more defensively, but the only possible response to a fatal error would be cutting off the user program from its input and output. > (void) lseek(fdsess,(long) 0,0); This is guaranteed to work. The third 0 could be changed to some symbolic constant (like L_SET), but as you know 0 is more portable. > (void) read(fdsess,(char *) &newuid,sizeof(int)); This is guaranteed to work, because fdsess is a regular file that was created with at least the int. The only possible problem is a hard I/O error or other fatal system error. > (void) sprintf(newsuid,"%d",newuid); This is guaranteed to work; newsuid has enough space. > (void) chdir(".."); This is guaranteed to work. > if (chdir(newsuid) == -1) > { > (void) mkdir(newsuid,0700); This cannot fail unless some renegade sysadmin changes the mode of the session directory while pty is running. 0700 could be made a symbolic constant too, but see above. > (void) chdir(newsuid); > } Cannot fail. > (void) sprintf(foo,"../%d/%s",uid,fnsess); > (void) rename(foo,fnsess); Guaranteed to work, as above. > (void) sprintf(foo,"../%d/%s",uid,fnre); > (void) rename(foo,fnre); /* in case we're already disconnected */ Ditto. ---Dan
jfh@rpp386.cactus.org (John F Haugh II) (02/02/91)
In article <1991Feb2.045119.22199@zoo.toronto.edu> geoff@zoo.toronto.edu (Geoffrey Collyer) writes: >Dan Bernstein, in response to Ozan Yigit: >> To rephrase: You don't seem to know what good programming techniques are. > >From pty/master.c: I'm sure it was very hard to write. Therefore, it should be hard to read, and hard to maintain, thus promoting job security. What exactly does it do? -- John F. Haugh II UUCP: ...!cs.utexas.edu!rpp386!jfh Ma Bell: (512) 832-8832 Domain: jfh@rpp386.cactus.org "I've never written a device driver, but I have written a device driver manual" -- Robert Hartman, IDE Corp.
dhesi%cirrusl@oliveb.ATC.olivetti.com (Rahul Dhesi) (02/03/91)
In <5883:Feb102:05:4991@kramden.acf.nyu.edu> brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: >Tradeoffs between ``multiple passes'' and ``single pass'' are entirely >irrelevant when they aren't reflected in speed, space, or human effort. It's often easier to adapt a single-pass program to handle previously-unforeseen needs than to similarly adapt a multipass program. A good example is what happens if you decide later that you would like to handle standard input. Now if you have a multipass implementation, you either have to completely redo it, or use a possibly huge temp file, or do buffering in blocks which may involve a heavy penalty in speed and memory usage. -- Rahul Dhesi <dhesi%cirrusl@oliveb.ATC.olivetti.com> UUCP: oliveb!cirrusl!dhesi
ok@goanna.cs.rmit.oz.au (Richard A. O'Keefe) (02/04/91)
In article <14994:Feb207:10:4791@kramden.acf.nyu.edu>, brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: > > if (chdir(newsuid) == -1) { > > (void) mkdir(newsuid,0700); > This cannot fail unless some renegade sysadmin changes the mode of the > session directory while pty is running. Er, what about the disc running out of I-nodes? Or going over quota? Or a file server dropping dead? > > (void) chdir(newsuid); > > } > Cannot fail. Can fail if the preceding mkdir() failed -- see above. There is *always* another way for things to go wrong. -- The Marxists have merely _interpreted_ Marxism in various ways; the point, however, is to _change_ it. -- R. Hochhuth.
oz@yunexus.yorku.ca (Ozan Yigit) (02/04/91)
In article <5883:Feb102:05:4991@kramden.acf.nyu.edu> (Dan Bernstein) burps: >In article <21548@yunexus.YorkU.CA> oz@yunexus.yorku.ca (Ozan Yigit) writes: >> To rephrase: ... you are using poor programming techniques. No program >> should make multiple passes over a disk file when it could accomplish its >> task with a single pass with hardly any effort on the programmer's part. > >If you spent a minute putting your priorities into order you'd realize >that this is an entirely ridiculous generalization. Well, I am sure you would know about ridiculous generalizations. Those were your own words plus a trivial meaning-preserving transformation. keep up the cyclically self-disputing silliness. oz
igb@fulcrum.bt.co.uk (Ian G Batten) (02/04/91)
In article <14994:Feb207:10:4791@kramden.acf.nyu.edu> brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: > > > if (chdir(newsuid) == -1) > > { > > (void) mkdir(newsuid,0700); > > This cannot fail unless some renegade sysadmin changes the mode of the Ah! I must go out and buy the patented ``Dan Bernstein never-fill'' file system, which has infinite data space and none of those nasty limits of the number of inodes. You can never, ever, in user space ensure that you have space to do IO as there will always be a timing window between your statfs (or whatever) and the actual IO. > > (void) chdir(".."); > > This is guaranteed to work. Really. Tell me about it. It was some utility --- I forget what --- screaming ``can't find ..'' which alerted me on the occasion that a renegade disk controller sprayed a data block accross a few directories on my news machine. Seeing that and rapidly shutting the machine down allowed me to rescue it.
chip@tct.uucp (Chip Salzenberg) (02/04/91)
According to brnstnd@kramden.acf.nyu.edu (Dan Bernstein): >> (void) read(fdsess,(char *) &newuid,sizeof(int)); > >This is guaranteed to work, because fdsess is a regular file that was >created with at least the int. The only possible problem is a hard I/O >error or other fatal system error. That particular "fatal" error -- disk failure -- will not kill the program. Instead, it will result in a pty program continuing to run using wrong data that could have been detected and ignored. This is "good programming practice?" >> if (chdir(newsuid) == -1) >> { >> (void) mkdir(newsuid,0700); > >This cannot fail unless some renegade sysadmin changes the mode of the >session directory while pty is running. Not to protect against that possibility is a strong temptation, but it certainly is not "good programming practice." >> (void) chdir(newsuid); >> } > >Cannot fail. Unless the mkdir() failed. -- Chip Salzenberg at Teltronics/TCT <chip@tct.uucp>, <uunet!pdn!tct!chip> "I want to mention that my opinions whether real or not are MY opinions." -- the inevitable William "Billy" Steinmetz
tchrist@convex.COM (Tom Christiansen) (02/05/91)
From the keyboard of oz@yunexus.yorku.ca (Ozan Yigit):
:In article <5883:Feb102:05:4991@kramden.acf.nyu.edu> (Dan Bernstein) burps:
:>In article <21548@yunexus.YorkU.CA> oz@yunexus.yorku.ca (Ozan Yigit) writes:
:>> To rephrase: ... you are using poor programming techniques. No program
:>> should make multiple passes over a disk file when it could accomplish its
:>> task with a single pass with hardly any effort on the programmer's part.
:>
:>If you spent a minute putting your priorities into order you'd realize
:>that this is an entirely ridiculous generalization.
:
:Well, I am sure you would know about ridiculous generalizations. Those
:were your own words plus a trivial meaning-preserving transformation.
:
:keep up the cyclically self-disputing silliness.
Well, that's what you get when you trade postings with Dan. It's
like playing patty cake with a schizoid King Kong.
--tom
--
"Still waiting to read alt.fan.dan-bernstein using DBWM, Dan's own AI
window manager, which argues with you 10 weeks before resizing your window."
### And now for the question of the month: How do you spell relief? Answer:
U=brnstnd@kramden.acf.nyu.edu; echo "/From: $U/h:j" >>~/News/KILL; expire -f $Utchrist@convex.COM (Tom Christiansen) (02/05/91)
From the keyboard of dhesi%cirrusl@oliveb.ATC.olivetti.com (Rahul Dhesi):
:In <5883:Feb102:05:4991@kramden.acf.nyu.edu>
:brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes:
:
:>Tradeoffs between ``multiple passes'' and ``single pass'' are entirely
:>irrelevant when they aren't reflected in speed, space, or human effort.
:
:It's often easier to adapt a single-pass program to handle
:previously-unforeseen needs than to similarly adapt a multipass
:program. A good example is what happens if you decide later that you
:would like to handle standard input. Now if you have a multipass
:implementation, you either have to completely redo it, or use a
:possibly huge temp file, or do buffering in blocks which may involve a
:heavy penalty in speed and memory usage.
Very true. Consider Henry Spencer's fiendishly clever "nroff -man in awk"
program (awf), to my mind the pinnacle of awk programming. He runs it in
three stages for various reasons, but this makes communication and feedback
between passes difficult at best. See his fascinating paper on ``Awk as a
Major System's Programming Language'' in the proceedings from last month's
Dallas USENIX in which he discusses this and a bunch of other issues.
It's a similar problem, by the way, to the one that occurs in streams like:
a | b | c | d | e | f | g | h | i > omega
It's a bummer when you realize that portions c and g need to talk
to each other.
It occurs to me that this discussion is about general programming, not
just C programming, so I've directed followups to comp.unix.programmer.
Of course, if you don't like this, edit your headers; for example, the
overly motivated might reasonably select alt.religion.computers. :-)
--tom
--
"Still waiting to read alt.fan.dan-bernstein using DBWM, Dan's own AI
window manager, which argues with you 10 weeks before resizing your window."
### And now for the question of the month: How do you spell relief? Answer:
U=brnstnd@kramden.acf.nyu.edu; echo "/From: $U/h:j" >>~/News/KILL; expire -f $Ubrnstnd@kramden.acf.nyu.edu (Dan Bernstein) (02/05/91)
Ya know, shouldn't there be a comp.programming group for programming discussions that aren't related to any particular language? We have IBM and UNIX and games programming groups, but no general comp.programming to be the practical side of comp.software-eng. I wonder. In article <21594@yunexus.YorkU.CA> oz@yunexus.yorku.ca (Ozan Yigit) writes: > >In article <21548@yunexus.YorkU.CA> oz@yunexus.yorku.ca (Ozan Yigit) writes: > >> No program > >> should make multiple passes over a disk file when it could accomplish its > >> task with a single pass with hardly any effort on the programmer's part. > Those > were your own words plus a trivial meaning-preserving transformation. Be serious. Time, space, and human effort are by far the most important three factors in programming. When you can get a noticeable improvement in one without noticeably hurting another, you should do so. That's what I've been saying all along. Now you bring in the fourth god of ``number of passes.'' Wtf are you talking about? The number of passes matters *only* in how it affects time, space, and human effort. Lots of examples bear this out. I still say that your generalization about the number of passes is ridiculous. It doesn't match anything I've ever said by a ``trivial meaning-preserving transformation,'' and I wish you would stop putting words into my mouth. ---Dan
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) (02/05/91)
In article <W~%&'N@uzi-9mm.fulcrum.bt.co.uk> igb@fulcrum.bt.co.uk (Ian G Batten) writes: > > > (void) chdir(".."); > > This is guaranteed to work. > Really. Tell me about it. I started the article with a blanket disclaimer saying that the code still had to handle hard I/O errors. Do I need to say ``other than the fatal errors mentioned above'' in every sentence so that you don't quote me out of context? Sheesh. ---Dan Stupidity, n.: An overwhelming desire to rewrite one-line shell scripts as 36-line Perl scripts so they run 6% faster. See Christiansen, Tommy.
ambar@ora.com (Jean Marie Diaz) (02/05/91)
From: brnstnd@kramden.acf.nyu.edu (Dan Bernstein) Date: 2 Feb 91 07:10:47 GMT > (void) read(fdsess,(char *) &newuid,sizeof(int)); This is guaranteed to work [...] > (void) chdir(".."); This is guaranteed to work. > (void) chdir(newsuid); > } Cannot fail. [and so forth] I commend to you a paper titled "Can't Happen, or /*NOTREACHED*/, or Real Programs Dump Core", by Ian Darwin & Geoff Collyer. It can be found in the proceedings of the Winter Usenix Conference, Dallas 1985, pages 136-151. Or FTP to cs.toronto.edu will get you the Postscript version under doc/programming/canthappen.PS. Since "anything that can go wrong will go wrong", it is a programmer's responsibility to deal with Murphy as gracefully as possible. Dumping core is never graceful. AMBAR
gwyn@smoke.brl.mil (Doug Gwyn) (02/05/91)
In article <29778:Feb419:26:3791@kramden.acf.nyu.edu> brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: >Now you bring in the fourth god of ``number of passes.'' Wtf are you >talking about? The number of passes matters *only* in how it affects >time, space, and human effort. Lots of examples bear this out. An /etc/utmp inspector could easily be foiled is it relies on the contents of /etc/utmp being unchanged between its first and second passes. It's bad enough not being able to lock the file for one pass, but that's probably easier to cope with than a phase error.
dkeisen@Gang-of-Four.Stanford.EDU (Dave Eisen) (02/05/91)
In article <1991Feb4.195805.16710@ora.com> ambar@ora.com (Jean Marie Diaz) writes: > >Since "anything that can go wrong will go wrong", it is a programmer's >responsibility to deal with Murphy as gracefully as possible. Dumping >core is never graceful. I disagree. There's a tradeoff between handling every single unlikely error condition imaginable and getting my next project started. It's a programmer's responsibility to figure out which error conditions merit careful handling and which ones do not. -- Dave Eisen "Une revolution qui demande que l'on se sacrifice 1447 N. Shoreline Blvd. pour elle est une revolution a la papa." Mountain View, CA 94043 --- French students' slogan, 1968. (415) 967-5644 dkeisen@Gang-of-Four.Stanford.EDU (for now)
oz@yunexus.yorku.ca (Ozan Yigit) (02/05/91)
In article <29778:Feb419:26:3791@kramden.acf.nyu.edu> Dan Bernstein writes: >Ya know, shouldn't there be a comp.programming group for programming >discussions that aren't related to any particular language? Why? So you can endlessly and vituperatively argue the good programming techniques within a 40-line C-boredom that core dumps when its required file is missing? >... and I wish you would stop putting words into my mouth. I couldn't possibly do that. Your foot is in there. oz
mike@BRL.MIL (Michael John Muuss <mike>) (02/05/91)
In article <14994:Feb207:10:4791@kramden.acf.nyu.edu>, brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: |> Are you worried about all the ignored return codes? I'll go through them |> one by one. Sure, I could've programmed more defensively, but the only |> possible response to a fatal error would be cutting off the user program |> from its input and output. |> |> Cannot fail. |> Guaranteed to work, as above. |> Ditto. |> |> ---Dan Dear Dan - I have seen every one of the system-calls you claim as "guaranteed to work" FAIL, in similar usage. (I've seen sprintf() fail too, but checking it's return code is problematic, due to the differences between BSD and SYSV implementations, so portability usually wins here). Filesystems filling up, memory failures, failing disk drives, corrupted program memory images, NFS outages and server malfunctions, etc, etc. ".." can unexpectedly vanish, or take you to unexpected places. ".." may not even be of type=directory! In my 17 years of using UNIX, I've seen it all. Not often, but not "never". "The problem with making programs foolproof is that the fools are so damned clever!" That goes double for hardware. Having programs not checking the error returns can make the job of troubleshooting and repairing malfunctions much more difficult. Yes, error checking can demand 3-4 times as much code as the actual task at hand. For essential systems programs, and criticial user applications, the error checking is well worth it. I don't think that I would consider the application under discussion "critical", so your (void)ing the error returns in this application was entirely appropriate. But, I didn't want you (or other readers of this list) to think that the sys-calls which you considered "guaranteed" to work actually would be, under all circumstances. They are not. Best, -Mike Muuss Advanced Computer Systems Ballistic Research Laboratory APG, MD 21005-5066
ksb@nostromo.cc.purdue.edu (Kevin Braunsdorf) (02/05/91)
In article <29778:Feb419:26:3791@kramden.acf.nyu.edu> brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: >Be serious. Time, space, and human effort are by far the most important >three factors in programming. Yeah, the time and space I have to spend tracking down sloppy code that: (a) has no error checks (b) doesn't identify itself (c) outputs the *wrong* error string... The *small* time it takes to type in if (0 != chdir(acTo)) { fprintf(stderr, "%s: chdir: %s: %s\n", progname, acTo, strerror(errno)); exit(9); } (which is a macro in vi, like: map! ^F fprintf(stderr, "%s: ;: %s: %s\n", progname, ;, strerror(errno));^[F; or: map! ^P fprintf(stderr, "%s: ;: %s\n", progname, strerror(errno));^[F; ) is better than either (void)chdir(acTo); or if (-1 == chdir(acTo)) { perror("chdir"); exit(1); } the second one is the one that really pisses me off. With a little more care the program could lead off with a *name* so long pipes wouldn't be so hard to debug under the UNIX shell. Yeah, Dan, the time I care about is not the first coder, it is the maintainer (most of the time is spent in maintaining a system), and the user writting his first pipeline (who thinks: chdir: Not a typewriter is not a great error message, and thinks all of UNIX is cryptic). -- "We'll crash and burn, it's hard to look away!!" kayessbee, Kevin Braunsdorf, ksb@cc.purdue.edu, pur-ee!ksb, purdu!ksb
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) (02/05/91)
Summary: Chip complains that a read() from disk into a ``newuid'' integer may fail and hence have pty switch to the wrong uid. But I explicitly initialized newuid to uid just to handle this possibility. Can people stop taking my code out of context? Fgs, right above the section that Geoff quoted is the comment /* XXX: We should have some error recovery here! */ I freely admit that the code isn't perfect. But I already have enough safeguards in place that I'm not worried about any huge disasters, and I don't need a dozen people telling me that my programming technique has problems when they haven't even bothered to read my code. In article <27AD7A46.343B@tct.uucp> chip@tct.uucp (Chip Salzenberg) writes: > >> (void) read(fdsess,(char *) &newuid,sizeof(int)); > That particular "fatal" error -- disk failure -- will not kill the > program. Instead, it will result in a pty program continuing to run > using wrong data that could have been detected and ignored. This is > "good programming practice?" So how would you like to handle it, Chip? Die with a fatal error, possibly killing a truly critical system program running under pty? You should already have known that the value read in---newuid---is initialized to uid. Now I know there are some wacko UNIX systems out there, but I've never seen an I/O error on the first block of a disk file that resulted in read() returning less than 4 bytes of that block. Either the entire block gets read in, or none of it does. (Sure, it's still worth testing for the error---if you can figure out a sane way to handle it.) So, Chip, all that will happen upon an I/O error in that section is that pty will continue with the same uid as it had before. If communication (or the disk) breaks down entirely then the signaller process will lose track of the master process, but it is practically impossible that the section of code you quoted will result in a wrong uid, even in the already disastrous situation that there are physical disk errors. You say ``it will result in a pty program continuing to run using wrong data''; I say that you don't know what you're talking about. I put that newuid = uid initialization in for a reason, and I am getting rather sick of people drawing incorrect conclusions about my code because Geoff took it out of context. Nobody's programming is perfect; the best I hope I can say is that I've gotten better over the years. I do think, however, that I have some degree of common sense about what can fail and what can't, and when I've already put a comment into my program saying ``We should test for foo'' it gets rather annoying to see a dozen people saying ``Dan, you don't use good programming technique, because you don't test for foo. Don't you realize what a programming criminal you are?'' Sheesh. Don't you read code before you criticize it? > >> if (chdir(newsuid) == -1) > >> (void) mkdir(newsuid,0700); > >This cannot fail unless some renegade sysadmin changes the mode of the > >session directory while pty is running. > Not to protect against that possibility is a strong temptation, but it > certainly is not "good programming practice." Well, golly gee, I suppose every program should encrypt all its internal data and throw away the key, just in case some renegade sysadmin is trying to corrupt its operation with ptrace()! Yeah! Chip, be reasonable. You can't demand of system programs that they check for external system consistency at every step. Defensive programming is fine, but screwing around with correct code just because ``a sysadmin might go out of his way to damage the internal state of the system'' can be pretty stupid. ---Dan
flint@gistdev.gist.com (Flint Pellett) (02/06/91)
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: >Summary: Chip complains that a read() from disk into a ``newuid'' >integer may fail and hence have pty switch to the wrong uid. But I >explicitly initialized newuid to uid just to handle this possibility. >Can people stop taking my code out of context? ... >So, Chip, all that will happen upon an I/O error in that section is that >pty will continue with the same uid as it had before. If communication >(or the disk) breaks down entirely then the signaller process will lose >track of the master process, but it is practically impossible that the >section of code you quoted will result in a wrong uid, even in the >already disastrous situation that there are physical disk errors. You >say ``it will result in a pty program continuing to run using wrong >data''; I say that you don't know what you're talking about. >I put that newuid = uid initialization in for a reason... I haven't followed this entire thread, but I think I've seen enough of it to make a couple general observations that may or may not apply directly here, which other people might be able to learn from. 1. If you post any code on usenet, expect it to be criticized. If you can't handle lots of criticism, don't post the code. 2. It would be nice, if people criticizing code did so as if were a professional code review. In the first place, remember that in a code review it is the code that is being criticized, NOT the programmer. There is a difference between someone "being wrong on this point" and "not knowing what they are talking about". If you don't have any respect for the person whose code you are criticizing, my suggestion is to not bother doing it: you aren't going to help them or yourself. 3. Regarding the code: I didn't see the entire example code, but from the ensuing discussion, I conclude that there was a problem in the comments in the code. It was stated that the initialization was put in "for a reason". If the comments were complete, that would have been obvious, and in fact that whole paragraph that appears above explaining what the reasons were, and the reasons why the error conditions are being ignored, should have been there as a comment, even if the result was 15 lines of comment for 1 line of code. When code is commented like that, then a maintainer who comes along trying to get/keep the code working in a slightly different environment can quickly determine if the "reason" for the check is still valid, instead of having to spend who knows how much time trying to figure out what the reason for it was. 4. A good maintainer (or criticizer) always assumes the original author did things for reasons. If you assume that the original author was just stupid and didn't know what they were talking about, it may save you the time of figuring out what the reason for having a check was, making it ok to remove it or alter it. Then you "test" it, it works ok, and you can go on your way, leaving a time-bomb sitting around waiting to blow up when the reason for the test you screwed up becomes apparent.) -- Flint Pellett, Global Information Systems Technology, Inc. 1800 Woodfield Drive, Savoy, IL 61874 (217) 352-1165 uunet!gistdev!flint or flint@gistdev.gist.com
terryl@sail.LABS.TEK.COM (02/06/91)
In article <14994:Feb207:10:4791@kramden.acf.nyu.edu> brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: >In article <1991Feb2.045119.22199@zoo.toronto.edu> geoff@zoo.toronto.edu (Geoffrey Collyer) writes: >> Dan Bernstein, in response to Ozan Yigit: >> > To rephrase: You don't seem to know what good programming techniques are. >> From pty/master.c: > >Yes, what's your point? I have several hundred notes on pieces of pty >that could be improved. In the list I find ``check for phys I/O errors >on reading fdsess''; that's the only problem with the section of code >you quoted. (Delete MANY lines of him defending his code) Well, I have no idea what his point was, but I have one: If I were your manager, I'd ding you REAL hard for writing semi-obscure code with such a dearth of comments. Now, if he deleted them, then fine, but since I didn't see you post any in your defense, I'll have to assume that they don't exist. __________________________________________________________ Terry Laskodi "There's a permanent crease of in your right and wrong." Tektronix Sly and the Family Stone, "Stand!" __________________________________________________________
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) (02/06/91)
In article <1123@gistdev.gist.com> flint@gistdev.gist.com (Flint Pellett) writes: > 1. If you post any code on usenet, expect it to be criticized. You're right, I overreacted. Sorry. I was hoping for some constructive criticism, though, and it doesn't help to be told something that I had already noted in the code. > 2. It would be nice, if people criticizing code did so as if were a > professional code review. Yes. Among other things, please check the original source, and when it has a comment like /* XXX: We should have some error recovery here! */ then assume that the author does realize that the code can be improved. > 3. Regarding the code: I didn't see the entire example code, but from > the ensuing discussion, I conclude that there was a problem in the > comments in the code. It was stated that the initialization was put > in "for a reason". If the comments were complete, that would have > been obvious, Agreed, and I think that the comment makes it clear enough in context. Geoff, why did you feel compelled to take the code out of context? > 4. A good maintainer (or criticizer) always assumes the original > author did things for reasons. Although in practice most things are done for no reason at all. :-) ---Dan
greywolf@unisoft.UUCP (The Grey Wolf) (02/06/91)
In article <5883:Feb102:05:4991@kramden.acf.nyu.edu> brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: > >To rephrase: You don't seem to know what good programming techniques >are. Tradeoffs between speed, space, and human effort are important. >Tradeoffs between ``multiple passes'' and ``single pass'' are entirely >irrelevant when they aren't reflected in speed, space, or human effort. As I'm sure someone will/has pointed out already (haven't seen it yet, but then news has strange timing problems), multiple-pass contemplation of a file is stable IFF the file is guaranteed not to be physically rearranged between passes. (Of course the same can be said of a single pass thru a file, but the window is (usually) *much* smaller). Examples where multiple passes work is on (normally) stable files, such as running them thru a compiler or running a disk thru fsck. (Of course, even there, anomalies happen...) Side issue: -- has anyone managed a single-pass fsck? That would be tricky at best, possibly disastrous. I thought for a moment that instead of doing five passes, one could use five file descriptors to handle each phase, seeing as the size of a disk partition is effectively fixed (last I checked, anyway...). But I guess the passes need to happen in a particular order... > >---Dan -- thought: I ain't so damb dumn! | Your brand new kernel just dump core on you war: Invalid argument | And fsck can't find root inode 2 | Don't worry -- be happy... ...!{ucbvax,acad,uunet,amdahl,pyramid}!unisoft!greywolf
geoff@zoo.toronto.edu (Geoffrey Collyer) (02/06/91)
Dan, I didn't exclude the XXX comment from the pty code excerpt to hide anything, it merely seemed redundant given the code that followed (surely all those (void) casts are a reminder that this code is incomplete and needs to be finished; to intend otherwise would be unprofessional). From pty.1: .SH BUGS None known, but they're probably hiding somewhere. Fascinating; this would seem to be at odds with the XXX comment. I also don't consider that code excerpt to be at all atypical of pty; I have read the entire pty source, and the excerpted code is fairly typical, in fact it was picked at random. Given your cavalier disregard for errors and apparent incomprehension of possible sources of error, I don't think you are in any position to criticise another's programming technique, let alone Ozan's. I shall be polite here and refrain from any further comment on the programming technique so stunningly displayed by pty. And I'm not going to argue this for ten weeks. Don't you have a job or thesis to keep you busy? -- Geoff Collyer utzoo!geoff, zoo.toronto.edu!geoff
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) (02/06/91)
In article <1991Feb6.051432.2846@zoo.toronto.edu> geoff@zoo.toronto.edu (Geoffrey Collyer) writes: > I didn't exclude the XXX comment from the pty code excerpt to hide > anything, it merely seemed redundant given the code that followed But people drew several wrong conclusions exactly because you took the code out of context. > From pty.1: > .SH BUGS > None known, but they're probably hiding somewhere. > Fascinating; this would seem to be at odds with the XXX comment. What do you mean? You can have an infinite amount of defensive programming, down to running a = b + c in triplicate and getting the majority-vote result. Yes, defensive programming is A Good Thing, but it's not required for a program to be correct. > I also don't consider that code excerpt to be at all atypical of pty; I > have read the entire pty source, and the excerpted code is fairly > typical, in fact it was picked at random. Geoff, maybe you believe what you're saying, but here are the facts. There are 154 (void) casts in the pty source, and every ignored return code is marked by a (void). 26 of those are in the section of code you quoted and in two sister sections, and as I've explained (and as is mentioned in the code) there should be more error checking there. I really doubt that you would have chosen any other sections, and I really find ``at random'' hard to believe. C'mon, Geoff, you've made a sweeping generalization about my code, and I say you're wrong until you provide some details. Here are the justifications I used when writing the source: 21 of the ignored return codes are for sprintf(), strcpy(), and strncpy(). Any complaints? 42 of those are for close(); 30 of the latter are upon descriptors that haven't been used for I/O. In 4 of the remaining cases it might make sense to report an error to the user upon failing close(); I am somewhat convinced by now that it is worth checking close() in those cases to deal with Sun's buggy network filesystem. Tally: 12 reasonable sources of complaint, 4 where I'm inclined to agree. 8 are for setreuid() between the current real and effective userids. Now there is absolutely no way they can fail, but as anyone who has the code can verify, I was so concerned about the chance of security holes that I made sure setreuid() worked before exec()ing a user program. Yes, Geoff, that's such poor programming practice I could just cry. Continuing on: 8 are for one-byte read(), one-byte write(), and fcntl(), all to a pipe which spends its entire lifetime internal to one process; I am tempted to say that those calls truly cannot fail, though I'm sure that Sun will manage to mess this up too. Tally: 20/4. 14 are for kill() upon processes that must exist for the subsystem to be internally consistent (such as the current pid). If, e.g., a renegade sysadmin kills one of the processes, the worst that happens is that the others block waiting for it. In 6 of those cases it's possible for an error to occur and it would make sense to report the error. Tally: 20/10. 3 are for similar internal communication; any error in these cases will actually be caught later. Tally: 23/10. 4 are for errors while printing an error message. 6 are for return codes that don't make sense as such: exit(), sleep(), execvp() (which always returns -1), etc. 11 are for u area modifications (like signal() upon a known signal number) that I really don't think will ever fail in any sane system. I may be wrong, but unless someone can propose a sensible action to take upon failure of signal(), I'm not going to check for it. Any complaints? 2 are for unlink()ing a bound UNIX-domain socket, in a section where error reporting is difficult. The unlink()s can only fail if someone's been futzing with the directory, and the worst that happens if they do fail is that a few garbage inodes get used up and cleaned up later. Tally: 25/10. 4 are for tty fchown()/fchmod() by root. In 2 of those cases an error report makes sense, and these days I'm inclined to report such things. Tally: 29/12. 4 are for restoring tty modes before printing an error message and dying, and the final one is for changing window size---another case where I'm not going to go crazy worrying about errors. (There's actually a bug related to one of the tty restores---in one of the situations where pty detects that the system is set up incorrectly---and I consider that bug a hell of a lot more important than whether I catch close() errors.) Tally: 29/12. Okay, Geoff. There are a few related, isolated sections of code where I really didn't catch errors; you just happened to pick one of those three sections to quote out of context. Now you say that those sections---with some 26 ignored return codes altogether---are typical of the entire program. Are you willing to go back through the code and think that through again? I simply don't believe that you can possibly give reasonable complaints about more than 29 of the other return codes, none disastrous and all perfectly justifiable. And remember that we're talking about a program with around 2000 calls altogether. Compared to ``professional'' Berkeley source, any code that checks practically every return code is what some people would call paranoid. > Given your cavalier disregard for errors and apparent incomprehension > of possible sources of error, Before this you were giving reasonable criticism, but now I'm insulted. Cavalier disregard for errors? Wtf are you talking about? Apparent incomprehension of possible sources of error? What higher plane did you come from? In my response to your article I began by noting the possibility of hard I/O errors; the same thing is in the comment above the code that you ripped out of context. I even mentioned the obscure possibility that a sysadmin is actively trying to destroy the internal communication of the program. Duh, you're so right, Geoff, it's perfectly apparent from my articles that I have complete faith in nothing ever going wrong. Hell, that's why I'm a syslog fanatic: security holes and reliability problems don't exist. > I don't think you are in any position to > criticise another's programming technique, let alone Ozan's. When someone says that a small number of passes is an important goal in and of itself, he's being silly. Unless reducing passes actually has a good effect (like reducing time, space, or human effort), nobody cares. As others have pointed out, you can make programs run just awfully by aiming for single-pass solutions. I think even the worst programmer would be justified in criticizing Ozan's statement about the number of passes as long as he doesn't make the mistake himself. > I shall > be polite here and refrain from any further comment on the programming > technique so stunningly displayed by pty. No, you've already stopped being polite. I once again invite anyone who wants to see the actual programming technique of pty---rather than its inaccurate renderings by Geoff---to look at the code for himself. > And I'm not going to argue this for ten weeks. But you're making statements that, for example, imply that pty ignores most return codes. That's simply wrong, and I hope you have the integrity to correct what you said. If I didn't have as much respect for you and your work, this article would be a lot shorter (and a hell of a lot less polite). Something like ``[Expletive] you, [expletive], you don't know what the [expletive] you're talking about. Except for sprintf(), close(), signal(), and strncpy(), pty checks for errors on 96% of its calls. Did you write any packages last May with 5500 lines of documentation and 4500 lines of code? How many errors did you check? How many years have you been a systems programmer?'' Surely you admit that this would be a natural response to someone who says ``These twenty lines of code [which don't check return codes] are typical of the entire package.'' Can you see how that would anger me? It's neither constructive nor accurate. It doesn't help me improve my code, and its main effect would be to damage my reputation. When I saw your first article, I assumed that you were just a tad annoyed by my comments about Ozan's new programming principle. I saw that you had selected one of the few remaining spots in the code which did very little error checking. Fair enough: I deserved the reminder. I didn't realize that the context you took away---the comment, and the variables, and the fact that the process doesn't have stderr---would give people such misimpressions about the code. But it did, and you were wrong to remove the context. Now you say that the section of code you quoted is ``fairly typical'' of the entire program. How so? In the sparsity of comments? There are comments throughout the code, and there's a big file with design notes on the structure of pty. No, you're saying that my program doesn't check its errors, and you're wrong about that. Sure, I'm glad that you had the time and interest to read the pty code. I'm sorry that I flamed Ozan---in the amoral, objectivist sense that I see more loss than gain in what I did, and I've made a mental note to do it more politely next time. I'm glad that you pointed out a section of code you don't like in pty. But all of these pale beside the spectacle of someone I respect giving an almost entirely inaccurate assessment of my biggest (and, I'd say, best) program last year. I can't stand by emotionlessly and watch you do this. I hope you'll at least send me a note saying that pty isn't the programming disaster you've made it out to be. > Don't you have a job or > thesis to keep you busy? The former---but at this time in the morning? Be serious. :-) Followups to the nonexistent group comp.programming---although this thread is somewhat related to both programming in C and programming in UNIX, it's really about more general issues. I don't expect followups from anyone except Geoff, and I assume he'll pick the right groups. ---Dan
boyd@necisa.ho.necisa.oz.au (Boyd Roberts) (02/07/91)
Come on Dan, I've told you before. Check those error conditions. Weird things happen. Just chmod 0 /dev/tty and watch your UNIX machine go absolutely ga-ga on you, and _none_ of tools that break will say anything of any worth about the problem. Presumably because they were written with the philosophy that states `X will ever happen'. And I thought it was warm here. There's nothing quite like a Dan flame-fest. Boyd Roberts boyd@necisa.ho.necisa.oz.au ``When the going gets wierd, the weird turn pro...''
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) (02/07/91)
In article <2013@necisa.ho.necisa.oz.au> boyd@necisa.ho.necisa.oz.au (Boyd Roberts) writes: > Come on Dan, I've told you before. Check those error conditions. Boyd, I *am* checking those error conditions. There are a few isolated sections of code in pty (one being the section Geoff quoted) where errors that might be handled are ignored. I realize that this 4% of the code needs to be redone to handle errors better. How do you think it comes across, Boyd, when a programmer puts a comment into some code saying ``this needs foo''; someone quotes the code without the comment; and then you pop in and say ``Come on, I've told you before. You need foo.'' I think it casts some doubt on your academic standards: you haven't bothered to read code before you criticize it, even in public. > Weird things happen. Just chmod 0 /dev/tty and watch your UNIX > machine go absolutely ga-ga on you, and _none_ of tools that break > will say anything of any worth about the problem. pty: fatal: cannot reopen pty for /dev/tty: Permission denied. Boyd, fuck off. Or learn to read the code you criticize. Your choice. ---Dan
stripes@eng.umd.edu (Joshua Osborne) (02/08/91)
In article <1991Feb4.195805.16710@ora.com>, ambar@ora.com (Jean Marie Diaz) writes: [...] > Since "anything that can go wrong will go wrong", it is a programmer's > responsibility to deal with Murphy as gracefully as possible. Dumping > core is never graceful. It isn't normally graceful, but in bizzare failure cases I *want* a core dump so I can see what went on to get me to the "can't happen" default in a switch (I want to know more then just the improper switch control variable). That's why my code has a ton of asserts, or for more end-user style code a assert-like call "make_core(int expr, char *reason)" which prints some standard message, the reason string, dumps some stuff in a file, dumps a core the tells the user to save the core & the error file, and who to mail or phone... (the core file isn't kept with that name, too many autodeleters nail core files...) Of corse more normal error conditions get more normal messages! I have redirected follow-ups to me because I don't know what newsgroup generic programing flames/tips should go to. Hopefully I did this right, I couldn't find it in the manpages, so I'll have to rely on my faulty memmory... -- stripes@eng.umd.edu "Security for Unix is like Josh_Osborne@Real_World,The Multitasking for MS-DOS" "The dyslexic porgramer" - Kevin Lockwood "CNN is the only nuclear capable news network..." - lbruck@eng.umd.edu (Lewis Bruck)
dbc@cimage.com (David Caswell) (02/09/91)
.From pty/master.c: . . (void) lseek(fdsess,(long) 0,0); . (void) read(fdsess,(char *) &newuid,sizeof(int)); . (void) sprintf(newsuid,"%d",newuid); . should be (void) lseek(fdsess,0L,0); saves some keystokes, after all what else in the world is important?