tchrist@convex.COM (Tom Christiansen) (02/05/91)
From the keyboard of dhesi%cirrusl@oliveb.ATC.olivetti.com (Rahul Dhesi):
:In <5883:Feb102:05:4991@kramden.acf.nyu.edu>
:brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes:
:
:>Tradeoffs between ``multiple passes'' and ``single pass'' are entirely
:>irrelevant when they aren't reflected in speed, space, or human effort.
:
:It's often easier to adapt a single-pass program to handle
:previously-unforeseen needs than to similarly adapt a multipass
:program. A good example is what happens if you decide later that you
:would like to handle standard input. Now if you have a multipass
:implementation, you either have to completely redo it, or use a
:possibly huge temp file, or do buffering in blocks which may involve a
:heavy penalty in speed and memory usage.
Very true. Consider Henry Spencer's fiendishly clever "nroff -man in awk"
program (awf), to my mind the pinnacle of awk programming. He runs it in
three stages for various reasons, but this makes communication and feedback
between passes difficult at best. See his fascinating paper on ``Awk as a
Major System's Programming Language'' in the proceedings from last month's
Dallas USENIX in which he discusses this and a bunch of other issues.
It's a similar problem, by the way, to the one that occurs in streams like:
a | b | c | d | e | f | g | h | i > omega
It's a bummer when you realize that portions c and g need to talk
to each other.
It occurs to me that this discussion is about general programming, not
just C programming, so I've directed followups to comp.unix.programmer.
Of course, if you don't like this, edit your headers; for example, the
overly motivated might reasonably select alt.religion.computers. :-)
--tom
--
"Still waiting to read alt.fan.dan-bernstein using DBWM, Dan's own AI
window manager, which argues with you 10 weeks before resizing your window."
### And now for the question of the month: How do you spell relief? Answer:
U=brnstnd@kramden.acf.nyu.edu; echo "/From: $U/h:j" >>~/News/KILL; expire -f $Ubrnstnd@kramden.acf.nyu.edu (Dan Bernstein) (02/05/91)
In article <1991Feb04.161829.9385@convex.com> tchrist@convex.COM (Tom Christiansen) writes: > From the keyboard of dhesi%cirrusl@oliveb.ATC.olivetti.com (Rahul Dhesi): > :In <5883:Feb102:05:4991@kramden.acf.nyu.edu> > :brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: > :>Tradeoffs between ``multiple passes'' and ``single pass'' are entirely > :>irrelevant when they aren't reflected in speed, space, or human effort. > :It's often easier to adapt a single-pass program to handle > :previously-unforeseen needs than to similarly adapt a multipass > :program. Meta-comment: Rahul is saying that single-pass programs are better because they often reduce human effort. Doesn't this prove my point? What Rahul really cares about isn't the number of passes. What he cares about is human effort. > It's a similar problem, by the way, to the one that occurs in streams like: > a | b | c | d | e | f | g | h | i > omega > It's a bummer when you realize that portions c and g need to talk > to each other. It's a bummer when you don't have tools (like multitee) meant to handle this sort of thing. The UNIX pipeline model doesn't force you to use just one direction of data flow. > It occurs to me that this discussion is about general programming, not > just C programming, so I've directed followups to comp.unix.programmer. Hmmm. Shouldn't there be a comp.programming group for this type of discussion? It doesn't fit into any language group, it's too general for the IBM or UNIX (or games :-)) programming groups, and it's too mundane for comp.software-eng. ---Dan Stupidity, n.: An overwhelming desire to rewrite one-line shell scripts as 36-line Perl scripts so they run 6% faster. See Christiansen, Tommy.
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) (02/06/91)
In article <1991Feb6.051432.2846@zoo.toronto.edu> geoff@zoo.toronto.edu (Geoffrey Collyer) writes: > I didn't exclude the XXX comment from the pty code excerpt to hide > anything, it merely seemed redundant given the code that followed But people drew several wrong conclusions exactly because you took the code out of context. > From pty.1: > .SH BUGS > None known, but they're probably hiding somewhere. > Fascinating; this would seem to be at odds with the XXX comment. What do you mean? You can have an infinite amount of defensive programming, down to running a = b + c in triplicate and getting the majority-vote result. Yes, defensive programming is A Good Thing, but it's not required for a program to be correct. > I also don't consider that code excerpt to be at all atypical of pty; I > have read the entire pty source, and the excerpted code is fairly > typical, in fact it was picked at random. Geoff, maybe you believe what you're saying, but here are the facts. There are 154 (void) casts in the pty source, and every ignored return code is marked by a (void). 26 of those are in the section of code you quoted and in two sister sections, and as I've explained (and as is mentioned in the code) there should be more error checking there. I really doubt that you would have chosen any other sections, and I really find ``at random'' hard to believe. C'mon, Geoff, you've made a sweeping generalization about my code, and I say you're wrong until you provide some details. Here are the justifications I used when writing the source: 21 of the ignored return codes are for sprintf(), strcpy(), and strncpy(). Any complaints? 42 of those are for close(); 30 of the latter are upon descriptors that haven't been used for I/O. In 4 of the remaining cases it might make sense to report an error to the user upon failing close(); I am somewhat convinced by now that it is worth checking close() in those cases to deal with Sun's buggy network filesystem. Tally: 12 reasonable sources of complaint, 4 where I'm inclined to agree. 8 are for setreuid() between the current real and effective userids. Now there is absolutely no way they can fail, but as anyone who has the code can verify, I was so concerned about the chance of security holes that I made sure setreuid() worked before exec()ing a user program. Yes, Geoff, that's such poor programming practice I could just cry. Continuing on: 8 are for one-byte read(), one-byte write(), and fcntl(), all to a pipe which spends its entire lifetime internal to one process; I am tempted to say that those calls truly cannot fail, though I'm sure that Sun will manage to mess this up too. Tally: 20/4. 14 are for kill() upon processes that must exist for the subsystem to be internally consistent (such as the current pid). If, e.g., a renegade sysadmin kills one of the processes, the worst that happens is that the others block waiting for it. In 6 of those cases it's possible for an error to occur and it would make sense to report the error. Tally: 20/10. 3 are for similar internal communication; any error in these cases will actually be caught later. Tally: 23/10. 4 are for errors while printing an error message. 6 are for return codes that don't make sense as such: exit(), sleep(), execvp() (which always returns -1), etc. 11 are for u area modifications (like signal() upon a known signal number) that I really don't think will ever fail in any sane system. I may be wrong, but unless someone can propose a sensible action to take upon failure of signal(), I'm not going to check for it. Any complaints? 2 are for unlink()ing a bound UNIX-domain socket, in a section where error reporting is difficult. The unlink()s can only fail if someone's been futzing with the directory, and the worst that happens if they do fail is that a few garbage inodes get used up and cleaned up later. Tally: 25/10. 4 are for tty fchown()/fchmod() by root. In 2 of those cases an error report makes sense, and these days I'm inclined to report such things. Tally: 29/12. 4 are for restoring tty modes before printing an error message and dying, and the final one is for changing window size---another case where I'm not going to go crazy worrying about errors. (There's actually a bug related to one of the tty restores---in one of the situations where pty detects that the system is set up incorrectly---and I consider that bug a hell of a lot more important than whether I catch close() errors.) Tally: 29/12. Okay, Geoff. There are a few related, isolated sections of code where I really didn't catch errors; you just happened to pick one of those three sections to quote out of context. Now you say that those sections---with some 26 ignored return codes altogether---are typical of the entire program. Are you willing to go back through the code and think that through again? I simply don't believe that you can possibly give reasonable complaints about more than 29 of the other return codes, none disastrous and all perfectly justifiable. And remember that we're talking about a program with around 2000 calls altogether. Compared to ``professional'' Berkeley source, any code that checks practically every return code is what some people would call paranoid. > Given your cavalier disregard for errors and apparent incomprehension > of possible sources of error, Before this you were giving reasonable criticism, but now I'm insulted. Cavalier disregard for errors? Wtf are you talking about? Apparent incomprehension of possible sources of error? What higher plane did you come from? In my response to your article I began by noting the possibility of hard I/O errors; the same thing is in the comment above the code that you ripped out of context. I even mentioned the obscure possibility that a sysadmin is actively trying to destroy the internal communication of the program. Duh, you're so right, Geoff, it's perfectly apparent from my articles that I have complete faith in nothing ever going wrong. Hell, that's why I'm a syslog fanatic: security holes and reliability problems don't exist. > I don't think you are in any position to > criticise another's programming technique, let alone Ozan's. When someone says that a small number of passes is an important goal in and of itself, he's being silly. Unless reducing passes actually has a good effect (like reducing time, space, or human effort), nobody cares. As others have pointed out, you can make programs run just awfully by aiming for single-pass solutions. I think even the worst programmer would be justified in criticizing Ozan's statement about the number of passes as long as he doesn't make the mistake himself. > I shall > be polite here and refrain from any further comment on the programming > technique so stunningly displayed by pty. No, you've already stopped being polite. I once again invite anyone who wants to see the actual programming technique of pty---rather than its inaccurate renderings by Geoff---to look at the code for himself. > And I'm not going to argue this for ten weeks. But you're making statements that, for example, imply that pty ignores most return codes. That's simply wrong, and I hope you have the integrity to correct what you said. If I didn't have as much respect for you and your work, this article would be a lot shorter (and a hell of a lot less polite). Something like ``[Expletive] you, [expletive], you don't know what the [expletive] you're talking about. Except for sprintf(), close(), signal(), and strncpy(), pty checks for errors on 96% of its calls. Did you write any packages last May with 5500 lines of documentation and 4500 lines of code? How many errors did you check? How many years have you been a systems programmer?'' Surely you admit that this would be a natural response to someone who says ``These twenty lines of code [which don't check return codes] are typical of the entire package.'' Can you see how that would anger me? It's neither constructive nor accurate. It doesn't help me improve my code, and its main effect would be to damage my reputation. When I saw your first article, I assumed that you were just a tad annoyed by my comments about Ozan's new programming principle. I saw that you had selected one of the few remaining spots in the code which did very little error checking. Fair enough: I deserved the reminder. I didn't realize that the context you took away---the comment, and the variables, and the fact that the process doesn't have stderr---would give people such misimpressions about the code. But it did, and you were wrong to remove the context. Now you say that the section of code you quoted is ``fairly typical'' of the entire program. How so? In the sparsity of comments? There are comments throughout the code, and there's a big file with design notes on the structure of pty. No, you're saying that my program doesn't check its errors, and you're wrong about that. Sure, I'm glad that you had the time and interest to read the pty code. I'm sorry that I flamed Ozan---in the amoral, objectivist sense that I see more loss than gain in what I did, and I've made a mental note to do it more politely next time. I'm glad that you pointed out a section of code you don't like in pty. But all of these pale beside the spectacle of someone I respect giving an almost entirely inaccurate assessment of my biggest (and, I'd say, best) program last year. I can't stand by emotionlessly and watch you do this. I hope you'll at least send me a note saying that pty isn't the programming disaster you've made it out to be. > Don't you have a job or > thesis to keep you busy? The former---but at this time in the morning? Be serious. :-) Followups to the nonexistent group comp.programming---although this thread is somewhat related to both programming in C and programming in UNIX, it's really about more general issues. I don't expect followups from anyone except Geoff, and I assume he'll pick the right groups. ---Dan
tchrist@convex.COM (Tom Christiansen) (02/07/91)
From the keyboard of terryl@sail.LABS.TEK.COM:
: There's nothing I hate more than programs that silently fail, because I
:can't tell WHY they failed. Even just a simple "I can't do this because of that"
:is preferable (like perror(filename)). At least I can say "Aha, over quota;
:better delete some files." Also, don't tell me "Cannot open file", even if file
:is a real live filename. Tell me WHY you can't open it.
Well, perror(filename) is far better than nothing, but there two major
problems:
1) It's not enough information.
2) Sometimes you have no stderr.
Regarding problem 1, I like to see error messages in this form:
program: operations on object: reason
as in:
my_prog: cannot creat /foo/bar: permission denied
The my_prog is especially important in programs in pipelines. I once
grepped through all the system source (ok, it was BSD, but that doesn't
let the rest of the world off the hook) for things that said effectively:
printf(stderr, "cannot open: %s\n", filename);
and was horrified and angered. They all deserve fixing.
On problem #2, I think that if you have no stderr, (and even if you do
for system programs) you should also syslog the problem. It's not a
perfect solution, but it's surely better than nothing. I'm aware of
Dan's arguments about syslog not being perfect, such as: you can spoof
it, because you can get denial of service problems, because it's not
(indefinitely) extensible. While these may be all true, it sure beats
a boot to the head. I just hate system programs that don't syslog.
And you don't want to hear what I have to say about systems without
syslog, and a modern one at that. I simply don't use them.
As was mentioned earlier, *every* system call should *always*
be checked, even if you "know" it can't fail. Look in the classic
paper referenced earlier. One of its examples was from the init code;
something like:
open("/", 0);
dup2(0,1);
dup2(0,2);
These can all fail. I'd like the program to know it. I'd also
like it if it found some way to tell me, but at least it shouldn't
ignore them just because it can't think of something better to do.
I know people say not to check for errors that you don't know how to
handle. I think they're wrong. That's what asserts are all about.
I've spent more in the last 10 years tracking down program that don't
detect error conditions, or if they do, don't give me reasonable
messages, than I like to think about, all because some programmer
was too lazy or short-sighted to deal with the problem in an
adequate and informative manner.
Yes, I've attempted to move the thread again. I don't think it
belongs in BSD bugs anymore, although maybe Bostic et alii will
notice the BSD reference above and amend any remaining deficiencies.
My check was many, many releases ago -- I don't how bad recent
Berkeley code is at this.
--tom
--
"Still waiting to read alt.fan.dan-bernstein using DBWM, Dan's own AI window
manager, which argues with you for 10 weeks before resizing your window."
### And now for the question of the month: How do you spell relief? Answer:
U=brnstnd@kramden.acf.nyu.edu; echo "/From: $U/h:j" >>~/News/KILL; expire -f $Ubrnstnd@kramden.acf.nyu.edu (Dan Bernstein) (02/07/91)
In article <1991Feb07.013637.6542@convex.com> tchrist@convex.COM (Tom Christiansen) writes: > Regarding problem 1, I like to see error messages in this form: > program: operations on object: reason It's even better when you're given an error level: debug, info, warning, critical, fatal. > On problem #2, I think that if you have no stderr, (and even if you do > for system programs) you should also syslog the problem. It's not a > perfect solution, but it's surely better than nothing. Okay, you're right. In alt.sources is my first distributed program that actually makes some use of syslog. I'm still worried that people will think syslog is reliable or secure, and I'm very worried that someone may come to depend on syslog or believe that it isn't worth using better systems. But for the moment I guess it is the best alternative. > I know people say not to check for errors that you don't know how to > handle. I think they're wrong. That's what asserts are all about. Huh? An assertion is just one type of error handler, and a very clumsy one at that. Dumping core is rarely the right way to handle an error in a production program. ---Dan Stupidity, n.: An overwhelming desire to rewrite one-line shell scripts as 36-line Perl scripts so they run 6% faster. See Christiansen, Tommy.
tchrist@convex.COM (Tom Christiansen) (02/07/91)
From the keyboard of brnstnd@kramden.acf.nyu.edu (Dan Bernstein):
:In article <1991Feb07.013637.6542@convex.com> I wrote:
:> Regarding problem 1, I like to see error messages in this form:
:> program: operations on object: reason
:
:It's even better when you're given an error level: debug, info, warning,
:critical, fatal.
Gee, where did we get those from? :-)
:> On problem #2, I think that if you have no stderr, (and even if you do
:> for system programs) you should also syslog the problem. It's not a
:> perfect solution, but it's surely better than nothing.
:
:Okay, you're right. In alt.sources is my first distributed program that
:actually makes some use of syslog.
Congrats.
:I'm still worried that people will
:think syslog is reliable or secure, and I'm very worried that someone
:may come to depend on syslog or believe that it isn't worth using better
:systems. But for the moment I guess it is the best alternative.
Right.
:> I know people say not to check for errors that you don't know how to
:> handle. I think they're wrong. That's what asserts are all about.
:
:Huh? An assertion is just one type of error handler, and a very clumsy
:one at that. Dumping core is rarely the right way to handle an error in
:a production program.
Well, maybe so. But even the kernel does this sometimes:
panic: trap: unresolved kernel pte violation
(proceeds to dump to swap or some such)
Which to me is just a big "segmentation fault -- core dumped".
Maybe a core dump is too strong, but to keep running when your
internal assumptions have been proven wrong is just asking for
terrible trouble.
--tom
--
"Still waiting to read alt.fan.dan-bernstein using DBWM, Dan's own AI window
manager, which argues with you for 10 weeks before resizing your window."
### And now for the question of the month: How do you spell relief? Answer:
U=brnstnd@kramden.acf.nyu.edu; echo "/From: $U/h:j" >>~/News/KILL; expire -f $Utorek@h2opolo.ee.lbl.gov (Chris Torek) (02/07/91)
In article <1991Feb07.013637.6542@convex.com> tchrist@convex.COM (Tom Christiansen) writes: [sample error message] > my_prog: cannot creat /foo/bar: permission denied > >The my_prog is especially important in programs in pipelines. I once >grepped through all the system source (ok, it was BSD, but that doesn't >let the rest of the world off the hook) for things that said effectively: > > fprintf(stderr, "cannot open: %s\n", filename); > >and was horrified and angered. They all deserve fixing. >... Yes, I've attempted to move the thread again. I don't think it >belongs in BSD bugs anymore, although maybe Bostic et alii will >notice the BSD reference above and amend any remaining deficiencies. >My check was many, many releases ago -- I don't how bad recent >Berkeley code is at this. Many of these have been fixed. We% tend to agree that programs should include their name, the operation and parameters, and the error string; many more programs now say (void) fprintf(stderr, "foo: cannot open %s for reading: %s\n", file, strerror(errno)); instead of perror(file); (I find it rather appalling to see how much longer the `good' version is in a side by side, er, bottom by top, comparison like this. Still, it seems worth it.) ----- % `We' == myself and Keith Bostic, at least (though we differ on some details :-) ). I have only indirect influence on new BSD releases, but that is better than none.... -- In-Real-Life: Chris Torek, Lawrence Berkeley Lab EE div (+1 415 xxx xxxx) Berkeley, CA Domain: torek@ee.lbl.gov
mjr@hussar.dco.dec.com (Marcus J. Ranum) (02/08/91)
torek@ee.lbl.gov (Chris Torek) writes: >many more programs now say > > (void) fprintf(stderr, > "foo: cannot open %s for reading: %s\n", file, strerror(errno)); > >instead of > > perror(file); > >(I find it rather appalling to see how much longer the `good' version is >in a side by side, er, bottom by top, comparison like this. Still, it >seems worth it.) In the case of most of the code I've seen, I'm thrilled to death to see perror() being used. Fortunately, code like: if((fd = open("foo",flags,mode)) < 0) { printf("can't open file, errno is %d\n",errno); exit(1); } seems to be on its way out. mjr.
jim@segue.segue.com (Jim Balter) (02/08/91)
In article <9646@dog.ee.lbl.gov> torek@ee.lbl.gov (Chris Torek) writes: >(I find it rather appalling to see how much longer the `good' version is >in a side by side, er, bottom by top, comparison like this. Still, it >seems worth it.) Appalled? Are you a programmer or a wimp? Code it as PrintError(ProgramName, CANT_READ, file); or make ProgramName a global implicit argument. Sheesh.
allbery@NCoast.ORG (Brandon S. Allbery KB8JRR) (02/12/91)
As quoted from <1991Feb07.013637.6542@convex.com> by tchrist@convex.COM (Tom Christiansen): +--------------- | And you don't want to hear what I have to say about systems without | syslog, and a modern one at that. I simply don't use them. +--------------- So how many of these have you run in to, aside from Xenix? (Before you spout the eternal Berzerk b*tch, "System V", try RTFMing the System V manuals.) ++Brandon -- Me: Brandon S. Allbery VHF/UHF: KB8JRR on 220, 2m, 440 Internet: allbery@NCoast.ORG Packet: KB8JRR @ WA8BXN America OnLine: KB8JRR AMPR: KB8JRR.AmPR.ORG [44.70.4.88] uunet!usenet.ins.cwru.edu!ncoast!allbery Delphi: ALLBERY