arnold%audiofax.com@mathcs.emory.edu (Arnold Robbins) (10/22/90)
Submitted-by: arnold%audiofax.com@mathcs.emory.edu (Arnold Robbins) (It's about time the subject line on this one got changed, don't you think?) [Seems plausible to me. -mod] Anyway, since we're discussing what is and isn't in the POSIX name space, I'd like to put in a plug for the /dev/fd directory. Opening /dev/fd/7 is equivalent to doing a dup(7); it is a generalization of the "treat '-' as stdin" hack used by cat and awk (and others) and allows at least two shells (ksh and rc [see your nearest V10 manual]) to do interesting things like set up non-linear pipelines. (At least I think rc does it. I know ksh does.) There's lot of existing practice on this one; it originated in V8, circa 1984 or earlier, and PD versions for various, more popular, Unix incarnations have been around for some time as well. (In fact, in V8 - V10, /dev/stdin, /dev/stdout, /dev/stderr, and /dev/tty are links to /dev/fd/0, /dev/fd/1, /dev/fd/2, and /dev/fd/3, respectively. The last, in particular, is a nice generalization, and eliminates an ugly special case in the kernel; init just does one more dup.) It's going to be fun watching how /dev/fd will be presented as both for and against the case for "fd-centric" Unix... :-) Personally, I'm in the put-it- in-the-filesystem camp. -- Arnold Robbins AudioFAX, Inc. | Laundry increases 2000 Powers Ferry Road, #200 / Marietta, GA. 30067 | exponentially in the INTERNET: arnold@audiofax.com Phone: +1 404 933 7612 | number of children. UUCP: emory!audfax!arnold Fax-box: +1 404 618 4581 | -- Miriam Robbins Volume-Number: Volume 21, Number 209
chet@cwns1.INS.CWRU.Edu (Chet Ramey) (10/25/90)
Submitted-by: chet@cwns1.INS.CWRU.Edu (Chet Ramey) Arnold Robbins writes: >Anyway, since we're discussing what is and isn't in the POSIX name space, >I'd like to put in a plug for the /dev/fd directory. I agree; it makes things like join <(prog1) <(prog2) > joined-output-of-progs-1-and-2 possible. >There's lot of existing practice on this one; it originated in V8, circa >1984 or earlier, and PD versions for various, more popular, Unix incarnations >have been around for some time as well. Keith Bostic has stated that /dev/fd will be in the next release of BSD; for all I know, it might already be in 4.3-reno. Chet -- Chet Ramey ``As I recall, Doug was keen on boxing. But Network Services Group when he learned to walk, he took up puttin' Case Western Reserve University the boot in the groin.'' chet@ins.CWRU.Edu Volume-Number: Volume 22, Number 2
craig@b11.ingr.com (Craig Presson) (10/25/90)
Submitted-by: craig@b11.ingr.com (Craig Presson) In article <13878@cs.utexas.edu>, arnold%audiofax.com@mathcs.emory.edu (Arnold Robbins) writes: |> Submitted-by: arnold%audiofax.com@mathcs.emory.edu (Arnold Robbins) |> |> Anyway, since we're discussing what is and isn't in the POSIX name space, |> I'd like to put in a plug for the /dev/fd directory. Opening /dev/fd/7 is |> equivalent to doing a dup(7); it is a generalization of the "treat '-' as |> stdin" hack used by cat and awk (and others) and allows at least two shells |> (ksh and rc [see your nearest V10 manual]) to do interesting things like set |> up non-linear pipelines. (At least I think rc does it. I know ksh does.) |> |> There's lot of existing practice on this one; it originated in V8, circa |> 1984 or earlier, and PD versions for various, more popular, Unix incarnations |> have been around for some time as well. |> |> (In fact, in V8 - V10, /dev/stdin, /dev/stdout, /dev/stderr, and /dev/tty are |> links to /dev/fd/0, /dev/fd/1, /dev/fd/2, and /dev/fd/3, respectively. The |> last, in particular, is a nice generalization, and eliminates an ugly special |> case in the kernel; init just does one more dup.) |> |> It's going to be fun watching how /dev/fd will be presented as both for and |> against the case for "fd-centric" Unix... :-) Personally, I'm in the put-it- |> in-the-filesystem camp. |> -- |> Arnold Robbins AudioFAX, Inc. | Laundry increases Ah, roger, that's a big "ditto" on the virtues of One Big Namespace for all Permanent Objects. Use subspaces to separate classes (he said tautologically) *. But for those of us without access to every Unix manual ever published (I do have a Version 7 Volume 1), could you fill in a bit more on the semantics of this hybrid /dev entry? Like what do you get when you open "/dev/fd/7" and there is no open file using that slot? Does the system make these entries "invisible" to processes not using them? Do you just get a classic "It's an error from Unix, you're not supposed to understand" type return? Or am I Missing Something? -- ****************************************************** ** Craig Presson pressonc@ingr.com ** ** Intergraph Corporation MS CR1104 ** ** Huntsville, AL 35894-0001 (205) 730-6176 ** ** FAX: (205) 730-6011 ** ****************************************************** * Those not old enough to remember "Tom Swifties" are encouraged to forgive my lapse of taste ... Volume-Number: Volume 22, Number 3
addw@phcomp.co.uk (Alain Williams) (10/25/90)
Submitted-by: addw@phcomp.co.uk (Alain Williams) > Anyway, since we're discussing what is and isn't in the POSIX name space, > I'd like to put in a plug for the /dev/fd directory. Opening /dev/fd/7 is > equivalent to doing a dup(7); it is a generalization of the "treat '-' as What happens if you do an ``ls -l'' on /dev/fd, do you see the fds which are open to the ls program or all possible fds, even those which aren't opened ? > (In fact, in V8 - V10, /dev/stdin, /dev/stdout, /dev/stderr, and /dev/tty are > links to /dev/fd/0, /dev/fd/1, /dev/fd/2, and /dev/fd/3, respectively. The > last, in particular, is a nice generalization, and eliminates an ugly special > case in the kernel; init just does one more dup.) I always thought that /dev/tty was a means of getting hold of the tty when you couldn't be certain that 0,1,2 was connected to it. What you are really saying is that the UNIX convention of 0,1,2 having ``pre defined uses'' be extended to `3 always connected to the terminal and used for nothing else'. It isn't a /dev/fd issue, it is a UNIX convention issue. The other thing is the /dev/tty is a guaranteed way of getting the terminal & not something else (that is why the passwd program uses /dev/tty). Alain Williams +44 734 461232 phLOGIN our Turnkey Security Login utility is available NOW - ask me for info. Volume-Number: Volume 22, Number 5
seanf@sco.COM (Sean Fagan) (10/29/90)
Submitted-by: seanf@sco.COM (Sean Fagan) In article <14014@cs.utexas.edu> addw@phcomp.co.uk (Alain Williams) writes: >What happens if you do an ``ls -l'' on /dev/fd, do you see the fds which are >open to the ls program or all possible fds, even those which aren't opened ? You get something that looks like kithrup 10> ls -l /dev/fd total 0 crw-rw-rw- 5 bin bin 46,0 Jun 11 15:48 0 crw-rw-rw- 5 bin bin 46,1 Jun 11 15:48 1 crw-rw-rw- 2 bin bin 46,2 Jun 11 15:48 2 crw-rw-rw- 1 bin bin 46,3 Jun 11 15:48 3 crw-rw-rw- 1 bin bin 46,4 Jun 11 15:48 4 crw-rw-rw- 1 bin bin 46,5 Jun 11 15:48 5 And so on. They are normal device drivers; stat'ing them doesn't do anything strange, just opening them. (/dev/stdin.o, /dev/stdin.s, /dev/stdin.c, /dev/stdin, and /dev/fd/0 are all linked on my system; similarly with /dev/stdout. /dev/stderr is linked to /dev/fd/0, and all the others [through 60 on my system] only have one link.) -- -----------------+ Sean Eric Fagan | "Quoth the raven," seanf@sco.COM | "Eat my shorts!" uunet!sco!seanf | -- Lisa and Bart Simpson (408) 458-1422 | Any opinions expressed are my own, not my employers'. Volume-Number: Volume 22, Number 6
addw@phcomp.co.uk (Alain Williams) (10/29/90)
Submitted-by: addw@phcomp.co.uk (Alain Williams) > I agree; it makes things like > > join <(prog1) <(prog2) > joined-output-of-progs-1-and-2 > > possible. Such things are possible *now*, you don't need /dev/fd to do it, just an intelligent shell that doesn't mind making & destroying fifos. Anyway, the above wouldn't work with a straight /dev/fd as has been talked about here recently. Why ? The trouble is that if /dev/fd contains files with names "0", "1", ... "19", the programs prog1 & prog2 would both have a file /dev/fd/1 as their stdout, join would see another /dev/fd/1. What is really needed to do what is suggested above is for /proc to contain an fd directory, thus the command line above would be ``re written'' by the shell to something like: join /proc/1234/fd/1 /proc/1235/fd/1 > joined-output-of-progs-1-and-2 (where 1234 & 1235 are the process ids of prog1 and prog2). If we adopt the /proc/nnn/fd idea, more questions are raised, for instance what are the file permissions on /proc/nnn/fd ? Let me remind the original purpose for which /dev/fd was proposed: provide a mechanism whereby programs could handle `-' to mean stdin/out as does cat, but without trying. Alain Williams +44 734 461232 phLOGIN our Turnkey Security Login utility is available NOW - ask me for info. Volume-Number: Volume 22, Number 7
arnold%audiofax.com@mathcs.emory.edu (Arnold Robbins) (10/30/90)
Submitted-by: arnold%audiofax.com@mathcs.emory.edu (Arnold Robbins) In article <13878@cs.utexas.edu>, arnold@audiofax.com (that's me) writes lots of stuff about the wonders of /dev/fd, including this point: >|> Opening /dev/fd/7 is equivalent to doing a dup(7); In article <14012@cs.utexas.edu> craig@b11.ingr.com (Craig Presson) writes: >But for those of us without access to every Unix manual ever published >(I do have a Version 7 Volume 1), could you fill in a bit more on the >semantics of this hybrid /dev entry? Like what do you get when you open >"/dev/fd/7" and there is no open file using that slot? Does the system >make these entries "invisible" to processes not using them? Do you just >get a classic "It's an error from Unix, you're not supposed to understand" >type return? Or am I Missing Something? What happens when you do a dup(7) and 7 isn't a valid file descriptor? It returns -1 and errno is set to EBADF. So, something along those lines happens if you open /dev/fd/7, and 7 isn't open. Now, in article <14014@cs.utexas.edu> addw@phcomp.co.uk (Alain Williams) writes: >What happens if you do an ``ls -l'' on /dev/fd, do you see the fds which are >open to the ls program or all possible fds, even those which aren't opened ? Yes; the files in /dev/fd are character device files, with major and minor device numbers, just like the entries in /dev for all your ttys. The major number is for the fd device driver routine, and the minor number indicates which fd you're trying to open. It is a different animal than the /proc, which is mounted on an in-kernel filesystem. >> (In fact, in V8 - V10, /dev/stdin, /dev/stdout, /dev/stderr, and /dev/tty are >> links to /dev/fd/0, /dev/fd/1, /dev/fd/2, and /dev/fd/3, respectively. The >> last, in particular, is a nice generalization, and eliminates an ugly special >> case in the kernel; init just does one more dup.) > >I always thought that /dev/tty was a means of getting hold of the tty when >you couldn't be certain that 0,1,2 was connected to it. What you are really >saying is that the UNIX convention of 0,1,2 having ``pre defined uses'' be >extended to `3 always connected to the terminal and used for nothing else'. >It isn't a /dev/fd issue, it is a UNIX convention issue. Exactly! One of the major thrusts of recent versions of Research Unix has been to simplify, and become as minimalist as possible. By having fd 3 be the tty and /dev/tty a link to /dev/fd/3, the code in the kernel for doing /dev/tty goes away. >The other thing is the /dev/tty is a guaranteed way of getting the terminal >& not something else (that is why the passwd program uses /dev/tty). In general, unless someone went to the trouble, fd 3 will be attached to the terminal, so opening /dev/tty is pretty safe. Nothing's foolproof; it's entirely possible that a process can get started off with no controlling terminal if its parent closed all open files and did the appropriate incantations to set the process group. You're no worse off than before when /dev/tty was built into the kernel. -- Arnold Robbins AudioFAX, Inc. | Laundry increases 2000 Powers Ferry Road, #200 / Marietta, GA. 30067 | exponentially in the INTERNET: arnold@audiofax.com Phone: +1 404 933 7612 | number of children. UUCP: emory!audfax!arnold Fax-box: +1 404 618 4581 | -- Miriam Robbins Volume-Number: Volume 22, Number 8
gwyn@smoke.brl.mil (Doug Gwyn) (10/30/90)
Submitted-by: gwyn@smoke.brl.mil (Doug Gwyn) In article <14101@cs.utexas.edu> seanf@sco.COM (Sean Fagan) writes: >>What happens if you do an ``ls -l'' on /dev/fd, ... >You get something that looks like ... That's the most common implementation. However, /dev/fd could also be implemented as a filesystem type of its own, and I'd actually prefer that. Then an "ls /dev/fd" would show just the in-use file descriptors. Volume-Number: Volume 22, Number 9
jfh@rpp386.cactus.org (John F. Haugh II) (10/30/90)
Submitted-by: jfh@rpp386.cactus.org (John F. Haugh II) In article <14110@cs.utexas.edu> gwyn@smoke.brl.mil (Doug Gwyn) writes: >That's the most common implementation. However, /dev/fd could also be >implemented as a filesystem type of its own, and I'd actually prefer >that. Then an "ls /dev/fd" would show just the in-use file descriptors. Which brings up the issue of "whose in-use file descriptors?". It would work just fine for the application itself, but "ls" would be useless if you define "in-use" to be the current process' in-use descriptors. Gee, how many times do you want to see which file descriptors "ls" has open. This works with /proc because processes are system wide, while file descriptors are per-process. My fd0 has nothing in common with your fd0 - so either I distinguish between my fd0 and your fd0 and get stuck with fondling every user-page in the system, or I just cop out. A more complex inplementation buys little or nothing in terms of function at a very high cost in terms of overhead. -- John F. Haugh II UUCP: ...!cs.utexas.edu!rpp386!jfh Ma Bell: (512) 832-8832 Domain: jfh@rpp386.cactus.org "SCCS, the source motel! Programs check in and never check out!" -- Ken Thompson Volume-Number: Volume 22, Number 10
ske@pkmab.se (Kristoffer Eriksson) (10/30/90)
Submitted-by: ske@pkmab.se (Kristoffer Eriksson) In article <14102@cs.utexas.edu> addw@phcomp.co.uk (Alain Williams) writes: >> join <(prog1) <(prog2) > joined-output-of-progs-1-and-2 >Anyway, the above wouldn't work with a straight /dev/fd as has been talked about >here recently. Why ? The trouble is that if /dev/fd contains files with >names "0", "1", ... "19", the programs prog1 & prog2 would both have a file >/dev/fd/1 as their stdout, join would see another /dev/fd/1. That's not how you do it. Prog1 and prog2 just output to their standard outputs, as they always do, and the shell sets up pipes from prog1 and prog2 to join. The /dev/fd names for these pipes, as seen by join, are then passed as argv[] parameters to join, to make it read them. Prog1 and prog2 never see them. -- Kristoffer Eriksson, Peridot Konsult AB, Hagagatan 6, S-703 40 Oerebro, Sweden Phone: +46 19-13 03 60 ! e-mail: ske@pkmab.se Fax: +46 19-11 51 03 ! or ...!{uunet,mcsun}!sunic.sunet.se!kullmar!pkmab!ske Volume-Number: Volume 22, Number 12
arnold@mathcs.emory.edu (Arnold Robbins) (10/30/90)
Submitted-by: arnold@mathcs.emory.edu (Arnold Robbins) In article <14102@cs.utexas.edu> addw@phcomp.co.uk (Alain Williams) writes: >> I agree; it makes things like >> >> join <(prog1) <(prog2) > joined-output-of-progs-1-and-2 >> >> possible. >Such things are possible *now*, you don't need /dev/fd to do it, just an >intelligent shell that doesn't mind making & destroying fifos. Yes, up to a point. What happens though if you do nohup join <(prog1) <(prog2) > joined-output-of-progs-1-and-2 & and then log out? The temporary fifos will still be around when the program finally exits and the shell won't be around to clean them up. A cron job of some sort could maybe do it, but it's still not as clean as /dev/fd. >Anyway, the above wouldn't work with a straight /dev/fd as has been talked about >here recently. Why ? The trouble is that if /dev/fd contains files with >names "0", "1", ... "19", the programs prog1 & prog2 would both have a file >/dev/fd/1 as their stdout, join would see another /dev/fd/1. No. The programs have their respective file descriptors 1 attached as pipes, by the shell via dup, to other "files" in /dev/fd. The join program would see join /dev/fd/4 /dev/fd/5 on its command line. The scheme requires both /dev/fd and knowledge by the shell to work completely. Graphically, you have prog1[fd1]>------\ \ join[fd4][fd5][fd1]--> output / prog2[fd1]>-----------/ >What is really needed to do what is suggested above is for /proc to contain >an fd directory, thus the command line above would be ``re written'' by the >shell to something like: > > join /proc/1234/fd/1 /proc/1235/fd/1 > joined-output-of-progs-1-and-2 >(where 1234 & 1235 are the process ids of prog1 and prog2). > >If we adopt the /proc/nnn/fd idea, more questions are raised, for instance what >are the file permissions on /proc/nnn/fd ? As explained above, this already happens. Permissions on /dev/fd/* should be 666 since no process can affect any other's file descriptors. >Let me remind the original purpose for which /dev/fd was proposed: >provide a mechanism whereby programs could handle `-' to mean stdin/out >as does cat, but without trying. That's exactly what existing implementations do. Having ported a version of a /dev/fd driver into NFS based systems, and used KSH with process substitution turned on, I can say from experience that it works just fine. It is such a nice thing to have that I make gawk (gnu awk) recognize file names of that type internally and "do the right thing" even if the underlying system does not support /dev/fd. -- Arnold Robbins AudioFAX, Inc. | Laundry increases 2000 Powers Ferry Road, #200 / Marietta, GA. 30067 | exponentially in the INTERNET: arnold@audiofax.com Phone: +1 404 933 7612 | number of children. UUCP: emory!audfax!arnold Fax-box: +1 404 618 4581 | -- Miriam Robbins Volume-Number: Volume 22, Number 11
peter@ficc.ferranti.com (Peter da Silva) (10/31/90)
Submitted-by: peter@ficc.ferranti.com (Peter da Silva) In article <14137@cs.utexas.edu> arnold@audiofax.com writes: > nohup join <(prog1) <(prog2) > joined-output-of-progs-1-and-2 & > and then log out? The temporary fifos will still be around when the > program finally exits and the shell won't be around to clean them up. You'd have to nohup the whole thing in either case because you'll clobber prog1 and prog2 when you log out. So this is a non-problem. -- Peter da Silva. `-_-' +1 713 274 5180. 'U` peter@ferranti.com Volume-Number: Volume 22, Number 14
arnold@audiofax.com (Arnold Robbins) (11/01/90)
Submitted-by: arnold@audiofax.com (Arnold Robbins) >In article <14137@cs.utexas.edu> arnold@audiofax.com writes: >> nohup join <(prog1) <(prog2) > joined-output-of-progs-1-and-2 & > >> and then log out? The temporary fifos will still be around when the >> program finally exits and the shell won't be around to clean them up. In article <14175@cs.utexas.edu> you write: >Submitted-by: peter@ficc.ferranti.com (Peter da Silva) >You'd have to nohup the whole thing in either case because you'll clobber >prog1 and prog2 when you log out. So this is a non-problem. I didn't think about that, but it's still a problem. Consider: nohup join <(nohup prog1) <(nohup prog2) > joined-output & I admit to the feasibility of using temporary fifos for process substitution. It is more work than /dev/fd, but doable. /dev/fd, though, does make the whole thing much cleaner. And it still provides things like $ ln -s /dev/stdin foo.c $ cc foo.c main () { printf("hello, world\n"); ^D $ a.out hello, world I'm not on a crusade here or anything --- I simply think /dev/fd is a neat idea and I'd like to see it become commonplace. I don't know how much more there is to say on the subject... -- Arnold Robbins AudioFAX, Inc. | Laundry increases 2000 Powers Ferry Road, #200 / Marietta, GA. 30067 | exponentially in the INTERNET: arnold@audiofax.com Phone: +1 404 933 7612 | number of children. UUCP: emory!audfax!arnold Fax-box: +1 404 618 4581 | -- Miriam Robbins Volume-Number: Volume 22, Number 17