gwyn@BRL.ARPA (VLD/VMB) (08/18/86)
Here is additional information about streams and pipes to add to my posting of 11-Aug-1986. It's the result of subsequent discussion with Dennis Ritchie, who kindly granted me permission to quote him. I hope this will help those who are trying to implement the full streams concept (beyond what is in SVR3) to see how it's best done. I was originally going to summarize Dennis's information, but upon attempting that, I found that he has, as usual, expressed the ideas already in about the best possible way, so the following are direct quotes from DMR. (I found the bit about implementing FIFOs with streams particularly instructive, and hope that the AT&T UNIX product developers in Summit NJ will incorporate this approach in a future release of UNIX System V, along with the pipes and terminal handler I previously mentioned.) [start DMR quotes] ... Pipes, for example, really do become much cleaner when they are just cross-connected streams. SIGPIPE is not a special case; instead it is a signal generated by writing on any stream whose other end has gone away. No-delay IO likewise can be handled in the single appropriate place (the stream read and write routine) instead of being spread around among the various drivers; so too for select. Although FIFOs (named pipes) aren't in V8 proper, I reinstalled them recently in our Cray system, which is SVR2 with ingrafted streams; it added about 10 lines of code, in one place, and I took the opportunity of stripping out pages of old, special-case FIFO goo scattered all over the place. The disengagement of terminal processing from device and network IO was the most important reason I went into the stream business at all, so the fact that a separable TTY module hasn't yet made it into SV is genuinely disconcerting. ... First, there is, and I think has to be, a real distinction between stream heads and stream ends; the difference is that some sort of intelligence (viz., locally-written code that can be executed by a process) is associated with a stream head, whereas a stream end is a stupid (or even malevolent) device. On the other hand, a stream like a pipe can have two heads. Second: all this traffic seems to have resulted from a question about the detailed fstat behavior of pipes. In v8, the result of an fstat on a pipe tells you that it is an ordinary file whose size is always 0. A seek on any stream file generates ESPIPE. If there were any ill effects from this, I don't remember them; it doesn't seem to be an important issue. ... Well, here are some details. The inode table has a new i_sptr slot that can point to a stream table entry, that in turn connects to the first of the possibly several stream modules. The various operations (read, write, ioctl, open close) check for i_sptr!=NULL. When a stream device is opened (they have an extra column in the cdevsw table) i_sptr is filled in. This connection, ip->i_sptr, defines a stream head. The stream end is the module for the device itself (the end). This is, I think, true in both V8 and SVr3. The V8 pipe call allocates two inodes. Most recently, they are on a special pipe file system type that has essentially no semantics, just serves as a label, so they don't have to be put on a real device. Each inode has a stream head, but the connections are set up so each head points to the other (this means, incidentally, that pipes are full-duplex, because there is an independent connection in each direction.) They have unique inumbers. Most of the code is symmetrical, and doesn't depend on the difference between "head" and "end." Some does. For example, when a stream is closed, one starts at the head, popping off processing modules, and stops at the end. When a pipe end is closed, though, it is necessary to detect the crossover and "seal off the other end" by attaching a fake device; you can't just leave the pointers dangling. fstat is pretty much transparent to all of this. Stream devices look like character devices because they are. Pipes look like ordinary files because that is what their inodes are marked as, but they could be anything; the stream pointer dominates the type when IO operations occur. Some side points: FIFOs on the Cray involve only a single inode (a real one, created in the usual way SV way). When it is opened, the usual stream head is allocated, but it is set up to point to itself. Thus they're half duplex like traditional pipes.