housel@en.ecn.purdue.edu (Peter S. Housel) (09/05/89)
In Minix versions 1.0 onward, when a pipe(2) system call is done, the filesystem allocates an inode on the same filesystem as the current working directory, and uses the resulting file to hold the pipe data. (See the book, lines 10409-10411, or lines 61-63 of the 1.3 fs/pipe.c.) There are several real and potential problems with this. The first is even though a filesystem is mounted read-only, the pipe inode and its associated blocks are created there anyway. The read-only flag is checked at a high level in the filesystem, and several paths into the write mechanism have been missed. If the device is actually writable, then the pipe file will write on it with no problems, except for the fact that you requested "don't scribble on this disk!" with the mount "-r" flag. If it is not writable (i.e. a floppy with a write-protect tab), then the system will eventually try to write on the disk when the cache is flushed, you will receive several I/O errors, and the operation will fail. Another problem is that much more writing to disk is done than is necessary. I was rather surprised to see blocks being sync'ed to disk in the middle of terminal (term(1)) sessions, until I discovered that term(1) uses pipes. Still another is that these files stay around after a system crash or sloppy shutdown, and occasionally have to be cleaned up with fsck. In BSD systems, pipes are really socketpairs and have little to do with the "lower" (inode and disk block) half of the filesystem at all. In Version 7 and System V, there is a specific pipe device which is determined at boot time, and is usually the same as the root filesystem device. One possible solution would be to adopt the V7 approach. The advantages of this are that it would be easy to make the change, and that no disk I/O would be done at all for people who keep there root filesystems on ramdisk. While I'm fond of the idea of keeping the root FS in RAM, it isn't cast in stone and other people may want the root FS on a real disk, which would still be just as slow. Another would be to have a list of filesystems which could potentially be used to keep pipe files, and reject ones which are read-only or don't have enough free zones. However, if you ended up rejecting all of them, what do you do? Another would be to make sure that pipe-related blocks never get flushed to disk but stay in the buffer cache. This sounds dangerous and difficult to implement. Any comments? -Peter S. Housel- housel@ecn.purdue.edu ...!pur-ee!housel
n62@nikhefh.nikhef.nl (Klamer Schutte) (09/05/89)
In article <15116@ea.ecn.purdue.edu> housel@en.ecn.purdue.edu (Peter S. Housel) writes: >> a discussion about where to put the pipe'd data << >Another would be to make sure that pipe-related blocks never get flushed >to disk but stay in the buffer cache. This sounds dangerous and difficult >to implement. don't do. This puts a constrain on the maximum data size of a pipe. And what to do when one more block is needed in the cache and the rest are pipes? Abort job with Broken Pipe signal? > >Any comments? > Why not configuring at compile-time at which device the pipes should go? ( at my system this should be RAM; but this also puts a constraint on the maximum pipe size (512k on my system)). >-Peter S. Housel- housel@ecn.purdue.edu ...!pur-ee!housel Klamer. -- ____________________Yes, mail address changed again :-(_________________________ Klamer Schutte mcvax!nikhefh!{n62,Schutte} {Schutte,n62}@nikhef.nl
evans@ditsyda.oz (Bruce Evans) (09/07/89)
In article <252@nikhefh.nikhef.nl> Schutte@nikhefh.nikhef.nl (Klamer Schutte) writes: >Why not configuring at compile-time at which device the pipes should go? >( at my system this should be RAM; but this also puts a constraint on the maximum > pipe size (512k on my system)). Pipe sizes are limited to 7 blocks by the implementation of pipes using inodes. I hate compile-time decisions (limitations). How about putting pipes on /dev/pipe? It would be acceptable to cache the value found for the first pipe open. I'm not sure if this would allow reconfiguring /dev/pipe just after booting. As Peter Housel pointed out, it is unreasonable to hold all pipes in the cache (holding them on a disk delays the problem :-)). However, it would be nice to avoid flushing discarded pipe blocks and inodes. Once a block is marked dirty, it is normally only cleaned by writing. The same applies to deleted files ... The limitation of writing only 7K to pipes is very annoying (not device- independent). In article <2234@nucleus.UUCP>, msm@nucleus.UUCP (Michael S. Mattone) posted a fix. It didn't quite work for me. There was a bug with skipping a block in the pipes. I only saw it while writing a fairly complicated directory structure using PD-tar. It seemed to have something to do with the extra i/o for system("mkdir"). This is the fate of many fixes posted to the newsgroup. There is not enough feedback about source code and fixes. I'm not suggesting that everyone follow up everything, but a few should send mail to the author and the author should follow up after a few weeks. No mail is signifigant and probably means no-one was interested; the newsgroup is not that big. Bruce -- Bruce Evans evans@ditsyda.oz.au