rcodi@yabbie.rmit.oz (Ian Donaldson) (10/09/88)
From article <30506@bbn.COM>, by mesard@bbn.com (Wayne Mesard): > Indeed. And more generally, a consistent way of handling directories > (i.e. directory files) needs to be established. In SunOS 3.4: > > cp <dir> <fn> ==> "cp: <dir>: Is a directory (not copied). > head, tail, cat <dir> ==> {works} > more <dir> ==> "*** <dir>: directory ***" > My preference is for cp to do the right thing (i.e, nix the -r flag) and > for cat, head and tail (and any other file display* programs) to behave > as more(1) does, with the addition of a new option to cat(1) to > explicitly tell it to treat a directory as an ordinary file. No, this is all wrong. Directories contain OS-specific information and should not be readable by any normal program. There shouldn't be any reason why anybody would need to read a directory directly at all when there are routines for reading directories in a portable way (see directory(3)). (mind you this has not yet trickled through to all UNIXen yet unfortunately) The decision as to whether a program has the right to read a directory or not should be done consistently by one part of the OS: the kernel by returning EPERM or something. (maybe the restriction should be relaxed for root, I doubt if it would matter much unless you were debugging a flakey filesystem, for which tools already exist in some UNIXen for this purpose, and they generally don't require reading directories other than via the raw special device). There should NOT be gratuitous mods to almost every program in the OS that opens a file for reading to check if its a directory first. Opening directories for writing is already disallowed universally (obviously), and reading a directory special file is already disallowed over NFS anyway (under SunOS 3.5 EISDIR is returned by read(2)) (why the -open- isn' t disallowed too I can't figure out). So what's the problem with disallowing opening directory special files generally? Ian D
kjones@talos.UUCP (Kyle Jones) (10/14/88)
In article <884@yabbie.rmit.oz> rcodi@yabbie.rmit.oz (Ian Donaldson) writes: >So what's the problem with disallowing opening directory special files >generally? It violates the UNIX principle of `a file is a file is a file...'. It is clear why directories should not be arbitrarily writable but I don't see any such reason why they shouldn't be readable. The portable directory reading routines could use the existing read(2) system call, instead of adding (yet another) system call just to read directories.
henry@utzoo.uucp (Henry Spencer) (10/14/88)
In article <884@yabbie.rmit.oz> rcodi@yabbie.rmit.oz (Ian Donaldson) writes: >So what's the problem with disallowing opening directory special files >generally? How can the directory access routines work then? (Actually, the long-term answer is that directory reading really ought to go via the kernel so that a standard interface to different kinds of file systems can be provided, but just walling off the contents of directories is not the answer just yet.) -- The meek can have the Earth; | Henry Spencer at U of Toronto Zoology the rest of us have other plans.|uunet!attcan!utzoo!henry henry@zoo.toronto.edu
daveb@gonzo.UUCP (Dave Brower) (10/17/88)
In article <331@talos.UUCP> kjones@talos.UUCP (Kyle Jones) writes: >In article <884@yabbie.rmit.oz> rcodi@yabbie.rmit.oz (Ian Donaldson) writes: >>So what's the problem with disallowing opening directory special files >>generally? > >It violates the UNIX principle of `a file is a file is a file...'. It >is clear why directories should not be arbitrarily writable but I >don't see any such reason why they shouldn't be readable. The portable >directory reading routines could use the existing read(2) system call, >instead of adding (yet another) system call just to read directories. The problem that is fixed by having system call directory access rather than plain ol' read(2) is this: What do you do when you have a filesystem type, presumably imported from some other OS, where directories are *not* just files? This struck the folks at AT&T like a bullet when they introduced the files system switch (FSS) in V.3 (or was it V.2?). In particular, it is what would let you mount a VMS file system and still be able to read the directories. -dB
guy@auspex.UUCP (Guy Harris) (10/18/88)
> (Actually, the long-term answer is that directory reading really ought to > go via the kernel so that a standard interface to different kinds of file > systems can be provided, In SunOS since SunOS release 2.0 (as well as, I suspect, many UNIX systems that have picked up NFS), and in System V Release 3 and later versions, it does precisely that, using "getdirentries" in SunOS prior to 4.0 and "getdents" in S5R3 and SunOS 4.0. Programs shouldn't use those calls directly, of course; they should use "readdir".
liam@cs.qmc.ac.uk (William Roberts) (10/20/88)
In article <331@talos.UUCP> kjones@talos.UUCP (Kyle Jones) writes: >It is clear why directories should not be arbitrarily writable but I >don't see any such reason why they shouldn't be readable. The portable >directory reading routines could use the existing read(2) system call, >instead of adding (yet another) system call just to read directories. This is too narrow a view in the world of networks and distributed systems - the "portable directory routines" exist so that the code *above* them is portable, not so that the routines themselves are portable. We have lots of grief because of programs which try to interpret directories via read(2), not noticing that they are actually coming from 1) a fileserver with a different byte ordering, e.g. a Sun mounted on a Sequent anything, 2) a different type of UNIX, e.g. an A/UX (== SysV.2.2) filestore mounted on a Sun (== BSD 4.2) 3) (God help us) a completely different kind of beast altogether, such as the NFS-served MacOS filesystem that we have here at QMC. The grief, incidentally, comes from the effect when the errant program fails to notice that its information is garbage - the server spends lots of time printing messages such as "NFS read request on non-file" on the console.... If someone out there is listening - please fix "tar" and "ranlib" so that they don't expect to use read(2) on directories, and furthermore, make *your* NFS implementation follow the rule that open(2)ing a directory is not permitted. -- William Roberts ARPA: liam@cs.qmc.ac.uk (gw: cs.ucl.edu) Queen Mary College UUCP: liam@qmc-cs.UUCP LONDON, UK Tel: 01-975 5250
forsyth@minster.york.ac.uk (10/21/88)
There is no trouble with using read() to replace getdirentries (or whatever). Inodes (or vnodes in some cults) have a type. The kernel obviously knows when you are reading a directory, and can put whatever information it likes into your buffer. For instance, it could format the information in the same way as getdirentries does now, if that were a sensible format. Each file system type's read implementation would map its file system dependent structure into the portable one. In fact, the Newcastle Connection read() has been doing something similar for years! I once considered changing directory read() to return just the list of file names, separated by newlines. Then a simple ls X == cat X | sort. The representation is convenient for programs, portable, and completely hides file system dependent information. (Think carefully about pwd before rushing off to try this, though.) Any good scheme should not have odd restrictions: ``nbytes must be greater than or equal to the block size associated with the file...sizes less than this may cause errors on certain filesystems'' [getdirentries(2)].
jim@cs.strath.ac.uk (Jim Reid) (10/21/88)
In article <744@sequent.cs.qmc.ac.uk> liam@cs.qmc.ac.uk (William Roberts) writes: >In article <331@talos.UUCP> kjones@talos.UUCP (Kyle Jones) writes: >>It is clear why directories should not be arbitrarily writable but I >>don't see any such reason why they shouldn't be readable........... > >This is too narrow a view in the world of networks and >distributed systems - the "portable directory routines" exist >so that the code *above* them is portable, not so that the >routines themselves are portable. We have lots of grief because >of programs which try to interpret directories via read(2), not >noticing that they are actually coming from The solution to that problem is to fix the broken programs, not to kludge the NFS implementation by preventing programs from reading the directory. In an ideal world, all the programs that need to know the contents of a directory would use the "portable directory routines". However, NFS should not prevent programs from reading remote directories directly. There are some circumstances where this could be useful: to see the underlying format of some NFS server's directory (assuming it has one), or perhaps to look at empty directory slots to see what files used to be there (assuming this is a reasonable thing to do). NFS makes a big thing of representing heterogeneous files as random access byte streams. Since UNIX implements directories as a "special" type of file - but still a byte stream - they should be visible as byte streams through NFS. If that breaks programs, tough. Preventing remote directory reads will break these programs anyway. In any case, such programs should have been fixed long ago considering how long have readdir(), scandir() and friends been around. I can see there are cases where remote directory reading is not meaningful - from an NFS server that doesn't have directories for instance. Dealing with cases like that for a generalised remote file access protocol is going to be tricky. It is a kludge that NFS simply imposes a blanket ban to avoid this hard and intractable problem. I don't claim to know what NFS (or any other comparable protocol) should do in these cases. I don't think it is reasonable to disallow the remote directory as a byte-stream paradigm when the remote directory is implemented in precisely that manner. NFS should allow this even if the clients and servers have different byte sex and/or directory formats. NFS has no business preventing user processes from doing something that, IMHO, most people would consider reasonable. Jim -- ARPA: jim%cs.strath.ac.uk@ucl-cs.arpa, jim@cs.strath.ac.uk UUCP: jim@strath-cs.uucp, ...!uunet!mcvax!ukc!strath-cs!jim JANET: jim@uk.ac.strath.cs "JANET domain ordering is swapped around so's there'd be some use for rev(1)!"
kjones@talos.UUCP (Kyle Jones) (10/24/88)
In <331@talos.UUCP> kjones@talos.UUCP (Kyle Jones) writes: >It is clear why directories should not be arbitrarily writable but I >don't see any such reason why they shouldn't be readable. The portable >directory reading routines could use the existing read(2) system call, >instead of adding (yet another) system call just to read directories. In <744@sequent.cs.qmc.ac.uk> liam@cs.qmc.ac.uk (William Roberts) writes: >This is too narrow a view in the world of networks and >distributed systems - the "portable directory routines" exist >so that the code *above* them is portable, not so that the >routines themselves are portable. I didn't say that the portable directory reading routines needed to be portable. I questioned the need for these ruotines to have to use another UNIX system call *just* to read directories when UNIX file semantics make it possible to use an existing system call. (And behold, what does getdirentries(2) do but fill a buffer, just as read(2) does?) If I can use read(2) to get input from something as idiosyncratic as a teletype, why can't I (or the portable directory reading routines) use it to read directories? Wouldn't it have been just a simple to make read(2) (when reading a directory) fill the buffer pointed to by its second argument with the same information that the getdirentries(2) call provides? kyle jones <kjones@talos.UUCP>
guy@auspex.UUCP (Guy Harris) (10/25/88)
>It is clear why directories should not be arbitrarily writable but I >don't see any such reason why they shouldn't be readable. The portable >directory reading routines could use the existing read(2) system call, >instead of adding (yet another) system call just to read directories. It would probably have been possible, in principle, to have "read()" on a directory not give you the raw data in the directory file, but give you the directory entries in a "standard" format; over NFS, it would do a NFS "readdir" call rather than a "read" call. However, that wasn't what was done; I don't know the reason why not, but one possibility I can think of is that they wanted SunOS 1.x binaries to still be able to read directories on local files (so that your 1.x binaries weren't immediately obsolete, and you or the vendor would have time to recompile. Thus, instead of modifying "read()" in that fashion, they added "getdirentries". You can probably make philosophical arguments either way, but they're irrelevant because 1) either way works well enough, and neither one works significantly better (as several people have pointed out, old programs that "knew" what directory entries looked like would break if they thought they looked like V7-style directories, so keeping "read()" wouldn't have let them continue to work) and 2) Sun already introduced "getdirentries", and AT&T added "getdents" in S5R3 (which Sun picked up in SunOS 4.0), so the choice has already been made. Adding "yet another" system call is hardly *ipso facto* a bad thing.
guy@auspex.UUCP (Guy Harris) (10/25/88)
>The solution to that problem is to fix the broken programs, not to >kludge the NFS implementation by preventing programs from reading the >directory. In an ideal world, all the programs that need to know the >contents of a directory would use the "portable directory routines". The reason why the NFS implementation was kludged was to *catch* the broken programs in question. The 2.0 NFS implementation allowed it.