nathan@eddie.MIT.EDU (Nathan Glasser) (05/24/88)
On my vax running 4.3bsd, apparently calling the rename(2) function
with the same filename for both arguments causes the function to
remove the file, and return 0 indicating success.
e.g. The program
main()
{
rename("foo","foo");
}
will remove the file foo.
This is definitely contrary to the statement of the man page which
says that an instance of the file is guaranteed to exist, even
if the system crashes.
--
Nathan Glasser
fnord nathan@{mit-eddie.uucp, xx.lcs.mit.edu}
"A tribble is the only love that money can buy."
Presently there is insufficient evidence to conclude that tribbles spread AIDS.chris@mimsy.UUCP (Chris Torek) (05/25/88)
In article <9312@eddie.MIT.EDU> nathan@eddie.MIT.EDU (Nathan Glasser) writes: >On my vax running 4.3bsd ... rename("foo","foo"); ... >will remove the file foo. Sounds like you have a hacked 4.3BSD. From /sys/sys/ufs_syscalls.c: /* * Short circuit rename(foo, foo). */ if (xp->i_number == ip->i_number) goto bad; This is, of course, not quite right: link("foo", "bar"); rename("foo", "bar"); leaves both foo and bar around, where one would expect it to remove file foo. -- In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 454 7163) Domain: chris@mimsy.umd.edu Path: uunet!mimsy!chris
alex@aiva.ed.ac.uk (Alex Zbyslaw) (05/25/88)
In article <9312@eddie.MIT.EDU> nathan@eddie.MIT.EDU (Nathan Glasser) writes: >> >On my vax running 4.3bsd, apparently calling the rename(2) function >with the same filename for both arguments causes the function to >remove the file, and return 0 indicating success. >This is definitely contrary to the statement of the man page which >says that an instance of the file is guaranteed to exist, even >if the system crashes. RENAME(2) UNIX Programmer's Manual RENAME(2) [...] Rename causes the link named from to be renamed as to. If to exists, then it is first removed. Need I point out the importance of the second sentence? --Alex -- JANET: alex@uk.ac.ed.eusip ARPA: alex%ed.eusip@nss.cs.ucl UUCP: ...{uunet, decvax, ihnp4}!mcvax!ukc!eusip!alex [CSNET BITNET]: alex%ed.eusip%nss.cs.ucl@[csnet-relay cunyvm] Who needs opinions anyway?
nathan@eddie.MIT.EDU (Nathan Glasser) (05/27/88)
In article <467@aiva.ed.ac.uk> alex@eusip.ed.ac.uk (Alex Zbyslaw) writes: >RENAME(2) UNIX Programmer's Manual RENAME(2) > Rename causes the link named from to be renamed as to. If > to exists, then it is first removed. >Need I point out the importance of the second sentence? From the same man page, a couple of sentences later, it says: Rename guarantees that an instance of to will always exist, even if the system should crash in the middle of the opera- tion. I don't know about you, but I think there's an inconsistency in the man page. First of all, the operation is in consistent with the sentence I've included. Secondly, in the section Alex included, it says that 'to' will be removed, but if it removes it in the case where they're the same file, then it won't be able rename 'from' to be 'to' like it claims to do without great difficulty. -- Nathan Glasser fnord nathan@{mit-eddie.uucp, xx.lcs.mit.edu} "A tribble is the only love that money can buy." Presently there is insufficient evidence to conclude that tribbles spread AIDS.
jfh@rpp386.UUCP (John F. Haugh II) (05/28/88)
In article <9341@eddie.MIT.EDU> nathan@eddie.MIT.EDU (Nathan Glasser) writes: >I don't know about you, but I think there's an inconsistency in the >man page. i don't know about you, but i keep getting that fortune cookie which defines a university as a place to get software where if it doesn't work the first time, there is always someone willing to fix it. now, do you trust an operating system written at a university? oh god, i make myself laugh some times ... - john. -- John F. Haugh II | "If you aren't part of the solution, River Parishes Programming | you are part of the precipitate." UUCP: ihnp4!killer!rpp386!jfh | -- long since forgot who DOMAIN: jfh@rpp386.uucp |
les@chinet.UUCP (Leslie Mikesell) (05/29/88)
In article <2144@rpp386.UUCP> jfh@rpp386.UUCP (The Beach Bum) writes: >now, do you trust an operating system written at a university? oh >god, i make myself laugh some times ... > Hmm.. Maybe you like the SysV rename() better? Oops, I forgot, there isn't one. Then maybe you like the interval between unlink() and link() when your files don't exist. Or maybe you don't update files that other people might want to read... Les Mikesell
ron@topaz.rutgers.edu (Ron Natalie) (06/01/88)
> Then maybe you like the interval between unlink() and link() > when your files don't exist. This is completely absurd. From day 1 on UNIX the way to rename files is to link the file to the new name and unlink the old one. There is no way to do otherwise (what arguments are you going to provide to link?). There is not now, or ever has been a period of time that the file "doesn't exist." Once a file has no links in UNIX, it requires FSCK or someone very talented at manipulating the raw disks to put it back. -Ron
les@chinet.UUCP (Leslie Mikesell) (06/02/88)
In article <May.31.17.12.57.1988.3256@topaz.rutgers.edu> ron@topaz.rutgers.edu (Ron Natalie) writes: >> Then maybe you like the interval between unlink() and link() >> when your files don't exist. > >This is completely absurd. From day 1 on UNIX the way to rename >files is to link the file to the new name and unlink the old one. >There is no way to do otherwise (what arguments are you going to >provide to link?). There is not now, or ever has been a period of >time that the file "doesn't exist." Once a file has no links in >UNIX, it requires FSCK or someone very talented at manipulating the >raw disks to put it back. sigh... I suppose I should have been explicit about the arguments to the unlink() and link() functions, but I thought it was obvious. Given an existing file xx which several people may be reading, and new data in a file TMP4653 that is supposed to replace file xx, and no atomic rename() function, what do you do? It is kind of anti-social to copy on top of a file that someone is reading since they might see part of the old file and part of the new, so you are left with: unlink("xx"); /* anyone who has it open will still see the old file */ link("TMP4653","xx"); /* opening now will get the new file */ unlink("TMP4653"); /* cleanup */ This leaves an interval between unlink("xx") and link("TMP4653","xx") when an attempt to open("xx") will fail. I work with a system that has about 10,000 files on line that are mostly updated daily, some every 10 minutes, and it seems like a real kludge that I had to make the retrieval program attempt open() twice with a sleep() in between to be sure of finding the file. Can it be done otherwise? Les Mikesell
mouse@mcgill-vision.UUCP (der Mouse) (06/02/88)
In article <11658@mimsy.UUCP>, chris@mimsy.UUCP (Chris Torek) writes: > In article <9312@eddie.MIT.EDU> nathan@eddie.MIT.EDU (Nathan Glasser) writes: >> On my vax running 4.3bsd ... rename("foo","foo"); ... >> will remove the file foo. > Sounds like you have a hacked 4.3BSD. From /sys/sys/ufs_syscalls.c: > [code intended to shortcircuit renaming a file to itself] I tried this on various systems, and in the presence of NFS. The machines involved are: apollo: Sun-3 running Sunix 3.5 (not an Apollo machine, but a Sun called "apollo") ra: another Sun-3 also running Sunix 3.5 larry: VAX 750 running mtXinu 4.3+NFS curly: MicroVAX-II running Berkeley 4.3, with my NFS server Apollo: Local filesystem: rename fails, errno is set to -1 (!). Ra's filesystem: same as local. Larry's filesystem: same as Larry with local filesystem. Curly's filesystem: same as Curly with local filesystem. Larry: Local filesystem: file disappears (original poster's position). Apollo's filesystem: same as Apollo with local filesystem. Curly's filesystem: same as Curly with local filesystem. Curly: Local filesystem: no error, file remains (Chris' claim) So it seems that NFS renames aren't locally checked, but are shipped over to the remote system to be interpreted there. On the basis of this, I made one more NFS test. The client was larry, the server was an Ultrix 2.x machine (not sure of the minor version number). Ultrix seems to have gotten this right: the rename succeeded but didn't destroy the file. Both Sun and mtXinu seem to be broken. Sun at least causes the rename to fail, though they return an undocumented value for errno as it does so. mtXinu's failure is the most dangerous. Chris' claim is correct, as usual, though it was arguably reasonable for the original poster to claim to be running "4.3" if the system was actually something similar to the above (a vendor modification of real 4.3). Yet more reason not to use NFS: implementations of it tend to be broken. In this case it's even worse: the breakage shows even when NFS is not in use! (Not all implementations; note the Ultrix test. DEC got this one right - a kudos or two to them.) der Mouse uucp: mouse@mcgill-vision.uucp arpa: mouse@larry.mcrcim.mcgill.edu
mouse@mcgill-vision.UUCP (der Mouse) (06/02/88)
In article <2144@rpp386.UUCP>, jfh@rpp386.UUCP (John F. Haugh II) writes: > i don't know about you, but i keep getting that fortune cookie which > defines a university as a place to get software where if it doesn't > work the first time, there is always someone willing to fix it. ...and a vendor is a place to get software where if it doesn't work, you get to pay large amounts of money to get the next major release, where the bug might be fixed if enough customers have complained. > now, do you trust an operating system written at a university? oh > god, i make myself laugh some times ... No, but I trust one written by a company out to make money even less. der Mouse uucp: mouse@mcgill-vision.uucp arpa: mouse@larry.mcrcim.mcgill.edu
guy@gorodish.Sun.COM (Guy Harris) (06/03/88)
> apollo: Sun-3 running Sunix 3.5 > (not an Apollo machine, but a Sun called "apollo") > ra: another Sun-3 also running Sunix 3.5 > larry: VAX 750 running mtXinu 4.3+NFS > curly: MicroVAX-II running Berkeley 4.3, with my NFS server > > Apollo: > Local filesystem: rename fails, errno is set to -1 (!). Bug, fixed in 4.0. > Yet more reason not to use NFS: implementations of it tend to be > broken. In this case it's even worse: the breakage shows even when NFS > is not in use! Umm, in the SunOS 3.5 case, the conclusion that it's an NFS bug cannot be drawn from the evidence. Calling an NFS problem, given that it shows up "even when NFS is not in use", is a bit silly. In fact, it was a bug in the *UFS* code, and had nothing whatsoever to do with NFS. The bug was introduced by a bug fix to what is, I suspect, the bug you saw in the Mt. Xinu system, where rename("foo", "foo") causes "foo" to disappear. If this is the case with the Mt. Xinu system, it's another UFS bug, not an NFS bug.
limes@sun.uucp (Greg Limes) (06/03/88)
In article <1126@mcgill-vision.UUCP> mouse@mcgill-vision.UUCP (der Mouse) discusses a problem with "rename", noting that on a Sun-3 running "Sunix" 3.5, the call 'rename ("foo", "foo")' returns an error and sets errno to -1. This seemed a bit strange to me, so I tried it. TESTBED: threefive, a Sun 3/50 running SunOS 3.5 /tmp: local filesystem /fouroh/tmp: NFS mounted from fouroh:/tmp fouroh, a Sun 3/60 running SunOS 4.0 /tmp: local filesystem /threefive/tmp: NFS mounted from threefive:/tmp "ren", a trivial utility: extern int errno; main (ac, av) int ac; char **av; { int rv = rename (av[1], av[1]); printf ("rename returns %d; errno is %d\n", rv, errno); } RESULTS: fouroh% ren /tmp/foo rename returns 0; errno is 0 fouroh% ren /threefive/tmp/foo rename returns -1; errno is -1 threefive% ren /tmp/foo rename returns -1; errno is -1 threefive% ren /fouroh/tmp/foo rename returns 0; errno is 0 Looks like der Mouse's conclusions are correct, in that the rename request is passed on to the host system; also, I have to agree that some implementations of NFS may have bugs, but in this case NFS is only transporting the strange error code from one system to the other so we can pretty much remove NFS from our list of possible culprits. We can also see from the above that, whatever the bug was, it is fixed in release 4.0 of SunOS. In article <1128@mcgill-vision.UUCP> der Mouse writes: >... I trust one written by a company out to make money even less. Funny, I would expect exactly the reverse. If the operating system does not work properly, the company gets bug reports and has to fix them (this costs bucks), and if the bugs are bad enough, the company starts to lose customers to competitors who can provide better functionality. Thus is it in the best interests of the for-profit corporation to provide software that is as reliable and bug-free as possible. -- Greg Limes [limes@sun.com]
gordon@sneaky.UUCP (06/03/88)
> > Then maybe you like the interval between unlink() and link() > > when your files don't exist. > > This is completely absurd. From day 1 on UNIX the way to rename > files is to link the file to the new name and unlink the old one. > There is no way to do otherwise (what arguments are you going to > provide to link?). There is not now, or ever has been a period of > time that the file "doesn't exist." Once a file has no links in There is a time when no file by the destination filename exists (using the unlink/link/unlink method). This is the problem. The BSD rename() call is supposed to guarantee that if a file by the destination filename existed before the call, a file by the destination filename will exist after the call, and at all times in between. > UNIX, it requires FSCK or someone very talented at manipulating the > raw disks to put it back. Suppose you have a file, say, /etc/passwd. You make a copy of it, /etc/passwd.new, and edit it. (Let us assume there is locking somehow to prevent *TWO* people editing /etc/passwd at the same time. This is not the issue. I also do not care about the people who need to re-generate a hashed database whenever the password file is modified. This is a different problem.) You need to rename /etc/passwd.new back to /etc/passwd. In the process of doing so, /etc/passwd must always exist, so you don't get all sorts of nasty error messages out of programs trying to use it. It must always exist, and exist in complete form, so doing a copy instead of a rename is also unacceptable.
karish@denali.stanford.edu (Chuck Karish) (06/03/88)
In article <5730@chinet.UUCP> les@chinet.UUCP (Leslie Mikesell) writes:
This leaves an interval between unlink("xx") and link("TMP4653","xx")
when an attempt to open("xx") will fail. I work with a system that
has about 10,000 files on line that are mostly updated daily, some
every 10 minutes, and it seems like a real kludge that I had to make
the retrieval program attempt open() twice with a sleep() in between
to be sure of finding the file. Can it be done otherwise?
Why not have each program that uses the shared files lock them
while they're in use? If your system doesn't support real locks,
an ad hoc advisory locking system would be adequate if the
programs were willing to cooperate with one another.
Chuck Karish ARPA: karish@denali.stanford.edu
BITNET: karish%denali@forsythe.stanford.edu
UUCP: {decvax,hplabs!hpda}!mindcrf!karish
USPS: 1825 California St. #5 Mountain View, CA 94041les@chinet.UUCP (Leslie Mikesell) (06/04/88)
In article <22180@labrea.Stanford.EDU> karish@denali.stanford.edu (Chuck Karish) writes: > > In article <5730@chinet.UUCP> les@chinet.UUCP (Leslie Mikesell) writes: > This leaves an interval between unlink("xx") and link("TMP4653","xx") > when an attempt to open("xx") will fail. I work with a system that >Why not have each program that uses the shared files lock them >while they're in use? Well, some of the programs are "cat", "pg", "compress", "uucp" and a print spooler as well as my interactive access program. But the real reason I didn't consider locks is that the 10 minute updates cannot be deferred until no one happens to be reading the previous copy. I suppose I could make a copy each time someone reads a file which would limit the time the file needs to be locked, but that would cause a real performance problem. This is currently running on a 3B2 (actually 2 of them with the files duplicated over RFS) and the updating machine stays pretty busy now. Trying the open() twice seemed like the only reasonable thing to do, although it does take a bit longer to respond to invalid requests. How do other people deal with files that are frequently updated? Les Mikesell
mouse@mcgill-vision.UUCP (der Mouse) (06/04/88)
[ >> = me, writing about how rename("foo","foo") works and/or fails ]
[ > = Guy Harris ]
>> Yet more reason not to use NFS: implementations of it tend to be
>> broken. In this case it's even worse: the breakage shows even when
>> NFS is not in use!
> Umm, in the SunOS 3.5 case, the conclusion that it's an NFS bug
> cannot be drawn from the evidence. Calling an NFS problem, given
> that it shows up "even when NFS is not in use", is a bit silly. In
> fact, it was a bug in the *UFS* code,
This much was clear from the fact that it worked/failed the same for
any given filesystem regardless of whether the filesystem was accessed
over NFS or locally.
> and had nothing whatsoever to do with NFS.
This I argue with, see below.
> The bug was introduced by a bug fix to what is, I suspect, the bug
> you saw in the Mt. Xinu system, where rename("foo", "foo") causes
> "foo" to disappear. If this is the case with the Mt. Xinu system,
> it's another UFS bug, not an NFS bug.
When I called it a reason not to use NFS, I didn't say it was an NFS
bug. It is a bug in an NFS-capable kernel which wouldn't exist if it
weren't for the NFS capability. In that sense, NFS is responsible for
it, and that is the sense in which I meant my comment about NFS. I
consider the whole vfs-based filesystem as an NFS thing, since it
exists only to support NFS.
der Mouse
uucp: mouse@mcgill-vision.uucp
arpa: mouse@larry.mcrcim.mcgill.eduguy@gorodish.UUCP (06/05/88)
> I consider the whole vfs-based filesystem as an NFS thing, since it > exists only to support NFS. Which demonstrates that you *don't* know why the VFS mechanism exists. IT does not "exist only to support NFS." It exists to support *multiple* file systems; in other words, to permit several different file system types to share the same system call interface, and to permit new file system types to be added without rewhacking the system call interface. The following file system types are currently provided by Sun for the VFS interface: UFS, NFS, and an MS-DOS file system implementation provided by the Sun consulting group. AT&T's RFS, "/proc", and other file system types could also be implemented atop the VFS mechanism. I think some knowledgable customers have also looked at implementing, or are implementing, their own file system types. Neither the VFS mechanism nor Sun RPC exist solely to support NFS, despite the fact that some people *think* they do (and even publicly claim that they do). They were both designed as general tools, and NFS was implemented atop them.
les@chinet.UUCP (Leslie Mikesell) (06/05/88)
In article <-64039385@sneaky> gordon@sneaky.UUCP writes: >There is a time when no file by the destination filename exists (using >the unlink/link/unlink method). > >Suppose you have a file, say, /etc/passwd. You make a copy of it, >/etc/passwd.new, and edit it. Excellent example. A few years ago I was at an AT&T shell programming class and the twenty or so people in the class all signed in and changed their password at the same time. It managed to crash the 3b20 they were using for the classrooms. Since I had recently been exposed to a sales pitch about the reliability of the 3b20, I thought it was kind of amusing. Les Mikesell
res@ihlpe.ATT.COM (Rich Strebendt, AT&T-DSG @ Indian Hill West) (06/06/88)
In article <5730@chinet.UUCP>, les@chinet.UUCP (Leslie Mikesell) writes: > > Given an existing file xx which several people may be reading, and > new data in a file TMP4653 that is supposed to replace file xx, and > no atomic rename() function, what do you do? It is kind of anti-social > to copy on top of a file that someone is reading since they might see > part of the old file and part of the new, so you are left with: > > unlink("xx"); /* anyone who has it open will still see the old file */ > link("TMP4653","xx"); /* opening now will get the new file */ > unlink("TMP4653"); /* cleanup */ > > This leaves an interval between unlink("xx") and link("TMP4653","xx") > when an attempt to open("xx") will fail. I work with a system that > has about 10,000 files on line that are mostly updated daily, some > every 10 minutes, and it seems like a real kludge that I had to make > the retrieval program attempt open() twice with a sleep() in between > to be sure of finding the file. Can it be done otherwise? This kind of problem was handled routinely on IBM TSS/360/370 with Generation Data Groups. Perhaps something similar should be developed for UNIX. All a GDG was was a collection of files with funny names that the operating system recognized and handled differently from ordinary files when new members of the group were created. There were two ways a file in a GDG could be referred to, by absolute generation number and by relative generation number. A GDG file named FILENAME.G0017V00 was absolute generation 17 of the GDG named FILENAME. Rarely was the absolute number used, however, since most procdefs (similar to shell procedures) were written to use the relative numbers. The current version of GDG FILENAME was referred to as FILENAME(0). The previous version of FILENAME was referred to as FILENAME(-1). When a new version of FILENAME was created it was referred to as FILENAME(+1) prior to being completely created, and became FILENAME(0) when it was closed after being written. When a new member joined a GDG one or more of its predecessors was automatically deleted, according to the option selected when the GDG was declared. For example, one could specify that no more than 3 generations were to be maintained, with the oldest being removed when a new one was created. This would keep many generations of the file from accumulating. How did this solve the problem described above? When a user referenced the program or data file, they requested relative generation 0 of that file. In this way they got the most current of the moment. When we created a new one, we named it relative generation +1 until it had been written and closed. At that point it became relative generation 0 and the next user to request it would get this brand new generation. Rich Strebendt ...!ihnp4![iwsl6|ihlpe|ihaxa]!res
mike@turing.unm.edu (Michael I. Bushnell) (06/16/88)
In article <55239@sun.uucp> limes@sun.UUCP (Greg Limes) writes: >In article <1128@mcgill-vision.UUCP> der Mouse writes: >>... I trust one written by a company out to make money even less. > >Funny, I would expect exactly the reverse. If the operating system does >not work properly, the company gets bug reports and has to fix them >(this costs bucks), and if the bugs are bad enough, the company starts >to lose customers to competitors who can provide better functionality. >Thus is it in the best interests of the for-profit corporation to >provide software that is as reliable and bug-free as possible. OK...here's the 10,000,000 question: who competes with SUN to provide software for SUNs? No one. There is, therefore, no impulse on the part of sun to provide the best functionality. Once someone's bought the box, sun is a power to stick-it-to-em. DEC has a competitor for Ultrix, but the response has not been to improve Ultrix, it has been to keep hardware manuals secret to UCB can't write drivers. -- N u m q u a m G l o r i a D e o Michael I. Bushnell HASA - "A" division mike@turing.unm.edu {ucbvax,gatech}!unmvax!turing.unm.edu!mike
mike@turing.unm.edu (Michael I. Bushnell) (06/16/88)
In article <55437@sun.uucp> guy@gorodish.Sun.COM (Guy Harris) writes: >> I consider the whole vfs-based filesystem as an NFS thing, since it >> exists only to support NFS. > >Which demonstrates that you *don't* know why the VFS mechanism >exists. IT does not "exist only to support NFS." It exists to >support *multiple* file systems; in other words, to permit several >different file system types to share the same system call interface, >and to permit new file system types to be added without rewhacking >the system call interface. And thus, fails. The discussion is about a manner in which those different file systems do NOT provide the same system call interface. Some of them give an error for rename("foo","foo"), some delete "foo", some do nothing. flock(...) works on some, not on others. And then there's the celebrated mkdir bug. If I do open(fname, O_RDWR | O_CREAT | O_EXCL, mode), NFS doesn't guarantee exclusive creation. NFS and VFS do not guarantee the same semantics for different file systems. -- N u m q u a m G l o r i a D e o Michael I. Bushnell HASA - "A" division mike@turing.unm.edu {ucbvax,gatech}!unmvax!turing.unm.edu!mike
guy@gorodish.Sun.COM (Guy Harris) (06/17/88)
> And thus, fails. The discussion is about a manner in which those > different file systems do NOT provide the same system call interface. > Some of them give an error for rename("foo","foo"), some delete "foo", > some do nothing. Geez, talk about "people unclear on the concept...." The discussion is about a bug in some UFS implementations! This does NOT mean that there is some inherent flaw in the VFS mechanism. This means that there is a flaw in some UFS implementations; that particular flaw is fixed in SunOS 4.0. > NFS and VFS do not guarantee the same semantics for different file > systems. No shit, Sherlock! VFS was *NEVER INTENDED* to support the exact same semantics on all file systems; in some sense, that's the *point* of it! If, as, and when "/proc" is implemented under VFS, for example, try doing a "rename" on it; you're likely to be rudely surprised if you expect it to work - UNIX generally considers it impolite to change process IDs of processes on the fly. It might be amusing to consider having "unlink" do a "kill(pid, SIGKILL)", but that wouldn't conform to UNIX semantics either, unless you made the "/proc" directory sticky. Making a particular file system fully or partially support the same semantics as a UNIX on-disk file system is the responsibility of the author of the file system code; it is NOT the responsibility of the VFS mechanism. NFS was not designed to fully support UNIX file system semantics; some design compromises were made in order to achieve goals other than full UNIX file system semantics. You can debate whether these design compromises were correct or not (I don't plan to do so; it's not germane to this discussion), but that's a different matter.