dg@lakart.UUCP (David Goodenough) (01/31/89)
There is doubtless a good reason for it, but why is it not possible to unlink a demand paged program that is executing. I realise that the system needs to keep the program text somewhere, but why can it not do the same as is done with a normally open file: just remove the directory entry, but hang on to the I-node and disk blocks until such time as the process execl()'s or exit()'s. I am probably missing something, but I don't see the difference between a d.p. executable being used, and me opening a file and going ... seek(); read(); ... seek(); read(); ... up and down it. So what's the scoop? -- dg@lakart.UUCP - David Goodenough +---+ IHS | +-+-+ ....... !harvard!xait!lakart!dg +-+-+ | AKA: dg%lakart.uucp@xait.xerox.com +---+
maart@cs.vu.nl (Maarten Litmaath) (02/02/89)
dg@lakart.UUCP (David Goodenough) writes: \... why is it not possible to unlink a demand paged program that is executing. Huh? On 4.3BSD: % cat > c.c main() { pause(); } % cc c.c % file a.out a.out: demand paged pure executable not stripped % a.out & [1] 25056 % : > a.out a.out: Text file busy % /bin/rm a.out % jobs -l [1] + 25056 Running a.out % kill %1 [1] Terminated a.out % -- "Does she play, er, tennis? |Maarten Litmaath @ VU Amsterdam: Wink wink, notch notch!" |maart@cs.vu.nl, mcvax!botter!maart
guy@auspex.UUCP (Guy Harris) (02/02/89)
>There is doubtless a good reason for it, No, not really. >but why is it not possible to unlink a demand paged program that is >executing. It has nothing to do with demand paging. In fact, V7 imposed this restriction, and as distributed by AT&T, V7 didn't support demand paging. 4.xBSD *does* support demand paging, and Berkeley *removed* this restriction because it was dumb. >I realise that the system needs to keep the program text somewhere, but >why can it not do the same as is done with a normally open file: just >remove the directory entry, but hang on to the I-node and disk blocks >until such time as the process execl()'s or exit()'s. I am probably missing >something, No, you are not missing anything. You are 100% correct. The "reason", as I'd heard it, is that they didn't want to have files with no links lying around after a crash, and if they let you unlink a file while it's running, and the system crashes before the last process running that program exits or "exec"s some other program, the file for that program will be such a file. It is left as an exercise to the reader to figure out why the restriction in question won't make this "problem" go away. (Hint: the original poster's argument applies here, too....) The correct way to deal with those files is to use a file system salvager better than the abysmal "icheck" and "dcheck" that came out with the original V7; "fsck", fortunately, appeared on an addendum tape to V7, and will clean up files such as that quite nicely.
les@chinet.chi.il.us (Leslie Mikesell) (02/02/89)
In article <409@lakart.UUCP> dg@lakart.UUCP (David Goodenough) writes: >There is doubtless a good reason for it, but why is it not possible to >unlink a demand paged program that is executing. I realise that >the system needs to keep the program text somewhere, but why can it not >do the same as is done with a normally open file: just remove the >directory entry, but hang on to the I-node and disk blocks until such >time as the process execl()'s or exit()'s. I am probably missing >something, but I don't see the difference between a d.p. executable >being used, and me opening a file and going >... seek(); read(); ... seek(); read(); ... I don't have the answer to the question but there is a work-around. Just make a link to the running program somewhere else, then you can remove the original link. If you link under the same name but later in users PATH you can assure that anyone executing the program will get either the old or the new copy (instead of an error) when you install new programs. Now, if a.outs need this kind of protection (and they do) why isn't the same thing done for shell scripts - i.e prevent them from being modified while running? Les Mikesell
brad@polaris.ucsc.edu (Brad Smith) (02/03/89)
Two questions/points on this: - Wouldn't there be some problems with other links to a file being invalidated (pointing to the old version) if the file was unlinked, a new inode allocated, and a new link created to copy over an executing program? - It seems the reason for not locking shell scripts is that virtual memory for the executable is not being paged out of the file containing the script. Some program (typically a shell) is being run with the contents of the script as input... or something to that effect. Brad Smith CIS Board, UC Santa Cruz brad@saturn.ucsc.edu
trt@rti.UUCP (Thomas Truscott) (02/03/89)
In article <1997@solo12.cs.vu.nl>, maart@cs.vu.nl (Maarten Litmaath) writes: > dg@lakart.UUCP (David Goodenough) writes: > > why is it not possible to unlink a demand paged program that is executing. > > Huh? On 4.3BSD: ... it works fine This is a non-portable feature. As far as I can tell it is non-portable due to buggy implementations, or maybe just paranoid implementations. In most System V systems you cannot unlink the last on-disk link to an executing program. There is no particular reason for this, just remove the check (easiest if you have kernel source!) and everything works fine. In Ultrix 2.x you could unlink an executing demand-paged program, but the file blocks would not be returned to the free list and must be reclaimed with fsck (really!). This is fixed in Ultrix 3.0 -- unlinking an executing program works just fine now. In HP-UX 6.2 you cannot unlink an executing demand paged (or shared) program, regardless of how many other on-disk links there are. I have no idea why this change was made (HP-UX 6.0 did not have the restriction), perhaps someone with access to the source can tell us. (e.g. did this fix a problem ala Ultrix 2.x?). Perhaps some HP-UX wizard could get in touch with me, I sure would like a work-around for this. Tom Truscott (919) 541-7005
jfh@rpp386.Dallas.TX.US (John F. Haugh II) (02/03/89)
In article <7599@chinet.chi.il.us> les@chinet.chi.il.us (Leslie Mikesell) writes: >Now, if a.outs need this kind of protection (and they do) why isn't >the same thing done for shell scripts - i.e prevent them from being >modified while running? Shell scripts should be protected from modification during execution, but you weren't dicussing preventing pure-text files from being opened for writing. There _is_ a need to prevent text prototypes from being modified. There is not a need to prevent a pure-text file from being unlinked. The kernel is capable of recovering the disk space when the last reference [ loaded process ] goes away. Disallowing write permission on pure-text files should be all that is needed. [ And disallowing execution of pure-text files which are opened for writing ] -- John F. Haugh II +--Quote of the Week:------------------ VoiceNet: (214) 250-3311 Data: -6272 | "Get it through your head: InterNet: jfh@rpp386.Dallas.TX.US | CARS ARE THE ENEMY." UucpNet : <backbone>!killer!rpp386!jfh +------ -- Bob Fishell ----------
ray@spca6.UUCP (ray) (02/03/89)
In article <2765@rti.UUCP> trt@rti.UUCP (Thomas Truscott) writes: >In article <1997@solo12.cs.vu.nl>, maart@cs.vu.nl (Maarten Litmaath) writes: >> dg@lakart.UUCP (David Goodenough) writes: >> > why is it not possible to unlink a demand paged program that is executing. >> >> Huh? On 4.3BSD: ... it works fine >In most System V systems you cannot unlink the last >on-disk link to an executing program. That is, only if you link the program so that it has a shared text segment. This is the default on the Sys Vs I work on, but in all of them you can force the linker to put text and data in the same region(segment.) When linked in this fashion not only can you unlink the a.out you can overwrite it. >There is no particular reason for this, just remove the check >(easiest if you have kernel source!) and everything works fine. Problem I see for a.outs that have shared text. This is all from reading the Bach book, I have no access to source. From the Bach book page 223, "...the kernel searches the active region list for the file's text region, identifying it as the one whose inode pointer matches the inode of the executable file." Consider what might happen if an a.out with shared text is started (process A). Next the actual file is allowed to be removed. Later an executable is created and happens to land at the same inode that the previous a.out used. Now, with process A still running (and thus its text region is still in the active list,) some one trys to run the new a.out. Seems to me, from reading Bach, the new process gets the old text segment and the new data segment. This SHOULD be LOADS of fun (particularly when data or bss in the new a.out is smaller than in the original (at least a core dump :-)).) A.outs with non-shared text, would not experience this problem since no search is made of the in memory active region list. Just my 2 cents worth. = ray mit-eddie!uccba!spca6!ray
jfh@rpp386.Dallas.TX.US (John F. Haugh II) (02/03/89)
In article <314@spca6.UUCP> ray@spca6.UUCP (ray) writes: >Problem I see for a.outs that have shared text. >This is all from reading the Bach book, I have no access to source. >From the Bach book page 223, >"...the kernel searches the active region list for the file's text region, >identifying it as the one whose inode pointer matches the inode of the >executable file." This is fairly accurate. There is a table of active text segments. It contains information regarding the number of references to a pure-text file, how many are loaded, if each is swapped, and so on. Stare at /usr/include/sys/text.h for more details. >Consider what might happen if an a.out with shared text is started (process A). >Next the actual file is allowed to be removed. >Later an executable is created and happens to land at the same inode that the >previous a.out used. Can't happen, unless the entire inode is scavenged. Consider that when an open file is removed, the system waits until the last reference goes away before recovering the inode and associated data blocks. Thus, that i-number is not available until the last reference to the inode is removed. Same thing should work with pure-text prototypes. -- John F. Haugh II +--Quote of the Week:------------------ VoiceNet: (214) 250-3311 Data: -6272 | "Get it through your head: InterNet: jfh@rpp386.Dallas.TX.US | CARS ARE THE ENEMY." UucpNet : <backbone>!killer!rpp386!jfh +------ -- Bob Fishell ----------
debra@alice.UUCP (Paul De Bra) (02/04/89)
In article <6211@saturn.ucsc.edu> brad@polaris.UUCP (Brad Smith) writes: }Two questions/points on this: } - Wouldn't there be some problems with other links to } a file being invalidated (pointing to the old version) } if the file was unlinked, a new inode allocated, and } a new link created to copy over an executing program? If you try to write over an existing file that is being executed that should fail. You can remove the file and create a new one, but indeed all other links to the old file will still point to the old file. If your system supports them it is better to use symbolic links to executable files, to avoid ending up with multiple versions of programs that once were the same. } - It seems the reason for not locking shell scripts is } that virtual memory for the executable is not being } paged out of the file containing the script. Some } program (typically a shell) is being run with the } contents of the script as input... or something to } that effect. The shell only reads and interprets the shell script. The script does not contain machine-code. The shell does not read the whole script before executing it however, so when you change a large script while it is being executed (i.e. read) the shell may find syntax errors because it suddenly continues to read the new script from an arbitrary point (a block boundary usually). Paul. -- ------------------------------------------------------ |debra@research.att.com | uunet!research!debra | ------------------------------------------------------
peter@ficc.uu.net (Peter da Silva) (02/04/89)
In article <314@spca6.UUCP>, ray@spca6.UUCP (ray) should probably go back to Bach's book and re-read the stuff about what happend when you delete an open file... > Consider what might happen if an a.out with shared text is started (process A). > Next the actual file is allowed to be removed. > Later an executable is created and happens to land at the same inode that the > previous a.out used. Can't happen. This is exactly equivalent to what happens when you delete an open file. What would happen is that the inode and file would hang around with the open count >= 1. When the last process using that image terminated, open count would == 0 and link count would == 0, and the file would be reclaimed. New files would get new inodes until reclamation. -- Peter da Silva, Xenix Support, Ferranti International Controls Corporation. Work: uunet.uu.net!ficc!peter, peter@ficc.uu.net, +1 713 274 5180. `-_-' Home: bigtex!texbell!sugar!peter, peter@sugar.uu.net. 'U` Opinions may not represent the policies of FICC or the Xenix Support group.
guy@auspex.UUCP (Guy Harris) (02/04/89)
>Now, if a.outs need this kind of protection (and they do) They most definitely do *not* need protection against unlinking, as Mr. Goodenough pointed out - the system is definitely smart enough not to blow away the file until the last *active* reference goes away, even if you remove the last link. They may, or may not, need protection against being written to; SunOS 4.0 provides no such protection, but I haven't found that to be a fatal problem - just use "mv" (or something that does the same sort of thing) to install a new version, which means processes currently using the image will continue to use an unmolested copy of the old version, and processes that use it subsequently will get the new version, and when the last process stops using the old version *poof* it goes away. >why isn't the same thing done for shell scripts - i.e prevent them from being >modified while running? Because the kernel has no idea that a shell script is "running". All it knows is that somebody has it open for reading. It would be considered rude to forcibly disallow writes to *any* file that somebody has open for reading, and UNIX, for better or worse, doesn't have the notion of "N-readers/1-writer" exclusion on "open"s.
guy@auspex.UUCP (Guy Harris) (02/04/89)
>In HP-UX 6.2 you cannot unlink an executing demand paged (or shared) >program, regardless of how many other on-disk links there are. >I have no idea why this change was made (HP-UX 6.0 did not have the >restriction), Amazingly enough, this stupid restriction is required by the SVID if you support shared text. (Check out UNLINK(BA_OS).) Sometimes "SVID" stands for "System V Implementation Description".... One way around this is to say you don't support shared text. SunOS will probably do so by Sun saying - quite correctly - that they don't support shared text; instead, they support read-only copy-on-write mappings to the file (yes, that really *is*, in effect, the way an "exec" of a 0413 executable is done; the net effect, of course, is that the pages are *still* shared - unless you try to run a debugger on the program and set breakpoints, in which case private copies are made of the pages in which breakpoints are set).
guy@auspex.UUCP (Guy Harris) (02/04/89)
>Consider what might happen if an a.out with shared text is started >(process A). >Next the actual file is allowed to be removed. >Later an executable is created and happens to land at the same inode that the >previous a.out used. Shouldn't happen on any reasonable implementation. The on-disk inode in question should *not* freed until the last *active* reference to it goes away, and shared texts using that file count as active references, so the inode should not be freed on disk until all processes running the program have stopped running it (either by exiting or "exec"ing some other program). I know this is the case in 4.xBSD, and was probably the case in V7 and S3 and older S5 implementations as well. I think it's true of paging S5 implementations as well. If it is *not* the case, the implementation is buggy; consider the case of somebody opening a file and then unlinking it (this is done by lots of programs). If the inode is recycled when the unlink is done, subsequent references to the file won't work, especially if the inode is then grabbed by another file....
les@chinet.chi.il.us (Leslie Mikesell) (02/04/89)
In article <6211@saturn.ucsc.edu> brad@polaris.UUCP (Brad Smith) writes: >Two questions/points on this: > > - Wouldn't there be some problems with other links to > a file being invalidated (pointing to the old version) > if the file was unlinked, a new inode allocated, and > a new link created to copy over an executing program? No - the sequence is: link old_name new_name unlink old_name (assuming no symlinks..) This gives new_name a pointer to the original inode and makes it possible to remove old_name since it is no longer the last link to the inode. mv will do this if the destination is on the same filesystem as the source. > - It seems the reason for not locking shell scripts is > that virtual memory for the executable is not being > paged out of the file containing the script. Some > program (typically a shell) is being run with the > contents of the script as input... or something to > that effect. Actually it is totally unrelated because scripts are treated as an ordinary file even though they are marked as executable. I just mentioned it because the person who thought of protecting demand paged executables should also have realized that scripts need equal protection. Of course, that person probably didn't have any control over the way shells handle their input. Does anyone have any useful trick examples of modifying a running shell script? How about horror stories of editing a program while cron is running it? Les Mikesell