huff@svax.cs.cornell.edu (Richard Huff) (02/28/90)
I guess that most people are used to using fork() and not having a shared address space between processes. We could eliminate fork() and exec() in favor of create() and activate(), where: create(program, procedure, argv, argc) ---- creates a new process to execute your favorite procedure out of your favorite program. The stack of the child process is initially empty; i.e., if the procedure returns, then the child terminates. Here's the twist --- the process is initially suspended. activate(child) ---- yep, it awakens the initially suspended child! :-) Here's the key point: between the create() and the activate(), the parent process can do all of those wonderful things that others have moaned and groaned need to be done between a fork() and an exec(). We need only have separate kernel calls that allow a parent process to manipulate the environment of its suspended children. So you could set up your child's stdin, stdout, etc. In fact, this method is even MORE general, because the parent process could (if your OS was willing) modify the child's global variables before letting it run; whereas with exec(), the child's global variables are reinitialized from the new core image. (This raises issues of protection that I don't really care about; so let's not flame over that issue, ok?) And of course this method, which requires true sharing of data between processes, does NOT need to copy the parent's stack or global data. If you really need the child to have the same global data as you have, then you could write your own block copy loop; if you don't need such copying, then create() and activate() won't do it --- whereas fork() always will, whether by COW or otherwise. So this new methodology is more efficient that fork()/exec(). So, as I said before, we do NOT need COW to efficiently support the spawning of new processes. We need only avoid a brain dead fork(). Richard Huff huff@svax.cs.cornell.edu
johnl@esegue.segue.boston.ma.us (John R. Levine) (03/01/90)
In article <37873@cornell.UUCP> huff@cs.cornell.edu (Richard Huff) writes: >We could eliminate fork() and >exec() in favor of create() and activate(), where: > ... >Here's the key point: between the create() and the activate(), the >parent process can do all of those wonderful things that others have >moaned and groaned need to be done between a fork() and an exec(). We >need only have separate kernel calls that allow a parent process to >manipulate the environment of its suspended children. Uh huh. We get rid of fork() and add a hundred new remote context manipulation calls, hoping that we remember every single thing that a process would want to do to itself, and trying to remind ourselves that every time in the future we add some new item to the process context, we have to provide a way not just for a process to change its own context, but also a different way to change the context of other processes, and then invent a whole new set of protection rules about what processes are allowed to change what context bits in what other processes. This is progress? No thanks. -- John R. Levine, Segue Software, POB 349, Cambridge MA 02238, +1 617 864 9650 johnl@esegue.segue.boston.ma.us, {ima|lotus|spdcc}!esegue!johnl "Now, we are all jelly doughnuts."
terry@uts.amdahl.com (Lewis T. Flynn) (03/02/90)
In article <1990Feb28.174312.7640@esegue.segue.boston.ma.us> johnl@esegue.segue.boston.ma.us (John R. Levine) writes: >In article <37873@cornell.UUCP> huff@cs.cornell.edu (Richard Huff) writes: >>We could eliminate fork() and >>exec() in favor of create() and activate(), where: >> ... >>Here's the key point: between the create() and the activate(), the >>parent process can do all of those wonderful things that others have >>moaned and groaned need to be done between a fork() and an exec(). We >>need only have separate kernel calls that allow a parent process to >>manipulate the environment of its suspended children. > >Uh huh. We get rid of fork() and add a hundred new remote context >manipulation calls, hoping that we remember every single thing that a process >would want to do to itself, and trying to remind ourselves that every time in >the future we add some new item to the process context, we have to provide a >way not just for a process to change its own context, but also a different way >to change the context of other processes, and then invent a whole new set of >protection rules about what processes are allowed to change what context bits >in what other processes. This is progress? No thanks. Mr Huff's scenario is approximately what I was used to doing before I switched to working on Unix. I found it to be a bit more cumbersome to use than fork(), but considerably more efficient and flexible. It lends itself well to automation by means of a library of functions. Terry #include <disclaimer.std>
pasek@ncrcce.StPaul.NCR.COM (Michael A. Pasek) (03/03/90)
In article <37873@cornell.UUCP> huff@cs.cornell.edu (Richard Huff) writes: >I guess that most people are used to using fork() [remainder of a very > long article regarding a Unix system call, its faults, and a proposed > solution have been deleted] What does this have to do with comp.arch ? PLEASE keep OS-specific or language-specific stuff in its appropriate (its own) newgroup. If there isn't one, use comp.misc..... M. A. P.
phg@cs.brown.edu (Peter H. Golde) (03/03/90)
In article <37873@cornell.UUCP> huff@cs.cornell.edu (Richard Huff) writes: >I guess that most people are used to using fork() and not having a >shared address space between processes. We could eliminate fork() and >exec() in favor of create() and activate(), where: >......... I fail to see what is wrong the simplest solution of all: Just a spawn(program, argv, envp) call. To change the childs file handles, for example, you a) switch your file handles around b) spawn the child c) switch your file handle back This is easy to do with dup() and dup2(). Other things can be handled similarly.. More complex uses of fork can be done better using multiple threads (lightweight processes). OS/2 for one uses this approach, and it seems to be pretty easy to use, without the goofiness and peformance problems of fork(), which can be considerable on some architectures. --Peter Golde
lkaplan@bbn.com (Larry Kaplan) (07/09/90)
I wrote: >> The idea behind >> copy-on-write is that a proper fork() requires NO EXTRA physical memory for >> the child process (except that required for kernel data structures and page >> tables). You probably end up needing one stack page pretty soon. >> This implementation of fork() is >> what fork() was always intended to be, as far as I can tell. By doing fork() >> correctly, the need for a separate vfork() disappears (as stated in the BSD >> man pages). In article <3830012@hpcupt1.HP.COM> renglish@hpcupt1.HP.COM (Robert English) writes: >Ain't necessarily so, for a couple of reasons. > >First, while vfork() was a hack intended to get around the absence of a >copy-on-write scheme, it has different semantics from a regular fork(), >and those semantics can sometimes be useful. /bin/csh, for example, >takes advantage of them. Such use may not be wise, but it does exist, >and if you just get rid of vfork(), there will be programs that break. >Not many, but some. While this is true, it is very unfortunate. This problem is of the RTFM (read the @#$%& manual) form. It says right there: "Users should not depend on the memory sharing semantics of vfork() ..." If they do, the code is "incorrect". We did have to spend some small amount of time addressing this problem in various system programs. >Second, and more important from an architectural point of view, >vfork-exec is usually faster than fork-exec, even when copy-on-write >is implemented. Whenever you fork a process, you have to copy its data >structures. For a large process, the work of setting up an entire >virtual memory structure for a process and then immediately tearing it >down can be significant, even when the process's data is not actually >copied. Copying system data structures is indeed a non-zero cost operation. However, typical Unix programs have a relatively simple memory structure. Using MACH style maps, entries and objects requires very little to describe the process's address space. The data structure allocations would require about six zone allocations (from preallocated linked lists) and filling in the fields of these structures. This can't take very long and seems worth the effort. As long as the process has this Unix style map (only 3 segments), this method is virtually fixed in cost no matter what the address space size. Adjusting the page tables of the parent to support copy-on-write (COW) can take some time proportional to the address space currently used in the parent but is again a relatively cheap operation. While I can't claim the fork-exec with COW is faster than vfork-exec, I claim that it is not significantly slower in most cases and the COW implementation provides significant advantages in other areas such as parallel programming. Note that vfork() has other problems in certain multiprocessing memory architectures. > >Finally, all of this discussion misses the point that the normal >performance-critical sequence is a fork followed by some kernel data >structure manipulations and an exec, and that both performance and >purity could be achieved by providing a single system call that performs >the whole sequence. Seems true, though I don't know of anyone who has done this in UNIX. One problem might be the file descriptor maintanance and other little things the parent likes to have done in the child before it execs. #include <std_disclaimer> _______________________________________________________________________________ ____ \ / ____ Laurence S. Kaplan | \ 0 / | BBN Advanced Computers lkaplan@bbn.com \____|||____/ 10 Fawcett St. (617) 873-2431 /__/ | \__\ Cambridge, MA 02238
david@eng.sun.com (Big Ed's Gas Farm) (07/10/90)
>While I can't claim the fork-exec with COW >is faster than vfork-exec, I claim that it is not significantly slower in >most cases... I timed a simple test program on a Sun 4/60 running SunOS 4.1 (has COW): #include <signal.h> #include <vfork.h> main(argc) int argc; { int i, pid; signal(SIGCHLD, SIG_IGN); for (i = 0; i < 10000; i++) { pid = argc > 1 ? vfork() : fork(); if (pid == 0) exit(0); } } % time ./a.out (fork) 0.2u 4.3s 0:09 47% 0+140k 0+0io 0pf+0w % time ./a.out x (vfork) 0.0u 4.1s 0:04 92% 0+140k 0+0io 0pf+0w fork() seems to take twice as long as vfork() for a small process. Of course if exec() is sufficiently slow you're right, this will not be significant. -- David DiGiacomo, Sun Microsystems, Mt. View, CA david@eng.sun.com
chip@tct.uucp (Chip Salzenberg) (07/10/90)
According to david@eng.sun.com (Big Ed's Gas Farm): >>While I can't claim the fork-exec with COW >>is faster than vfork-exec, I claim that it is not significantly slower in >>most cases... > >I timed a simple test program on a Sun 4/60 running SunOS 4.1 (has COW): >fork() seems to take twice as long as vfork() for a small process. Thanks for the test. However, there are questions to be answered: How much memory is available? What are the static and dynamic sizes of the test program? Why doesn't the test program wait(), while most real programs do? (Resource usage will go up if many children are live simultaneously.) -- Chip Salzenberg at ComDev/TCT <chip@tct.uucp>, <uunet!ateng!tct!chip>
lkaplan@bbn.com (Larry Kaplan) (07/10/90)
In article <476@exodus.Eng.Sun.COM> david@eng.sun.com (Big Ed's Gas Farm) writes: >I timed a simple test program on a Sun 4/60 running SunOS 4.1 (has COW): > > (code was presented here) > >% time ./a.out (fork) >0.2u 4.3s 0:09 47% 0+140k 0+0io 0pf+0w >% time ./a.out x (vfork) >0.0u 4.1s 0:04 92% 0+140k 0+0io 0pf+0w > >fork() seems to take twice as long as vfork() for a small process. > >Of course if exec() is sufficiently slow you're right, this will not be >significant. > >-- >David DiGiacomo, Sun Microsystems, Mt. View, CA david@eng.sun.com I have some trouble with these numbers. The system times are virtually identical. In addition, calling exit() after a vfork() is not usually a good idea. Also, since the argument is fork-exec vs. vfork-exec, this timing is fairly inappropriate anyway. The proper thing would be a time call of some form before the (v)fork() call and one in the program being exec'd. Does SunOS 4.1 really have COW? Where is it documented? This would be nice to see. #include <std_disclaimer> _______________________________________________________________________________ ____ \ / ____ Laurence S. Kaplan | \ 0 / | BBN Advanced Computers lkaplan@bbn.com \____|||____/ 10 Fawcett St. (617) 873-2431 /__/ | \__\ Cambridge, MA 02238
peter@ficc.ferranti.com (Peter da Silva) (07/10/90)
In article <476@exodus.Eng.Sun.COM> david@eng.sun.com (Big Ed's Gas Farm) writes: > % time ./a.out (fork) > 0.2u 4.3s 0:09 47% 0+140k 0+0io 0pf+0w > % time ./a.out x (vfork) > 0.0u 4.1s 0:04 92% 0+140k 0+0io 0pf+0w > fork() seems to take twice as long as vfork() for a small process. I don't know about that... look at more than the 0:09. Both fork and vfork took about the same amount of system time, which is where you'd expect to see the extra overhead. There's also an extra second or so missing from the fork version... what else was running on the machine? -- Peter da Silva. `-_-' +1 713 274 5180. <peter@ficc.ferranti.com>
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (07/11/90)
In article <476@exodus.Eng.Sun.COM> david@eng.sun.com (Big Ed's Gas Farm) writes: >I timed a simple test program on a Sun 4/60 running SunOS 4.1 (has COW): >fork() seems to take twice as long as vfork() for a small process. But, you should have measured fork() time of a large process. >#include <signal.h> >#include <vfork.h> ! !#define SIZE (100*1024*1024) !char mem[SIZE]; > >main(argc) > int argc; >{ > int i, pid; > > signal(SIGCHLD, SIG_IGN); ! ! for(i=0;i<SIZE;i+=1024) /* to make pages dirty */ ! mem[i]=1; > > for (i = 0; i < 10000; i++) { > pid = argc > 1 ? vfork() : fork(); > if (pid == 0) ! _exit(0); ! else ! wait(0); /* unless you want to spwan 10000 processes ! at the same time */ > } >} Masataka Ohta
guy@auspex.auspex.com (Guy Harris) (07/24/90)
>Does SunOS 4.1 really have COW? Yes; so did 4.0. Was this really not generally known? >Where is it documented? *Virtual Memory Architecture in SunOS*, Summer 1987 USENIX Proceedings. *SunOS Virtual Memory Implementation*, some recent EUUG procedings (it even uses the phrase "copy-on-write", unlike some of the other references). SunOS 4.0 MMAP(2) manual page: MAP_SHARED and MAP_PRIVATE describe the disposition of write references to the memory object. If MAP_SHARED is speci- fied, write references will change the memory object. If MAP_PRIVATE is specified, the initial write reference will create a private copy of the memory object page and redirect the mapping to the copy. The mapping type is retained across a fork(2). etc.. (BTW, for those who insist on making some stupid BSD vs. S5 debate out of this, note that S5R4 has "vfork()", and that the S5R4 "vfork()" man page includes a warning from which one could possibly infer that it operates similarly to the 4.xBSD and SunOS 4.x "vfork()". This may have been done for the benefit of code written by people incapable of reading, or incapable of comprehending, the "don't depend on this sharing!" stuff in the BSD manual page....)
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) (07/24/90)
In article <3733@auspex.auspex.com> guy@auspex.auspex.com (Guy Harris) writes: >(BTW, for those who insist on making some stupid BSD vs. S5 debate out >of this, note that S5R4 has "vfork()", and that the S5R4 "vfork()" man >page includes a warning from which one could possibly infer that it >operates similarly to the 4.xBSD and SunOS 4.x "vfork()". NeXT (thus MACH) man page also have the same description, though, actually, vfork is identical to fork. Vfork on NeXT dosen't even wait for the completion or execing of child process. Masataka Ohta