hakanson@ogicse.ogc.edu (Marion Hakanson) (02/05/90)
In article <2276@uvaarpa.virginia.edu> kayvan@mrspoc.transact.com writes: >I wrote the following function to snatch pty pairs from the system, >which I put in pty.pl: >. . . >I then used it in a perl test of ptys and got strange results. >. . . > Ready> > abcd > Sent: 'abcd' > Got: 'I got abcd' > Child<abcd> > Child<I got abcd> > foo > Child<foo> > Sent: 'foo' > Got: 'I got I got abcd' > Child<I got I got abcd> >. . . >Looking at the code below, can anyone tell me what I'm doing wrong? >(Or is this a bug?) >. . . ># Copyright (C) 1990 Kayvan Sylvan <kayvan@mrspoc.transact.com> # ># Copying permitted under the terms of the GNU General Public License. # I was hoping someone else would answer this, but I guess everyone thinks I'm the "Perl-pty" expert. Hah! Anyway, I don't think this is a problem with pty's. It somewhat resembles the problem I had when I first tried using pty's, and I tracked it down to not being related to pty's at all. If you rewrote your program to use just plain pipes, you may see the same behavior (I recommend that you try it). The problem I have seen comes from "forked" file-handles, in my opinion. It does not affect all systems -- for example, 4.2bsd and 4.3bsd does not exhibit the behavior, but SunOS-4.0.3 does (in my tests). I recently had an insight into what may be the cause, for SunOS-4, anyway. I suspect that one would see the same problem if the file-handles were forked using "vfork" instead of plain fork. And in fact, Larry has done things so Perl cleverly avoids using vfork if the child is not going to immediately do an exec (or otherwise share some data which shouldn't be shared) -- instead he uses plain fork, which in a "normal" Unix system makes a complete copy of the parent process' address space for the child to run in. But, some modern Unix systems don't have a vfork anymore -- instead these systems with modern memory-management (with hardware support) do a copy-on-write fork. Thus the fork takes place, and no copy is done until the child tries to write to its copy of the data. It's really quite efficient, since you never have to copy anything until you actually need a separate copy, and you only copy those pieces (pages, usually) which are necessary. But I think that Perl operates under the assumption that after a fork (as opposed to a vfork), none of its data structures are shared between the parent and child. I haven't worked out the nitty-gritty details, but the behavior I have seen can be described as the parent writing to a forked filehandle, after which the parent sees both what it wrote to that filehandle as well as what the child wrote to it. In other words, the data gets duplicated, instead of being consumed by (just) the child. I'm not sure that you are seeing the same thing, but it looks similar. My intuition tells me that the implementation of the stdio subsystem is at fault here, but I have not examined any source code to confirm that hypothesis. Larry, I must say that this is one issue that makes me wish Perl used Unix file descriptors instead of C file pointers, but I certainly don't blame you for wanting to avoid implementing your own buffered I/O system. But it certainly would make forking a little less painful. -- Marion Hakanson Domain: hakanson@cse.ogi.edu UUCP : {hp-pcd,tektronix}!ogicse!hakanson
lwall@jpl-devvax.JPL.NASA.GOV (Larry Wall) (02/06/90)
In article <7115@ogicse.ogc.edu> hakanson@ogicse.UUCP (Marion Hakanson) writes:
: I haven't worked out the nitty-gritty details, but the behavior I have
: seen can be described as the parent writing to a forked filehandle,
: after which the parent sees both what it wrote to that filehandle as
: well as what the child wrote to it. In other words, the data gets
: duplicated, instead of being consumed by (just) the child. I'm not
: sure that you are seeing the same thing, but it looks similar.
:
: My intuition tells me that the implementation of the stdio subsystem
: is at fault here, but I have not examined any source code to confirm
: that hypothesis. Larry, I must say that this is one issue that makes
: me wish Perl used Unix file descriptors instead of C file pointers,
: but I certainly don't blame you for wanting to avoid implementing
: your own buffered I/O system. But it certainly would make forking
: a little less painful.
Note the disclaimer in the manual entry on fork:
fork Does a fork() call. Returns the child pid to the
parent process and 0 to the child process. Note:
unflushed buffers remain unflushed in both
processes, which means you may need to set $| to
avoid duplicate output.
The disclaimer should probably be propagated to other constructs that
do fork as well, such as open().
Larryhakanson@ogicse.ogc.edu (Marion Hakanson) (02/07/90)
In article <6989@jpl-devvax.JPL.NASA.GOV> lwall@jpl-devvax.JPL.NASA.GOV (Larry Wall) writes: >. . . >Note the disclaimer in the manual entry on fork: > > fork Does a fork() call. Returns the child pid to the > parent process and 0 to the child process. Note: > unflushed buffers remain unflushed in both > processes, which means you may need to set $| to > avoid duplicate output. This does look like the cause of the problem experienced by the original poster. In fact, I have reviewed my test scripts, and the only one I have which exhibits the weird feedback behavior I described is the "ptytst5.pl" script which I posted awhile back (you know, the one for which I'm "eternally beloved"). The example may very well be a special case, since it is one of the few ways I've seen for a Perl script to both read from and write to the same filehandle. I thought I understood some of what's going on, until I tried out the following: ==========fhtty.pl======== #!/usr/bin/perl # TTY filehandle test program -- 90/02/06 # Marion Hakanson (hakanson@cse.ogi.edu) # Oregon Graduate Institute of Science and Technology open(TTY, "+>/dev/tty") || die 'Cannot open /dev/tty, aborted'; select(TTY); $| = 1; select(STDOUT); $| = 1; for ($i=0; $i<3; $i++ ) { print TTY "TTY: TYPE Hello $i\n" || die "Cannot print to TTY, aborted"; sleep(5); $ttyin = <TTY>; $ttyin =~ tr/A-Z/a-z/; $ttyin =~ s/\r//g; print STDOUT "STDOUT: $ttyin"; } print TTY "$$: Done\n"; close(TTY); ========================== The idea is that you type "Hello x" as prompted as soon as you see the prompt, and wait for the next prompt before typing the next line. Results: ===========DYNIX-3.0 (Sequent-4.2bsd) output======== ogicse 127% ( stty -echo; ./fhtty.pl; stty echo ) TTY: TYPE Hello 0 STDOUT: tty: type hello 0 TTY: TYPE Hello 0 TTY: TYPE Hello 1 STDOUT: tty: type hello 0 TTY: TYPE Hello 0 TTY: TYPE Hello 2 STDOUT: tty: type hello 0 TTY: TYPE Hello 0 17069: Done ogicse 128% Hello: Command not found. ogicse 129% Hello: Command not found. ogicse 130% Hello: Command not found. ogicse 131% ======================== Note that the script never gets around to reading what I typed, and in fact never seems to get past reading back the first line it printed out (straight 4.3bsd behaves the same). It looks like some file pointers are being clobbered (or not updated), while the chars you type keep getting stuffed in the input buffer and not read. ===========SunOS-4.0.3 (Sun-3) output========= ansel 78% ( stty -echo; ./fhtty.pl; stty echo ) TTY: TYPE Hello 0 STDOUT: hello 0 Hello 0 TTY: TYPE Hello 1 STDOUT: hello 1 Hello 1 TTY: TYPE Hello 2 STDOUT: hello 2 Hello 2 10124: Done ansel 79% ============================== And here, although it looks a lot cleaner and more regular than the 4.xbsd case, I can't come up with a good explanation as to where the "Hello x" all-by-itself lines come from. But it is completely consistent with the ptytst5.pl behavior. Aren't these interactions between tty drivers and stdio wonderful? Anyway, note that the following works as expected (by me) in both of my test OS's. ==============fhfork3.pl============= #!/usr/bin/perl # Forked filehandle test program -- 90/02/06 # Marion Hakanson (hakanson@cse.ogi.edu) # Oregon Graduate Institute of Science and Technology open(TTY, "+>/dev/tty") || die 'Cannot open /dev/tty, aborted'; if ( fork ) { # parent select(TTY); $| = 1; select(STDOUT); $| = 1; for ($i=0; $i<3; $i++ ) { print TTY "$$: TYPE Hello $i\n" || die "Cannot print to TTY, aborted"; sleep(5); } print TTY "$$: Done\n"; close(TTY); } else { # child select(TTY); $| = 1; select(STDOUT); $| = 1; while ($ttyin = <TTY>) { $ttyin =~ tr/A-Z/a-z/; $ttyin =~ s/\r//g; print STDOUT "$$: $ttyin"; } close(TTY); } ================================== =========fhfork3.pl output============ ogicse 138% ( stty -echo; ./fhfork3.pl; stty echo ) 18840: TYPE Hello 0 18843: hello 0 18840: TYPE Hello 1 18843: hello 1 18840: TYPE Hello 2 18843: hello 2 18840: Done ogicse 139% ================================== So I guess I'm back to suspecting the SunOS-4.0.3 pty driver. Even if I had sources to SunOS, I doubt I'd want to track this one down.... -- Marion Hakanson Domain: hakanson@cse.ogi.edu UUCP : {hp-pcd,tektronix}!ogicse!hakanson