[comp.soft-sys.andrew] "typescript" and "tm" under SunOS 4.1 - a workaround

guy@auspex.auspex.com (Guy Harris) (10/05/90)

The reason why "typescript" and "tm" fail under 4.1 is, in a word:

POSIX.

From IEEE Standard 1003.1:

	2.3 General Terms.  The following are definitions of terms
	peculiar to this standard:

	...

	controlling process.  The session leader that established the
	connection to the controlling terminal. ...

	...

	3.2 Process Termination.  There are two kinds of process
	  termination:

	   (1) "Normal termination" occurs by a return from "main()" or
	   when requested with the "exit()" or "_exit()" functions.

	...

	  ... Both "exit()" and "abort()" shall terminate a process wiht
	the consequences specified in "_exit()" section 3.2.2 ...

	3.2.2  Terminate a Process.

     Function: _exit()

	3.2.2.2 Description.  The "_exit()" function shall terminate the
     calling process with the following consequences:

	...

       (6) If the process is a controlling process, the SIGHUP signal
       shall be sent to each process in the foreground process group of
       the controlling terminal belonging to the calling process.

	...

	7.1.1.3 The Controlling Terminal.

	...

	  When a controlling process terminates, the controlling
	terminal is disassociated from the current session, allowing it
	to be acquired by a new session leader.  Subsequent access to
	the terminal by other processes in the earlier session may be
	denied, with attempts to access the terminal treated as if modem
	disconnect had been sensed.

And from "atk/typescript/tscript.c"

	#if SY_AIX221 || SY_AIX12
	    if (! GetPtyandName(&masterChannel, &ptyChannel,ptyname,MAXPATHLEN))  {
	#else /*if SY_AIX221 */
	    if (! GetPty(&masterChannel, &ptyChannel))  {
	#endif /*if SY_AIX221 */
		printf ("Can't connect subchannel\n");
		return FALSE;
	    }

	...

	    if ((pid = osi_vfork ()) < 0) {
		printf ("Fork failed\n");
		return FALSE;
	    }
	    if (pid == 0) {
		...
		execvp (tp->progname, arglist);
		...
	    }

Before the GetPty or GetPtyandName is executed, "typescript" has
arranged that it will be a session leader (at least in SunOS 4.1, and
possibly in other POSIX-conforming systems as well); see the
"Disassociate thie process from its controling(sic) tty" comment, which
states that this "Must be done after opening the childs pty because that
will become the controlling tty if we don't have one.", except that it's
done *before* opening the child's pseudo-tty, not *after*.

"GetPty" and "GetPtyandName" will search for a free pseudo-tty by
opening all the pseudo-tty masters; if it finds one that it can open, it
also tries to open the slave.  Since this open does *not* use the POSIX
O_NOCTTY flag, on many UNIX systems (SunOS 4.1 among them), this means
that if the process doing the "GetPty*" call is a session leader, it
becomes a controlling process and the pseudo-tty becomes that session's
controlling terminal.

Now, this doesn't cause immediate problems for, say, "xterm" (it may
cause some problems, but to first order it works).  That's because
"xterm" doesn't try to help the user out by forking itself off.  Andrew
applications, however, *do* fork themselves off, and in particular
"typescript" and "tm" do so *after* they've become a controlling
process.  The parent - i.e., the controlling process - exits, which
causes all the consequences described above; SIGHUPs and possibly denial
of access to the pseudo-tty slave by processes in the session, including
the shell.

The promised workaround is "use the '-d' flag to {typescript,tm}" and
run the {typescript,tm} in the background yourself, e.g. 'typescript&'."
That seemed to work OK when I tried it.

If you don't care whether Andrew builds on systems not having O_NOCTTY,
you can modify "GetPty*" to open the slave pty with O_NOCTTY, and modify
"typescript" and "tm" to do what the comment mentioned above says it
should, namely open the pseudo-tty for real (i.e., without O_NOCTTY) in
the child process (i.e., the one that runs the shell); this may fix the
problem (at least on *some* POSIX-conforming systems; others, including
I think 4.4BSD, will *never* give you a controlling terminal merely as a
result of opening a terminal, you have to issue an explicit "ioctl" to
get a controlling tty - this is what UNIX should have done when
controlling ttys first showed up, but, well, ...).

I haven't spent much time trying to come up with a scheme that lets you,
for example, build Andrew under SunOS 4.0[.x] and have the resulting
Andrew work under 4.0[.x] and 4.1.  I'm still looking forward to the day
when the last non-POSIX-compliant UNIX system is plowed under, along
with the millions of #ifdefs that lack of a standard such as POSIX made
necessary....