[comp.unix.wizards] Bug in System V setpgrp

xsimon@its63b.UUCP (06/09/87)

There would seem to be a rather serious flaw in the System V setpgrp()
system call (for disassociating oneself from a terminal and setting oneself
up as a process-group header). Setpgrp is pretty much like:
	setpgrp()
	{
		...
		if (p->p_pid != p->p_pgrp)
			u.u_ttyp = NULL;
		p->p_pgrp = p->p_pid;
	}

The problem occurs when you try to grab yourself a new terminal with open().
In openi(), it sees if what you are opening is a character-special, and if
so uses its open routine (and sets the indirect tty to be this, if you don't
have a tty already):
	openi(...)
	{
		...
		case IFCHR:
			...
			if (u.u_ttyp == NULL) u.u_ttyd = dev;
			(*cswdev[major(dev).open)(...);
			...
		...
	}

And of course, in ttopen (and also sxtopen, for opening an SXT device):

	if ( p->p_pid == p->p_pgrp
	  && u.u_ttyp == NULL
	  && tp->t_pgrp == 0 ) {
		u.u_ttyp = &tp->t_pgrp;
		tp->t_pgrp = u.u_procp->p_pgrp;
	}

So, the acquiring of a new controlling terminal (with u.u_ttyp and u.u_ttyd)
only occurs when opening a new device (which isn't already somebody else's
controlling tty), when you are a process-group header (ie, pid==pgrp), and
when your u.u_ttyp is NULL.

********** THE COMPLAINT ... ***********

BUT... setpgrp() only sets u.u_ttyp to be NULL if pid!=pgrp (ie, you weren't
already a process-group leader)! This means that any process that is a
process group leader already (such as a login-shell) CANNOT change its
controlling terminal. WHY SHOULD THIS BE?

****************************************

(I came across this problem when I wanted to change the controlling terminal
of a login-shell to be an SXT device, so that job-control would work).

Any suggestions why setpgrp() is done this way? Is it a bug, or is it actually
necessary? Somebody must have deliberately decided to make the test for
"if (p->p_pid != p->p_pgrp) ..." in setpgrp(), so it would seem a strange
sort of bug to creep in unnoticed!



-- 
----------------------------------
| Simon Brown 		         | UUCP:  seismo!mcvax!ukc!{its63b,cstvax}!simon
| Department of Computer Science | JANET: simon@uk.ac.ed.{its63b,cstvax}
| University of Edinburgh,       | ARPA:  simon%{its63b,cstvax}.ed.ac.uk ...
| Scotland, UK.			 |				@cs.ucl.ac.uk
----------------------------------	 "Life's like that, you know"