[net.bugs.v7] TIOCSETN and CBREAK/RAW

rrg (03/26/83)

There is a bug in the V7 tty driver involving the use of the TIOCSETN
ioctl to switch into and out of CBREAK or RAW mode.  The problem occurs
when a line is typed ahead while the terminal is in cooked mode, but is
read by the program after a switch has been made to CBREAK/RAW.  On
input a "delimiter" (ASCII 0377) is entered into the raw queue after
the newline is typed, and the delimiter-count (tp->t_delct) is
incremented.  After the mode switch, the 0377 is still in the queue and
will be visible to the program; the manual warns of "garbage input" in
this case, which is exactly what happens.  But t_delct, which is
non-zero, is never touched while the terminal is in RAW or CBREAK mode,
and when the switch back to cooked mode is made (via another TIOCSETN)
the non-zero t_delct triggers the raw-to-canonical list-processing as
soon as another read is issued on the terminal.  Since the raw queue is
empty at this point, the program issuing the read sees a false
indication of end-of-file.  The problem doesn't occur if TIOCSETP is
used because in that case the queue is flushed and t_delct is zeroed.

The bug showed up most often locally when using "vi" or "more" and
typing a search-pattern while the program was starting up; upon exit
from the program the Shell would immediately see end-of-file and you
would be logged off.

A suggested fix is shown below; the delimiter-count is zeroed when
switching into CBREAK or RAW mode.  (And to make sure the same thing
doesn't happen in reverse--a character introduced in RAW mode being
treated as a delimiter after the switch to cooked mode--t_delct is
decremented in cooked mode upon encountering a delimiter only if it's
still positive.  This is much less likely to cause problems, however.)

A better fix would be to do some sort of processing on the raw queue
when the switch is made to CBREAK/RAW, to remove the delimiters from
the input (along with something similar on return to cooked mode); this
would fix the "garbage input" problem as well.

The bug may also exist in the old and new tty drivers in 4.1bsd, but I
haven't specifically checked for it.

The exact line numbers in the diff listing which follows probably don't
correspond to the "original".

----------------------------------------------------------------------
*** tty.c.old	Fri Mar 25 22:40:11 1983
--- tty.c	Fri Mar 25 22:40:22 1983
***************
*** 231,244
	case TIOCSETP:
		wflushtty(tp);
	case TIOCSETN:
		if (copyin(addr, (caddr_t)&iocb, sizeof(iocb))) {
			u.u_error = EFAULT;
			return(1);
		}
		tp->t_ispeed = iocb.ioc_ispeed;
  		tp->t_ospeed = iocb.ioc_ospeed;
  		tp->t_erase = iocb.ioc_erase;
  		tp->t_kill = iocb.ioc_kill;
  		tp->t_flags = iocb.ioc_flags;
  		break;
  

--- 253,271 -----
	case TIOCSETP:
		wflushtty(tp);
	case TIOCSETN:
		if (copyin(addr, (caddr_t)&iocb, sizeof(iocb))) {
			u.u_error = EFAULT;
			return(1);
		}
		tp->t_ispeed = iocb.ioc_ispeed;
		tp->t_ospeed = iocb.ioc_ospeed;
  		tp->t_erase = iocb.ioc_erase;
  		tp->t_kill = iocb.ioc_kill;
+ 		/*
+ 		 * Reset delimiter count if switching into CBREAK or RAW.
+ 		 */
+ 		if (iocb.ioc_flags&(CBREAK|RAW))
+ 			tp->t_delct = 0;
  		tp->t_flags = iocb.ioc_flags;
  		break;
  
***************
*** 364,370
  	while ((c=getc(&tp->t_rawq)) >= 0) {
  		if ((tp->t_flags&(RAW|CBREAK))==0) {
  			if (c==0377) {
! 				tp->t_delct--;
  				break;
  			}
  			if (bp[-1]!='\\') {

--- 391,398 -----
  	while ((c=getc(&tp->t_rawq)) >= 0) {
  		if ((tp->t_flags&(RAW|CBREAK))==0) {
  			if (c==0377) {
! 				if (tp->t_delct > 0)
! 					tp->t_delct--;
  				break;
  			}
  			if (bp[-1]!='\\') {
----------------------------------------------------------------------

				Ron Gomes
				Human Computing Resources Corp.