[comp.bugs.sys5] transient bug in vi nailed down

friedl@vsi.COM (Stephen J. Friedl) (01/02/89)

Pre-note note:

	In this note, `function key' means a terminfo description
	for a function key, an arrow key, or any other kind of
	special key.

Description: 

	Vi produces a vgoto error or a core dump if a function-key
	sequence is interrupted for terminals that have a
	character-insert mode.  Is this strange or what?

Easily reproduced on:

	Machine		Operating System
	--------	------------------------------------
	3B2/310		System V Release 2.0.4
	3B2/400		System V Release 3.1
	3B2/600		System V Release 3.2.1
	3B15		System V Release 3.1.1
	6386E		AT&T System V Release 3.1
	6386 		Interactive System V/386  Rel 1.0.5
	AST Prem/286	Microport System V/AT Rel 2.3

	(enough?)

Terminals:

	ANSI terminals with (1) some function keys defined that
	begin with ESC and (2) any kind of character-insert mode
	(smir/rmir or ich).  Note that this usually does NOT
	include the vt100 because it lacks the character-insert
	capability.

How to reproduce:

	Put the following in a file.  Don't include the `cut here'
	lines, and the file should have 11 lines and 35 characters.

#------ cut here -----
x
x
x
x
x
x
x
x
x
x
xxxxxxxxxxxxxx
#------ cut here -----

     Now, go to the bottom line and enter insert mode.  As soon
as you hit ESCAPE to exit insert mode, *immediately* hit DELETE or
whatever is your local interrupt character.  On all systems tried,
we get a `vgoto error' from vi and get stuck in ex mode.

     One some machines (3B2/400 and Microport, at least) deleting
one of the `x' lines and doing the same things will cause a core
dump -- maybe this will help you developer-kind-of-folks track
this down.

     This problem goes away if there are no function key
sequences that start with ESCAPE (\E) or if the terminal has no
kind of character insert mode (either smir/rmir or ich1/ich).
Unfortunately, this is not a very useful fix for the ANSI
terminals that many of use :-(.

     The problem seems to relate to the confusion between ESCAPE
as the exit-insert-mode character and ESCAPE as the leading
character for function keys.  If vi sees an ESCAPE character, it
reads the next character with some kind of timeout (say, one
second).  If a character is read *before* this timeout
terminates, it is assumed to be part of a function key definition
and processed accordingly.  If the timeout does terminate, it
must have been a regular old ESCAPE key from the user, and it
usually terminates insert or append mode.

     The bug seems to show up if the DELETE key is used to
terminate a timeout read -- perhaps a signal handler is messing
up somewhere (where is `volatile' when you need it :-).

     If you are trying to track down this bug, there is an
additional different and *very* unfriendly manifestation of same.
Edit an empty file, enter append mode, hit ESCAPE and *quickly*
interrupt.  Vi will start spewing characters at high speed,
ignoring XOFF and interrupts, and will generally lock up the
terminal.  DO NOT DO THIS unless you have another terminal nearby
to kill your job.  It will sometimes run forever and will
sometimes core dump.

     If anybody tracks this down, I would love to hear about
the source of the problem.  If anybody is unable to reproduce
this on the abovementioned systems, I can send the terminfo
source that causes the problem.

     Steve

-- 
Stephen J. Friedl        3B2-kind-of-guy            friedl@vsi.com
V-Systems, Inc.        I speak for me only      attmail!vsi!friedl
Santa Ana, CA  USA       +1 714 545 6442    {backbones}!vsi!friedl
-------Nancy Reagan on

chris@mimsy.UUCP (Chris Torek) (01/05/89)

Chances are this bug is at least related to one I just fixed (?)
in the 4BSD vi.  There is a better way; but this was easy.

RCS file:        RCS/ex_vget.c,v;   Working file:    ex_vget.c
head:            1.3
locks:           ;  strict
access list:
symbolic names:
comment leader:  " * "
total revisions: 3;    selected revisions: 3
description:
ex/vi ex_vget.c: input for open/visual
----------------------------
revision 1.3
date: 89/01/05 01:59:18;  author: chris;  state: Exp;  lines added/del: 14/14
beware the longjmp botch, my son
the traps that bite, the race in CATCH
beware the signal call, and shun
the high-load schedule (natch)
[well, at least it rhymes]
----------------------------
revision 1.2
date: 89/01/04 08:30:28;  author: chris;  state: Exp;  lines added/del: 32/29
4.3tahoe prerelease edition: changes for ANSI C and VMS
----------------------------
revision 1.1
date: 89/01/03 17:42:23;  author: chris;  state: Exp;
Initial revision
=============================================================================
RCS file: RCS/ex_vget.c,v
retrieving revision 1.2
diff -c2 -r1.2 ex_vget.c
*** /tmp/,RCSt1018927	Thu Jan  5 02:09:13 1989
--- ex_vget.c	Thu Jan  5 02:07:44 1989
***************
*** 656,684 ****
  #ifdef MDEBUG
  	if (trace)
! 		fprintf(trace,"\nfastpeekkey: ",c);
  #endif
  	Oint = signal(SIGINT, trapalarm);
! 	if (value(TIMEOUT) && inopen >= 0) {
! 		signal(SIGALRM, trapalarm);
  #ifdef MDEBUG
! 		alarm(10);
! 		if (trace)
! 			fprintf(trace, "set alarm ");
  #else
! 		alarm(1);
  #endif
! 	}
! 	CATCH
  		c = peekkey();
  #ifdef MDEBUG
! 	if (trace)
! 		fprintf(trace,"[OK]",c);
  #endif
- 		alarm(0);
  	ONERR
  		c = 0;
  #ifdef MDEBUG
! 	if (trace)
! 		fprintf(trace,"[TIMEOUT]",c);
  #endif
  	ENDCATCH
--- 656,684 ----
  #ifdef MDEBUG
  	if (trace)
! 		fprintf(trace,"\nfastpeekkey: ");
  #endif
  	Oint = signal(SIGINT, trapalarm);
! 	CATCH
! 		if (value(TIMEOUT) && inopen >= 0) {
! 			signal(SIGALRM, trapalarm);
  #ifdef MDEBUG
! 			alarm(10);
! 			if (trace)
! 				fprintf(trace, "set alarm ");
  #else
! 			alarm(1);
  #endif
! 		}
  		c = peekkey();
+ 		alarm(0);
  #ifdef MDEBUG
! 		if (trace)
! 			fprintf(trace,"[OK]");
  #endif
  	ONERR
  		c = 0;
  #ifdef MDEBUG
! 		if (trace)
! 			fprintf(trace,"[TIMEOUT]");
  #endif
  	ENDCATCH
-- 
In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 454 7163)
Domain:	chris@mimsy.umd.edu	Path:	uunet!mimsy!chris