[net.unix-wizards] alleged Disgusting kernel hack

geoff@utcsstat.UUCP (Geoffrey Collyer) (01/16/84)

The utcsstat (v7) kernel has a *simple* paginator in the kernel tty
driver, originally written by Ron Gomes of HCR (utzoo!hcr!rrg).
If the 4.2BSD terminal driver is sufficiently similar to v7's,
the same paginator will soon be inserted into the 4.2BSD systems
here; I cannot abide more.

This paginator is selectable by ioctl and the screen length is
settable.  It consists of fourty-four (44) lines of source
(including comments but excluding blank lines) and adds two new
ioctl codes (one to get and one to set the paging parameters).
This paginator makes *very* few decisions of its own and will
soon be extended slightly to make fewer decisions by making more
of its behavior selectable.  The paginator certainly does belong in the
terminal or the terminal driver and this fits in quite nicely with the
UNIX philosophy, thank you.  On utcsstat, only one program paginates
terminal output: the kernel.  It does it quickly and uniformly.

More is not ``a bit slow''; it is a huge, lumbering slug of a program.
(Since we switched from more to kernel pagination in readnews, news
reading, for example, is *much* faster.) It is abysmally written and
unmaintainable.  It is the poorest example of a filter that I can think
of, offhand.  More has countless bugs and does umpteen different
things unrelated to the job of pagination or each other.

As for putting the shell in the kernel, Berkeley does seem to be headed
that way, though they have a blind spot for the primary job done by
more: pagination.

Pipes and filters are still wonderful ideas; signals were a large
improvement at the time since they turned all sorts of phenomena into a
single mechanism.

With the kind permission of Ron Gomes and as a public service I offer
here the changes needed to add a simple kernel paginator to the v7
terminal driver;  4.2BSD changes may follow.  This code contains
no Bell nor AT&T code and is hereby declared public domain.

---only code and documentation follow this line---
In tty.h, add the following definitions:
before the definition of `struct tty',
struct tpage {
	unsigned short	t_pagsiz;	/* Page size for automatic paging */
	char	t_pgparm[4];	/* Reserved for future bells and whistles */
};

at the end of the definition of `struct tty',
	struct	tpage tpg;	/* Paging parameters */
	unsigned short	t_xstate;	/* Extra state bits */
	unsigned short	t_plc;	/* Line count for paging */

after the other definitions of bits in t_state,
/* Extra internal state bits (encoded in t_xstate) */
#define PAGEFULL 01		/* Screen full, wait till something is typed */

after the other ioctl codes (19 and 20 can be any unused terminal ioctl codes),
#define TIOCSPAG	(('t'<<8)|19)
#define TIOCGPAG	(('t'<<8)|20)
---
In tty.c,
at the end of ttyclose,
	tp->t_xstate = 0;
	tp->tpg.t_pagsiz = 0;
	tp->t_plc = 0;

in ttioccomm's switch, before the default: label,
	/*
	 * set and fetch paging parameters
	 */
	case TIOCSPAG:
		if (copyin(addr, (caddr_t)&tp->tpg, sizeof(struct tpage)))
			u.u_error = EFAULT;
		break;

	case TIOCGPAG:
		if (copyout((caddr_t)&tp->tpg, addr, sizeof(struct tpage)))
			u.u_error = EFAULT;
		break;

in ttyinput, before the test for c equal to tun.t_quitc or tun.t_intrc,
		tp->t_plc = 0;
		if (tp->t_xstate&PAGEFULL) {
			tp->t_xstate &= ~PAGEFULL;
			wakeup((caddr_t)&tp->t_plc);
			if (c=='\n' || (c=='\r' && t_flags&CRMOD))
				return;
		}

in ttwrite, before the call to ttyoutput,
		while (tp->t_xstate & PAGEFULL)
			sleep((caddr_t)&tp->t_plc, TTOPRI);
		spl5();
		if ((tp->t_flags&(RAW|CBREAK)) == 0 && tp->tpg.t_pagsiz > 0 &&
		    c == '\n' && ++(tp->t_plc) >= tp->tpg.t_pagsiz) {
			ttyoutput('\r', tp);	/* distinctive pause */
			ttstart(tp);
			tp->t_xstate |= PAGEFULL;
			while (tp->t_xstate & PAGEFULL)
				sleep((caddr_t)&tp->t_plc, TTOPRI);
		}
		spl0();
---
Here are the changes to tty(4):
.PP
In raw mode, every character is passed immediately
to the program without waiting until a full line has been typed.
No erase or kill processing is done;
the end-of-file indicator (EOT), the interrupt character
(ETX) and the quit character (FS) are not treated specially.
There are no delays, no stop/start handshaking, no paging,
and no replacement of
one character for another;
characters are a full 8 bits for both input and output
(parity is up to the program).
...
.PP
CBREAK is a sort of half-cooked (rare?) mode.
Programs can read each character as soon as typed, instead
of waiting for a full line,
but quit, interrupt, stop, and start work, and output delays, case-translation,
CRMOD, XTABS, ECHO, and parity work normally.
On the other hand there is no erase or kill,
no paging,
and no special treatment of \e or EOT.
...
.PP
The terminal interface can be put into a ``paging mode''
in which output will halt when a full ``page''
has been displayed;
output resumes when any character,
other than the start and stop characters,
is typed.
Usually this will be a newline, which is absorbed;
any other character will also restart output but in addition
it will be transmitted to the program reading from the terminal.
The structure referenced in the
.I ioctl
call is defined in
.I <sgtty.h>
as follows:
.PP
.nf
.ft 3
struct tpage {
	unsigned short	t_pagsiz;	/* Page size for automatic paging */
	char	t_pgparm[4];	/* Reserved for future bells and whistles */
};
.fi
.ft R
.PP
The
.I t_pagsiz
value is the number of lines which will be printed before output
stops to wait for a character to be typed;
if this is zero the paging feature is turned off.
The calls are:
.TP
TIOCSPAG
Set the paging parameters for the terminal according to the
pointed-to structure.
.TP
TIOCGPAG
Fetch the paging parameters and store them in the pointed-to structure.
...
.SH HISTORY
.PP
The paging feature was implemented at BNR by Ron Gomes.
...
.PP
The Bell documentation did not say whether
stop and start worked in CBREAK mode,
and claimed that RAW mode did not echo.
...
.SH BUGS
.PP
The paging feature should really take account of a terminal's
line length, to allow for extra newlines caused by wraparound
on over-long lines.
.PP
Raw mode does not do stop/start protocol, which some terminals
absolutely must have at all times.
---

Geoff Collyer, U. of Toronto

kre@mulga.SUN (Robert Elz) (01/18/84)

I don't believe it!  I had considered the original article to be
a joke, & had considered mailing a complaint at those who took
it seriously & sent articles ridiculing the idea.  Now we have code ????

The kernel simply does not have enough information to get pagination
right (apart from problems of terminal that do, or do not wrap long
lines).

What you want paginated is the output from some program, and then only
really when its coming quickly (when I do "cc *.c" I don't want, or
expect, it to stop after 24 files, cause it thinks that I want to
look at the names of the files its compiling - but I do want the output,
cause I like the security of knowing that its still happening out there).

Similarly, if someone else writes at me, while I'm taking a half hour
or so to gaze at some particularly interesting piece of news (or perhaps
if one of the less interesting has sent me to sleep), I don't think they
will take it too kindly if their message request gets held up until I
decide to scroll the screen.

Way back in v6 days, we had a terminal driver at Melbourne that did
all that (it did *everything*).  Its one unused, & very soon discarded
"feature" was pagination.

To all sites out there who have managed to resist this bug in their
tty drivers:  Don't be misled by "free" code, it isn't often that you
get something for free that's worth having, & this isn't one of those
rare occasions.

Finally, to those who criticize "more" as being a slow & bulky paginator:
It isn't a paginator.  Its a program to display data on a terminal, which
includes pagination, re-display, pattern searches, ...  Maybe that shouldn't
all be in one program, but then again, maybe it should.  It has a well
defined job to do, & does just that.  However, please don't assume
that I'm defending its bugs, it has had, & continues to have rather a lot
of them, perhaps it is a good target for re-implementation, maybe
someone can get the code right.

If all you want is a paginator, about 15 minutes & less than a page
of code will give you one.

Robert Elz,		decvax!mulga!kre

rpw3@fortune.UUCP (01/20/84)

#R:utcsstat:-166800:fortune:11600042:000:1141
fortune!rpw3    Jan 19 15:34:00 1984

In our port we (mostly Dave Yost) also put "page mode" into the driver.
This version solved the really big gripe I'd always had with it in
systems I had seen try it before, which is not knowing that the driver
had stopped the output. (Try a slow "find" which outputs a line per minute
or so, and tell me how mad you are when you discover that tty output was
stopped ten minutes ago!) Ours prints <<STOPPED>> at the bottom of the
screen, and then overwrites it with data when output starts again. Three new
characters are added to the magic set (all of which can be disabled or
changed like 4.1 "jobs" stuff); the defaults are:

			^F restart for "pagelen" more lines
			^B restart for pagelen/2 lines
			^E restart for one more line
plus
			^Q restart and turn off page mode (continuous stream)
			^S stop (^B/E/F work as above)

The line counts are reset by going into input I/O wait. This all interacts
surprisingly well. (Editors do have to be taught to turn it off.)

Rob Warnock

UUCP:	{sri-unix,amd70,hpda,harpo,ihnp4,allegra}!fortune!rpw3
DDD:	(415)595-8444
USPS:	Fortune Systems Corp, 101 Twin Dolphins Drive, Redwood City, CA 94065

chris@umcp-cs.UUCP (01/21/84)

Boy, some people are gonna have fun with HP 26xx's on your system, if they
use them at 9600 baud.  They send an enormous number of ^S/^Q pairs and
are likely to send a ^Q in response to <<STOPPED>>....
-- 
In-Real-Life: Chris Torek, Univ of MD Comp Sci
UUCP:	{seismo,allegra,brl-bmd}!umcp-cs!chris
CSNet:	chris@umcp-cs		ARPA:	chris.umcp-cs@CSNet-Relay

sysman@glasgow.UUCP (System Manager) (01/27/84)

It is interesting to see the wheel being reinvented, with corners this time.
The 4.1BSD driver and 'more' still pain me, although admittedly not enough to
do anything about it.
The Glasgow terminal driver stopped at the bottom of the page in 1977 and gave
rise to the current EUUG driver. Some credits are probably in order:

Glasgow (v6) driver: Emrys Jones and Bill Findlay
EUUG (v7) driver: Jim McKie

The size of the screen was part of the sgtty structure. Line folding and
page stopping was then trivial.
The v6 driver also had variable size tabs which was another feature sorely
missed when we moved to BSD.

On the other hand the reliability of 4.1 is so much better than V7 that
the terminal driver is bearable.

The other side of the coin was also somewhat tarnished by the need to modify
lots of programs to switch the pageing and line folding off. If you buy
binaries only there are obvious problems. There are two obvious solutions:

1/ Set up a standards committee to produce 'THE STANDARD DRIVER'

2/ make terminals behave intelligently and stop at the bottom of the page


The 2nd option does seem to be the best since the only thing the driver would
need to know is what the terminal says when it doesn't want any more output.

Looking forward to your comments:

Zdrav


Zdravko Podolski, Comp Sci Dept, Univ. of Glasgow, Scotland
{...!decvax!mcvax | ...!vax135 }!ukc!edcaad!edee!glasgow!{ zp | sysman }
or better still:  glasgow!zp%edee%rco%ucl-cs@CSNet-Relay

norskog@fortune.UUCP (Lance Norskog) (01/30/84)

It's horse's mouth time!

Yes, the Fortune 32:16 console screen is memory-mapped, the best-case time
for writing characters is ~45Kbaud.  

The page mode feature keeps page length as the only terminal-knowledge
parameter.  Using the restart character (default ^Q) as the
page-mode restart character was a mistake, it interacts with the
^S/^Q used by terminals.  Currently it 2 other problems:
You can't change the <STOPPED> prompt, and it doesn't know
about logical-line-length-that-is-greater-than-physical-line-length.

Other than that, I love it.  One other thing: you can start a
long program that prints out a lot of stuff, leave it going for
20 minutes, and discover that its been <STOPPED> for 19 minutes!
Maybe a timeout would be nice...

Lance C. Norskog
Fortune Systems, 101 Twin Dolphin Drive, Redwood City, CA
{cbosgd,hpda,harpo,sri-unix,amd70,decvax!ihnp4,allegra}!fortune!norskog