z@cca.UUCP (Steve Zimmerman) (03/05/84)
As the author of one of the original versions of terminal paging in the kernel (known locally as "page mode"), I have been rather surprised by the strong opposition to it by a number of people in this newsgroup. To a large extent, the arguments opposed to page mode remind me lot of a few years ago when many people were arguing against the use of screen editors. In both cases, most people opposed to the new feature had never used it on a regular basis; the few people who had had used a particularly brain damaged implementation, and had then begun arguing against all implementations. I think the most important people to listen to are the users of a system, who vote with their keyboards. When page mode was about to be brought up here at CCA three years ago, our users were split fairly evenly between those who were in favor of of it, those who had no opinion, and those who had their doubts. Page mode was installed as the default for all users, with each user having the ability to disable it by putting an "stty -page" in his or her .login file. After the first few days, there were only two users in our user population of several dozen who decided to disable it; even these two ended up changing their minds and enabling it over the next several months. Over the three years since then, as our user population has multiplied many times, no one else has ever disabled page mode, and no one has even complained about it. This record would seem to say that the objections against page mode can be met quite successfully. The version we are now running has evolved over the years, and contains input from a number of other sites as well. Here are my answers to the objections I've heard: Certain special programs (such as those that send data over a terminal line) would get totally messed up by page mode. This is similar to an argument that screen editors should not be used because there are still a lot of hardcopy terminals out there. I would guess that none of the people who have made this argument have ever used a page mode, because in every one I've ever seen there is a simple command (usually an option of stty) to turn it off or on. In several implementations such as ours, if you don't want the output of a particular command to be paged, you can just hit a single control character (^A in our case) any time after entering the command, and page mode is then disabled until you start typing again. Contrast the ability to turn off page mode temporarily in the middle of a command with the inability on vanilla Unix systems to start paging output in the middle of a command. Lots of programs such as editors will need to be modified so that they temporarily turn off page mode. Page mode is automatically disabled in RAW and CBREAK modes, which is what these programs typically run in. On our system, not a single program had to be modified to be made compatible with page mode. It's a real annoyance to have to type the space bar every 24 lines, no matter what is happening. Some terminals have a hardware scrolling mode, which stops output after every 24 lines. I have the same complaint about these terminals, and never use this feature on them. However, a reasonable page mode does not work this way. Output is only stopped if a screenful has gone by since the user last typed something. It's hard to tell whether output has stopped at the end of a page, or whether the program has died, or the system has crashed. This is another complaint from people who have apparently never used a reasonable page mode. Some implementations of page mode print <<STOPPED>> in the lower left hand corner of the screen; ours does a carriage return but not a linefeed after the last line, so that the cursor is underlining the character in the lower left hand corner of the screen. Both behaviors are quite unique, and I have never hear a user of either of these systems make the above complaint. Page mode does not belong in the kernel. Or, to quote from a recent article that raised this argument: "It is the job of the device driver to SIMPLY and DIRECTLY control the resource, not to try to "help" me when I may not want or need to be helped." By that definition, the erase and kill functions of the terminal driver should be moved out of the kernel, maybe into "more". Do people think that that would be a good idea? Page mode should be in the terminal driver for exactly the same reasons that erase and kill processing are there: it is useful for an overwhelming number of programs, it is more efficiently implemented in the kernel than anywhere else, and it is simple to implement in the kernel. My first implementation of page mode was a couple of dozen lines of code; the current version is somewhat larger, but it is still on the order of the size and complexity of most of the other features in the terminal driver. The overall effect of putting page mode into the terminal driver is to reduce the load on the system; on a heavily loaded system such as ours with dozens of logged in users, getting rid of all the "more" processes and the memory and context switching associated with them makes a noticeable difference in the load. Steve Zimmerman
ignatz@ihuxx.UUCP (Dave Ihnat, Chicago, IL) (03/06/84)
In response to Steve Zimmerman's defense of page mode in the kernel driver: Steve, you've answered a number of objections cleanly and convincingly. I would like to take this opportunity to point out two important items: - ANY time anyone talks about adding *anything* to the Unix(Tm) kernel, it's most emphatically a subject for protracted discussion. One of the most important things Unix forced on an operating system was excision of 'features' that weren't critically required in the kernel, e.g., record maintenance in the file system). Any move to add something to the kernel is a move in the other direction; thus, whether they yelped for an intelligent, reasoned reason or from dogmatism, those who questioned your move were *RIGHT* to do so. Their, and your, following discussions justify or invalidate the change. -When considering such modifications to the kernel, you have to consider if the benefits, as analyzed by either you or the user community, is a real benefit because of the quality of your change, or because of the shortcomings of Unix! Why did I bring up such self-obvious questions? Simply because I still believe you're wrong to put such code in the kernel. Why? First, the kernel should only contain code that manages critical system resources--and the paging of your terminal isn't a critical system resource. (Why? Because the system is already managing access to your terminal, which *is* a critical resource. Why is raw/cooked in the kernel, if paging shouldn't be? To be honest, I'm not totally certain that it belongs there!) So, what are the reasons you *shouldn't* put such things in the kernel? Because of just exactly the type of thing you mention--the arguments that came up when full-screen editors began to proliferate. *You* are making assumptions about what a terminal is, in a piece of common code--that it's 24 lines long, and maybe 80 characters wide. Are there other types of terminals? Is there not a trend toward larger screens, or bitmapped, variable font terminals?? Maybe...so do you change the kernel code---again and again---or make this performance-critical piece of code dependent on conditional tests, external config files, etc.? You mention the expense of all the 'more' processes--totally true!! But doesn't this point to a need for a better way of handling the use of identical processes on Unix, rather than embedding more necessarily hardware- or assumption-dependent code in the kernel?? (Like, maybe shared-text process slots for system-wide shared text? Or whatever.) I guess the end of all of this is that it's horribly easy to add twenty or thirty lines of code to the kernel to handle a special case. But, eventually, this way lies the large, over-specialized OS's of the sixties and early seventies, such as OS-360 or even GCOS-6. We've already got the problem of divergent Unix systems: Berkeley and Bell. (DON'T get me started on BSD...) Let's not stumble BACK into the errors made 20 years ago! Glad to accept mail on this topic, Dave Ihnat ihuxx!ignatz
trb@masscomp.UUCP (03/06/84)
*You* are making assumptions about what a terminal is, in a piece of common code--that it's 24 lines long, and maybe 80 characters wide. Are there other types of terminals? Is there not a trend toward larger screens, or bitmapped, variable font terminals?? Maybe...so do you change the kernel code---again and again---or make this performance-critical piece of code dependent on conditional tests, external config files, etc.? Yea, and *we* make assumptions about what disk drives are. In a piece of common code. That it's 24 sectors around and maybe 80 tracks deep. Is there not a trend toward larger disks, or controllers with built-in addressing smarts? We change the kernel code again and again... I like having terminal paging in the kernel. I think that the folks who don't want it there could be characterized as UNIX prudes, the Jerry Foulwells of system software. They have a right to their opinion as long as they stay out of my bedroom and don't terrorize my children. Dennis Ritchie has done work with the notion of stacked line disciplines, which are sort of an extension of the UNIX pipe philosophy to lower level I/O line interfaces. This would be the place for terminal paging if AT&T (or someone else) would ever make it available. This stacking notion should be used for disk drivers too, so that we could layer per-device notions on top of a generic disk driver. Andy Tannenbaum Masscomp Inc Westford MA (617) 692-6200 x274
cfh@cca.UUCP (Christopher Herot) (03/06/84)
As a user of Steve's terminal paging code, I am very grateful that he put it in the kernel. I am sure all of the other users of the two VAX's at CCA would agree with me. It would be unfortunate if we let some misguided notions of mathematical purity get in the way of getting our jobs done. The bottom line is that life here is better with the change than it was without it. You can wait for the researchers to develop a more elegant approach if you want to - I'll use kernel paging while I wait. Thank Ken & Dennis that they made it possible to make such changes ourselves while the rest of the world is locked in this endless debate.
z@cca.UUCP (Steve Zimmerman) (03/07/84)
Dave Ihnat brings up an objection to page mode that I neglected to answer in my original message and which, for the sake of completeness, I feel obliged to answer: *You* are making assumptions about what a terminal is, in a piece of common code--that it's 24 lines long, and maybe 80 characters wide. Page mode has never made any such assumptions. Even the original, two dozen line version had code for an ioctl to set the terminal screen size. This ioctl is typically used by stty; when the user says "stty page", the stty program uses termcap to find the size of the screen, and then uses this as an argument to the ioctl. Alternatively, you can do something like "stty page 66" to override what termcap believes. The important thing to note is that you have full functionality and generality here, yet it is the user program (in this case stty) that does most of the work. I've always been for keeping the kernel simple; I've saved all my complicated code for my EMACS! Steve Zimmerman
henry@utzoo.UUCP (Henry Spencer) (03/07/84)
Our experiences with kernel paging here parallel Steve Zimmerman's:
extensive skepticism in the beginning (including me), total acceptance
after a little bit of real experience. Ours isn't even as fancy as
Steve's, and we love it just the same. (Ours is essentially the one
Geoff Collyer posted a while ago; he got it from us. We do not yet
have a turn-paging-off-temporarily character, but plan to add one.)
Try it, you'll like it. Really.
I would like to add one thing to Steve's fairly complete rebuttal of
the critics. Another criticism that has been seen is:
The current kernel doesn't know anything about all the
strange terminals out there. It is dreadfully unclean
to add virtual-terminal stuff to it.
Well, I would argue the last part at some length, if the virtual-
terminal stuff is being added for some useful purpose like implementing
a window system. But I don't need to argue that here, because the
first part of this objection is full of s**t. Have you ever looked
at the definition of, say, the NL1 delay bit? This is found in very
nearly every Unix in existence, and it is custom-tuned for a terminal
that nobody in his right mind has used for a decade. The V7 Unix
kernel contains a fairly complete virtual-terminal system for hardcopy
terminals circa 1972. About half the bits in the tty mode are dedicated
to this exact purpose. So please don't complain about putting code
that knows about terminals into the kernel; it's been there all along.
The problem is that it was never updated to match modern terminals.
--
Henry Spencer @ U of Toronto Zoology
{allegra,ihnp4,linus,decvax}!utzoo!henrymats@dual.UUCP (Mats Wichmann) (03/09/84)
<>
Without getting too involved in the terminal paging issue (other than
to note without justification that I oppose it) I wanted to comment
on the statement that we are making assumptions about the sizes
of disk drives in the kernel. WE are most definitely not making any
assumptions about disk sizes in the kernel - we encode this information
on a reserved area on the disk. On the first access to a drive, this
reserved area is read, and the number of heads-tracks-sectors, the
device partitions, and the bad block map (if any) is pulled in.
I think you would find that many of the micro-UNIX vendors are
doing the same thing: we usually have to support too many different
configurations to be able to have a different kernel for each one -
massive distribution headaches would ensue (especially for bad block
handling). If there were a way to obtain this information for each
terminal WITHOUT hardcoding it into the kernel, this portion of the
paging dispute would go away - but I don't see any way to do that.....
Mats Wichmann
Dual Systems Corp.
...{ucbvax,amd70,ihnp4,cbosgd,decwrl,fortune}!dual!matsphil%rice@sri-unix.UUCP (03/09/84)
From: William LeFebvre <phil@rice>
Yea, and *we* make assumptions about what disk drives are. In a piece
of common code. That it's 24 sectors around and maybe 80 tracks
deep. Is there not a trend toward larger disks, or controllers with
built-in addressing smarts? We change the kernel code again and
again...
Given a typical system, there are a wider variety of terminals than
disk drives. Mainly because there are about 20 times more terminals on
the system than disk drives. Also, given a typical system, terminal
types tend to change more often than disk drive types. Mainly because
there is an order of magnitude complexity difference between changing a
terminal and changing a disk drive. Not to mention the fact that
people who dial up a computer use whatever terminal they have at home.
When was the last time you changed the type of drive you are using?
When was the last time you added or swapped out a terminal? When was
the last time someone connected a disk drive to your system over a
phone? Your argument is fairly vacuous. It's quite true that the disk
drive characteristics are hard coded in the kernel. But they don't
change very often. You can afford to hard code disk drive
characteristics. I'm not saying it's a good idea, but you can get away
with it. You CAN'T afford to do that with terminals! And anyone that
does so is painting himself into a corner.
William LeFebvre
Department of Mathematical Sciences
Rice University
<phil@rice>Lepreau@UTAH-20.ARPA (03/09/84)
From: Jay Lepreau <Lepreau@UTAH-20.ARPA> Good grief, what some of you are beating is known, not as a "dead horse," but as a "strawman." Nobody who has advocated page mode EVER said that the kernel has to know terminal characteristics to do paging; a nay-sayer asserted it. Remember the tty CR delay bits that have been in the kernel forever? How did it find those out? stty, that's how, from your .profile. And how does it find out cols and rows to do paging and folding? stty or tset from the same place. The data happen to be readily available, you know, nicely encoded in a database so that even user code needn't be adapted to hardware changes-- it's called termcap. It's not as if Unix has to boot off your tty. -------
crl@pur-phy.UUCP (Charles LaBrec) (03/10/84)
There is a way to tell the tty driver paging info w/o hardcoding it--simply add or create a structure that tells it some terminal characteristics like line and page length. Another useful piece of info would be how to do a deleting backspace--I know of one type that uses '^H' as home cursor! This has shades of TOPS-20 in it--whenever it did a backwards delete on a terminal it knew, it actually did a move backwards and a delete to end of page. Not only did this make an erasing kill line or word less painful at slow speeds, but is also cleared the remainder of the page if some screen oriented program screwed up. Charles LaBrec UUCP: pur-ee!Physics:crl, purdue!Physics:crl INTERNET: crl @ pur-phy.UUCP
phil@amd70.UUCP (Phil Ngai) (03/11/84)
Oh good grief. In the implementation of terminal paging running on my
machine, there is an ioctl which specifies the number of lines the
terminal has. And you can say "stty page 24", or 60 or whatever.
I can't believe all the flames about so-called defects. I wish people
would spend more time solving problems and less time complaining.
--
Phil Ngai (408) 988-7777 {ucbvax,decwrl,ihnp4,allegra,intelca}!amd70!philhenry@utzoo.UUCP (Henry Spencer) (03/11/84)
Mats Wichmann observes, in part:
.......... If there were a way to obtain this information for each
terminal WITHOUT hardcoding it into the kernel, this portion of the
paging dispute would go away - but I don't see any way to do that.....
"Hardcoding into the kernel"??!?!?? I wish people would assume that
the advocates of in-kernel pagination aren't completely crazy. Our
system takes some pains to get the value of $TERM correct, and the code
that enables paging uses $TERM to find out (from termcap) the size and
whatnot of the terminal screen. This isn't hardcoded into the kernel
any more than vi's terminal handling is!!
--
Henry Spencer @ U of Toronto Zoology
{allegra,ihnp4,linus,decvax}!utzoo!henrybarmar@mit-eddie.UUCP (Barry Margolin) (03/11/84)
If there were a way to obtain this information for each
terminal WITHOUT hardcoding it into the kernel, this portion of the
paging dispute would go away - but I don't see any way to do that.....
Sure there is - the user tells the kernal how many lines and columns
there are. I have seen references to "stty page 24". You could
simplify it by allowing "stty type vt100"; stty would look vt100 up in
termcap to find out the number of lines, and then make the appropriate
system call. This way you don't need the type table in the kernal, just
the ability to make use of the information in the table. In some cases
you can even do better than requiring the user to use stty: many
terminals have a recognizable answerback, so when it dials up Unix can
send an ENQ and see if it recognizes the answerback.
Those of you who are against terminal paging in the kernal have not been
very constructive. Anyone have any suggestions for a way to get this
affect without doing it in the kernal? I am very sympathetic to those
who believe that the kernal should only be used for the low-level device
control that requires privileged instructions. However, the pipe
mechanism doesn't provide enough functionality to do what we want. I
just experimented with
ncsh|more
and discovered that I didn't see anything until I filled up a page or
sent EOF (which was better than what happened when I tried
csh|more
which didn't even echo and didn't recognize ^D). In addition, there is
no way to perform ioctl's on filters (I could be wrong - I am not that
much of a wizard), and many programs use isatty, which will fail on
pipes. I suppose it could be done with pseudo-tty's, but that will put
a much larger burden on the system than one would want.
So, either put up or shut up. The fact is that someone has made very
minor kernal modifications to add a very useful feature.
--
Barry Margolin
ARPA: barmar@MIT-Multics
UUCP: ..!genrad!mit-eddie!barmardudek@utcsrgv.UUCP (Gregory Dudek) (03/12/84)
I still think that "advanced" features of the terminal driver could
be supported by user processes hooked into the kernel. This would
allow users to customize their own features. This would be a lot
several network file systems (eg. cocanet) which work under UNIX.
Gregory Dudek
utcsrgv!dudek
--
Gregory Dudek
{cornell,decvax,ihnp4,linus,utzoo,uw-beaver}!utcsrgv!dudekgeoff@utcsstat.UUCP (Geoffrey Collyer) (03/13/84)
[non-null first line] From: barmar@mit-eddie.UUCP (Barry Margolin) Those of you who are against terminal paging in the kernal have not been very constructive. Anyone have any suggestions for a way to get this affect without doing it in the kernal? I am very sympathetic to those who believe that the kernal should only be used for the low-level device control that requires privileged instructions. Much of the discussion of pagination by the terminal driver lately seems to revolve around whether the kernel is an appropriate place for a terminal paginator or not. In a well-structured operating system, it isn't. Look at TUNIS or a number of operating systems more recent than UNIX*: in a well-structured operating system the kernel is small, a first-level interrupt handler and synchroniser with perhaps a smidgen of scheduling or memory management. In such a system, the UNIX common terminal code might be a terminal manager process. On UNIX, the kernel includes code that logically might be removed into separate processes under a different implementation. One would like the UNIX kernel to remain small, but let us remember that the UNIX kernel is conceptually a kernel (roughly speaking, the assembler code), device managers (the dev (a.k.a. io) directory), two levels of file system managers (flat inodes and directory trees), scheduler, swapper or pager, memory manager and some miscellany. I don't really want to put terminal pagination in the UNIX kernel, I want to put it in a terminal manager (which might even be, *gasp*, replacable by a user, in some implementations), but there isn't a terminal manager process in UNIX. However, the pipe mechanism doesn't provide enough functionality to do what we want. I just experimented with ncsh|more and discovered that I didn't see anything until I filled up a page or sent EOF (which was better than what happened when I tried csh|more which didn't even echo and didn't recognize ^D). In addition, there is no way to perform ioctl's on filters (I could be wrong - I am not that much of a wizard), and many programs use isatty, which will fail on pipes. I suppose it could be done with pseudo-tty's, but that will put a much larger burden on the system than one would want. I am aware that this has been beaten to death before in unix-wizards and have no wish to kindle another round. (I had hoped vainly that after the usual round of ad-hominem attacks, terminal pagination would be discussed only briefly. It is now almost two months since I posted the paginator.) I feel that isatty should have been named something like needcare (needs careful buffering) and returned true for pipes. The more serious problem is that many stdio implementations buffer output poorly to terminals and pipes (perhaps due to reliance on isatty). As previously noted, pipes were once used interactively. On v6, sh | pager worked as one would like. I think that I know how to make stdio correctly buffer output to terminals and pipes, but will refrain from further comment until I have tested my theory. So, either put up or shut up. The fact is that someone has made very minor kernal modifications to add a very useful feature. Agreed. The 44 lines of kernel terminal paginator that I posted seem insignificant next to the 1801 lines of our more.c. --- * UNIX is a trademark of AT&T Bell Laboratories, or something similar. Geoff Collyer, U. of Toronto
rpw3@fortune.UUCP (03/16/84)
#R:cca:-684100:fortune:11600071:000:1092
fortune!rpw3 Mar 16 01:53:00 1984
I agree with Steve Zimmerman -- paging in the kernel with parameter-setting
in "stty". Notice that this obeys the VERY IMPORTANT software design
principle called "separation of policy and mechanism". Here, "stty" sets
policy, all the kernel does is implement it.
Other examples: operating systems which put only a low-level process switcher
in the kernel and put the long-term scheduler and swapper in user mode; network
protocols which put the data moving in the kernel but put session control in
user mode; systems which put the magtape driver in the kernel but all of the
label handling and volume switching in user mode; error-flagging vs. error-
logging and reporting; process accounting vs. logging and reporting; etc, etc.
The key is that as long as the mechanism is clean (and general) enough, one
can implement many policies on it. How many different UNIX shells have there
been? They all use "fork" and "exec".
Rob Warnock
UUCP: {sri-unix,amd70,hpda,harpo,ihnp4,allegra}!fortune!rpw3
DDD: (415)595-8444
USPS: Fortune Systems Corp, 101 Twin Dolphin Drive, Redwood City, CA 94065mats@dual.UUCP (Mats Wichmann) (03/20/84)
I have learned something from this discussion. Number one is to try harder
to keep my foot out of my mouth. I had not really meant to comment on the
topic at hand, only to respond to the comment about the disk partitions
and sizes being wired into the driver. I am now in the process of pulling
together a summary of what I have seen on this topic, this will be more
carefully thought out than my previous submission. Expect posting in a few
days (hold breath in eager anticipation....and it won't all be anti-paging).
I do have to respond to the flames I have gotten, however, because I feel
badly scorched. I worried about informing the pager about the characteristics
of the terminal. I got flames to the effect of `EVER HEARD OF TERMCAP?'.
Okay, I have heard of termcap. I even use it. The pager is informed through
an ioctl how many lines/characters the screen will hold. Now - how is this
ioctl performed? First you have to find out what the terminal type is...
this problem has not really been solved. Notice that Berkeley used to use
ttytype (is this still the case for 4.2). Have you noticed Bell releases
with this feature in them? All UniSoft System III/V releases are also
missing this handy little feature...(although I quickly put it in for our
system). What about dialups - TSET options handle specifying the teriminal
type on login to a dialup, but not everybody supports tset either. A pager
such as more knows when output is directed to a terminal and when it isn't,
and behaves appropriately. Have you built the same knowledge into the
kernel? I presume you have, since otherwise you would have had lots of
complaints - or is every program responsible for sending out the ioctl
to stop paging (which would be a real mess).
I am not arguing against the idea of kernel paging here - just showing
my ignorance and asking a few questions that still bother me. Answers
anyone (WITHOUT the flames, this time).
Mats Wichmann
Dual Systems Corp.
...{ucbvax,amd70,ihnp4,cbosgd,decwrl,fortune}!dual!mats
It now became apparent (despite the lack of library paste)
that something had happened to the vicar; [ Edward Gorey ]z@cca.UUCP (Steve Zimmerman) (03/21/84)
More answers to criticisms of page mode: First you have to find out what the terminal type is... this problem has not really been solved. Although there may be no 100% foolproof way of doing this that works automatically every time, the problem certainly has been solved in the general case. Whether people are using Berkeley Unix and ttytype or USG Unix and some other mechanism, the fact is that regular users of terminal dependent programs such as screen editors or "more" (and that should include just about everybody) already have some mechanism for getting their terminal type set correctly, either by the system or in their .login or .profile file. Once this is done, issuing the "stty page" command I described previously, typically in the .login or .profile file, works just fine. Although we run Berkeley Unix, most of our terminals hang off a port selector, so the ttytype file is of little use to us. Yet with a little intelligence in people's .login files, the terminal type is set correctly for whatever programs want to use it. In practice, no problems have arisen. A pager such as more knows when output is directed to a terminal and when it isn't, and behaves appropriately. Have you built the same knowledge into the kernel? Page mode is built into the terminal driver, and handles output from the terminal. When output is directed somewhere else, the page mode code is never called. Steve Zimmerman