MQH@CORNELLC.BITNET.UUCP (02/14/87)
Hello (and forgive me, this is awful long),
I'm trying to decide how to "correctly" implement TELNET option
negotiation for programs which can emulate both ASCII and EBCDIC.
I support Wiscnet (a VM implementation of TCP/IP) here at Cornell.
I've been working with one of our programmers who's been working on
Cornell versions of TN3270 which run on MACs and PCs. Our TN3270s
can successfully negotiate into 3270 terminal emulation with Wiscnet,
but we've discovered that they can't talk to KNET (yet another
implementation of TCP/IP for VM). It turns out that Wiscnet can't
talk to KNET either, and that Berkeley's latest release of TN3270
doesn't talk to KNET in 3270 mode.
I've spent some time researching this whole mess, and my results
are below (I'm of the opinion that everyone's at least a little
busted). Basically, the big problem is how to decide that we're
going to do the connection in EBCDIC. I don't think the current
RFCs for telnet supply enough information on how to properly handle
EBCDIC connections.
Here is a basic summary of the various "philosophies" of option
negotiation in the code I've dealt with. I'll warn you in advance
that these are my impressions, and best guesses, and that I've been
known to be wrong. If I am wrong, please let me know.
Wiscnet
After establishment of the connection, both user, and server must
"discuss" the following options, IN THIS ORDER, to form a
"transparent mode" (3270) connection. TERMINAL TYPE, END OF RECORD,
TRANSMIT BINARY. If any of the options are settled out of order,
or some of the responses are unacceptable, or data arrives before
option negotiation completes, the connection will irrevocably be
put into "line mode" (ASCII).
KNET
After establishment of the connection, KNET will send a one line
blurb in ASCII announcing itself and the host. Then option
negotiation begins. If KNET does a SEND TERMINAL TYPE, and the
user code responds with a valid 3270-ish terminal, KNET sends
DO BINARY, WILL BINARY, and begins sending EBCDIC data.
If it doesn't get an acceptable terminal type, the server puts
up a menu of ASCII terminals it's willing to do protocol conversion
for.
Cornell TN3270 (PC and MAC)
Always responds to SEND TERMINAL TYPE with "IBM-3278-2".
Hack 1:
If the server sends DO EOR, we enter 3270 emulation mode.
If not, we do H19 emulation.
Hack 2:
If the server sends DO BINARY, we enter 3270 emulation mode.
If not, we do H19 emulation.
Berkeley TN3270 (UN*X hosts) (I'm not at all positive about this one)
Always responds to SEND TERMINAL TYPE with "IBM-3278-2".
If we change an option, and we get to a state where we
have sent IBM-3278-2, and we are sending and receiving
in binary, we enter 3270 emulation.
Now, after all that, we can look at why some won't talk to others.
Wiscnet -> KNET
When the KNET server sends its one line blurb, Wiscnet decides
to do line mode emulation. Unfortunately, it will still respond
with options appropriate to 3270 emulation. The result is that
KNET server is in 3270 mode, and Wiscnet user code is in ASCII
line mode. I'm not sure of how the reverse case turns out.
Cornell TN3270 -> KNET
KNET doesn't initiate EOR option processing with the user.
Hack 1 of our TN3270 therefore never went into 3278 emulation.
Hack 2 does better, but we've come upon a bug at the TCP level.
We're looking into it.
Berkeley TN3270 -> KNET
An old release (March 22, 1985) works fine. A newer release,
(January 11, 1986) end's up talking to KNET in ASCII, and is
forced to use the KNET protocol conversion. I think it's
KNET's fault. The new TN3270 does a DO SUPPRESS GO AHEAD
before KNET gets a chance to send an option. I think this
confuses KNET, because it never gets around to sending a
SEND TERMINAL TYPE. Since it never gets a 3270-ish terminal
type from the user, it does ASCII emulation. Fortunately,
since TN3270 never sent it's terminal type, it also decides
to do ASCII.
Now that that's clear as mud....
It seems that most all the implementations seem to think that the
BINARY option is the way to do EBCDIC. Is that cool? It seems to me
that an "EBCDIC" option would be less confusing. I DON'T think that
the ASCII/EBCDIC question should be settled based on terminal type
'cause most UN*X implementations won't behave properly. If the
server doesn't know how to do the terminal type announced by the
user code, it's supposed to ask the user if it'll do something else.
In practice, I haven't seen a single implementation that does this.
Berkeley 4.2 never asked, and Berkeley 4.3 asks only once. It doesn't
notice, for example, that "IBM-3278-2" isn't in its termcap.
So, I guess I'd like to see an EBCDIC option added to telnet. Once
that's in place, it's a trivial matter to make one end or the other
announce that it WILL EBCDIC. If both sides DO EBCDIC, then
everything would be fine. The matter of 3270 emulation could be
decided with TERMINAL TYPE negotiation. If EBCDIC is refused, then
both ends would do the connection in ASCII.
If an EBCDIC option is the way to go, I have two questions. First,
would the server, or the user announce WILL EBCDIC? The TN3270
programs are a little wierd in that they can emulate more than
one kind of terminal. Second, would EBCDIC imply anything about
BINARY?
My thanks in advance for any comments you have on the subject.
I hope that we can come to a quick resolution to this problem so
we can get all our IBM implementations talking to one another.
Mike Hojnowski
Long-winded Wiscnet Grunt
Cornell Theory Centerminshall%opal.Berkeley.EDU@UCBVAX.BERKELEY.EDU.UUCP (02/18/87)
Hi all. The question of how one gets into "3270 mode" is a one
worth pondering. A follow up message by Bruce Crabill (which may
not have made it to the tcp-ip list) and the original message by
Mike Hojnowski are fairly worthwhile reading, but I would like
to add my two-cents worth.
First, there is, of course, absolutely NO standard for how "3270 over
telnet" should be done. The Wisconsin picked something, based possibly
on conversations with the UCLA (MVS) people. Doing this involved
inventing a new Telnet option (EOR's). The initial WISCNET just
negtotiated a 3270 terminal type and binary mode, and then assumed
EOR. After the implementation, the EOR (end of record) RFC came out and
said "thou shalt negotiate EOR prior to using it", so later versions
of WISCNET have negotiated EOR in addition to the terminal type and
binary mode.
Berkeley TN3270 does, in fact, always respond "ibm3278-X" to "send
terminal type". This is somewhat of a problem, since if we AREN'T
going to get into 3270 mode, we would probably REALLY like to say
"vt100", or whatever the user's terminal REALLY is. I would LIKE
to see this change, but it's not really too big of a deal.
Berkeley TN3270 does, in fact, check occasionally to see if "binary"
and "terminal type of 3270 sent" are true. If they are, then Berkeley
TN3270 magically slips into 3270 mode. If these are NOT true, then
Berkeley TN3270 magically slips out of 3270 mode (which transition
may not work very well - I've never had a host that wanted to leave
3270 mode on me, though I understand that the UCLA code does like
to do this at times). We DON'T check EOR option being set (though
the current Berkeley TN3270 is willing to have EOR set on either end).
Berkeley TN3270 IS sensitive to the fact that various IBM implementations
are order sensitive when doing the initial negotiations. The most recently
announced version of Berkeley TN3270 had a "bug" which caused it to
not work when talking with Fibronics/Spartacus/KNET. The current version
(available on arpa.berkeley.edu, or from me, but never announced)
has a kludge to get around the KNET operating characteristics.
The problem with order sensitivity is that on start up, a client
would really like to just send "do sga" and "will terminal type".
Doing this sort of "parallel" option negotiating can speed up the
user-perceived connect time a lot.
I would like an agreement on how to do things. Perhaps an RFC.
I don't think the issues are actually that easy to resolve (though
maybe I'm just slow).
Part of the constraints should be that if 3270 negotiation doesn't work,
we should slip back into "send terminal type" so that (for example)
an Amdahl-compatible machine with an onboard emulator (ala KNET) can
determine the terminal type without having to ask the user. Also,
the mode should be able to be switch back and forth (for the UCLA people,
if no one else).
The problem with doing an RFC, of course, is "who wants an RFC that only
pertains to one class of terminals" (though the telnet RFC, after all,
spends a lot of time making telnet work for IBM 2741's).
Greg Minshall
CFC 273 Evans Hall
UC Berkeley, Ca. 94720
(ps - this message is partly a sly way of announcing a "fixed" version
of tn3270 available on arpa.berkeley.edu in pub/tn3270tar; the message is
also to announce the availability of tn3270 to the bitnet community)braden@ISI.EDU.UUCP (02/20/87)
Mike, Your recital of the various IBM host implementations of full-screen 3278 operation did not mention the UCLA OS/MVS version, which (almost!) interoperates with the Wiscvm code. You incorrectly state that the choice of mode is between ASCII and EBCDIC encoding. Suppose there were an EBCDIC option in TELNET... this would simply be NVT encoded in EBCDIC; the only question would be whether to use CR LF or NL for end of line. Actually, the choice is between NVT and encapsulated IBM 3278 order streams. Now, the latter is about as close to binary as anything I know... it contains command code bytes, cursor positions, repeat counts, flags, and, encapsulated in all that framing and control, some EBCDIC text. It could as well be Display Code or Sanskrit. The 3278 stream is BINARY, man! You state that the Wiscnet implementation requires a fixed order of negotiation. That surprises me... in an exchange of mail several years ago Larry Landweber and I agreed on the following rules: The terminal type negotiation and the EOR negotiations may occur in any order. If EOR has been negotiated in both directions and if a 3278 terminal type has been negotiated, THEN negotiation of BINARY will cause the mode to change from NVT to full-screen-3278. Note that the BINARY is negotiated separately in each direction, which correctly synchronizes the mode change (any NVT data in the pipeline in each direction should be correctly handled). The UCLA code (v.1.6) does this correctly; I thought that Larry said the Wiscnet code did also. There is a problem with negotiating OUT of full-screen back to NVT. The MVS Telnet Server requires this; it negotiates OUT of BINARY, which returns to NVT. Returning to 3278 mode later in the Telnet connection requires only negotiating BINARY again; the EOR and Terminal Type negotiations are assumed to be persistent on the connection. The Wiscnet code, because of the hacky way they implemented Telnet using existing VM features, is unable to change a connection back to NVT mode if it begins in full-screen mode. The need for a little RFC on 3278 encapsulation in Telnet has been repeated pointed out over the past 4 years, but no one has ever felt they had the time (read $$) to pay for it. Not Wisconsin, not UCLA, not IBM FSD, not Berkeley. If would be wonderful if someone would write it down... but they had better understand the problem first. For example, the current spec uses the pre-SNA encoding, which is probably a mistake. And it is probably necessary in general to negotiate not just terminal type but also control unit type, since different IBM controllers support different wonderful features of the 3278/etc terminals. Welcome to the wonderful world of... Bob Braden
solomon@GJETOST.WISC.EDU.UUCP (03/05/87)
OK, here's the "Wisconsin view" on the 3270/telnet controversy. Sorry I took so long to respond, but I somehow got deleted from the TCP/IP list. I think I tracked down all the relevant mail on this subject from the list, but if I seem to be ignoring some past message, it may be because I missed it. I used to think that no new rfc was needed, since the existing telnet specs (especially rfc885 and rfc930) told the whole story. I now think that while a new spec may not be needed, an rfc might be called for just to make things clearer. I pretty much agree with Bob Braden about what SHOULD happen: > The terminal type negotiation and the EOR negotiations may occur > in any order. If EOR has been negotiated in both directions and if a 3278 > terminal type has been negotiated, THEN negotiation of BINARY will cause > the mode to change from NVT to full-screen-3278. Note that the BINARY > is negotiated separately in each direction, which correctly synchronizes > the mode change (any NVT data in the pipeline in each direction should > be correctly handled). The remainder of this (rather long) message contains my views about the philosophy of telnet, some of the practical problems in implementing it, and some explanation of why Wiscnet behaves the way it does. Telnet assumes that there is a set of "options". Corresponding to each option is a pair of Boolean variables, one for each end of the connection. The telnet spec (rfc854) gives a protocol for changing any one of these Boolean variables. It gives no information about what they may mean. The individual options are described in separate rfc's, some in more detail than others. Telnet also provides a "suboption" facility to send more complicated out-of-band information. Finally, telnet defines a "network virtual terminal" (NVT), which is a dumb teletype-like device that dictates the default syntax and semantics of data on the connection. Telnet has no facilities for tying together a package of options that are logically interdependent. The general strategy is to hold back data during a flurry of option negotiations until everything settles down, so that you don't send anything while the options are in an inconsistent state. The ASCII option is misnamed. It should be called "NVT", since it means that data sent (from an end where ASCII is "true") will conform to NVT conventions. The negation of ASCII is BINARY (perhaps "transparent" would be a better name), which means that the data is NOT assumed to have any particular telnet-defined structure. The 327x terminals have their own protocol, which is very unlike NVT. Therefore, it seems reasonable for two consenting telnet ends to exchange data in the "raw" 327x format. Other "bilateral" agreements could similarly be defined. The BINARY option almost, but not quite, fits the bill. There are two problems. First, the details of the protocol depend on the particular model of 327x terminal, and perhaps on other information, such as the type of controller and whether SNA is in use. Thus there has to be a way of exchanging terminal type information. The set of terminal types defined in rfc930 is small, but the intention is that this option negotiation be used to convey whatever information is necessary to set up whatever ad hoc protocol is desired. Berkeley's tn3270 always says "ibm3278-2". The 3270 emulator I wrote (included with the Wiscnet distribution), says 3278-2, -3, or -4 depending on the number of line on the screen (from termcap or from the tty driver on 4.3BSD UNIX systems). If the need arises, one can easily define other "terminal types" to describe SNA, or whatever. Second, the 327x protocol is a record-oriented rather than byte-stream protocol. There has to be a way of signaling the end of a record outside the set of 256 possible octets. (By the way, for the same reason, IMAGE mode in FTP is not adequate for transferring IBM files; the end of a record is not indicated by any of the 256 EBCDIC characters, but by out-of-band data. You have to use PAGE mode instead.) I originally suggested that we simply define an escape sequence IAC EOR (where EOR is some code distinct from WILL, WONT, etc) to mark the end of a record. Since this sequence could never legally occur according to the existing spec (even in BINARY mode!), it shouldn't cause any confusion. However, Jon Postel insisted that a telnet process shouldn't send IAC EOR without first getting permission via IAC DO EOR. Thus the general scheme is as follows: Consenting telnets use BINARY to agree that they will not use NVT, but rather their own private protocol agreed upon by bilateral negotiation. Before doing so, they have to establish the details of this bilateral protocol. Thus a telnet process may refuse BINARY if it feels adequate groundwork has not been laid out. In the present case, an acceptable terminal type and permission to use EOR are required. I can imagine similar variants of this scheme for other non-NVT protocols. Now for implementation problems. If you want to implement server telnet, you either have to modify every application so that it can talk to the telnet server rather than a terminal, or you need an operating-system facility for the server to masquerade as a terminal to another process. The latter facility is called a pseudo-tty (pty) in UNIX jargon; under VM, it's called "logical device support facility" (LDSF or ldev). Under LDSF, the information passed between the application and the server must be a 327x data stream. One of the parameters to the "initiate" LDSF call is the terminal type. Once the logical device is created, there is no way to change the terminal type. Thus the Wiscnet server tries to avoid creating the logical device as long as possible. If a telnet connection begins with an appropriate negotiation for BINARY mode, the server has all the necessary information. Otherwise, it creates the logical device with type "3278-2" and goes though the painful process of translating between that protocol and NVT. If any NVT data arrives before the terminal type is established, the server is forced to create a logical device, and so it must choose a terminal type. Subsequent attempts to go into BINARY mode are simply rejected. I suppose two enhancements are possible: First, if the other end sends some NVT data and then decides to advertise a terminal type that "just happens" to match the type chosen by the server, it should be possible to accept it. Second, the server could save up a "small" amount of NVT data before setting up the LDSF device, in hopes that the necessary info will soon arrive. Both are kludgy, but both might be very helpful in practice. A similar problem comes up in backing out of BINARY mode. Backing out of BINARY/3278-2 to NVT shouldn't be too hard in principle (simply a matter of programming :-). Backing out of some other 327x model would be a bit harder. Right now Wiscnet refuses to do either. Finally, I should discuss sequencing of negotiations. As I think I've made clear, the terminal type and EOR have to be settled before BINARY makes sense. The current Wiscnet code insists on EOR before terminal type. That's probably a bug. However, the Wiscnet code will also be willing to go into BINARY mode without first agreeing on EOR. That's a "feature"--really a historical curiosity (see the discussion above and the comment in the Wiscnet code), but it also conforms to the maxim, "be conservative in what you send and liberal in what you send". Telnet can understand IAC EOR perfectly well whether or not DO EOR has been established. Well Bob, can this longwinded discussion serve as a draft of the needed rfc, or should be rfc just be the single paragraph of yours that I quoted above?
minshall%opal.Berkeley.EDU@UCBVAX.BERKELEY.EDU (03/09/87)
All, First, as to Philip Budne's complaint about 3270 negotiation making it hard to pass the REAL terminal type across the telnet connection - true. But, you have to think about where this all came from. The original intent was that real, live, ibm 327x's, when initiating telnet from their real, live ibm host, should be able to talk to other ibm hosts in a "native" fashion. No one should complain about this - we let vt241 users get their vt241's connected in "native" fashion when talking between two Vax Unix systems. However, the people at Wisconsin, Rice, Berkeley, Yorktown labs, Cornell, and who knows where else have realized that "hey, we can do that from Unix/ms-dos/macintosh-land, too!" Then, there IS a problem. Maybe what we need is "do terminal-type" and also "do charlatan-terminal-type". Anyway, the point is that it WOULD be convenient if we knew we were trying for 3270 emulation; or we could pass both a series of 3270 types and of "real" terminal types. Say you are connecting to a system (any old system) from a PC. The system may have a "preferred" terminal type (VT241, say), and the PC may have a few emulation modes (adm3a, vt100, tvi950, say). So, the system keeps says "send terminal type", and the PC sends them all, and finally indicates end-of-file with "unknown". Now, we have just "lost", since the system might just wish it had been willing to take the best of what had been offered at the time (maybe vt100). This is something like "the price is right" (TV show, for the uncultured). "You may now take this vt100 and run with it, OR you may toss it away in the hopes that what lies hidden behind that door may be better for you". In fact, it may be more complicated than that. Still, I don't think so. So, maybe the ability to "see the list again" would allow for things to work. If so, then the server could see the whole list, keep track of "what seemed best", and then go back and ask for that (so, why not "send terminal LIST", and "SET terminal type"?). (The point is, right, that we are TRYING to learn; not that "we made a mistake!".) As to Marvin Solomon's comments, I doubt I can add much. I do believe that the server could buffer some data, waiting for the end of the terminal type/eor/binary negotiations to happen before committing to either NVT or 3270 mode. The server is probably in error in not accepting even some of the negotiation out of order (but, it isn't the only server with this bug). The bottom line in the Wiscnet case is, probably, people to do the various "small" things. To my knowledge, Wisconsin is no longer getting any IBM money to support (let alone enhance) Wiscnet. I think an RFC would be nice. I think, however, that maybe it needs to address a fairly large amount of issues. Greg
Kodinsky@MIT-MULTICS.ARPA.UUCP (03/29/87)
I wish to first apologize for not reading and responding to this note earlier. It obviously has great importance to the KNET telnet programs (both client and server). The KNET telnet client and server operate almost identically to the procedure laid out in Marvin's note (at least that is the way they should operate). There are a few exceptions - FIRST: we will renegotiate the terminal type at any time - though in ] the "No change" mode. As a matter of fact, if the terminal type were to be changed (within telnet server storage) there would be no effect - the logical device already exists. SECOND: we do not, as best as I can tell, require an EOR option to be negotiated. If we are in "3270" mode then we automatically assume that EOR is going to be understood by the other end. I think that an RFC would be a good thing for this. It should address two issues: the details of 3270 telnet AND (more importantly) the issues involved in tying together interdependent options (as Marvin Pointed out). We at Spartacus would be glad to work with the community on this RFC. We do not feel that we can take a lead in this effort since we have been on the net for only a few months. Regards, Frank Kastenholz - Manager KNET/VM Development - Spartacus/Fibronics