[comp.protocols.tcp-ip] Wollongong telnet and new line processing

wrk@abvax.UUCP (07/17/87)

I've been tyring to get our BSD 4.3 VAX to talk to a VAX/VMS
system using Wollongong telnet and have discovered an interesting
problem. Unix uses \n as the line terminator, but VMS interpetes
that as 'delete the word to the left of the cursor'. So imagine
this:
	% telnet vms_node

	Username: GUEST
		       ^
		       and here you type a return.

Unix translate the \r into a \n and sends it across the net
to the VMS machine which promptly deletes the word GUEST. This
makes logging in a little difficult.

I've modified telnet to send a \r instead of a \n for the line
terminator. Although I think this is only a work around.

I say this because if you are connect to the Unix machine
via a direct line everything works fine. This problem only
occurs when you are connected via a pty. We use Bridge
cs/1 terminal servers almost exclusivly, hence you are always
connected to a pty and never to a direct line. 

This leads me to think that the problem lies somewhere in the
pty driver. Anyone have any ideas? 

Thanks
Bill King
...!{pyramid,decvax}!abic!wrk

dab@oliver.cray.COM (Dave Borman) (07/20/87)

In reply to William R. King's message, where he is trying to connect
to a VAX/VMS system running Wollongong.  He claims that when using
telnet from a directly connected terminal on his 4.3BSD VAX things
would work, but when he was logged into the VAX via a Bridge cs/1
terminal, all of his \r's were turned into \n, which on VMS thought
that meant 'delete a word to the left of the cursor'.  Bill changed
telnet, and belives that it is a pty problem on the VAX.

	My guess is that it is either a Bridge or a 4.3 telnetd
problem, or mayby even a combination of them. When you run telnet,
it defaults to single character input, with remote echo.  In this
mode, every character that comes in from the terminal/pty is passed
straight through.  Telnet puts the the terminal/pty into CBREAK mode,
with ECHO & CRMOD turned OFF.  This means that the terminal driver
will do no translation between \r and \n.  Hence, if a \n is showing
up at the VMS node, it must be coming from before the pty, i.e. either
from telnetd or dirctly from the Bridge box.
	Be aware that between 4.2 and 4.3, changes were made to both
telnet and telnetd to deal with the \r/\n issue.  4.2 did not conform
to the TELNET spec in this area.
	A simple test that could be done would be to write a short
program that puts the terminal into CBREAK mode, and turns off ECHO
and CRMOD, and then just dumps it's input in octal.  Run that on the
4.3 VAX when logged in from the Bridge box, and see what it sees when
you type \r.
	Then write a dummy server that does the same thing, run it on
some other TCP port, and telnet to that alternate port from the Bridge
box (assuming it will let you do that...), and see what it says a \r is.
	If you get "\n" in the first case, and "\r" or "\r\n" in
the second case, the problem is in telnetd.  If you get "\n" in
both cases, then the problem is in the Bridge box.
	If the 4.3 telnetd is not in binary mode, then "\r\n" will be
translated to "\n".  If the bridge box is sending "\r\n" when you type
"\r", then the 4.3 telnetd is where the translation is taking place.
What should be sent is "\r\0", which is the correct sequence to send
according to the TELNET spec. (If the 4.3 telnetd is in BINARY mode,
then this translation will not take place.) "\r/\n" means "move the
cursor to the begining of the next line", usually tranlated to '\n'
on UNIX machines, "\r\0" means do a "carriage return", (octal 015)
and a '\n' (not preceeded by '\r') means do a "line feed" (octal 012).

	I hope I've helped to clarify things, not confuse them even
more.
		-Dave Borman,
		Cray Research, Inc.
		dab@umn-rei-uc.arpa

CASNER@VENERA.ISI.EDU (Stephen Casner) (07/20/87)

The TELNET spec (RFC 854, pp. 11-12) says that the line terminator is
to be the pair of characters CR-LF (\r\n if you prefer).  I have seen
problems in both UNIX user telnet (sending just \n) and server telnet
(not interpreting \r\n properly).  This particular bit of UNIX chauvinism
seems to be rather difficult to stamp out.  The problem has been discussed
on this list more than once in the past.  It often comes up when one tries
to communicate between a UNIX machine and something else.
-------

KVC@ENGVAX.SCG.HAC.COM (Kevin Carosso) (07/21/87)

> Unix translate the \r into a \n and sends it across the net
> to the VMS machine which promptly deletes the word GUEST. This
> makes logging in a little difficult.

> I say this because if you are connect to the Unix machine
> via a direct line everything works fine. This problem only
> occurs when you are connected via a pty. We use Bridge
> cs/1 terminal servers almost exclusivly, hence you are always
> connected to a pty and never to a direct line.

> This leads me to think that the problem lies somewhere in the
> pty driver. Anyone have any ideas?
> Bill King

We had the same problem here, but a different manifestation (couldn't
run a local editor or TELNET to a VMS system running CMU IP/TCP).

The problem occurs when you connect via TELNET to a 4.3 system and then
run any program which reads in raw mode and expects to see EXACTLY the
key the user typed.  I traced the problem to the fact that "telnetd.c"
has the following in it:

                        /*
                         * We map \r\n ==> \n, since \r\n says
                         * that we want to be in column 1 of the next
                         * printable line, and \n is the standard
                         * unix way of saying that (\r is only good
                         * if CRMOD is set, which it normally is).
                         */
                        if ((myopts[TELOPT_BINARY] == OPT_NO) && c == '\r') {
                                if ((ncc > 0) && ('\n' == *netip)) {
                                        netip++; ncc--;
                                        c = '\n';
                                } else {
                                        state = TS_CR;
                                }
                        }

I don't think it's right to map <CR><LF> to <LF>.  I changed it to map to <CR>
instead, and everything works just fine.  The terminal driver will map the <CR>
to <LF> when necessary and programs that think that are supposed to get a <CR>
now work fine.

My rationale was that the UNIX tty driver is set up to work with terminals
that usually send <CR> when the user hits "carriage return".  A pty works
just like a tty, so we should be sending <CR> into it.

I suppose another approach might be to say that the Bridge TELNET code is
broken and that it shouldn't be sending <CR><LF> when you type
"carriage return" but should send <CR><NUL>.  I can't fix the Bridge code,
though.

        /Kevin Carosso                     kvc@engvax.scg.hac.com
         Hughes Aircraft Co.               kvc%engvax@oberon.usc.edu

Mills@UDEL.EDU (07/22/87)

Steve,

Well, the annual raucus caucus over the TELNET new-line convention has
come around again, exactly a year after the previous one, which itself
was the latest in the annual series. last year it was Ford Research
squawking that Wollongongware did not love fuzzballs. Subsequently,
the Wollongongbunch responded to the complaint and fixed the problem.
The present instance suggests the gongfermers should be called. You
may look up that interesting word in a dictionary.

Dave

jjensen@atlas.UUCP (Jim Jensen) (07/22/87)

in article <25@abvax.icd.ab.com>, wrk@abvax.icd.ab.com (William R. King) says:
> 
> I've been tyring to get our BSD 4.3 VAX to talk to a VAX/VMS
> system using Wollongong telnet and have discovered an interesting
> problem. Unix uses \n as the line terminator,
>[deleted about sending a \r to VMS]
> Unix translate the \r into a \n and sends it across the net
> [problems this causes on VMS deleted]
> I've modified telnet to send a \r instead of a \n for the line
> terminator. Although I think this is only a work around.
> 
> I say this because if you are connect to the Unix machine
> via a direct line everything works fine. This problem only
> occurs when you are connected via a pty. We use Bridge
> cs/1 terminal servers almost exclusivly, hence you are always
> connected to a pty and never to a direct line. 
> 

(this is my first posting so I dont know if I am doing this right)

I just finished writing telnet for MPX (Gould's real time OS), and ran into 
the same problem.  The bridge box sends a \r\n which is the specifed in the
telnet rfc as the end of line character. which the unix telnet server turns
into a \n.

on telnet 4.2 the  telnet task, converts this back to \r\n 
which the VMS(or MPX)server converts back to a \r and everything works.

4.3 leaves it as a \n, which seems to me to be a clear violation of the 
protocol.  The rfc SAYS that \r\n is the end of line character, not \n.
Thats what protocols are for; To take care of differences between machines.
When a protocol is not followed problems like this happen.

I asked one of our unix people about it, and he said that  
BSD did it on purpose, and it is in telnet. I was unable to convince
him it was a mistake, so he would not change it.  that is until some people
in our unix group tried to use a bridge box. Thus:

 the latest patch to Gould Unix has a telnetd which has a variable that can be
set called Telnet_with_a_bridge_box(if I remember correctly) that if set fixes 
the problem.

I am not a unix person, so I may have gotten something wrong about unix, but
I don't think so.
                                              Jim Jensen
					      Gould inc. CSD 
					      6901 West Sunrise Boulevard
					      Ft. Lauderdale Fl 33313-4499
					      (305) 587-2900

jjensen@atlas.UUCP (Jim Jensen) (07/22/87)

in article <25@abvax.icd.ab.com>, wrk@abvax.icd.ab.com (William R. King) says:
> 
> I've been tyring to get our BSD 4.3 VAX to talk to a VAX/VMS
> system using Wollongong telnet and have discovered an interesting
> problem. Unix uses \n as the line terminator,
>[deleted about sending a \r to VMS]
> Unix translate the \r into a \n and sends it across the net
> [problems this causes on VMS deleted]
> I've modified telnet to send a \r instead of a \n for the line
> terminator. Although I think this is only a work around.
> 
> I say this because if you are connect to the Unix machine
> via a direct line everything works fine. This problem only
> occurs when you are connected via a pty. We use Bridge
> cs/1 terminal servers almost exclusivly, hence you are always
> connected to a pty and never to a direct line. 
> 

(this is my first posting so I dont know if I am doing this right)

I just finished writing telnet for MPX (Gould's real time OS), and ran into 
the same problem.  The bridge box sends a \r\n which is the specifed in the
telnet rfc as the end of line character. which the unix telnet server turns
into a \n.

on telnet 4.2 the  telnet task, converts this back to \r\n 
which the VMS(or MPX)server converts back to a \r and everything works.

4.3 leaves it as a \n, which seems to me to be a clear violation of the 
protocol.  The rfc SAYS that \r\n is the end of line character, not \n.
Thats what protocols are for; To take care of differences between machines.
When a protocol is not followed problems like this happen.

I asked one of our unix people about it, and he said that  
BSD did it on purpose, and it is in telnet. I was unable to convince
him it was a mistake, so he would not change it.  that is until some people
in our unix group tried to use a bridge box. Thus:

 the latest patch to Gould Unix has a telnetd which has a variable that can be
set called Telnet_with_a_bridge_box(if I remember correctly) that if set fixes 
the problem.

I am not a unix person, so I may have gotten something wrong about unix, but
I don't think so.
                                              Jim Jensen
					      Gould inc. CSD 
					      6901 West Sunrise Boulevard

mcc@ETN-WLV.EATON.COM (Crockett) (07/25/87)

I submitted this previously but probably performed the submission incorrectly.

*************

I don't have a copy of RFC 854; however, I do  have  MIL-STD-1782  which
defines  the implementation requirements for the TELNET Protocol for DDN
MILNET and other DDN subnetworks.  The end of line character is  a  <CR>
which  moves  the  cursor  or print head to the beginning of the current
line.

The standard became confusing when it attempted to  accomodate  physical
terminals that generate a <CR><LF> sequence when the ENTER or RETURN key
is struck.  To permit the use of these terminals a two octet end of line
function  (ELF)  was  defined.  For terminals that could generate a <CR>
and a <LF> independently, the ELF was defined to be  a  <CR><NUL>.   For
those that couldn't, the ELF was defined to be <CR><LF>.

Regardless  of  any echo options that may be in effect, the user (local)
host is responsible for transmitting only the keyboard  input  from  the
terminal  to  the  server  (remote) host.  If the RETURN key generates a
<CR><LF>, it is to be mapped to a <CR><NUL> prior to  transmission  over
the  network.  If the NVT is being operated in its default line buffered
full duplex mode  with  local  echo,  the  user  host  echoes  both  ELF
variations as a <CR><LF> to the terminal display.

At  the  server  host  <CR>,  <CR><NUL>,  and <CR><LF> are to be treated
identically and  mapped  to  the  host's  local  interpretation  of  the
character  or  character  sequence  that  is  passed  to the application
software when the RETURN key is struck.  When  the  virtual  circuit  is
being  operated in half duplex mode with the server echoing data back to
the user, it transmits either a <CR><NUL> or <CR><LF> with the ELF  form
being dependent upon the server's local convention.

The  original  problem  that  raised  the  question of the NVT ELF was a
difference in operation when a terminal (workstation) was connected on a
local  LAN  segment  or  on  a remote LAN segment requiring the use of a
gateway to make the transition between the LAN segments.  The answer  is
that  both  Woologong  and  the  Bridge  CS/1  have  errors or have been
configured incorrectly.

Any host in the network that functions as a gateway is  responsible  for
ensuring  that  the  data  transmitted is the same as what was received.
The sequences <CR><LF> and <CR><NUL><LF> are not necessarily equivalent.
The gateway can not compress the sequence <CR><NUL><LF> to <CR><LF>.

The  problem  I  have  seen  with  a  number of hardware and/or software
products that implement TCP/IP, TELNET, FTP, SMTP, and UDP is  that  the
vendor has based his product on the 4.x bsd implementation.  The problem
with the vendors that have done that is that they have failed to go back
through  the software and remove all the tacit assumptions that the host
system will be running a UNIX or UNIX derivative operating system.  Some
vendors  have  even  removed  code to support the options that are to be
negotiated always responding with a DO  or  WILL  when  they  should  be
responding  with  a DON'T or WON'T because they know that 4.x bsd always
attempts to negotiate to the same mode of operation.

Invariably these products have problems when one attempts to use them on
a  PDP11  or  VAX  based  AN/GYQ-21(V) systems which use the IAS and VMS
operating systems, respectively.

				Merton Campbell Crockett
				AN/GYQ-21(V) Program
				EATON Corporation
				Information Management Systems Division

Mills@UDEL.EDU (07/28/87)

Larry,

See the delightful little book "Cotswold Privies" by Mollie Harris and Sue
Chapman (Hogarth Press, London 1984) for a full, complete and satisfying
definition of the word. In sweet, a gongfermer is the poor (but well-paid)
lad who cleaned out the privies in yesteryesteryear. Any similarity, functional
or otherwise, to departmental of industrial institutions is stoutly denied.

Why did I use this wonderful 15th-century word? Because I hoped to get a message
like yours.

Dave

----- Forwarded message # 1:

Received: from Louie.UDEL.EDU by Huey.UDEL.EDU id aa05649; 27 Jul 87 14:25 EDT
Received: from bbncc-eur.arpa by Louie.UDEL.EDU id a010057; 27 Jul 87 11:23 EDT
Date: Mon, 27 Jul 87 17:02:50 GMT+0100
From: "Lawrence J. Kaufman" <lkaufman@bbncc-eur.arpa>
Subject: Re:  Wollongong telnet and new line processing
In-Reply-To: Your message of Tue, 21 Jul 87 22:03:44 EDT
To: Mills@louie.udel.edu
Cc: lkaufman@bbncc-eur.arpa
Message-ID:  <8707271123.a010057@Louie.UDEL.EDU>

Dave,

Would you please explain gongfermer to a poor city boy?  What does a
privy farmer do and why are you using a 15th century word?

--- Larry Kaufman
    lkaufman@bbn.com
    now in Germany, moving to Maryland in the next week or so.


----- End of forwarded messages