[net.unix-wizards] lost characters in serial I/O

jwp@uwmacc.UUCP (Jeffrey W Percival) (10/13/85)

We have an 11/70 running 2.8BSD, and are trying to send in data from
a remote site that supports only a crude protocol.  The remote person
logs into the 11/70, and types "cat > filename".  Then the remote person
causes the remote computer to send data down the line (2400 baud), and cat
stuffs the bytes into the file.  The remote site ends with a cntl/d.  The
remote computer uses XON/XOFF in the transfer.  Here's the problem:
if only the remote person is logged in, everything's fine.  If just
one other person logs in, then data dropouts appear in the file being
built by cat.  That is, just one additional login interferes with the
data transfer.  Can anybody suggest what the problem is?  If you want
or need more info, let me know by e-mail.

Oh yeah...  we also see something like this in another context.  When
the system is *extremely* busy, sometimes characters from users'
keyboards get dropped.  A user might type "abcdefg" and see only "abg"
on the terminal screen.  Are these problems related?  Why does very
light use cause problem #1, but problem #2 appears only with heavy use?

We will appreciate any hints you can give on this...

-- 
	Jeff Percival ...!uwvax!uwmacc!jwp

root%bostonu.csnet@CSNET-RELAY.ARPA (BostonU SysMgr) (10/15/85)

>From: jwp@uwmacc.UUCP (Jeffrey W Percival)
In summary: a file is sent over a serial line in a very simple manner,
chars are lost although XON/XOFF is enabled. Similar behavior is seen
with typing at the system at very high loads.

1. Check that the remote system really is sending the XOFFs

2. Check that the local system (or terminal) really is stopping
when an XOFF is received.

3. Check the points at which the XOFF is sent in the kernel (eg. tty.c)
which is going to be a little below where the system decides to throw
things away (it gives a little margin to let the other side stop sending.)
Raise the margin to allow more time for the other side to respond.

4. If the file you are sending does not have a newline after 255 (or
whatever the manifest limit is in your kernel) characters things start
getting thrown away (TTYHOG.) Putting the remote end into CBREAK may
help this (and will probably speed up the kernel's ability to pass
thru chars as a lot of code checking for things like rubouts is
bypassed when CBREAK'd, tho I have never measured this. What you really
need here is the SYSV TERMIO stuff to suppress making things worse with
lots of context switches.)

5. If you are going through some sort of switch and it buffers and
it does not respond to XOFF's (just passes them through) you might be
in trouble as it will continue pouring in chars waiting in the switch
although the remote has heard the XOFF, not likely but a guess.

6. Write something trivially more sophisticated that ACKs each time a
line is read and teach the other side to wait, lockstepping in each
line. Or put a delay after each line sent (this is tricky.)

7. Shut off echoing when transferring (stty -echo;cat > file;stty echo)
I have seen this cause all sorts of troubles.

8. Use something like KERMIT which is quite expert in all these problems
and is free and probably available for both sides (certainly UNIX.) It
has the added advantage of not only flow-controlling but also error checking.

9. As far as user's typing goes, either try to reduce the load, try allocating
more memory for clists in the kernel or make sure user's terminals are set
up to refuse keyboard input if they hear an XOFF (VTxxx's will do this),
this sounds like an extreme situation if you don't trace it down to a bug
somewhere.

	-Barry Shein, Boston University

peter@graffiti.UUCP (Peter da Silva) (10/17/85)

> We have an 11/70 running 2.8BSD, and are trying to send in data from
> a remote site that supports only a crude protocol.  The remote person
> logs into the 11/70, and types "cat > filename"...

I'd guess that you're using DEC serial cards, which are notoriously
flakey. A lot of them (no names come to mind) can only support better
than 300 baud for short bursts... that is, they can accept people's
typing but file transfer just fills their little silos and they crater.

If the device driver is trying to cover for this by throwing CPU cycles
at it, then strange things will start happening as the load goes up. First
file transfers will die, then you'll start losing characters from normal
typing... which seems to be what's happening.

I don't know of any solution except to go for a more accurately rated serial
card.