BILLW@SU-SCORE.ARPA (William "Chops" Westfield) (10/31/85)
The Tops20 TCP implementation attempts to be "sociable" by not sending bare ACKs. In order to do this, whenever it receives a packet that requires an ACK, it tells itself to wait for 250 milliseconds for some data to appear that is going in the opposite direction so that the ACK can go with it. On something like a telnet connection, this can essentially half the number of packets being sent, and so it is an excellent and widely implemented idea. Unfortunately, Many TCP connections are essentially "one-way". There is never going to be any data to send in the reverse direction to piggyback the ACK on, and the system ends up waiting when it should send and ack immediately. Furthermore, some TCPs (including TOPS20's) calculate their retransmission interval based on how long it takes them to receive the ACK for a packet they have sent. Delaying your ACK waiting for data disturbs this calculation, as pointed out by Alan Larson on the TOPS20 mailing list (27-Jun-85). Now, if the time it takes the other system to fill the window (presumably including data transmission time) is less than the time that your system waits to send the ACK, transmission becomes bursty, and you get serious performance degradation. This is aggravated by fast networks and small windows. Such was the case on Stanford Tops20 systems (which are on ethernets) running programs using the DEC JFN interface to TCP (which advertises a maximum 1456 byte window). Using FTP, for example, getting a file transfer speed of greater than 30K bps was rare. Using a BBN/CMU FTP (which advertises much larger windows), higher throughputs could be achieved (up to over 300Kbps, if irratically, and in extreme cases), which provided additional frustration. PUP FTP regularly beat 150Kbps. To improve upon this situation, I have implemented a concept I call "Interactivity". A connection is maximally interactive if a packet is sent every time a packet is received, and minimally interactive if it never sends any packets. In the first case, you want to wait a bit to try to piggyback acks on the outgoing data, but in the second case this introduces delays, and you want to send acks immediately. "Inverse interactivity" can be approximated by counting the number of packets received between each one sent. If greater than N, it is probably a good idea to send ACKs immediately. [Note: Immediately in this case mean AFTER to have chosen not to avoid sending the ACK to prevent the Silly Window Syndrome]. After performing this operation on the local tops20 systems, FTP data rates (using the JFN interface) as high as 150kbps were seen on the local ethernet. As might be expected from the above discussion, little increase in performance was seen on connections to hosts "further away" or on connections that used very large windows. I am CCing this message to various mailing lists besides TOPS20 since the idea may be useful within other TCP implementations. [The new tops20 sources (ANAUNV, TCPBBN, TCPTCP, STG) are available from Sierra in <6-1-monitor>, although these modules include some additional TCP fixes of a less general nature. They are also on Score, though those sources may be in transitory states. This monitor has been running on Score and Sushi since last friday without any apparent ill effects.] BillW -------
MILLS@USC-ISID.ARPA (11/01/85)
In response to the message sent Thu 31 Oct 85 01:31:59-PST from BILLW@SU-SCORE.ARPA Bill, It's refreshing to see that somebody is still brave enough to keep hacking at TCP trying to improve performance. We have seem some awful broken things wandering by our gateway, some of which caused massive congestion due imappropriate choices of retransmission timeouts, packetization, etc. Our darling fuzzballs were taught a very similar trick some time back for the same reason - to improve throughput on low-delay nets while sustaining good packetization efficiency on long-delay ones. However, we took a slightly different approach. If the left window edge moves an ACK will be guaranteed after no more than a nominal delay (about the same as yours). If the right window edge moves more than MSS octets since the last ACK, an ACK is forced immediately. Thus, high-volume traffic with max-size packets is not delayed, while interactive traffic with tinygrams is delayed for piggybacking opportunities. Using the Nagle Algorithm together with this technique, a typist can twinkle on a keyboard with a TOPS-20 or fuzzball (but not a 4.2) and result in one segment flipping end-for-end picking up data, remote-echoes and ACKs as it flips. The gateways love it. Dave -------