wiltzius@lll-lcc.UUCP (Dave P. Wiltzius) (02/13/90)
I'm testing my port of a version of 4.3BSD TCP/IP code. In doing so, I discard in the loopback driver every Nth packet (say, N=10). This results in the MBUF pool being quickly depleted. I discovered that the TCP input routine will save all out of order TCP packets waiting for the TCP retry to fill in the gaps. Hence most of the MBUF pool is enqueued on this TCP fragment/reassembly list for this TCPCB. Also, the sender continues to get ACKs advertising the largest window since the receive socket buffer is empty. Nothing particularly evil happens and since this is a very degenerate case, perhaps I should just shrug my shoulders and go on with life. But it does bother me a bit - rightfully so? (Couldn't the window advertised by the receiver close, or a limit made on the number of out-of-order TCP packets on a TCPCB?) Thanks for any help. Dave (wiltzius@lll-lcc.llnl.gov)
subbu@hpindda.HP.COM (MCV Subramaniam) (02/15/90)
> >I'm testing my port of a version of 4.3BSD TCP/IP code. In >doing so, I discard in the loopback driver every Nth packet >(say, N=10). This results in the MBUF pool being quickly >depleted. I discovered that the TCP input routine will >save all out of order TCP packets waiting for the TCP retry >to fill in the gaps. Hence most of the MBUF pool is >enqueued on this TCP fragment/reassembly list for this >TCPCB. Also, the sender continues to get ACKs advertising >the largest window since the receive socket buffer is >empty. > >Nothing particularly evil happens and since this is >a very degenerate case, perhaps I should just shrug >my shoulders and go on with life. But it does bother >me a bit - rightfully so? (Couldn't the window advertised >by the receiver close, or a limit made on the number >of out-of-order TCP packets on a TCPCB?) > Remember, the ACK number in all those ACK packets is the last sequence number recevied *in order*. So, decreasing the available window will mean shrinking the window, which is not acceptable as good behavior. It is the sender's responsibility to realize (on seeing old ACKs, and timing out for not receiving new ACKs) that old segments have to be retransmitted. Btw, are you using 4.3 BSD for the sender too? >Thanks for any help. Hope this helps. > Dave (wiltzius@lll-lcc.llnl.gov) PS: You could also tune your implementation to set a limit on the number or segments (or mbufs, or memory, whatever) on the reassembly queue. This way, you won't run out of mbufs for other protocols/applications. -Subbu email: subbu@hpindaw.hp.com
dab@OPUS.CRAY.COM (Dave Borman) (02/17/90)
> I'm testing my port of a version of 4.3BSD TCP/IP code. In > doing so, I discard in the loopback driver every Nth packet > (say, N=10). This results in the MBUF pool being quickly > depleted. I discovered that the TCP input routine will > save all out of order TCP packets waiting for the TCP retry > to fill in the gaps. Hence most of the MBUF pool is > enqueued on this TCP fragment/reassembly list for this > TCPCB. Also, the sender continues to get ACKs advertising > the largest window since the receive socket buffer is > empty. > > Nothing particularly evil happens and since this is > a very degenerate case, perhaps I should just shrug > my shoulders and go on with life. But it does bother > me a bit - rightfully so? (Couldn't the window advertised > by the receiver close, or a limit made on the number > of out-of-order TCP packets on a TCPCB?) > > Thanks for any help. > Dave (wiltzius@lll-lcc.llnl.gov) There are a few things to be aware of. First, the TCP reassembly queue will not grow for ever. It can only hold as much data as was offered in the window. So, in theory, this is no different that having the user process stop reading the socket, and letting the receive queue fill up with data. But in actuallity, there are some differences. The 4.3 code does not do any compression on the data in the reassembly queue, like it does on the receive queue for the socket. Thus, if you have a window of, say, 4k, and you have the data coming in in 1 byte packets, and you loose the first packet, you could wind up having over four thousand mbufs stuck in the reassembly queue. There are two things that should be done to improve this situation: 1) fix the reassembly code to do compaction. 2) When you run out of mbufs, flush the tcp reassembly queues. Both of these features have been in our networking code for several releases. -Dave Borman, dab@cray.com