sanjay@wucs1.wustl.edu (Sanjay Kapoor) (08/03/90)
I don't know if this question has already been asked or not but anyway. Q I am trying to investigate the relationship (if any exists) between FDDI frame size and the effective throughput on the ring for a node. How does the throughput vary as a function of the number of nodes? Does anyone know of any classical papers/articles that address this issue? I don't know if this question has already been asked or not but anyway. Thnkxs -Sanjay Kapoor kapoor@wuee1.wustl.edu
rpw3@rigden.wpd.sgi.com (Rob Warnock) (08/04/90)
In article <1990Aug3.015719.11688@cec1.wustl.edu> sanjay@wucs1.wustl.edu (Sanjay Kapoor) writes: +--------------- | Q I am trying to investigate the relationship (if any exists) between FDDI | frame size and the effective throughput on the ring for a node. +--------------- The relationship is fairly direct. Let's assume you are using TCP/IP with RFC-1103 encapsulation (SNAP #0 over LLC UI Type 3), as most do. Then I assume by "throughput" you mean user data bytes per second. All you need to do is add up the overhead bytes, and then maximum throughput is just 100 Mb/s * UserBytesPerPacket / (UserBytesPerPacket + OverheadBytesPerPacket). OverheadBytePerPacket comprise: Preamble ("idles") 8 bytes (min., from the transmitter's p.o.v.) FC byte 1 Destination MAC addr 6 (I'm assuming normal 48-bit addresses) Source MAC addr 6 802.2 LLC 3 RFC-1103 (SNAP) 5 (includes 2-byte "Ethernet type") IP 20 (min., no options) TCP 20 (min., no options) ...user data goes here... CRC-32 4 "T" & 3 "R/S" symbols 2 (no "optional Control Indicators") ===== 75 bytes I'll leave the rest of the arithmetic and choosing of packet sizes to you. Oh, I should at least mention a few common cases: - Max. FDDI frame = 4500 bytes (incl. 2 bytes of preamble) = 4431 user data, throughput = 100 * 4431 / (4431 + 75) = 98.3 Mb/s. - Probable max user data = 4096 (nice power of 2), throughput = 98.2 Mb/s. - Typical TCP segment bridged from an Ethernet has 1024 user data bytes, throughput = 93.2 Mb/s - With one byte of user data (e.g. Telnet input), throughput = 1.3 Mb/s. +--------------- | How does the throughput vary as a function of the number of nodes? +--------------- It depends on whether hosts are capable of sending *large* amounts of data (~2 Mbytes among all the hosts) at each "service opportunity" (roughly speaking, "token capture"). If they are, then ring throughput is approximately T_Opr / (T_Opr + IdleRotTime) times the throughput calculated above based on packet size. "T_Opr" is the actual negotiated Target Token Rotation Time, and by "IdleRotTime" I mean the actual latency with a given ring configuration for a token rotation with no traffic. The worst case "IdleRotTime" is given in the FFDI spec as "D_max = 1.617ms", which is equivalent to ~20,000 bytes. (D_max occurs with a 100 km circumference double ring with 500 dual-attach single-MAC stations, as well as various other large configurations.) T_Opr (unless some station lowers it) will be equal to the default "T_max = 165 ms", or 2,062,500 bytes. With such a maximal ring, the maximum throughput will be 99% of the numbers calculated above for packet overhead. (Let's call those numbers MTAPS, or "maximum throughput adjusted for packet size".) WARNING: If each host in the above maximal ring sends T_Opr worth of data each time it's permitted to by the FDDI rules for a "service opportunity" -- that is, each host sends until it "uses up" its maximum permitted asynchronous allocation each time it gets the token -- it will be T_Opr * 500 hosts = 82.5 *seconds* between chances for a given host to send! That is, once a host has sent for T_opr, the rules on the FDDI MAC timers are such that that *no* host may send until the token has gone all the way around and just past the host that last sent. And if the *next* host sends for T_Opr... Thus the high throughput of 99% MTAPS is purchased at the cost of *horrible* latency! On the other hand, suppose there is only one host responsible for most of the traffic (say, with a big server) and that host can only send, say, 180 Kbytes of data each time it gets the token (perhaps due to some limits of FDDI controller buffer size, or window limits in its protocols), then the throughput of such a "big" ring is 90% MTAPS. [And average latency for a new host to be able to send is only (10 * D_max)/2 = 8 milliseconds.] And in the case where there is just one host sending to one other host and the most that can be sent per token is 16 Kbytes -- a TCP connection with a 32 KB receive window size which has settled down to opening one- half the window per token rotation, which is reasonable if you assume the receiving side can't generate an ACK between the time it receives the last packet of the burst before a token and the time it has to release the token (a couple of microseconds) -- then the max throughput is 45% MTAPS (or 44.2 Mb/s with 4096 bytes of user data per packet). [Avg. latency = 2.9 ms] Finally, if the sending host is off on some other network, and sending to a host on this "big" FDDI ring through a gateway, and the FDDI receiving host only opens up an 4096-byte window (some old BSD code did this), and the sending host only put 512 bytes of user data per packet (for safety, because it's routing off-net and it wants to avoid fragmenting -- many BSD implementations still do this), then MTAPS = 87.2 Mb/s, and the max throughput is (using the same assumptions as above about only 1/2-window-size per token rotation) (587 * 4) / ((587 * 4) + 20000) * MTAPS = 10.5% MTAPS = 9.2 Mb/s, which is still enough to keep an Ethernet busy on the other side of the gateway. Please note that all of these numbers have been for a "worst-case"-sized FDDI ring. Rings with much smaller numbers of nodes and much less fiber distance will have far less "virtual overhead" due to the gating effect of the token, and far lower average latencies. Or, as is often said in these newsgroups, "Your mileage may vary". Still, it pays to be very careful when setting FDDI ring parameters such as the number of nodes, length of fiber, and T_max/T_Opr. One more observation: Restricting protocol window sizes in the hosts may be a better way to control latency than restricting T_Opr. On a ring with a large number of hosts, restricting T_Opr may actually appear to *increase* latency, as it effectively reduces the ring bandwidth. That is, if some group of hosts starts hitting the T_Opr limit, hosts *upstream* of them will see a sharp increase in latency. Plus, hitting the T_Opr limit is regenerative -- once one host hits it, other hosts lose their service opportunities, and thus have more data ready to send the next time they *get* a service opportunity, which makes it more likely that *they* will hit the T_Opr limit, etc. -Rob ----- Rob Warnock, MS-9U/510 rpw3@sgi.com rpw3@pei.com Silicon Graphics, Inc. (415)335-1673 Protocol Engines, Inc. 2011 N. Shoreline Blvd. Mountain View, CA 94039-7311