jjones@cs.uiuc.edu (Joel Jones) (03/16/91)
pmartin@uceng.UC.EDU (Paul Martin) writes: >3) IRC uses a great deal of bandwitdh. > One of the replies I got back mentioned the as much as 3% of the > total NSFNET backbone was used by IRC packets last month. Here are some figures retrieved from merit.edu, thanks to Scott Cattanac: Service port # packets % month #bytes %bytes J91irc 6667 50541247 1.316 3942709477 0.640 F91irc 6667 107960902 2.776 8849122531 1.397 So Paul Martins figures are pretty much correct in terms of # of packets. However, if we look at bandwidth, irc only accounted for 1.397% of total bytes sent on the NSFnet. Joel Jones jjones@uiuc.edu -- Joel Jones As the advertisment for an exhibition on Leonardo da Vinci said, jjones@uiuc.edu "They called him a genius, a botanist, a demon, a philosopher, a practical joker, an eccentric, and a visionary. No wonder he was such a great engineer."
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) (03/18/91)
In article <1991Mar15.165133.18601@m.cs.uiuc.edu> jjones@uiuc.edu writes: > Service port # packets % month #bytes %bytes > J91irc 6667 50541247 1.316 3942709477 0.640 > F91irc 6667 107960902 2.776 8849122531 1.397 > So Paul Martins figures are pretty much correct in terms of # of packets. > However, if we look at bandwidth, irc only accounted for 1.397% of total bytes > sent on the NSFnet. But it's the packets that cost. A 2-byte packet costs almost as much as a 500-byte packet. Apparently IRC sends about twice as few bytes per packet as the average service; this certainly doesn't make it cheaper. ---Dan
jdarcy@seqp4.ORG (Jeffrey D'Arcy) (03/20/91)
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: >But it's the packets that cost. A 2-byte packet costs almost as much as >a 500-byte packet. Yeah, right. On an Ethernet, you have a 14-byte Ethernet header, 20-byte IP header, 20-byte TCP header, a couple of trailers...all told less than 60 bytes, and I can't imagine other data links are that much worse. That would seem to make a 500-byte packet about nine times as costly as a 2-byte packet in terms of transmission time. Certainly there are per-packet costs regardless of the packet length, but not enough to make your statement even nearly true.
schafer@devils.rice.edu (Richard Alan Schafer) (03/20/91)
In article <703@seqp4.UUCP>, jdarcy@seqp4.ORG (Jeffrey D'Arcy) writes: |> brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: |> >But it's the packets that cost. A 2-byte packet costs almost as much as |> >a 500-byte packet. |> |> Yeah, right. On an Ethernet, you have a 14-byte Ethernet header, 20-byte |> IP header, 20-byte TCP header, a couple of trailers...all told less than |> 60 bytes, and I can't imagine other data links are that much worse. That |> would seem to make a 500-byte packet about nine times as costly as a 2-byte |> packet in terms of transmission time. Certainly there are per-packet costs |> regardless of the packet length, but not enough to make your statement even |> nearly true. True, if all you're considering is the actual number of bytes send down the pipe. But if you consider the effect on anything that has to route the packet, everything I've seen suggests that the handling of the packet headers, etc., is much more of a load than the actual size of the data in the packet itself. I.e., if decoding the headers in a 1500 byte Ethernet packet takes up more time than actually sending the data portion of the packet, then the size of the data is not necessarily relevant.
torek@elf.ee.lbl.gov (Chris Torek) (03/21/91)
>brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: >>But it's the packets that cost. A 2-byte packet costs almost as much as >>a 500-byte packet. In article <703@seqp4.UUCP> jdarcy@seqp4.ORG (Jeffrey D'Arcy) writes: >Yeah, right. On an Ethernet, you have .... Two points are important here: Most wide-area links are not Ethernet (so you cannot calculate packet lengths using Ethernet particulars); and: Much of the Internet within the continental USA (and probably also outside it, but I do not know) uses truly horrid IP forwarding implementations. The NSS routers, for instance, have (or had; probably this has been fixed by now) such long code paths that they could not keep up with T1 (1.44 Mb/s) links. The huge latency introduced by existing Internet routers really does make small packets almost as expensive as large ones. This is entirely curable (and in software, no less). -- In-Real-Life: Chris Torek, Lawrence Berkeley Lab CSE/EE (+1 415 486 5427) Berkeley, CA Domain: torek@ee.lbl.gov
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) (03/22/91)
In article <703@seqp4.UUCP> jdarcy@seqp4.ORG (Jeffrey D'Arcy) writes: > brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: > >But it's the packets that cost. A 2-byte packet costs almost as much as > >a 500-byte packet. > Yeah, right. On an Ethernet, you have a 14-byte Ethernet header, 20-byte > IP header, 20-byte TCP header, a couple of trailers...all told less than > 60 bytes, and I can't imagine other data links are that much worse. By far the largest cost of a packet on the Internet is the cost of routing. If we upgraded NSFNET to trillion-megabyte-per-microsecond links, connections over it would run perhaps a few percent faster. You can shout all you want about ``data links,'' but there aren't any fixed data links in the Internet except on local networks. How much IRC data do you think *doesn't* get propagated outside its local network? > That > would seem to make a 500-byte packet about nine times as costly as a 2-byte > packet in terms of transmission time. Certainly there are per-packet costs > regardless of the packet length, but not enough to make your statement even > nearly true. Rather than making up statistics, why don't you measure what actually goes on? The cost per byte is very small compared to the fixed cost per packet. ---Dan
jdarcy@seqp4.ORG (Jeffrey d'Arcy) (03/23/91)
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: >Rather than making up statistics, why don't you measure what actually >goes on? The cost per byte is very small compared to the fixed cost per >packet. When he wasn't quoting parts of my previous article out of context, Chris Torek pointed out what I (and probably most people here) already knew: the routers used by NSFnet have pretty poor performance. Let's see just how poor they'd have to be for your original statement (that a 2-byte packet consumes nearly the same resources as a 500-byte packet) to be true. If I recall, T1 speed is 1.44 Mb/s, or 180 KB/s. That means a single byte takes ~5.5 microseconds to be transmitted. Therefore, the difference in transmission time between a 2-byte packet and a 500-byte packet is ~2.8ms. The per-packet processing time would have to be greater than this in order to be considered more significant than the transmission time. Since the reciprocal of the per-packet processing time is the packets-per-second rate, the router would have to be limited to a sustained rate of <360pps for your first statement to be true of routers attached to T1 links. Arguments about queuing delay do not apply since we're dealing with *sustained* packet rates and therefore constant queue lengths. Also, the break-even packet rate will be much lower for slower links such as the near-ubiquitous 56Kb. So, is anyone going to tell me that the NSFnet routers are unable to route 360pps? I don't think so. To debunk your specious little argument even further, you're attempting to compare the *transmission* cost per byte to the *routing* cost per packet, which is not what I was talking about. What's relevant here is the transmission cost *per packet* compared to the routing cost per packet. This relationship *will* vary according to packet size, and for large packets the the routing cost is the lesser of the two. Since you obviously need help finding valid arguments, I'll give you one for free. In the second paragraph above I failed to mention that the routing delay *for all routers on a path* (not for a single router) would need to be >2.8ms to be the governing factor. Thus, on a 10-hop path the routing delay on *each* router would have to be only 0.28ms, and the native packet rate would have to be 3600pps. Also, queuing delay does become a factor now since the router interfaces are probably not all the same speed. Nonetheless, I think your statement as regards IRC is far from proven, especially since (a) adjacent IRC servers are typically only a few IP hops apart, and (b) it's not IRC's fault that the Internet has to deal with many unnecessarily small packets generated by applications (or systems) that don't coalesce packets properly. Chris, do you know if the NSFnet routers can handle 3600pps in loopback?
torek@elf.ee.lbl.gov (Chris Torek) (03/23/91)
In article <704@seqp4.UUCP> jdarcy@seqp4.UUCP (Jeffrey d'Arcy) writes: >Chris, do you know if the NSFnet routers can handle 3600pps in loopback? This is all from my memory of one conversation on a noisy bus (the kind that carries people around cities, not the kind you find in a computer) from almost a year ago, and I understand things have been improved (I certainly hope so!). In other words, do not assume that this number represents reality: At the time the NSS routers peaked out at not much over 300 packets per second, and certainly under 1000 packets per second. Another word of warning: this is probably a deduction from a measurement showing that each NSS hop added an otherwise unaccountable 2 to 4 ms. of latency to each packet (regardless of packet size). At the time (and perhaps still), getting a packet from one end of an NSS to another actually involved sending it through 3 to 5 separate machines. There was also mention of someone `getting the packet forwarding cost down to about 14,000 instructions' (these being on IBM ROMPs)---apparently this was an improvement that was not yet in place. -- In-Real-Life: Chris Torek, Lawrence Berkeley Lab CSE/EE (+1 415 486 5427) Berkeley, CA Domain: torek@ee.lbl.gov
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) (04/05/91)
In article <704@seqp4.UUCP> jdarcy@seqp4.UUCP (Jeffrey d'Arcy) writes: [ routers ] > Let's see just how poor they'd > have to be for your original statement (that a 2-byte packet consumes nearly > the same resources as a 500-byte packet) to be true. I believe Chris's comments confirm that routers are just that poor. This isn't a big issue. It's just that people writing WAN applications should be much more worried about the number of packets sent than the number of bytes sent. Perhaps someday the Internet will consist solely of fast (10000 packets per second, say), modern routers, and a 2-byte packet will indeed cost noticeably less than a 500-byte packet. But until then, a service like IRC that uses 1.5% of the NSFNET bytes and 3% of the NSFNET packets isn't twice as cheap as average---it's twice as expensive. ---Dan
torek@elf.ee.lbl.gov (Chris Torek) (04/05/91)
In article <27773:Apr420:02:0691@kramden.acf.nyu.edu> brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: >I believe Chris's comments confirm that routers are just that poor. `Were', not `are'. (No doubt some are still in place; it takes time to throw out broken equipment and software that is in production use.) I have some more recent information. Old NSS routers took 6 to 8 ms. of cpu time to forward each packet, due to excessive stupidity in the device driver code. (The routing time was too small to measure in these systems; `Almost all the per-packet cpu time was spent in device drivers'. In addition the old routers added store-and-forward delays at each T1-to-NSS boundary, because the T1 boards had onboard 386s doing copies in and out of onboard buffers. [Better than Z8s, I guess. :-) ]) The new CNSS/ENSS T3 routers are better. My source has no timings yet but `we had no trouble using the T3 backbone to saturate a 10Mb/s ethernet from a machine 1000 miles away'---saturate here means get approximately 1 MB/s (8 Mb/s) actual data transfer, host-to-host. (One host was a Cray, the other a Sun SparcStation, both running experimental software.) In other words, the problems may not be completely fixed yet, but they have definitely receded a comfortable distance. -- In-Real-Life: Chris Torek, Lawrence Berkeley Lab CSE/EE (+1 415 486 5427) Berkeley, CA Domain: torek@ee.lbl.gov
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) (04/07/91)
In article <11813@dog.ee.lbl.gov> torek@elf.ee.lbl.gov (Chris Torek) writes: > The new CNSS/ENSS T3 routers are better. My source has no timings yet > but `we had no trouble using the T3 backbone to saturate a 10Mb/s > ethernet from a machine 1000 miles away'---saturate here means get > approximately 1 MB/s (8 Mb/s) actual data transfer, host-to-host. That's good to hear. The big question: Can they still do it with 2-byte packets? ---Dan