usenix@ucbtopaz.CC.Berkeley.ARPA (01/17/85)
;login: USENET in the Sky Satellite Distribution of Netnews: The Stargate Experiment Lou Katz Introduction Several thousand computer sites in the United States, Canada, Europe and Australia are linked together into a logical "network" which permits the transfer of messages directly from one individual to another (mail) and the posting of messages to be read by anyone who is interested (news). The many sites on this network which are involved with news transfers collectively are called USENET. More specifically, USENET is defined as all sites receiving the newsgroup net.announce. A USENET link between two sites is one that net.announce is sent over. Note that this is different from a uucp link, over which mail and file transfers may occur but not necessarily news. As more computer sites have gained access to this network a number of problems have arisen, in particular with respect to the communications costs incurred in the operation of this net and to difficulty of new sites obtaining access. As usage increases, USENET is faced with the spectre of increased costs possibly forcing curtailment of network activity, an eventuality which is causing great concern in the network community. Furthermore, the magnitude of the load which news places on a site is so large that new sites have great difficulty finding a site willing to feed news to them. Many new sites wish- ing to get such information are without connections. At the present time it is estimated that there are about a thousand sites in the network, with that number growing daily! Total network traffic is basically proportional to the number of sites, so that traffic is growing too. It is vital to realize that network services, to be useful, must connect to the machines a particular individual uses regularly and as a matter of course. For news and especially for mail, it doesn't work for the person to have to make an individual special call to a different machine just to see if there is mail or news for him/her, any more than it makes sense to walk two miles to the post office each day just to see if there is mail, when there often will be none. Note however, that USENET IS NOT A NETWORK in the formal sense! That is, unlike all other nets (ARPANET, CSNET, BITNET, etc), there is NO administra- tion, no central structure, no joining, and no membership to USENET. The net actually represents the human and professional network of personal, technical and business contacts, and PAIRWISE desires for groups or individuals to com- municate and share information easily. It is just this pairwise organization which gives the network its vital- ity. Without the burden of administration, all that is required is the tele- phone switched network, which permits any machine, anywhere to contact any other machine DIRECTLY, subject only to administrative and software agreement between its managers. Some pairs of sites are linked via dedicated high speed circuits, because of the volume of traffic between them. This linkage is not, however, crucial to the operation of USENET. Volume 9, Number 6 December 1984 1 ;login: News forwarding often represents a massive percentage of the overall data traffic flowing through a given USENET site. Some sites have taken this responsibility upon themselves for a variety of reasons, but most sites will only receive news or forward it to very specific recipients. Mail is treated somewhat differently, and many sites will forward mail as a professional cour- tesy to others, which improves overall mail performance, and helps ensure that others will forward mail to them. Estimates indicate that MAIL accounts for about 15% of the network "load" and NEWS for about 85%, although at high volume nodes or central sites which forward both news and mail, mail may reach 50%. For two machines to be networked, they have to be connected in some manner. This connection can be a dedicated link (leased phone line, internal wires within a site, infrared relay, fiber optics) or a shared link such as a dialup line. Dedicated links, except for the trivial case of running a wire between two machines in the same room, almost always involve dealing with large external entities such as local phone company and common carriers to get special dedicated wire services. These links are expensive, are rapidly grow- ing more expensive, and can involve very long (months or longer!) waiting periods for installation in many areas. The cost of network phone calls is hard to see directly. However, if one conservatively estimates that there is about 1 Mb of news every DAY, and if this is transmitted at 1200 bits per second with a an error-correcting transmission protocol, there are roughly 3 hours of transmission per day. Current phone rates run about $.15-$.30/minute in the dead of the night (the times usually selected for transmission, just to keep the costs down) either interstate or intrastate. If a site getting news initiated the call itself, it would spend about $36/day or a little over $1000/month on such phone calls. Unfortunately, many phone calls wind up with bad circuits, giving numerous retries and aborted messages. This can add up to a factor of two on call costs. If two sites attempt to utilize a single phone line for both a feed in and a feed out they are likely to utilize the entire 11pm-8am nighttime rate slot on one line. Since there is a pyramiding effect with each site servicing several below it, a single site could easily dedicate two or three lines just to network service, and often wind up using more expensive evening and daytime connections. If only one hundred sites have to make non-local calls for this purpose, the national phone bill attributed to network activity would be over $100,000/month! This amount may very well be much too conservative, as reli- able rumors suggest that the phone bill for one particular site is signifi- cantly in excess of $20,000/month! New technology is beginning to provide us with modems capable of working on the nationwide switched network at speeds of 2400 bps, or double the present common speed. The costs of these modems are much higher than the com- mon 1200 bps hardware. Installation of such devices could cut some phone bills considerably (though by less than half, due to various technical fac- tors), but only if both sides of a connection have them. It is unfortunately easier for many system administrators to justify rising phone bills than to receive approval for such a specific purchase, often from a different budget 2 December 1984 Volume 9, Number 6 ;login: category, so that faster modems may not provide any relief. In fact, even if phone bills COULD be cut in half in this manner, the costs would still remain very high, and would still grow constantly worse as the network grows and news traffic continues to increase. A Possible Solution Lauren Weinstein has presented at the Summer 1984 USENIX Conference in Salt Lake City (Cf. Conference Proceedings, p. 18) a promising technological solution to the most pressing part of the problem, the cost of news transmis- sion. The idea is as follows: portions of the video signal on TV transmis- sion are not used for picture information, and can carry other information, in particular, suitably encoded ASCII. The effective bandwidth of this type of transmission could easily exceed 65 Kbps. It should be possible to establish a computer system at the "headend" of a cable or satellite transmission sys- tem, and upload such information piggyback on the TV signal. Any site which wished to receive the data would get a decoder and either a cable link or a satellite receiver dish. The decoder would have sufficient internal memory to store a significant fraction of a day's news transmission (e.g. 500 Kbytes), so that the local computer could buffer and flow control the input and select and extract the information it wanted from the decoder at its own pace. Estimated costs for the decoder are about $1000 each (retail) and about $1300-$1500 for a satel- lite dish for most locations in the continental U.S. and parts of Canada, if the channel with the information were not also carried by a local cable TV company. The economics of netnews would then change radically. No longer would a fanout of news have to occur over the dialup network. Rather, each item could be transmitted ONCE to the head end distribution computer, then "broadcast" for all to receive over the satellite system. The TOTAL "national" phone bill for news then decreases to about $1000/month, instead of several hundred thousand dollars. The cost of the original transmission which occurs when an item is sub- mitted (the phone call from the submitter's computer to the satellite link) is obviously borne by the submitter. The costs of the reception equipment and decoders are either one-time costs to the installation, easily amortized over a few months of phone bills, or else handled as monthly rental fees. This scheme does not, in any way, cut off the current mode of transmission of net- news. However, as more and more sites have to examine their phone budgets, they will generate both dollars and justification for inclusion of more and more newsgroups via satellite transmission. Volume 9, Number 6 December 1984 3 ;login: The Experiment Lauren Weinstein has secured the cooperation of several corporations and institutions in conducting an experiment into the technical feasibility of this mode of transmission. The purpose of the experiment is to test the reception quality, error rates, flow control and system reliability and functionality. Reception will be tested both directly from a small reasonably priced microwave dish, and from ordinary cable-TV service in a number of locations. The USENIX Association is providing support for incoming phone lines at the transmitter site, a small microwave receiver dish to test that mode of reception and travel to the transmission site to set up the system. The Asso- ciation is also providing coordination of the efforts of Lauren Weinstein and the other participants, as well as dissemination of the results through writ- ten articles in ;login: and, of course, over USENET, and a presentation at the January technical meeting in Dallas. If technical conditions permit, there will also be a live demonstration of the system at that meeting. SSS (Southern Satellite Systems, Atlanta, Georgia) is supplying the experiment with continuous use of one scan line in their broadcast signal, with an effective baud rate of 1200 baud for a few months. They are also pro- viding access to the uplink encoder which will properly format the input ASCII information and insert it into the TV signal. There transmissions are going out under the TV signal of WTBS, the Atlanta-based "Superstation", which is widely available throughout the United States. They are also providing two sets of tuners and decoders for receiving the signal directly and extracting the ASCII stream from the video. Bell Communications Research (BCR) is providing modems for the uplink facility and other support. Fortune Systems Corporation (Redwood City, California) has provided the uplink computer, a Fortune Systems XT30 UNIX system, which will receive net- news articles from dial-in phone lines and format them for insertion into the video signal. If the experiment shows that we will achieve satisfactory performance from a technical point of view, the UNIX community at large will then be faced with the far more difficult problem: how to make this technology available so that USENET will flourish. The future organization of USENET is a more com- plex issue. For a stable network capable of functioning over the next few years, a host of legal, financial and organizational issues must be faced. How can even a modest effort be financed? What information or news groups could such a network distribute? Who would be responsible for content? These and other considerations must be worked through if satellite transmission is to become a viable facility. 4 December 1984 Volume 9, Number 6