brian@ucsd.EDU (Brian Kantor) (12/20/88)
RFC 1077 November 1988 not restrict the aggregate rates we can achieve over trunks, it prevents delivery of high data rate flows to the host-based applications, which will prevent the development of new applications needing high bandwidth. The host bottleneck is thus a serious impediment to networked use of supercomputers. To build a GN we need to create new ways for hosts and their high bandwidth peripherals to connect to networks. We believe that pursuing research in the ways to most effectively isolate host and LAN development paths from the GN is the most productive way to proceed. By decoupling the development paths, neither is restricted by the momentary performance of capability bottlenecks of the other. The best context in which to view this separation is with the notion of a network front end (NFE). The NFE can take the electronic input data at many data rates and transform it into gigabit light data appropriately packetized to traverse the GN. The NFE can accept inputs from many types of gateways, hosts, host peripherals, and LANS and provide arbitration and path set-up facilities as needed. Most importantly, the NFE can perform protocol arbitration to retain upward compatibility with the existing Internet protocols while enabling those sophisticated network input sources to execute GN specific high-throughput protocols. Of course, this introduces the need for research into high-speed NFEs to avoid the NFE becoming a bottleneck. 3.3.1. VLSI and Optronics Implementations In a host interface, unless the host is optical (an unlikely prospect in the near-term), the opportunities for optronic support are limited. In fact, with a serial-to-parallel conversion on reception stepping the clock rate down by a factor of 32 (assuming a 32-bit data path on the host interface), optronic speeds are not required in the immediate future. One exception may be for encryption. Current VLSI implementations of standard encryption algorithms run in the 10 Mbit/s range. Optronic implementation of these encryption techniques and encryption techniques specifically oriented to, or taking advantage of, optronic capabilities appears to be an area of some potential (and enormous benefit if achieved). The potential of targeted VLSI research in this area appears limited for similar reasons discussed above with its application in high- speed switching. The major benefits will arise from work that is well-motivated by other research (such as high-performance parallelism) and by strong commercial interest. Again, we need to be Gigabit Working Group [Page 24] RFC 1077 November 1988 open to imaginative opportunities not foreseen here while keeping ourselves from being diverted into low-impact research without further insights being put forward. 3.3.2. High-Performance Transport Protocols Current transport protocols exhibit some severe problems for maximal performance, especially for using hardware support. For example, TCP places the checksum in the packet header, forcing the packet to be formed and read fully before transmission begins. ISO TP4 is even worse, locating the checksum in a variable portion of the header at an indeterminate offset, making hardware implementation extremely difficult. The current Internet has thrived and grown due to the existence of TCP implementations for a wide variety of classes of host computers. These various TCP implementations achieve robust interoperability by a "least common denominator" approach to features and options. Some applications have arisen in the current Internet, and analogs can be envisioned for the GN environment, which need qualities of service not generally supported by the ubiquitous generic TCP, and therefore special purpose transport protocols have been developed. Examples include special purpose transport protocols such as UDP (user datagram protocol), RDP (reliable datagram protocol), LDP (loader/debugger protocol), NETBLT (high-speed block transfer protocol), NVP (network voice protocol) and PVP (packet video protocol). Efforts are also under way to develop a new generic transport protocol VMTP (versatile message transaction protocol) which will remedy some of deficiencies of TCP, without the need to resort to special purpose protocols for some applications. Research is needed in this area to understand how transport level protocols should be constructed for a GN which provide adequate qualities of service and ease of implementation. A new transport protocol of reasonable success can be expected to last for ten years more. Therefore, a new protocol should not be over optimized for current networks and must not ignore the functional deficiencies of current protocols. These deficiencies are essential to remedy before it is feasible to deploy even current distributed systems technology for military and commercial applications. Forward Error Correction (FEC) is a useful approach when the bandwidth/delay ratio of the physical medium is high, as can be expected in transcontinental photonic links. A degenerate form of FEC is to simply transmit multiple copies of the data; this allows Gigabit Working Group [Page 25] RFC 1077 November 1988 one to trade bandwidth for delay and reliability, without requiring much intelligence. In fact, it is generally true that reliability, bandwidth, and delay are interrelated and an improvement in one generally comes at the expense of the others for a given technology. Research is required to find appropriate operating points in networks using transmission components which offer extremely high bandwidth with very good bit-error-rate performance. 3.3.3. Network Adaptors With the promised speed of networks, the future network adaptor must be viewed as a memory interconnect, tying the memory in one host to another, at least if the data rate and the low latency made possible by the network is to be realized at the host-to-host or process-to- process level. The challenge is too great to be met by just implementing protocols in custom VLSI. Research is required to investigate the impact of network interconnection on a machine architecture and to define and evaluate new network adaptor architectures. Of key importance is integration of network adaptor into the operating system so that process-to- process communications performance matches that offered by the network. In particular, we conjecture that the transport level will be implemented largely, if not entirely, in the network adaptor, providing the host with reliable memory-to-memory transfer at memory speeds with a minimum of interrupt processing bus overhead and packet processing. Drawing an analogy to RISC technology again, maximal performance requires a well-designed and coordinated protocol, software, and hardware (network adaptor) design. Current standard protocols are significantly flawed for hardware compatibility, suggesting a need for considerable further research on high-performance protocol design. 3.3.4. Host Operating System Software Conventionally, communication has been an add-on to an operating system. With the GN, the network may well become the fastest "peripheral" connected to most nodes. High-performance process-to- process (or application to application) communication will not be achieved until the operating system is well designed for fast access to and from the network. For example, incorporating templates of the network packet header directly in the process descriptor may allow a Gigabit Working Group [Page 26] RFC 1077 November 1988 process to initiate communications with minimal overhead. Similarly, memory mapping can be used to eliminate copies between data arriving from the network and it being delivered to the applications. With a GN, an extra copy forced by the operating system may easily double the perceived transfer time for a packet between applications. Besides matching data transfer mechanisms, operating systems must be well-matched in security design to that supported by the host interface and network as well. Otherwise, all but the most trivial additional security actions by the operating system in common case communication can easily eliminate the performance benefits of the GN. For example, if the host has to do further encryption or decryption, the throughput is likely to be at least halved and the latency doubled. Research effort is required to further refine operating systems for the level of performance offered by the GN. This effort may well be best realized with coupling existing efforts in distributed systems with the GN activities, as opposed to starting new separate efforts. 3.4. Advanced Network Management Algorithms An important emphasis for research into network management should be on decentralized approaches. The ratio of propagation delay across the country to data rates in a GN appear to be too great to deal effectively with resource management centrally when traffic load is bursty and unstable (and if it is not, one might argue there is no problem). In addition, important principles of fault containment and minimal privilege for reliability and security suggest that a centralized management approach is infeasible. In particular, compromising the security of one portion of the network should not compromise the security of the whole network. Similarly, a failure or fault should affect at most a local region of the network. The challenge is clearly to provide decentralized management techniques that lead to good global behavior in the normal case and acceptable behavior in expected worst-case failures, traffic variations and security intrusions. 3.4.1. Control Flow vs. Data Flow Network operational communications can be separated into flow of user data and flow of management/control data. However, the user data must contain some amount of control data. One question that needs to Gigabit Working Group [Page 27] RFC 1077 November 1988 be explored in light of changes in communications and computing costs and performance is the trade-off between these two flows. An example of a potential approach is to use data units which contain predefined path indicators. The switch can perform a simple table look-up which maps the path indicator onto the preferred outbound link and transmits the packet immediately. There is a path set-up packet which fills in the appropriate tables. Path set-up occurs before the first data packet flows and then, while data is flowing, to improve the routes during the lifetime of the connection. This concept has been discussed in the Internet engineering group under the name of soft connections. We note that separating the data flow from the control flow in the GN has security and reliability advantages as well. We could encrypt most of the packet header to provide confidentiality within the GN and to limit the ability of intruders to perform traffic analysis. And, by separating the control flow, we can encrypt all the control exchanges between switches and the host front ends thereby offering confidentiality and integrity. No unauthorized entity will be able to alter or examine the control traffic. By employing a path set-up procedure, we can assure that the GN NFE-to-NFE path is functioning and also include user-specific requirements in the route. For example, we could request a certain bandwidth allocation and simplify the job of the switches in handling flow control. We could also set up backup paths in case the output link will be busy for so many microseconds that the packet cannot be stored until the link is freed. 3.4.2. Resource Management Algorithms Most current networks deliver one quality of service. X.25 networks deliver a reliable byte-stream. Most LANs deliver a best-effort unreliable service. There are few networks today that can support multiple types of service, and allocate their resources among them. Indeed, for many networks, such as best-effort unreliable service, there is little management of resources at all. The next generation of network will require a much more controlled allocation of resources. There will be a much wider range of desired types of service, with current services such as remote procedure call mixing with new services such as video streams. Unless these are separately recognized and controlled, there is little reason to believe that effective service can be delivered unless the network is very lightly loaded. Gigabit Working Group [Page 28] RFC 1077 November 1988 In order to support multiple types of service, two things must happen, both a change from current practice. First, the application must describe to the network what type of service is required. Second, the network must use this information to make resource allocation decisions. Both of these practices present difficulties. Past experience suggests that application code is not prepared to know or specify what service it needs. By custom, operating systems provide a virtual world, and the applications in this world are unaware of the relation between this and the reality of time and space. Resource requests must be in real terms. Allocation of resources in the network is difficult, because it requires that decisions be made in the network, but as network packet throughput increases, there is less time for decisions. The resolution of this latter conflict is to observe that decisions must be made on larger units than the unit of multiplexing such as the packet. This in turn implies that packets must be visible to the network as being part of a sequence, as opposed to the pure datagram model previously exploited. As suggested earlier in this report, research is required to support this more complex form of switch without compromising robustness. To permit the application to specify the service it needs, it will be necessary to propose some abstraction of service class. By clever design of this abstraction, it should be possible to allow the application to describe its needs effectively. For example, an application such as file transfer or mail has two modes of operation; bulk data transfer and remote procedure call. The application may not be able to predict when it will be in which mode, but if it just describes both of them, the system may be able to adapt by observing its current operation. Experimentation needs to be done to determine a suitable service specification interface. This experimentation could be done in the context of the current protocols, and could thus be undertaken at once. 3.4.3. Adaptive Protocols Network operating conditions can vary quickly and over a wide range. This is true of the current Internet, and is likely to affect the GN too. Protocols that can adapt to changing circumstances would provide more even and robust service than is currently possible. For example, when error rates increased, a protocol implementation might decide to use smaller packets, thus reducing the burden caused by Gigabit Working Group [Page 29] RFC 1077 November 1988 retransmissions. The environment in which a protocol operates can be described in terms of the service it is getting from the next lower layer. A protocol implementation can adapt to changes in that service by tuning its internal mechanisms (time-outs, retransmission strategies, etc.). Therefore, to design adaptive protocols, we must understand the interaction between protocol layers and the mechanisms used within them. There has been some work done in this area. For example, the SATNET measurement task force has looked at the interactions between the protocol used by the SIMP, IP, and TCP. What is needed is a more complete characterization of the interactions at various layer boundaries, and the development of appropriate protocol designs and mechanisms to provide for necessary adaptations and renegotiations. 3.4.4. Error Recovery Mechanisms Being large and complex, GNs will experience a variety of faults such as link or nodal failure, excessive buffer overflow due to faulty flow and congestion control, and partial failure of switching fabric. These failures, which also exist in today's networks, will have a stronger effect in GNs where a large amount of data will be "stored" in transit and, to expedite the switching, nodes will apply only minimal processing to the packets traversing them. In source routing, for example, a link failure may cause the loss of all packets sent until the source is notified about the change in topology. The longer is the delay in recovering from failures, the higher is the degradation in performance observed by the users. To minimize the effects of failures, GNs will need to employ error recovery mechanisms whereby the network detects failures and error conditions, reconfigures itself to adapt to the new network state, and notifies peripheral devices of the new configuration. Such protocols, which have to be developed, will respond quickly, will be decentralized or distributed to minimize the possibility of fatal failures, and will complement, rather than replicate, the error correction mechanisms of the end-to-end protocols, and the two must operate in coordinated manner. To this end, the peripheral devices will have to be knowledgeable about the intranet recovery mechanisms and interact continuously with them to minimize the effect on the connections they manage. Gigabit Working Group [Page 30] RFC 1077 November 1988 3.4.5. Flow Control As networks become faster, two related problems arise. First, existing flow control mechanisms such as windows do not work well, because the window must be opened to such an extent to achieve desired bandwidth that effective flow control cannot be achieved. Second, especially for long-haul networks, the larger number of bits in transit at one time becomes so large that most computer messages will fit into one window. This means that traditional congestion control schemes will cease to work well. What is needed is a combination of two approaches, both new. First, for messages that are small (most messages generated by computers today will be small, since they will fit into one round-trip time of future networks), open-loop controls on flow and congestion are needed. For longer messages (voice or video streams, for example), some explicit resource commitment will be required. 3.4.6. Latency Control and Real-Time Operations Currently, there are several distinct approaches to latency control. First, there are some networks which are physically short, more like multiprocessor buses. Applications in these networks are built assuming that delays will be short. Second, there are networks where the physical length is not constrained by the design and may differ by orders of magnitude, depending on the scope of the network. Most general purpose networks fall in this category. In these networks, one of two things happens. Either the application takes special steps to deal with variable latency, such as echo suppression in voice networks, or these applications are not supported. For most applications today, the latency in the network is not an obvious issue so long as the network is not overloaded (which leads to losses and long queues), because the protocol overhead masks the variation in the network latency. This balance will change. The latency due to the speed of light will obviously remain the same, but the overhead will drop (of necessity if we are to achieve high performance) which will leave speed of light and queueing as the most critical sources of delay. This conclusion implies that if queueing delay can be controlled, it will be possible to build networks with stable and controlled latency. If applications exist that require this class of service, Gigabit Working Group [Page 31] RFC 1077 November 1988 it can be supported. Either the network must be underloaded, so that queues do not develop at all, or a specific class of service must be supported in which resources are allocated to stabilize the delay. If this service is provided, it will still leave the application with delays that can vary by several orders of magnitude, depending on the physical size of the network. Research at the application level will be required to see how applications can be designed to cope with this variation. 3.4.7. High-Speed Internetworking and Administrational Domains Internetworking recognized that the value of communication services increases significantly with wider interconnection but ignored management and the role of administrations. As a consequence we see that: 1. The Internet is more or less unmanageable, as evidenced by performance, reliability, and security problems. 2. The Internet is being stressed by administrators that are building networks to match their organization rather than the geography. An example is a set of Ethernets at different company locations operating as a single Internet network but geographically dispersed and connected by satellite or leased lines. The next generation of internetworking must focus on administration and management. Internetworking must support cohesion within an administration and a healthy separation between administrations. To illustrate by analogy, the American and Soviet embassies in Mexico City are geographically closer to each other than to their respective home countries but further in administrational distance, including security, accounting, etc. The emerging revolution in WANs makes this issue that much more critical. The amount of communication to exchange the state of systems is bound to increase enormously. The potential cost of failures and security violations is frightening. A promising approach appears to be high-level gateways that guard between administrations and require negotiations to set up access paths between administrations. These paths are set up, and labeled with agreements on authorization, security, accounting, and possible resource limits. These administrative virtual circuits provide transparency to the physical and geographical interconnection, but need not support more than datagram packet delivery. One view is that of communication contracts with high-level gateways acting as Gigabit Working Group [Page 32] RFC 1077 November 1988 contract monitors at each end. The key is the focus on controlled interadministrational connectivity, not the conventional protocol concerns. Focus is required on developing an (inter)network management architecture and the specifics of high-level gateways. The structures of such gateways will have to take advantage of advances in multi-processor architectures to handle the processing load. Moreover, a key issue is being able to optimize communication between administrations once the contract is in place, but without losing control. Related is the issue of allowing high-speed interconnection within a single administration, although geographical dispersed. Another issue is fault-tolerance. High-level gateways contain state information whose loss typically disrupts communication. How does one minimize this problem? A key goal of these administrational gateways has to be failure containment: How to protect against external (to administration) problems and how to prevent local problems imposing liability on others. A particular area of concern is the self-organizing problems of large-scale systems, observed by Van Jacobson in the Internet. Gateways must serve to damp out oscillations and control wide load swings. Rate control appears to be a key area to investigate as a basis for buffer management and for congestion control, as well as to control offered load. Given the speed of new networks, and the sophistication of the gateways suggested above, another key area to investigate is the provision of high-speed network interface adaptors. 3.4.8. Policy-Based Algorithms Networks of today generally select routes based on minimizing some measure such as delay. However, in the real world, route selection will commonly be constrained at the global level by policy issues, such as access rights to resources and accounting and billing for usage. It is difficult for connectionless protocols such as Internet to deal with policy controls, because a lack of state in the gateway implies that a separate policy decision must be made for each packet in isolation. As networks get faster, the cost of this processing will be intolerable. One possible approach, discussed above, is to move to a more sophisticated model in which there is knowledge in the gateways of the ongoing flows. Alternatively, it may be possible to design gateways that simply cache recent policy evaluations and apply Gigabit Working Group [Page 33] RFC 1077 November 1988 them to successive packets. Routing based on policy is particularly difficult because a route must be globally consistent to be useful; otherwise it may loop. This implies that the every policy decision must be propagated globally. Since there can be expected to be a large number of policies, this global passing of information might easily lead to an information explosion. There are at least two solutions. One is to restrict the possible classes of policy. Another is to use some form of source route, so that the route consistent with some set of policies is computed at one point only, and then attached to the packet. Both of these approaches have problems. A two-pronged research program is needed, in which mechanisms are proposed, and at the same time the needed policies are defined. The same trade-off can be seen for accounting and billing. A single accounting metric, such as "bytes times distance", could be proposed. This might be somewhat simple to implement, but would not permit the definition of individual billing policies, as is now done in the parts of the telephone system. The current connectionless transport architectures such as TCP/IP or the connectionless ISO configuration using TP4 do not have good tools for accounting for traffic, or for restricting traffic from certain resources. Building these tools is difficult in a connectionless environment, because an accounting or control facility must deal with each packet in isolation, which implies a significant processing burden as part of packet forwarding. This burden is an increasing problem as switches are expected to operate faster. The lack of these tools is proving a significant problem for network design. Not only are accounting and control needed to support management requirements, they are needed as a building block to support enforcement of such things as multiple qualities of service, as discussed above. Network accounting is generally considered to be simply a step that leads to billing, and thus is often evaluated in terms of how simple or difficult it will be to implement. Yet an accounting and billing procedure is a mechanism for implementing a policy considered to be desirable for reasons beyond the scope of accounting per se. For example, a policy might be established either to encourage or discourage network use, while fully recovering operational cost. A policy of encouraging use could be implemented by a relatively high monthly attachment charge and a relatively low per-packet charge. A policy of discouraging use could be implemented by a low monthly charge and a high per-packet charge. Gigabit Working Group [Page 34] RFC 1077 November 1988 Network administrators have a relatively small number of variables with which to implement policy objectives. Nevertheless, these variables can be combined in a number of innovative ways. Some of the possibilities include: 1. Classes of users (e.g., large or small institutions, for- profit or non-profit). 2. Classes of service. 3. Time varying (e.g., peak and off-peak). 4. Volume (e.g., volume discounts, or volume surcharges). 5. Access charges (e.g., per port, or port * [bandwidth of port]). 6. Distance (e.g., circuit-miles, airline miles, number of hops). Generally, an accounting procedure can be developed to support voluntary user cooperation with almost any single policy objective. Difficulties most often arise when there are multiple competing policy objectives, or when there is no clear policy at all. Another aspect of accounting and billing procedures which must be carefully considered is the cost of accumulating and processing the data on which billing is based. Of particular concern is collection of detailed data on a per-packet basis. As network circuit data rates increase, the number of instructions which must be executed on a per-packet basis can become the limiting factor in system throughput. Thus, it may be appropriate to prefer accounting and billing policies and procedures which minimize the difficulty of collecting data, even if this approach requires a compromise of other objectives. Similarly, node memory required for data collection and any network bandwidth required for transmission of the data to administrative headquarters are factors which must be traded off against the need to process user packets. 3.4.9. Priority and Preemption The GN should support multiple levels of priority for traffic and the preemption of network resources for higher priority use. Network control traffic should be given the highest priority to ensure that it is able to pass through the network unimpeded by congestion caused by user-level traffic. There may be additional military uses for multiple levels of priority which correspond to rank or level of Gigabit Working Group [Page 35] RFC 1077 November 1988 importance of a user or the mission criticality of some particular data. The use of and existence of priority levels may be different for different types of traffic. For example, datagram traffic may not have multiple priority levels. Because the network's transmission speed is so high and traffic bursts may be short, it may not make sense to do any processing in the switches to deal with different priority levels. Priority will be more important for flow- (or soft-connection-) oriented data or hard connections in terms of permitting higher priority connections to be set up ahead of lower priority connections. Preemption will permit requests for high priority connections to reclaim network resources currently in use by lower priority traffic. Networks such as the Wideband Satellite Network, which supports datagram and stream traffic, implement four priority levels for traffic with the highest reserved for network control functions and the other three for user traffic. The Wideband Network supports preemption of lower priority stream allocations by higher priority requests. An important component of the use of priority and preemption is the ability to notify users when requests for service have been denied, or allocations have been modified or disrupted. Such mechanisms have been implemented in the Wideband Network for streams and dynamic multicast groups. Priority and preemption mechanisms for a GN will have to be implemented in an extremely simple way so that they can take effect very quickly. It is likely that they will have to built into the hardware of the switch fabric. 3.5. User and Network Services As discussed in Section 2 above, there will need to be certain services provided as part of the network operation to the users (people) themselves and to the machines that connect to the network. These services, which include such capabilities as white and yellow pages (allowing users to determine what the appropriate network identification is for other users and for network-available computing resources) and distributed fault identification and isolation, are needed in current networks and will continue to be required in the networks of the future. The speed of the GN will serve to accentuate this requirement, but at the same time will allow for new architectures to be put in place for such services. For example, Ethernet speeds in the local environment have allowed for more usable services to be provided. Gigabit Working Group [Page 36] RFC 1077 November 1988 3.5.1. Impact of High Bandwidth One issue that will need to be addressed is the impact on the user of such high-bandwidth capabilities. Users are already becoming saturated by information in the modern information-rich environment. (Many of us receive more than 50 electronic mail messages each day, each requiring some degree of human attention.) Methods will be needed to allow users to cope with this ever-expanding access to data, or we will run the risk of users turning back to the relative peace and quiet of the isolated office. 3.5.2. Distributed Network Directory A distributed network directory can support the user-level directory services and the lower-level name-to-address mapping services described elsewhere in this report. It can also support distributed systems and network management facilities by storing additional information about named objects. For example, the network directory might store node configurations or security levels. Distributing the directory eases and decentralizes the administrative burdens and provides a more robust and survivable implementation. One approach toward implementing a distributed network directory would be to base it upon the CCITT X.500/ISO DIS 9594 standard. This avoids starting from ground zero and has the advantage of facilitating interoperability with other communications networks. However, research and development will be required even if this path is chosen. One area in which research and development are required is in the services supplied by the distributed network directory. The X.500 standard is very general and powerful, but so far specific provisions have been made only for storing information about network users and applications. As mentioned elsewhere, multilevel security is not addressed by X.500, and the approach taken toward authentication must be carefully considered in view of DoD requirements. Also, X.500 assumes that administration of the directory will be done locally and without the need for standardization; this may not be true of GN or the larger national research network. The model and algorithms used by a distributed network directory constitute a second area of research. The model specified by X.500 must be extended into a framework that provides the necessary flexibility in terms of services, responsiveness, data management Gigabit Working Group [Page 37] RFC 1077 November 1988 policies, and protocol layer utilization. Furthermore, the internal algorithms and mechanisms of X.500 must be extended in a number of areas; for example, to support redundancy of the X.500 database, internal consistency checking, fuller sharing of information about the distribution of data, and defined access-control mechanisms. 4. Avenues of Approach Ongoing research and commercial activities provide an opportunity for more rapidly attacking some of the above research issues. At the same time, there needs to be attention paid to the overall technical approach used to allow multiple potential solutions to be explored and allow issues to be attacked in parallel. 4.1. Small Prototype vs. Nationwide Network The central question is how far to jump, and how far can the current approaches get. That is, how far will connectionless network service get us, how far will packet switching get us, and how far do we want to go. If our goal is a Gbit/s net, then that is what we should build. Building a 100 Mbit/s network to achieve a GN is analogous to climbing a tree to get to the moon. It may get you closer, but it will never get you there. There are currently some network designs which can serve as the basis for a GN prototype. The next step is some work by experts in photonics and possibly high-speed electronics to explore ease of implementation. Developing a prototype 3-5 node network at a Gbit/s data rate is realistic at this point and would demonstrate wide-area (40 km or more) Gbit/s networking. DARPA should consider installing a Gbit/s cross-country set of connected links analogous to the NSF backbone in 2 years. A Gbit/s link between the east and west coasts would open up a whole new generation of (C3I), distributed computing, and parallel computing research possibilities and would reestablish DARPA as the premier network research funding agency in the country. This will require getting "dark" fiber from one or more of the common carriers and some collaboration with these organizations on repeaters, etc. With this collaboration, the time to a commercial network in the Gbit/s range would be substantially reduced, and the resulting nationwide GN would give the United States an enormous technical and economic advantage over countries without it. Gigabit Working Group [Page 38] RFC 1077 November 1988 Demonstrating a high-bandwidth WAN is not enough, however. As one can see from the many research issues identified above, it will be necessary to pursue via study and experiment the issues involved in interconnecting high-bandwidth networks into a high-bandwidth internet. These experiments can be done through use of a new generation of internet, even if it requires starting at lower speeds (e.g., T1 through 100 Mbit/s). Appropriate care must be given, however, to assure that the capabilities that are demonstrated are applicable to the higher bandwidths (Gbit/s) as they emerge. 4.2. Need for Parallel Efforts/Approaches Parallel efforts will therefore be required for two major reasons. First is the need to pursue alternative approaches (e.g., different strategies for high-bandwidth switching, different addressing techniques, etc). This is the case for most research programs, but it is made more difficult here by the costs of prototyping. Thus, it is necessary that appropriate review take place in the decisions as to which efforts are supported through prototyping. In addition, it will be necessary to pursue the different aspects of the program in parallel. It will not be possible to wait until the high-bandwidth network is available before starting on prototyping the high-bandwidth internet. Thus, a phased and evolutionary approach will be needed. 4.3. Collaboration with Common Carriers Computer communication networks in the United States today practically ignore the STN (the Switched Telephone Network), except for buying raw bandwidth through it. However, advances in network performance are based on improvements in the underlying communication media, including satellite communication, fiber optics, and photonic switching. In the past we used "their" transmission under "our" switching. An alternative approach is to utilize the common-carrier switching capabilities as an integral part of the networking architecture. We must take an objective scientific and economic look and reevaluate this question. Another place for cooperation with the common carriers is in the area of network addressing. Their addressing scheme ("numbering plan") has a few advantages such as proven service to 300 million users [4]. Gigabit Working Group [Page 39] RFC 1077 November 1988 On the other hand, the common carriers have far fewer administrative domains (area codes) than the current plethora of locally administered local area networks in the internet system. It is likely that future networks will eventually be managed and operated by commercial communications providers. A way to maximize technology transfer from the research discussed here to the marketplace is to involve the potential carriers from the start. However, it is not clear that the goals of commercial communications providers, who have typically been most interested in meeting the needs of 90+ percent of the user base, will be compatible with the goals of the research described here. Thus, while we recommend that the research program involve an appropriate amalgam of academia and industry, paying particular attention to involvement of the potential system developers and operators, we also caution that the specific and unique goals of the DARPA program must be retained. 4.4. Technology Transfer As we said above, it is our belief that future networks will ultimately be managed and operated by commercial communications providers. (Note that this may not be the common carriers as we know them today, but may be value-added networks using common carrier facilities.) The way to assure technology transfer, in our belief, is to involve the potential system developers from the start. We therefore believe that the research program would benefit from an appropriate amalgam of university and industry, with provision for close involvement of the potential system developers and operators. 4.5. Standards The Internet program was a tremendous success in influencing national and international standards. While there were changes to the protocols, the underlying technology and approaches used by CCITT and ISO in the standardization of packet-switched networks clearly had its roots in the DARPA internet. Nevertheless, this has had some negative impact on the research program, as the evolution of the standards led to pressure to adopt them in the research environment. Thus, it appears that there is a "catch-22" here. It is desirable for the technology base developed in the research program to have maximal impact on the standards activities. This is expedited by doing the research in the context of the standards environment. However, standards by their very nature will always lag behind the Gigabit Working Group [Page 40] RFC 1077 November 1988 research environment. The only reasonable approach, therefore, appears to be an occasional "checkpointing" of the research environment, where the required conversions take place to allow a new plateau of standards to be used for future evolution and research. A good example is conducting future research in mail using X.400 and X.500 where possible. 5. Conclusions We hope that this document has provided a useful compendium of those research issues critical to achieving the FCCSET phase III recommendations. These problems interact in a complex way. If the only goal of a new network architecture was high speed, reasonable solutions would not be difficult to propose. But if one must achieve higher speeds while supporting multiple services, and at the same time support the establishment of these services across administrative boundaries, so that policy concerns (e.g., access control) must be enforced, the interactions become complex. Gigabit Working Group [Page 41] RFC 1077 November 1988 APPENDIX A. Current R and D Activities In this appendix, we provide pointers to some ongoing activities in the research and development community of which the group was aware relevant to the goal of achieving the GN. In some cases, a short abstract is provided of the research. Neither the order of the listing (which is random) nor the amount of detail provided is meant to indicate in any way the significance of the activity. We hope that this set of pointers will be useful to anyone who chooses to pursue the research issues discussed in this report. 1. Grumman (at Bethpage) is working on a three-year DARPA contract, started in January 1988 to develop a 1.6 Gbit/s LAN, for use on a plane or ship, or as a "building block". It is really raw transport capacity running on two fibers in a token-ring like mode. First milestone (after one year?) is to be a 100 Mbit/s demonstration. 2. BBN Laboratories, as part of its current three-year DARPA Network-Oriented Systems contract, has proposed design concepts for a 10-100 Gbit/s wide area network. Work under this effort will include wavelength division multiplexing, photonic switching, self-routing packets, and protocol design. 3. Cheriton (Stanford) research on Blazenet, a high-bandwidth network using photonic switching. 4. Acampora (Bell Labs) research on the use of wavelength division multiplexing for building a shared optical network. 5. Yeh is reserching a VLSI approach to building high-bandwidth parallel processing packet switch. 6. Bell Labs is working on a Metropolitan Area Network called "Manhattan Street Net." This work, under Dr. Maxemchuck, is similar to Blazenet. It is in the prototype stage for a small number of street intersections; ultimately it is meant to be city-wide. Like Blazenet, is uses photonic switching 2 x 2 lithium niobate block switches. 7. Ultra Network Technologies is a Silicon Valley company which has a (prototype) Gbit/s fiber link which connects backplanes. This is based on the ISO-TP4 transport protocol. 8. Jonathan Turner, Washington University, is working on a Batcher-Banyan Multicast Net, based on the "SONET" concept, Gigabit Working Group [Page 42] RFC 1077 November 1988 which provides 150 Mbit/s per pipe. 9. David Sincowskie, Bellcore, is working with Batcher-Banyan design and has working 32x32 switches. 10. Stratacom has a commercial product which is really a T1 voice switch implemented internally by a packet switch, where the packet is 192 bits (T1 frame). This switch can pass 10,000 packets per second. 11. Stanford NAB provides 30-50 Mbit/s throughput on 100 Mbit/s connection using Versatile Message Transaction Protocol (VMTP) [see RFC 1045] 12. The December issue of IEEE Journal on Selected Areas in Communications, provides much detail concerning interconnects. 13. Ultranet Technology has a 480 Mbit/s connection using modified ISO TP4. 14. At MIT, Dave Clark has an architecture proposal of interest. 15. At CMU, the work of Eric Cooper is relevant. 16. At Protocol Engines, Inc., Greg Chesson is working on an XTP- based system. 17. Larry Landweber at Wisconsin University is doing relevant work. 18. Honeywell is doing relevant work for NASA. 19. Kung at CMU is working on a system called "Nectar" based on a STARLAN on fiber connecting dissimilar processors. 20. Burroughs (now Unisys) has some relevant work within the IEEE 802.6 committee. 21. Bellcore work in "Switched Multimedia Datanet Service" (SMDS) is relevant (see paper supplied by Dave Clark). 22. FDDI-2, a scheme for making TDMA channel allocations at 200 Mbit/s. 23. NRI, Kahn-Farber Proposal to NSF, is a paper design for high- bandwidth network. 24. Barry Goldstein work, IBM-Yorktown. Gigabit Working Group [Page 43] RFC 1077 November 1988 25. Bell Labs S-Net, 1280 Mbit/s prototype. 26. Fiber-LAN owned by Bell South and SECOR, a pre-prototype 575 Mbit/s Metro Area Net. 27. Bellcore chip implementation of FASTNET (1.2 Gbit/s). 28. Scientific Computer Systems, San Diego, 1.4 Gbit/s prototype. 29. BBN Monarch Switch, Space Division pre-prototype, chips being fabricated, 64 Mbit/s per path. 30. Proteon, 80 Mbit/s token ring. 31. Toronto University, 150 Mbit/s "tree"--- really a LAN. 32. NSC Hyperchannel II, reputedly available at 250 Mbit/s. 33. Tobagi at Stanford working on EXPRESSNET; not commercially available. 34. Columbia MAGNET-- 150 Mbit/s. 35. Versatile Message Transaction Protocol (VMTP). 36. ST integrated with IP. 37. XTP (Chesson). 38. Stanford Transport Gateway. 39. X.25/X.75. 40. Work of the Internet Activities Board. Gigabit Working Group [Page 44] RFC 1077 November 1988 B. Gigabit Working Group Members Member Affiliation Gordon Bell Ardent Computers Steve Blumenthal BBN Laboratories Vint Cerf Corporation for National Research Initiatives David Cheriton Stanford University David Clark Massachusetts Institute of Technology Barry Leiner (Chairman) Research Institute for Advanced Computer Science Robert Lyons Defense Communication Agency Richard Metzger Rome Air Development Center David Mills University of Delaware Kevin Mills National Bureau of Standards Chris Perry MITRE Jon Postel USC Information Sciences Institute Nachum Shacham SRI International Fouad Tobagi Stanford University Gigabit Working Group [Page 45] RFC 1077 November 1988 End Notes [1] Workshop on Computer Networks, 17-19 February 1987, San Diego, CA. [2] "A Report to the Congress on Computer Networks to Support Research in the United States: A Study of Critical Problems and Future Options", White House Office of Scientific and Technical Policy (OSTP), November 1987. [3] We distinguish in the report between development of a backbone network providing gigabit capacity, the GB, and an interconnected set of high-speed networks providing high- bandwidth service to the user, the Gigabit Network (GN). [4] Incidentally, they already manage to serve 150 million subscribers in an 11-digit address-space (about 1:600 ratio). We have a 9.6-digit address-space and are running into troubles with much less than 100,000 users (less than 1:30,000 ratio). Gigabit Working Group [Page 46]