farber@uunet.UU.NET (David J. Farber) (02/14/89)
In a November 1987 paper by David Farber titled Some Thoughts on the Impact of Very High Speed Networking on Processor Interfaces inspired by the research successes of the Memnet effort carried out by Gary Delp as part of his Phd, it was suggested that an approach to the design of very high speed protocols may lie in exploring the idea of treating the local memories of the participating computers as partially shared memory spaces. That is, at least a portion of the memory addressing space in each of the collaborating computers collectively form a distributed common memory. This memory is accessible by the networked processors in a manner which, in effect, forms a rather decentralized shared-memory, multiprocessor environment. Conceptually, "communications" is merely interprocess communication in such a multi-processor environment. The 1988 National Academy Report Toward a National Research Network, which said "it is interesting to view this networking environment (ed: gigabit level) as similar to a multiprocessor system" thus supporting the approach. The issue of the very large memory address spaces implied by such a national distributed system clearly is significant. It is worth noting that, in almost every modern computer architecture, such a huge memory address space exists. Simplistically, one can consider that we have created yet another tier of a segmented address structure. The new tier addresses the segmentation of memory among hosts rather than within a host. Based on early examination of the problem, we believe that the addressing structure found in contemporary machines can be straight-forwardly extended to this model. In a real sense, dealing with foreign memory is very similar to dealing with externally-paged memory. A major problem always facing the designers of distributed systems and networking systems, is the heterogeneous computer problem. Namely, different machines have different word lengths, different data representations, etc. Research in solving this problem is as old as inter-computer communications. It is not a simple problem. However, it does bear certain similarities to the software engineering problems found in the software reusability field. Initial Thoughts on the Architecture of an Object Based Distributed System by Razi and Farber 1983, suggested such a direction. The current rebirth of the object-oriented programming field, and its growing success in dealing with software reusability, strongly suggests that solutions for the heterogeneous computer problem may grow out of a study of this work. Where are we, and what is the future? We suggest that the ultra-high-speed network of the future can be considered to be a multiprocessor, shared-memory computer with a redundant, fault-tolerant bus system. We are in the process of designing such an architecture, based on Memnet experiences, as part of our CapNet research effort. A forthcoming paper CapNet - an Experiment in Ultra High Speed Networking by Tam and Farber addresses CapNet in more detail and explores the hardware interface designs needed to support the higher level view. Briefly, the idea of CapNet is that each host donates some of its memory as a part of the system-wide shared memory. Thus interprocess communication is done by manipulating the shared data. Data that resides in the remote node can be viewed as a page being swapped out. A memory request from a host is simply wrapped inside the link layer information and broadcasted to the other hosts. Consequently, the remote systems appear to the local host as a memory with large latency. With the advance of the very- high-speed network technology, the transmission time between two hosts is comparable to the amount of time it would take to service a page fault using a local disk, making it feasible to implement a physically distributed shared memory across the network (at least in terms of speed.)