darrell@midgard.ucsc.edu (Darrell Long) (01/30/89)
I'd like to get some opinions on this: How much sense does a distributed version of Multics make? That is, a memory mapped file system with shared segments across machines? Regards, DL
darrell@midgard.ucsc.edu (Darrell Long) (01/31/89)
Here is a summary of the responses to my "question for the week." Thanks to everyone who responded. DL In article <6154@saturn.ucsc.edu> you write: >I'd like to get some opinions on this: How much sense does a distributed >version of Multics make? %A W. A. Montgomery %T Measurements of Sharing in Multics %J PROC 6th SOSP (in Operating Systems Review) %V 11 %N 5 %D NOV 1977 %P 85-90 %X little empirical use (by users) of Multics segment sharing %X subset of sharing probably better for distributed systems Would it be better to ask: what subset of Multics makes sense for distributed systems? -- Paul M. Aoki, aoki@postgres.Berkeley.EDU In article <6154@saturn.ucsc.edu> darrell@midgard.ucsc.edu (DL) writes: >[Memory-mapped file system with shared segments across machines] See several papers by Kai Li. Only one I have handy: Kai Li Paul Hudak Memory Coherence in Shared Virtual Memory Systems. Proceedings 1986 ACM Conference on Principles of Distributed Systems pp. 229-239 -- Pardo, pardo@cs.washington.edu In article <6154@saturn.ucsc.edu> you write: # #I'd like to get some opinions on this: How much sense does a distributed #version of Multics make? That is, a memory mapped file system with shared #segments across machines? As an abstract concept, it makes sense. It would probably be *HELL* to implement, though. Many of Multics' ideas still make sense; memory- mapped files does. The biggest problem is sharing the actual files; in Multics, all processes that mapped in a segment used the same physical memory. In a distributed system, that would be a very bad idea, since the physical memory would likely be on another node and thus be very costly to access. So, we could have a Multics-type OS on each machine, and have files available to all, but that isn't what you were looking for, is it? replies welcomed ethan -- ethan miller, bandersnatch@ernie.berkeley.edu In article <6154@saturn.ucsc.edu> darrell@midgard.ucsc.edu (DL) writes: >I'd like to get some opinions on this: How much sense does a distributed >version of Multics make? That is, a memory mapped file system with shared >segments across machines? I hope it makes a lot of sense; I am proposing just such a scheme for my doctoral research. Specifically, a virtual storage memory management scheme that is very much like that of Multics (and therefore NOS/VS as well), but with extension to support segments located anywhere in the network. When a reference to a non-local segment is made, a special lightweight transport protocol fetches a copy across the network for local access. If you're interested in more details, send me e-mail. -- James Sterbenz, jps@wucs1.wustl.edu
brad@polaris.ucsc.edu (Brad Smith) (02/02/89)
In an article from the latest issue of Unix Today discussing a Honywell/Univ of Calgary project to create software to aid in the migration of Multics based software to Unix they mention that "...a MULTICS system is capable of tying together between one and six mainframe-level processors...". Does anyone know if this is in the "distributed Multics" sense? Brad Smith
yak%dove@gatech.edu (Yousef A. Khalidi) (02/04/89)
I have been working on memory management support for operating systems with
my advisor, Kishore Ramachandran, for the last year and half. Specifically,
we are looking at memory management support for the programming model of
Clouds/Ra, at each node in the distributed system, as well as network-wide
memory management. Distributed shared memory (DSM) is the tool used for
network-wide memory management, and it fits nicely with the Ra kernel. The
Ra kernel have segment manipulation primitives that borrow ideas from
Multics. Using DSM, any segment in the network may be accessed regardless of
its location. We use the mechanisms to implement remote object invocation (as
an alternative for RPC) by paging-in (portions of) the required segments and
performing the invocation locally. (The system also provides RPC and the
choice of which mechanism to use is left for the programmer/compiler/OS.)
The DSM mechanisms are also used for all paging across the network, and to
implement process and object migration. We combine locking with memory
coherency, and address the issue of synchronization. DSM mechanisms provide
semaphore operations that can be performed at any node in the system (by
moving the segment containing the semaphore to that node). I have several
tech reports that describe the design and implementation of DSM and Ra if you
are interested.
Yousef
--
Yousef A. Khalidi --- The Clouds/Ra Project
School of Information & Computer Science, Georgia Tech, Atlanta GA 30332
Internet: yak@gatech.edu
uucp: ...!{akgua,allegra,hplabs,ihnp4,linus,seismo,ulysses}!gatech!yakkrakowia@uunet.UU.NET (Sacha Krakowiak) (02/04/89)
In article <6154@saturn.ucsc.edu> darrell@midgard.ucsc.edu (Darrell Long) writes: > >I'd like to get some opinions on this: How much sense does a distributed >version of Multics make? That is, a memory mapped file system with shared >segments across machines? This is a very good idea. There is at least one distributed OS that more or less implements it : Apollo Domain. See description in P.J.Leach et al., The architecture of a distributed integrated local network, IEEE Journal on Selected areas in Communication, nov.1983,pp.842-856. -- ______________________________________________________________________________ Sacha Krakowiak, Lab. Genie Informatique | krakowiak@imag.imag.fr or IMAG,BP53X, 38041 Grenoble Cedex, France | {uunet.uu.net|mcvax}!imag!krakowiak ------------------------------------------------------------------------------
bga@shamash.cdc.com (Bruce Albrecht) (02/05/89)
CDC's NOS/VE is a Multics derivative, with a memory mapped file system with shared segments, and segments may be shared by multiple machines. The only catch is that currently the segment may be shared for read access by all machines, or have exclusive access by a single machine for write access. Theoretically, the shared segment could be available for concurrent write access (or non-exclusive access for write) on more than one machine, but it would be a real performance killer. CDC's NOS/VE has a file server, which is a process that communicates with the file servers on the other machines. When a shared segment is accessed, the file server sends requests for reads to the machine on which the segment actually resides (i.e., the machine to which the disk is connected.) If concurrent write accesses were allowed, the file servers would have to be responsible for invalidating pages on other mainframes, just as cache is invalidated on a multiple CPU machine. As long as there is some mechanism provided by the file servers to emulate it, even test_and_set instructions could be done across mainframes. I think the multi-mainframe file server for NOS/VE has been available for a year or so. We don't use it for all of our development systems, though, because our development systems tend to be running slightly different variants, and they only run it when two mainframes are scheduled to be in sync, OS-wise. In a few weeks, though, our communications development organization will be migrating to 2 dual CPU 860's, running with the multi-mainframe file server. I don't believe they support load-leveling between mainframes, but I think it's a natural extension of the multi-mainframe shared segments.
horn@uunet.UU.NET (Chris Horn) (02/10/89)
We also have been doing work on distributed shared memory for about the last 18 months. Our basic approach is that when an application requires access to a "segment", it can either be faulted in locally (possibly from a remote store, ie across the net) or the process can "diffuse" to use the segment at a remote site. When synchronous remote execution occurs, the same thread of control is involved at both the calling and called site - ie threads of control may be distributed. There are primitives to for example create, suspend, resume, kill such threads, independently of where current execution site is. By default, there is only at most one updateable image of a segment mapped into virtual memory anywhere in the network. Threads which simultaneously access this segment may therefore be diffused to reach this common site. Segments also are persistent by default - ie they behave as memory mapped updateable files. Defaults can be over-ruled to eg disable persistence (and so get a segment which behaves like a UNIX data/bss region), or support copy-on-write (thus disabling sharing updates in virtual memory, and useful for eg UNIX forking). Whether diffusion or remote fetch is used when a segment is first touched is under programmer/compiler/OS policy control. The work is under the auspices of the Esprit Comandos project, and our prototype was demonstrated on NS32332s at the last Esprit conference in Brussels (nov 88). We are currently improving and gaining experience with the prototype, and are porting to Vax. Our colleagues at IMAG grenoble have prototyped aspects of the system as a series of libraries on top of UNIX SysV (with bsd extensions) without kernel changes; our colleages at INESC lisbon have prototyped on PC/AT compatibles. Chris Horn --------- horn@cs.tcd.ie Dept Computer Science, Trinity College, IRL-Dublin 2
jps@wucs1.wustl.edu (James Sterbenz) (02/11/89)
In article 6238 raspail!bga@shamash.cdc.com (Bruce Albrecht) writes: >CDC's NOS/VE is a Multics derivative... I'm somewhat familiar with NOS/VE, and am very glad that with the demise of Honeywell supported Multics, the important ideas live on in a commercial product (there are other Multics derivatives, but NOS/VE seems to be the closest). >CDC's NOS/VE has a file server, which is a process that communicates with >the file servers on the other machines. When a shared segment is accessed, >the file server sends requests for reads to the machine on which the segment >actually resides (i.e., the machine to which the disk is connected.) I'd be interested in more information on this. I know nothing about CDC networking. My research involves virtual memory management across the network. I use a Multics-like base, but am also very concerned with the host hardware architecture and transport protocol design, which are optimised to work efficiently with the memory management. How does CDC integrate NOS/VE memory management with network hardware and protocols? Is there anything published on this? (The CDC manuals I have do not discuss the remote file access or any aspect of communications). Is a CDC NOS/VE or Cyber 800 communications type person on the net that could enlighten me? -- James Sterbenz Computer and Communications Research Center Washington University in St. Louis 314-726-4203 INTERNET: jps@wucs1.wustl.edu UUCP: wucs1!jps@uunet.uu.net