jack@citcom.UUCP (Jack Waugh) (11/15/87)
I use the "RAM:" disk driver on my Amiga computer. I suspect everyone does (on their Amiga). If you move the commands you use frequently into RAM, they will load faster. When you have moved to a phase of work where the files in RAM are no longer frequently used, they are a waste of fast memory (as opposed to floppy, not as opposed to chip memory). So you get rid of your cached files and cache the ones you are using now. What you have is a system that depends on the *user* to decide what permanent objects should be cached in fast memory and what should not, moment to moment. This sucks. The computer should automatically perform this function. I. e., minimal virtual memory. A user could ask for more than minimal virtual memory. Now, the only permanent objects on the Amiga are files and directories. A file is a growable sequence of mutable bytes. A directory is a function from names to things that are each a file or directory, with the ability to add and delete pairs. If the programmer wants to have data objects that don't fit that paradigm, they cannot be permanent and must take up the limited fast memory resource. I would like a general mechanism, supported in some combination of hardware and software, to support PERMANENT objects that could point to each other arbitrarily. In other words, a Lisp or Smalltalk-type memory that is permanent (as in Smalltalk). The collection of objects should be growable to span more than one floppy. Caching of most recently used objects should be in RAM. If you have a hard disk, it would come next. Next most recently used stuff would be on the floppy that's usually in the drive, and the very least recently used objects would live on other floppies. If the memory system also organized the keeping of enough redundant copies of everything that if any single disk got trashed, the user would only loose at most 10 min. work, the user would be releived from doing backups manually, too. There is no reason that task can't be taken care of algorithmically (except of course loading the floppies; but that's not the distracting part). Such a system would make it very easy for the application programmer to build useful things such as calendar programs, phone lists, document editors (word processors), compilers, shells, etc. and not have the application programming process slowed down by having to do the memory management design over and over. An issue with implementing such a system on the Amiga is native code management. A piece of native code that is to be executed must be in primary memory (RAM). It must also be somewhat relocated; the Amiga has a scatter loader that does this. A good virtual memory system would unload such code when it is no longer in use and the memory is more needed for something else. Can the author of the referenced article opposing virtual memory explain how to do this with a safe language and compiler?
dillon@CORY.BERKELEY.EDU (Matt Dillon) (11/15/87)
>memory). So you get rid of your cached files and cache the ones you >are using now. What you have is a system that depends on the *user* to >decide what permanent objects should be cached in fast memory and what >should not, moment to moment. This sucks. The computer should >automatically perform this function. I. e., minimal virtual memory. If your permanent files were originally on floppy, and the VM system decides to remove some of them from RAM:, then what happens when you request a file that has been removed? You get a disk requestor. Yucc. Back to square one. Also, you cannot use this VM file system to keep, say, source files which you are constantly updating in RAM: because the VM does not update the floppy originally containing the source (what happens when the system crashes?). If it did, it means the original floppy must be in the drive all the time anyway. So this VM of yours is actually a glorified disk cache. Thank you, I'll stick with Facc (a disk cache utilitiy). >If the memory system also organized the keeping of enough redundant >copies of everything that if any single disk got trashed, the user >would only loose at most 10 min. work, the user would be releived from >doing backups manually, too. There is no reason that task can't be >taken care of algorithmically >.. Not in a microcomputer you don't, unless you enjoy running out of memory and disk space. >Such a system would make it very easy for the application programmer to >build useful things such as calendar programs, phone lists, document >editors (word processors), compilers, shells, etc. and not have the >application programming process slowed down by having to do the memory >management design over and over. Unless... you don't do memory management design at all. Really now, memory management in a calendar program? Have you ever heard of disk based databases? Anything using so much memory as to need a software VM system will already be so large that adding the VM in software will cost virtually nothing. >An issue with implementing such a system on the Amiga is native code >management. A piece of native code that is to be executed must be in >primary memory (RAM). It must also be somewhat relocated; the Amiga >has a scatter loader that does this. A good virtual memory system >would unload such code when it is no longer in use and the memory is >more needed for something else. Can the author of the referenced >article opposing virtual memory explain how to do this with a safe >language and compiler? Unload it to where? On your hard drive? I would want a dedicated HD for something like that. What do you do about shared memory? You have Intuition allocating Window structures accessed by N other tasks, what happens when the Window structure gets relocated? I don't know of a single program which doesn't depend on address uniqueness over all tasks in the system. Even DOS depends on it. UNIX doesn't because tasks don't have access to each other's memory spaces except under tightly controlled conditions. But as far as the Amiga goes, any swapping would have to go back into the same memory block as it left. The purpose of scatter loading is to allow large programs to be loaded even when memory is badly fragmented, and doesn't apply to a VM system. I think a critical point here is being missed. The question is simply: Why do I want virtual memory? The answer is always (A) memory protection, and (B) utilization of a virtual address space larger than physical memory. (B) requires a dedicated hard disk for a clean implementation on a microcomputer. So we have (A): memory protection. On the Amiga, this is an extremely difficult task to accomplished because both the OS and most application programs depend on the ability to read AND WRITE memory 'owned' by other applications. The issue of private vs public memory is nice in concept, but useless in reality because most objects need to be public anyway. Falling back onto (B). Utilization of a virtual address space larger than main memory is easily supported by the Amiga, but you need a 68020 to do it (the 68000/68010 has only a 16Mbyte address space and most of that is already assigned.) What one would have to have on the Amiga (68000) would be a couple of virtual object allocation system calls... allocate, lock, unlock, and deallocate. You allocate an object of X bytes and get a key for that object. The object may or may not be in memory. By locking the object you get an actual pointer to the object and may read/write it. By unlocking the object you inform the OS that it can move the object out of memory if memory gets tight. You need to lock the object before you can access it again. With VM being tightly controlled in this way, objects can be shuffled around in memory and need not occupy one absolute location... but ONLY while the object is not locked and thus is not in use. Such a VM system would be distinguished from real memory. That is, unlike a real paged VM system, each task has full control over when an object is or is not in physical memory, or can use the normal memory allocation routines to get permanent memory (aka AllocMem()) with low overhead. -MDari
hsgj@batcomputer.tn.cornell.edu (Dan Green) (11/16/87)
In article <112@citcom.UUCP> jack@citcom.UUCP (Jack Waugh) writes: >I use the "RAM:" disk driver on my Amiga computer. I suspect everyone >does (on their Amiga). If you move the commands you use frequently >into RAM, they will load faster. When you have moved to a phase of >work where the files in RAM are no longer frequently used, they are a >waste of fast memory (as opposed to floppy, not as opposed to chip >memory). So you get rid of your cached files and cache the ones you >are using now. What you have is a system that depends on the *user* to >decide what permanent objects should be cached in fast memory and what >should not, moment to moment. This sucks. The computer should >automatically perform this function. I. e., minimal virtual memory. I don't think you are talking about virtual memory. Instead it looks like you are talking about intelligent caching. The only way I can see of doing this is by having RESIDENT work. This already works to some extent -- witness the libraries. For instance, you can load in diskfont from disk and narrator from disk and they hang around in RAM. Theoretically when the RAM free drops too low, one or the other library will be purged out. So what you have described in the above-quoted paragraph would be resident loading for programs. I think this has already been discussed, and I believe the concensus was that it wasn't do-able without going through a lot of contortions. I would love to have resident stuff, but not at the expense of having to buy another compiler or having all my old s/w break... -- Dan Green -- ARPA: hsgj@tcgould.tn.cornell.edu UUCP: ihnp4!cornell!batcomputer!hsgj BITNET: hsgj@cornella
dca@kesmai.COM (David C. Albrecht) (11/17/87)
Why are we hashing out the virtual memory bit again? > What you have is a system that depends on the *user* to > decide what permanent objects should be cached in fast memory and what > should not, moment to moment. This sucks. The computer should > automatically perform this function. I. e., minimal virtual memory. Me? I spend the $30 to buy FACC which performs caching of frequently used disk blocks. It isn't equivalent to paging of memory but it achieves most of what I need it for. Most things come off a hard disk fast enough that I wouldn't want to cache them in RAM:. ANY resource allocater will throw out something you haven't used in a while even if you don't want it to. A guaranteed way to lock items in memory ala. a RAM: disk is a plus even in virtual memory systems. > > lot's more stuff about some bizarro application which the author feels > wouldn't be supported well without virtual memory. > Who cares? I certainly don't. It is true that virtual memory systems make it possible to more efficiently use memory. It is difficult to have them do so without a memory speed access penalty (not to say that it can't be done). Virtual memory can make accessing of memory a much more time erratic phenomenon. It is a REQUIREMENT for virtual memory systems to have a hard disk, paging memory to floppies is preposterous and there are a hell of a lot of Amigas out there without a hard disk. In any case with a large enough memory space, virtual memory isn't really necessary. My Amiga has 2.5 meg which is enough memory for me to do virtually everything I want (I dare say the same is true for 98% of the rest of the people that use the machine). You can certainly achieve the task you so ardently desire with existing compilers 'safely', efficiently might be another matter. You could write a TSR style object manager just like which you would communicate with through a message port to get pointers to objects. It would handle swapping in/out of objects etc. You could 'lock' the object as you make your reference then 'unlock' it. It would probably be slow as hell, but it would work. I don't really miss not having virtual memory. I do miss not having interprocess protection. Certainly, you can come up with applications for which having virtual memory is a plus. You can also come up with reasons why being guaranteed that every page is in memory and immediately available is a plus. For the Amiga virtual memory wouldn't be a great idea. On any machine guaranteed to come with a hard disk, however, in my opinion virtual memory is always a win just because it will let you more efficiently use the memory you have. With enough memory, however, it becomes a moot point because you don't need to use it efficiently. David Albrecht
peter@sugar.UUCP (Peter da Silva) (11/20/87)
Just a quick interjection. Some of youse guys seem to be confusing: 1. Virtual memory. 2. Protected memory. 3. Memory Management. Memory management is when the address space seen by a program differs from the real address space. Virtual memory is what you have when part of the memory a program uses isn't in real RAM, but is copied in from secondary storage on demand. This usually requires memory management. Protected memory is when part of the address space is protected from reading and/or writing. This usually is part of memory management systems. Protected memory is a good idea. Memory management may be a good idea. Virtual memory probably isn't a good idea. Protected memory should be fairly easy to integrate with the Amiga O/S... just make MEMF_PUBLIC mean shared memory, MEMF_PRIVATE private and protected memory. Memory management will be tougher, but not too difficult. Just map managed memory above the 16 meg 68000 limit, and keep all the IO in the 68000 address space. Virtual memory is going to be pretty tough, and is probably Not even desirable, given the low cost of real memory. -- -- Peter da Silva `-_-' ...!hoptoad!academ!uhnix1!sugar!peter -- Disclaimer: These U aren't mere opinions... these are *values*.
greg@ncr-sd.SanDiego.NCR.COM (Greg Noel) (11/24/87)
In article <1089@sugar.UUCP> peter@sugar.UUCP (Peter da Silva) tries to distinguish between virtual memory, protected memory, and memory management. The definitions he uses are basically correct, but he makes one small error: >Virtual memory is what you have when part of the memory a program uses isn't >in real RAM, but is copied in from secondary storage on demand. This usually >requires memory management. This is a definition for "swapping," and not for virtual memory, although the two are frequently found in the same system. A better rule of thumb is that if the operating system could move a process within real memory without the process having to be aware of it, then you have virtual memory. Note that this doesn't say that it \does/ relocate processes, just that it \could/ if it wished. Swapping or paging systems build on top of this the ability to move pieces of the process between real memory and secondary storage, or to run a partially unloaded program, but that is not inherent in the meaning of "virtual memory." Perhaps an example will make this clear: I once worked on a project that had a number of real-time tasks with variable storage requirements, so they each used a (common) free storage management package. These tasks all had to live on the same computer. Some non-techincal idiot had added up the maximum storage required by each task, and had specified a computer memory only a little larger than the sum of the three requirements. The results were predictible; memory got fragemented and the system would run out of free memory with about 1/3 of the memory unallocated (Knuth's result). There was no way to get the hardware upgraded (it was a military procurement, and the specifications were cast in concrete), so we had to come up with a way to not run out of storage. The thing had an MMU (for use by the background programs), so we just put the real-time tasks in a virtual name space about three times as large as the actual space, and made the free storage package smart about allocating things within pages. That, and a little tweaking of the software so that more requests were in page-sized increments, allowed us to keep running until the memory was 95% to 98% utilized. (The interesting thing is that the workload is directly applicable to the Amiga -- we had a relativly few large requests (corresponding to programs being loaded) and a lot of smaller requests (corresponding to the data being manipulated). This means that an Amiga that has been running for a while will usually run out of memory when it is less than 75% full. Adding a MMU to the Amiga would be equivalent to adding 20% to 25% more memory, not to mention the increased capability that a MMU would give. I don't know how much a MMU would cost, but if it is less than one-quarter of the memory cost, then you get more for your buck by adding adding a MMU. Note that the leveraging increases the more memory you have.) The point of all of this is that here we are pretty clearly using virtual memory but not doing any paging. Some of the proposals for a modified memory model for the Amiga have been similar to this; it's certainly an idea worth investigating. I'd certainly like to see it in a future OS release...... -- -- Greg Noel, NCR Rancho Bernardo Greg.Noel@SanDiego.NCR.COM or greg@ncr-sd
peter@sugar.UUCP (Peter da Silva) (11/27/87)
Me: >Virtual memory is what you have when part of the memory a program uses isn't >in real RAM, but is copied in from secondary storage on demand. This usually >requires memory management. In article <1912@ncr-sd>, greg@ncr-sd.SanDiego.NCR.COM (Greg Noel) writes: > This is a definition for "swapping," and not for virtual memory, although the > two are frequently found in the same system. Please, let's not have the "virtual definitions wars" here. We just got done with them elsewhere on the net. Program may be moved in memory without knowing about it. Some people call this virtual memory, some people call this mapped or managed memory. Depends on whether your school used IBM, DEC, CDC, or whatever other equipment. Program may be copied entirely to disk and then back into memory. I call this swapping. Program may be partially copied to disk and reloaded on demand: when the program needs that code or data. Some people call this virtual memory. The people who call what I call mapped memory vitual memory call this demand paged virtual memory. The point of my article remains: to wit... a) Mapped memory, b) Protected memory, c) Demand Paged Virtual Memory, May be mixed and matched. You can have any combination of these any way you like, and there have been implementations of same. a only: Minix has this. b only: Amiga has this (WCS RAM). c only: this appears to be what C= has in mind for VMAmiga. a+b: Most '70s 16-bit minis did this. a+c: FORTH's disk management uses a technique like this. b+c: This one I don't know about. C= may be thinking of this. a+b+c: "Classical" VM system, such as the Vax. -- -- Peter da Silva `-_-' ...!hoptoad!academ!uhnix1!sugar!peter -- Disclaimer: These U aren't mere opinions... these are *values*.