dillon@HERMES.BERKELEY.EDU (Matt Dillon) (05/13/89)
I propose we implement a new AllocMem() flag, MEMF_VIRTUAL, in a way that can easily be accomplished in 1.4 or 1.5 and that is fully compatible with the current Amiga system. I propose that a bit in the hunk type field for an executable be reserved to mean MEMF_VIRTUAL (allow parts of an executable's data or code to be loaded into virtual memory). I propose a common virtual store be implemented for 1.4 ... very simple. No per-task state stored or anything... all tasks allocate out of the same virtual store just as they currently allocate out of the amiga's memory. I suggest that this would only be implemented on a 68020 or beyond (68000's would simply ignore the MEMF_VIRTUAL flag and allocate memory normally). Virtual memory would be demand paged to an arbitrary file on hard disk (arbitrary file path would allow paging over a network relatively easily). Since the programmer has complete control over what is in virtual memory and what is not, no munging or worrying about critical sections need occur. ---------------- reasons The current direction for shared memory on unix systems is the mmap() system call... map a file into (possibly shared) memory. The at&t shm() standard is a subset of mmap(). The logical next step to the above description for the Amiga would be to allow a process to map an arbitrary file into virtual memory (where the file is used as swap for the VM in question, and updated on close or unmap). I suggest that a logical next step would be to implement per-task virtual memory by adding another file handle to the process structure... a hard disk (or any other seekable) file that would be mapped into memory and used by the process to allocate VM. Release of memory allocated in this way is done simply by closing the file handle. One would also have to allow the file system to unlink (Delete()) an open file (i.e., as in unix, the file doesn't actually go away until all open descriptors are closed, but the file IS removed from the directory). All current software would be upward compatible as it will not specify the MEMF_VIRTUAL flag, and most of the current software will be immediately modifiable (modify flags in the hunk type) to take advantage of sticking various parts in virtual memory. -Matt
peter@sugar.hackercorp.com (Peter da Silva) (05/13/89)
I agree completely with Matt. This is the best way of getting VM started. (read <8905121949.AA16989@hermes.Berkeley.EDU> to see what I'm talking about) -- Peter "Have you hugged your wolf today" da Silva `-_-' ...texbell!sugar!peter, or peter@sugar.hackercorp.com 'U`
dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) (05/14/89)
Refering back to my first article, I have a couple more implementation ideas: (1) Since a distinction between real and virtual memory exists, the user should be able to set how much real memory the virtual memory will be mapped to. For example, if we have 2MB of ram and a process that is using 20MB of virtual memory, we do not want to use all 2MB for VM simply because many programs will still need to allocate non-virtual memory. So, for example, the user (read: s:startup-sequence) would tell the system to use X amount of MB or X% of real memory for virtual memory. The beauty of it all is that no matter how little or how much you specify, no program using VM will know the difference. The more you reserve, the less paging will have to go on. (2) The Maximum amount of virtual memory in the system can be implemented in two ways. I would like to say something like this in s:startup-sequence : enablevm 30% hdv:virtual-memory 20M Turn on virtual memory, reserve 30% of real memory for virtual memory, use the file hdv:virtual-memory to page to and set the maximum amount of VM to 20MB. There are three possible ways to implement this: P.S. for those of you who are not familar with VM techniques, note that real memory isn't actually mapped until you reference the VM page. real memory is NOT mapped at allocation. (A) Open(file,1006) and write 20MB of zero's (or garbage), map 20MB worth of memory via the MMU, and then just treat the memory like almost-normal AllocMem'd memory... you can even use the same allocation algorithm! * my first choice, extremely simple. (B) Open(file,1006) and write nothing yet. Assume you will be able to extend the file as needed to 20MB, map 20MB worth of memory via the MMU and use a slightly modified algorithm for AllocMem() to handle the fact that no paging space has been allocated beyond a certain marked point (that is, disallow accesses beyond the mark point until AllocMem() gets to there). * my favorite, medium difficulty. One would eventually want to add a ftruncate() command to the filesystem to do this right though. File fragmentation is not a problem.. the user can put the file on a little-used or unused partition, or make a partition specifically to hold the VM-file (but it is his choice!). (C) Reserve a partition to be used for VM. I do not like this idea at all. In fact, I hate it. Not only do you need to reserve an entire partition, but you can't adjust the size of your VM without reformatting the HD! The one advantage is that on system crash the OS can raw-write VM to the swap partition without worrying about destroying files (cause there are none). Using a file for VM means that on system crash you cannot dump core. I don't we want to loose the functionality of files for paging just so we can dump system core. -Matt
451061@UOTTAWA.BITNET (Valentin Pepelea) (05/16/89)
Matt Dillon <dillon@postgres.berkeley.edu> in message
<8905132040.AA27524@postgres.Berkeley.EDU> as well as in other messages
introduces some ideas involving virtual memory. Let s analyse them one by one.
- That a new MEMF_VIRTUAL flag be added. The idea is to maintain compatibility
- for older programs which mishandle critical sections, while new programs
- may specifically request virtual memory pages.
This is a prime example of what should not be done. If we are to implement
virtual memory, let s do it right the first time. Do we want to create
compatibility cludges and end up with version 7.0 of AmigaDOS which will be
stuck in old technology and incapable of running the majority of applications?
Look at what Apple is stuck with. Let s rough up a few feathers, and illuminate
the right path.
I propose that a new MEMF_PHYSICAL flag be added. memory requested with this
flag will be guaranteed never to be swapped in and out, and thus would be the
ideal location for device drivers and other real-time programs. If a user
wishes to run a program that does not, all he has to do is not install the
virtual memory handler. This is a real neat feature of the upcoming virtual
memory and memory protection handlers; they can be installed long after the
Amiga has been booted and running.
- That as a SWAP: device, a file be used. The potential advantages of this are
- numerous, and include to ability to increase dynamically the size of the
- virtual memory, and the ability to locate the file anywhere.
But this is another prime example of what should not be done. It is much
preferable to use a reserved fixed partition of a hard drive for speed. Files
are great when you wish to store and access data sequentially. But when fixed
size blocks (pages) have to be accessed randomly, going through the file system
will slow down the access time by a factor of ten, optimistically. Since the
memory requirement versus access time speed is exponentially proportional
(inversly), we will need three times more physical ram to map into. Choose:
SWAP: is a DOS file Physical: 6 Mbytes Virtual: 40 Mbytes
SWAP: is a partiiton Physical: 2 Mbytes Virtual: 40 Mbytes
So you see, the hard disk s access time plays a primordial role in our
selection of how much physical ram we need to map into, and how large our page
size is going to be. It would be best to have a Quantum 19ms drive devoted
solely to virtual memory thus not even allowing other programs to play with
the postion of the drive s head.
Note that the size of the virtual addressing space (40 Meg) is not at all a
factor in the page size and physical ram size. But rather the number of
programs you are running, and their atomicity is. For example Smalltalk and
Forth programs would prefer 512 byte pages, while C programs would prefer
2K pages. Um, make that 4K. So allot as large a virtual addressing space as
your hard drive allows you too.
Valentin
_________________________________________________________________________
"An operating system without Name: Valentin Pepelea
virtual memory is an operating Phonet: (613) 233-1821
system without virtue." Bitnet: 451061@Uottawa.bitnet
Usenet: Use cunyvm.cuny.edu gate
- Ancient Inca Proverb Planet: 451061@acadvm1.UOttawa.CAdillon@POSTGRES.BERKELEY.EDU (Matt Dillon) (05/17/89)
I write: :- That a new MEMF_VIRTUAL flag be added. The idea is to maintain compatibility Valentin Pepelea 451061@acadvm1.UOttawa.CA writes: :This is a prime example of what should not be done. If we are to implement :virtual memory, let s do it right the first time. Do we want to create :compatibility cludges and end up with version 7.0 of AmigaDOS which will be :stuck in old technology and incapable of running the majority of applications? :Look at what Apple is stuck with. Let s rough up a few feathers, and illuminate :the right path. : :I propose that a new MEMF_PHYSICAL flag be added. memory requested with this :flag will be guaranteed never to be swapped in and out, and thus would be the You are clearly missing the point. Certainly I would much rather have a MEMF_PHYSICAL flag instead of a MEMF_VIRTUAL flag, but it would break too many programs... more than too many, a *huge* number of programs. I write: :- That as a SWAP: device, a file be used. The potential advantages of this are :- numerous, and include to ability to increase dynamically the size of the :- virtual memory, and the ability to locate the file anywhere. Valentin Pepelea 451061@acadvm1.UOttawa.CA writes: :preferable to use a reserved fixed partition of a hard drive for speed. Files :are great when you wish to store and access data sequentially. But when fixed :size blocks (pages) have to be accessed randomly, going through the file system :will slow down the access time by a factor of ten, optimistically. Since the :memory requirement versus access time speed is exponentially proportional :(inversly), we will need three times more physical ram to map into. Choose: Here you are wrong for many reasons. (1) Using a fixed partition restricts the USER's ability to configure the system the way he likes... i.e. he must reformat his hard drive to change the swap partition size. (2) It requires an entirely new module to be written to handle the independant trackdisk.device interface. (3) It is not implementable in the near future. (4) it does not ease the way for eventual file mapping which is where UNIX is going these days. (5) Your basic assumption that managing a custom swap partition would be faster than using a file is incorrect. The only point I need to embellish upon as the last one. Frankly, I think that with appropriate modifications to the fast file system, using the FFS would be nearly as efficient as a custom partition. NOT ONLY THAT BUT USING IT WITHOUT MODIFICATION WOULD BE VIABLE, and not much slower than a custom partition. Certainly not more than 10% slower and I'd wager the difference would be even less than that. :SWAP: is a DOS file Physical: 6 Mbytes Virtual: 40 Mbytes :SWAP: is a partiiton Physical: 2 Mbytes Virtual: 40 Mbytes : :So you see, the hard disk s access time plays a primordial role in our :selection of how much physical ram we need to map into, and how large our page You are assuming here that an FFS file is 3 TIMES SLOWER! Bullshit. :Valentin -Matt
451061@UOTTAWA.BITNET (Valentin Pepelea) (05/17/89)
Matt Dillon <dillon@postgres.berkeley.edu> writes in Message-ID: <8905162209.AA20997@postgres.Berkeley.EDU> > You are clearly missing the point. Certainly I would much rather > have a MEMF_PHYSICAL flag instead of a MEMF_VIRTUAL flag, but it would break > too many programs... more than too many, a *huge* number of programs. Would you not rather break many programs today, when there are so few potential A2500 owners and users of virtual memory, than three years from now when every body will be having MMUs in their Amigas? (wishful thinking) I would rather burn naughty devellopers now, when there is 1 user of VM, than three years from now, when there will be 2 million. > (1) Using a fixed partition restricts the USER's ability to configure the > system the way he likes... Agree with you there, although the driver for the SWAP: device does not necessarely have to be a harddisk.device. Flexibility is maintained, though not as elastic as if a file were used. > (2) It requires an entirely new module to be written to handle the > independent trackdisk.device interface. Definitely not. The SWAP: device should be a mere hard disk partition controlled by access through the harddisk.device. Ultimately a custom driver should be used only if the harddisk.device driver proves to be to slow. Unlikely > (3) It is not implementable in the near future. Preposterous! Virtual memory can be written within a week by somebody who knows what he is doing. A handler will be available by the end of the summer, I will bet my iron ring on that. > (4) it does not ease the way for eventual file mapping which is where UNIX is > going these days. Hmmm, you seem to have Unix etched in your head. I bet you are one of those guys who has written a CSH-like shell for the Amiga. Unix is a prime example of the direction the Amiga should not be going into. Unix is slow, Unix is big, Unix is cumbersome. The Amiga on the other hand is a real-time system, small, compact, fast. Useful. The Amiga provides a common addressing space for all concurrent tasks, a most useful feature than can only be provided in single- user systems. Unix intents to map the individual addressing spaces of each process to a file, which is ideal for multi-user systems where programs have to have their own addressing space. That is clearly not what we are dealing with on the Amiga. We have a single addressing space, and we know in advance the range to which we want this addressing space to be limited to. > (5) Your basic assumption that managing a custom swap > partition would be faster than using a file is incorrect. > The only point I need to embellish upon as the last one. Frankly, > I think that with appropriate modifications to the fast file system, using > the FFS would be nearly as efficient as a custom partition. NOT ONLY THAT > BUT USING IT WITHOUT MODIFICATION WOULD BE VIABLE, and not much slower than > a custom partition. Certainly not more than 10% slower and I'd wager the > difference would be even less than that. ...after my example... > You are assuming here that an FFS file is 3 TIMES SLOWER! Bullshit. Quite clearly, you do not understand. FFS is fast because after an initial sector is found, the following ones are loaded sequentially off the same track. Thus for large files, the tranfer rate is impressive. But for small scattered files, FFS is not better than the old file system. For paging, we have to transfer individual 512 bytes pages which are never needed in a sequential order. Thus the overhead that the filing system introduces to find the approriate sector on disk increases seek time by an order of magnitude. The example I gave said the filing system would be 10 times slower, not 3, thus requiring three times more physical memory. sqrt(10)=3 For swapping page 316, it is much simpler to transfer directly sector 316 through the trackdisk.device, than having to determine the location of the sector through a filing system. The only way a filing system could be used in this case would be by an indexed access method, or database access method filing system. A good example of such a marvel is INFOS II on Data General computers. I hear the Software Distillery is working on just such a beast. No wonder, since John Toebes used to hack on DG systems. Valentin _________________________________________________________________________ "An operating system without Name: Valentin Pepelea virtual memory is an operating Phonet: system without virtue." Bitnet: 451061@Uottawa.bitnet Usenet: Use cunyvm.cuny.edu gate - Ancient Inca Proverb Planet: 451061@acadvm1.UOttawa.CA
ewhac@well.UUCP (Leo 'Bols Ewhac' Schwab) (05/17/89)
In article <8905132040.AA27524@postgres.Berkeley.EDU> dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) writes: > (C) Reserve a partition to be used for VM. I do not like this idea > at all. In fact, I hate it. [ ... ] Doesn't UNIX do it this way? _-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_ Leo L. Schwab -- The Guy in The Cape INET: well!ewhac@ucbvax.Berkeley.EDU \_ -_ Recumbent Bikes: UUCP: pacbell > !{well,unicom}!ewhac O----^o The Only Way To Fly. hplabs / (pronounced "AE-wack") "Work FOR? I don't work FOR anybody! I'm just having fun." -- The Doctor
deven@rpi.edu (Deven Corzine) (05/17/89)
In article <8905132040.AA27524@postgres.Berkeley.EDU> dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) writes: > Refering back to my first article, I have a couple more >implementation ideas: Your first article was great... just the way to get VM going on the Amiga. > (1) Since a distinction between real and virtual memory exists, the > user should be able to set how much real memory the virtual memory > will be mapped to. For example, if we have 2MB of ram and a > process that is using 20MB of virtual memory, we do not want to > use all 2MB for VM simply because many programs will still need > to allocate non-virtual memory. Matt, Matt... Here you seem to suffer from a lack of foresight. This is NOT an ideal way to implement it. I'd say the proper way to implement it is to modify AllocMem and design the pager such that all available memory (i.e. unallocated real RAM) may be used for VM. If you have 2MB of RAM and a process is using 20MB of VM, and another process is using .5MB (including system overhead, we'll say) of real RAM, then the remaining 1.5MB of RAM should be used for the VM. Then, if the other process were to allocate another 1MB of real RAM, then AllocMem should call the pager to flush 1MB of the VM pages allocated, and reallocate them as real RAM for the other process. Similarly, upon a FreeMem, the system should return as much of the freed memory as is usable [that are of full VM pagesize, after freelist merge] to the pager to lessen paging required. In this way, if there is real RAM free, any program which needs it can allocate it, but if it is unused, then let it be used for VM as necessary, reclaiming the VM-paging real RAM pages on demand. You get dynamic usage of the available memory, and optimum efficiency at all times, without worrying how much memory you need to set aside for either real RAM or VM paging space. What's leftover is simply used for paging, automatically. Remember, think *dynamic*. Catch-word of the day. :-) > The beauty of it all is that no matter how little or how much you > specify, no program using VM will know the difference. The more > you reserve, the less paging will have to go on. But this dynamic method is even more beautiful, because you don't waste memory on either unallocated (but reserved) real RAM nor on unused (but reserved) VM paging memory. Again, the program using VM won't know the difference, except that speed will be affected. Instead of less paging when you reserve more RAM, you get less paging when you're USING less RAM. And when you use more, it will page more accordingly, yet not tell you the system is out of RAM. [Clearly, at least ONE page must remain to page to, but the rest can be reclaimed at need.] > (2) The Maximum amount of virtual memory in the system can be > implemented in two ways. I would like to say something like this > in s:startup-sequence : > enablevm 30% hdv:virtual-memory 20M Perhaps have VM automatically use some default file if a program attempts to use VM when no file has been thus set? Or, maybe have MEMF_VIRTUAL memory allocated in real RAM until an enablevm, at which time, all previously allocated MEMF_VIRTUAL pages would be remapped (or reallocated, if need be) into virtual memory pages... That would perhaps be the best way to go. There should be a disablevm program, also, which would remap/reallocate vitual pages into real RAM (if possible) and disable the pager until an enablevm. [...] > (A) Open(file,1006) and write 20MB of zero's (or garbage), [...] > (B) Open(file,1006) and write nothing yet. [...] > (C) Reserve a partition to be used for VM. [...] I propose enablevm use the following template: enablevm file/a size/a where "file" is either a file, volume or partition to page to, using "VM:" if file is unspecified, or "HD0:VM" [maybe] if no VM: exists. If no VM: exists, ASSIGN it to the file or partition actually used, whether specified on the command line or taken as the default default. ["HD0:VM" or whatever] The "size" argument would only be applicable if paging to a file and not a partition, device or volume. If paging to a file, "size" would specify how large the paging file must start. If no size is given, the paging file starts empty. In either case, the file would be expanded as necessary, assuming space is available for expansion. If VM is already enabled upon an enablevm, the new enablevm would change the paging file (device, partition, volume) to the specified (or default) one, copying over all the pages from the old paging file, and probably (but not necessarily) deleting the old paging file. Disablevm would need no arguments and would simply force all VM pages back into real RAM if possible and disable the pager. Well, perhaps a switch or two to define the action taken if not enough real RAM exists for the virtual pages. [leave remapped pages in real RAM and leave the pager going/remap all pages back to the VM file/partition and leave the pager enabled/dump unmapped pages, delete paging file and disable pager/etc.] This way, you could get the best of all worlds. True, it would be slightly more programming effort for C-A than a simpler method, but not that much, and I think it is the right way to do it. Comments? :-) Deven -- shadow@[128.113.10.2] <shadow@pawl.rpi.edu> Deven T. Corzine (518) 272-5847 shadow@[128.113.10.201] <shadow@acm.rpi.edu> 2346 15th St. Pi-Rho America deven@rpitsmts.bitnet <userfxb6@rpitsmts> Troy, NY 12180-2306 <<tionen>> "Simple things should be simple and complex things should be possible." - A.K.
daveh@cbmvax.UUCP (Dave Haynie) (05/17/89)
in article <8905162209.AA20997@postgres.Berkeley.EDU>, dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) says: > NOT ONLY THAT BUT USING [FFS] WITHOUT MODIFICATION WOULD BE VIABLE, and not > much slower than a custom partition. Certainly not more than 10% slower and > I'd wager the difference would be even less than that. Not only that, but if you're willing to live with a SWAP: file that doesn't move during the life of an active VM system, you can have you're cake and eat it too. Given an file in FFS, it's rather easy to chase down the device driver level location of evey block in that file. A translation table of sorts would allow a swapper to use direct device level access to the contents of the file, rather than SEEKing at the filesytem level. With a properly clever system, you'd have your choice, since it's really the same thing to the rest of the system. So you could have swap space out on your LAN somewhere if you really wanted it, or local access faster than is possible under FFS. > > -Matt -- Dave Haynie "The 32 Bit Guy" Commodore-Amiga "The Crew That Never Rests" {uunet|pyramid|rutgers}!cbmvax!daveh PLINK: D-DAVE H BIX: hazy Amiga -- It's not just a job, it's an obsession
bryan@cs.utexas.edu (Bryan Bayerdorffer @ Wit's End) (05/18/89)
In article <DEVEN.89May17023146@daniel.rpi.edu> deven@rpi.edu (Deven Corzine) writes: =- =-Matt, Matt... Here you seem to suffer from a lack of foresight. This =-is NOT an ideal way to implement it. I'd say the proper way to =-implement it is to modify AllocMem and design the pager such that all =-available memory (i.e. unallocated real RAM) may be used for VM. If =-you have 2MB of RAM and a process is using 20MB of VM, and another =-process is using .5MB (including system overhead, we'll say) of real =-RAM, then the remaining 1.5MB of RAM should be used for the VM. Then, =-if the other process were to allocate another 1MB of real RAM, then =-AllocMem should call the pager to flush 1MB of the VM pages allocated, =-and reallocate them as real RAM for the other process. Similarly, =-upon a FreeMem, the system should return as much of the freed memory =-as is usable [that are of full VM pagesize, after freelist merge] to =-the pager to lessen paging required. =- =-In this way, if there is real RAM free, any program which needs it can =-allocate it, but if it is unused, then let it be used for VM as =-necessary, reclaiming the VM-paging real RAM pages on demand. You get =-dynamic usage of the available memory, and optimum efficiency at all =-times, without worrying how much memory you need to set aside for =-either real RAM or VM paging space. What's leftover is simply used =-for paging, automatically. =- Yeah, I thought of posting the same response to Matt's article, but there is a problem. Neglecting chip RAM, the reason a task needs physical memory is to guarantee fixed execution time for instructions, and for calls to AllocMem. Now, you can lock the page frames occupied by the code at load time, but if you allow AllocMem to kick out a VM page, and the page is dirty, then you have to wait for the pager to run and write the page out to disk. I can think of a couple of solutions to this, but none are much better than statically partitioning memory. Of course, if you don't run any real-time programs, then the idea remains elegant, and I'm glad I had it first. :-) ______________________________________________________________________________ /_____/_____/_____/_____/_____/_____/_____/_____/_____/_____/_____/_____/_____/ |_____|_____|_____|_____|_____|_____|_____|_____|_____|_____|_____|_____|_____| _No dark sarcasm in the classroom|_____|_____|_____|_____|_____|_____|_____|___ |____Teachers leave the kids alone__|_____|_____|_____|_bryan@cs.utexas.edu___| ___|_____|_____|_____|_____|_____|_____|{vertebrae...}!cs.utexas.edu!bryan_|___ |_____|_____|_____|_____|_____|_____|_____|_____|_____|_____|_____|_____|_____|
new@udel.EDU (Darren New) (05/18/89)
In article <8905170500.AA25917@jade.berkeley.edu> 451061@UOTTAWA.BITNET (Valentin Pepelea) writes: >For swapping page 316, it is much simpler to transfer directly sector 316 >through the trackdisk.device, than having to determine the location of the >sector through a filing system. Alternately, the VM handler could, upon startup, in some way scan through the file to determine where all the sectors are, then bypass the file system and go to the driver. If the file is fixed size, this is real easy. If the file can grow, then it would have to be done on later allocations as well. This would give the best of both worlds, and certainly could not be harder to code than DiskSalv or DiskDoctor. -- Darren P.S., I always assumed MEMF_PUBLIC meant memory that would be shared between tasks, thus providing an upgrade for when memory mapping/protection is implemented (much like the MEMF_CHIP that most 1.1 programs failed to use). Are you saying that MEMF_PHYSICAL/MEMF_VIRTUAL is needed to lock pages in RAM or needed to allow sharing? -- Darren
new@udel.EDU (Darren New) (05/18/89)
Another thought to ponder: Would you want your RAM: drive to use virtual memory? :-) :-) Darren
451061@UOTTAWA.BITNET (Valentin Pepelea) (05/18/89)
Darren New <new@udel.edu> in Message-ID <15706@louie.udel.EDU> writes: > P.S., I always assumed MEMF_PUBLIC meant memory that would be shared between > tasks, thus providing an upgrade for when memory mapping/protection is > implemented (much like the MEMF_CHIP that most 1.1 programs failed to use). > Are you saying that MEMF_PHYSICAL/MEMF_VIRTUAL is needed to lock pages in > RAM or needed to allow sharing? -- Darren Here is are the definitions of the flags as they are/will/should implemented: MEMF_CHIP The lower 512 or 1024 kilobytes of memory accessible by the graphic coprocessors. MEMF_FAST Memory not accessible by the custom chips. MEMF_CLEAR The memory gets cleared before it is allocated. MEMF_PUBLIC Guarantees that the memory returned is accessible for both read an write operations by all tasks. If not specified, the memory returned is accessible only for reading by other tasks. This flag is to be used when memory protection is implemented, and has nothing to do with virtual memory. MEMF_PRIVATE The memory returned will not be allowed to be read bor written to by other tasks. Useful only for multi-user systems. Hopefully will never be implemented on the Amiga. There are so neat tricks you are not allowed to do on multi-user systems. MEMF_PHYSICAL Or better yet, MEMF_LOCKED, guarantees that the memory returned will never be paged out of physical ram. MEMF_VIRTUAL Returns a block of memory that is probably going to be in the virtual memory addressing range. Hopefully you will never see this; intended as a compatibility feature to allow old and new software to coexist. Also provides CPM-80 compatibility under AmigaDOS. :-) Timex-80 compatibility in the works. Dave Haynie just reminded me the virtual machine capibility that an MMU provides. Thus Forbid() and Disable() may turn out not to be a problem after all. Valentin _________________________________________________________________________ "An operating system without Name: Valentin Pepelea virtual memory is an operating Phonet: New! (613) 231-7476 system without virtue." Bitnet: 451061@Uottawa.bitnet Usenet: Use cunyvm.cuny.edu gate - Ancient Inca Proverb Planet: 451061@acadvm1.UOttawa.CA 3 weeks and counting...
451061@UOTTAWA.BITNET (Valentin Pepelea) (05/18/89)
Darren New <new@udel.edu> in Message-ID <15707@louie.udel.EDU> writes: > Another thought to ponder: > Would you want your RAM: drive to use virtual memory? > :-) :-) Darren Of course you would! (If you had a large physical ram space to map into) Basically it would be like accessing your hard drive through a disk cache, but without incurring the delay of forced writes to disk. Using a disk cache like FACC on the other hand would be a really stupid. Valentin _________________________________________________________________________ "An operating system without Name: Valentin Pepelea virtual memory is an operating Phonet: (613) 231-7476 (New!) system without virtue." Bitnet: 451061@Uottawa.bitnet Usenet: Use cunyvm.cuny.edu gate - Ancient Inca Proverb Planet: 451061@acadvm1.UOttawa.CA
deven@rpi.edu (Deven Corzine) (05/18/89)
In article <347@mohawk.cs.utexas.edu> bryan@cs.utexas.edu (Bryan Bayerdorffer @ Wit's End) writes: > Yeah, I thought of posting the same response to Matt's >article, but there is a problem. Neglecting chip RAM, the reason a >task needs physical memory is to guarantee fixed execution time for >instructions, and for calls to AllocMem. Now, you can lock the page >frames occupied by the code at load time, but if you allow AllocMem >to kick out a VM page, and the page is dirty, then you have to wait >for the pager to run and write the page out to disk. I can think of >a couple of solutions to this, but none are much better than >statically partitioning memory. Of course, if you don't run any >real-time programs, then the idea remains elegant, and I'm glad I had >it first. :-) Thpppt. I had it first. :-) Seriously, you could help that situation by reserving a set amount of free real RAM that should be available at any given time. It could act as a buffer so the pager could free more pages while the program merrily goes and uses the memory it allocated. Alternatively, (or conjunctively, or something) you could have a maximum number of "dirty" pages the pager may have... and allow it to use as much memory as is available for read-only pages, which can be (almost) instantly flushed. Basically, this comes down to minimum clean free pages, but without necessarily wasting them... That's the idea, anyhow. Deven -- shadow@[128.113.10.2] <shadow@pawl.rpi.edu> Deven T. Corzine (518) 272-5847 shadow@[128.113.10.201] <shadow@acm.rpi.edu> 2346 15th St. Pi-Rho America deven@rpitsmts.bitnet <userfxb6@rpitsmts> Troy, NY 12180-2306 <<tionen>> "Simple things should be simple and complex things should be possible." - A.K.
dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) (05/18/89)
:article, but there is a problem. Neglecting chip RAM, the reason a :task needs physical memory is to guarantee fixed execution time for :instructions, and for calls to AllocMem. Now, you can lock the page :frames occupied by the code at load time, but if you allow AllocMem :to kick out a VM page, and the page is dirty, then you have to wait :for the pager to run and write the page out to disk. I can think of :a couple of solutions to this, but none are much better than :statically partitioning memory. Of course, if you don't run any :real-time programs, then the idea remains elegant, and I'm glad I had :it first. :-) Well, the particular task which had the page fault will 'block' waiting for the page to come in, but remember this is a multitasking system! Other tasks will still run. Page outs wouldn't occur all that much unless you are running more active processes than you have memory for so it is pretty much restricted to the 'beginning to overload the machine' case. -Matt
dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) (05/18/89)
:>in article <8905162209.AA20997@postgres.Berkeley.EDU>, dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) says:
:>
:>> NOT ONLY THAT BUT USING [FFS] WITHOUT MODIFICATION WOULD BE VIABLE, and not
:>> much slower than a custom partition. Certainly not more than 10% slowerand
:>> I'd wager the difference would be even less than that.
:>
:Dave Haynie "The 32 Bit Guy" Writes:
:>Not only that, but if you're willing to live with a SWAP: file that doesn't
:>move during the life of an active VM system, you can have you're cake and
:>eat it too.
:>Given an file in FFS, it's rather easy to chase down the device driver level
:>location of evey block in that file. A translation table of sorts would allow
Yes, exactly! VM can easily be implemented using files and the
theoretical efficiency limit approaches that of a custom partition. I say
get it working first (even if only 80-90% efficient) and worry about
remembering all file blocks later. Remember, currently side sectors are
cached which is where I got my 80-90% efficiency number from.
-Mattdillon@POSTGRES.BERKELEY.EDU (Matt Dillon) (05/19/89)
:> (1) Since a distinction between real and virtual memory exists, the :> user should be able to set how much real memory the virtual memory :> will be mapped to. For example, if we have 2MB of ram and a :> process that is using 20MB of virtual memory, we do not want to :> use all 2MB for VM simply because many programs will still need :> to allocate non-virtual memory. : :Matt, Matt... Here you seem to suffer from a lack of foresight. This :is NOT an ideal way to implement it. I'd say the proper way to :implement it is to modify AllocMem and design the pager such that all Hey Hey! A guy can make mistakes! You are right of course... all (or most) of real memory should be under VM but have the ability to 'lock' pages into memory as has been mentioned. The only problem is that AllocMem() requests are normally quite small... much smaller than a page, so management becomes more difficult if all of real memory is in VM. Still, it isn't much of a problem. On this MEMF_PHYSICAL vs MEMF_VIRTUAL bit. Sure MEMF_PHYSICAL is a nice way to say 'I want memory and it should be locked', and I'd much rather have that than MEMF_VIRTUAL (which says 'I want this memory to be pagable'), but really the only way to guarentee upward compatibility is to make physical the default and MEMF_VIRTUAL the exception. Thus have a MEMF_VIRTUAL AllocMem() flag and not a MEMF_PHYSICAL flag. -Matt
dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) (05/19/89)
:In article <8905132040.AA27524@postgres.Berkeley.EDU> dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) writes:
:> (C) Reserve a partition to be used for VM. I do not like this idea
:> at all. In fact, I hate it. [ ... ]
:
: Doesn't UNIX do it this way?
:
Yup. As I said, learn from UNIX's mistakes. Remember that UNIX is
essentially something that people normally build up and up and up and up....
until it becomes a bloated mess.
In this case, the fact that there are currently at least THREE paging
mechanisms on many UNIXs now. (1) VM paging to swap device, (2) read-only
paging of text from the executable file image, (3) mmap() rw paging to a file.
That doesn't include zero-fill or other features that would be utilitized
anyway.
Frankly, I would rather see just one mechanism: rw paging to a file.
Now, unfortunately, you can't ro-page the text (code) segment directly from
an Amiga executable due to the relocation information but that doesn't preclude
combing (1), (2), and (3) to (3). P.S. the specs on mmap() do allow for
the file to grow, of course. See my original posting.
-Mattdillon@POSTGRES.BERKELEY.EDU (Matt Dillon) (05/19/89)
:> You are clearly missing the point. Certainly I would much rather :> have a MEMF_PHYSICAL flag instead of a MEMF_VIRTUAL flag, but it would break :> too many programs... more than too many, a *huge* number of programs. : :Would you not rather break many programs today, when there are so few potential :A2500 owners and users of virtual memory, than three years from now when every :body will be having MMUs in their Amigas? (wishful thinking) I would rather :burn naughty devellopers now, when there is 1 user of VM, than three years from :now, when there will be 2 million. Boy, not only did you miss the point, it flew right past you! If you had read my posting more carefully, you would have noted that I said that systems which do not implement VM would ignore the MEMF_VIRTUAL flag. Are we clear now? Do you understand why it must be MEMF_VIRTUAL and not MEMF_PHYSICAL? It's so that the systems that *do* implement virtual memory are upward compatible from the systems that do not. :> (1) Using a fixed partition restricts the USER's ability to configure the :> system the way he likes... : :Agree with you there, although the driver for the SWAP: device does not :necessarely have to be a harddisk.device. Flexibility is maintained, though :not as elastic as if a file were used. Even MORE flexible if it's a file, eh? :> (4) it does not ease the way for eventual file mapping which is where UNIX is :> going these days. : :Hmmm, you seem to have Unix etched in your head. I bet you are one of those :guys who has written a CSH-like shell for the Amiga. Unix is a prime example of :the direction the Amiga should not be going into. Unix is slow, Unix is big, :Unix is cumbersome. The Amiga on the other hand is a real-time system, small, I leave interpretation up to individual readers but would like to note that your own presumably bad experiences may be getting the best of you here. :> You are assuming here that an FFS file is 3 TIMES SLOWER! Bullshit. : :Quite clearly, you do not understand. FFS is fast because after an initial :sector is found, the following ones are loaded sequentially off the same track. :Thus for large files, the tranfer rate is impressive. But for small scattered :files, FFS is not better than the old file system. For paging, we have to :transfer individual 512 bytes pages which are never needed in a sequential :order. Thus the overhead that the filing system introduces to find the :approriate sector on disk increases seek time by an order of magnitude. The Amusing. This question has already been answered. The problem is to find the data sector right? We are talking 2 hash lookups in the fully cached cased (which will be most of the time). 2 hash table lookups on the sector cache are zilch compared to actual transfer time. :The only way a filing system could be used in this case would be by an indexed :access method, or database access method filing system. A good example of such :a marvel is INFOS II on Data General computers. I hear the Software Distillery :is working on just such a beast. No wonder, since John Toebes used to hack on :DG systems. Uh, you seem to be under the misconception that seeking in a file takes lots of time when in fact it takes nearly zero time for the fully cached case (2 or 3 control sectors cached). :Valentin -Matt
suggs@loki.pmel.edsg.hac.COM (Brian Suggs) (05/19/89)
Matt Dillon writes: > ... Certainly I would much rather > have a MEMF_PHYSICAL flag instead of a MEMF_VIRTUAL flag, but it would break > too many programs ... My two cents: It seems that even those advocating MEMF_VIRTUAL agree that MEMF_PHYSICAL is a better way (exept for maintaining compatability.) For the benefits of virtual memory to be realized, MEMF_VIRTUAL *has* to be the default. MEMF_PHYSICAL would only be specified for small routines in those few programs that need it (and you'd better have make sure you really need it!) What I would like to see in a future version of the OS (1.5? 2.0?) that supports virtual memory, is a way to invoke a process such that it doesn't use VM nor do any tasks or processes it invokes. I think this would be the best way to maintain compatability with old programs. And you would only resort to this if the program didn't work in VM. At least this way you can try it in VM first! Then again, if VM isn't part of the official OS until 2.0 (where everythin's gonna break anyway!) the point becomes moot. As someone poined out in a previous posting, main(){printf("Hello World!\n");} is a multitasking program on a "proper" multitasking OS. It should also run in virtual memory *by default* OK now everyone tell me why I'm wrong... -Brian suggs%loki.pmel.edsg.hac.com@hac2arpa.hac.com
deven@rpi.edu (Deven Corzine) (05/19/89)
In article <8905181705.AA08260@postgres.Berkeley.EDU> dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) writes:
Hey Hey! A guy can make mistakes! You are right of course... all
(or most) of real memory should be under VM but have the ability to 'lock'
pages into memory as has been mentioned.
Well, at least you'll admit to them... :-)
The only problem is that AllocMem() requests are normally quite
small... much smaller than a page, so management becomes more difficult if
all of real memory is in VM. Still, it isn't much of a problem.
Sure, they're small, but that needn't pose a problem. Simply force
the pager to free pages (as necessary) enough to satisfy the request.
if there's no memory left, and 8 bytes are allocated, then it should
free a (presumably) 512 byte page from VM and allocate 8 bytes leaving
the other 504 still free. [this is ignoring system overhead,
clearly.] Of course, it should try to free pages which would merge
with the already-free memory for efficiency. [I consider memory
efficiency to be more important than speed efficiency, for the most
part.]
On this MEMF_PHYSICAL vs MEMF_VIRTUAL bit. Sure MEMF_PHYSICAL
is a nice way to say 'I want memory and it should be locked', and I'd
much rather have that than MEMF_VIRTUAL (which says 'I want this
memory to be pagable'), but really the only way to guarentee upward
compatibility is to make physical the default and MEMF_VIRTUAL the
exception. Thus have a MEMF_VIRTUAL AllocMem() flag and not a
MEMF_PHYSICAL flag.
It helps guarantee compatibility, but would tend to discourage its
use. If people have to go out of their way to lock the memory
allocated into real memory, then they're not normally likely to do it
unless there's a need to. Similarly, people aren't likely to add an
MEMF_VIRTUAL flag unless there's particular reason for it.
Particularly susceptible would be small programs and most anything
written by someone who has no MMU and does not expect to get one.
Keep in mind that many, many programmers are lazy. Most, to be sure.
It's hard enough to get them to stay within system guidelines well
enough not to crash the entire machine. Consider the issue of screen
sizes... people who use morerows (such as myself) will often get
tired of resizing new windows that applications create (such as, say,
DME... :-) to the real screen size instead of the program's assumed
screen size. The programmer could include code to check the actual
size of the screen, yet usually does not. Particularly if the
programmer doesn't use anything but a 640x200 screen. They often
don't support well any screen size approaching the possible 704x440 or
so you can get.
With that aside aside, you can expect that many programmers will
ignore a MEMF_VIRTUAL flag, leaving people WITH an MMU in trouble. I
suggest that a preferable solution would be to add a MEMF_LOCKED
(and/or MEMF_PHYSICAL) flag, and make VM the default unless explicitly
requested locked memory is allocated. For compatibility, perhaps the
best solution would be the ability to mark a particular task as being
locked in memory, so everything it allocates (as well as the code and
data) will automatically have the MEMF_LOCKED flag set. And, of
course, there would need to be an easy way to run an old program
locked in memory. [a utility program, probably, or maybe a
modification of the loader format?] This way, you could get unchanged
operation (at slightly more effort) on those few programs where it's
likely to *really* matter, (probably would need some trial and error)
and other programs could take advantage of virtual memory without
having to be recompiled or modified.
Deven
--
shadow@[128.113.10.2] <shadow@pawl.rpi.edu> Deven T. Corzine (518) 272-5847
shadow@[128.113.10.201] <shadow@acm.rpi.edu> 2346 15th St. Pi-Rho America
deven@rpitsmts.bitnet <userfxb6@rpitsmts> Troy, NY 12180-2306 <<tionen>>
"Simple things should be simple and complex things should be possible." - A.K.deven@rpi.edu (Deven Corzine) (05/19/89)
In article <8905181714.AA08425@postgres.Berkeley.EDU> dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) writes:
Now, unfortunately, you can't ro-page the text (code) segment
directly from an Amiga executable due to the relocation information
[...]
I don't see that that necessarily holds. Isn't the relocation
information just a list of the points in the file where you need to
add the base address where loaded? I don't see why you couldn't have
the MMU map the memory to start at 0 [as far as the program/processor
is concerned] and ignore the relocation information entirely. Now,
you might have trouble getting something of the sort to cooperate with
the operating system, as Exec/AmigaDOS tend to assume everything is in
a single large address space, but I don't see that it necessarily
CAN'T be done.
Let it be a challenge. :-)
Deven
--
shadow@[128.113.10.2] <shadow@pawl.rpi.edu> Deven T. Corzine (518) 272-5847
shadow@[128.113.10.201] <shadow@acm.rpi.edu> 2346 15th St. Pi-Rho America
deven@rpitsmts.bitnet <userfxb6@rpitsmts> Troy, NY 12180-2306 <<tionen>>
"Simple things should be simple and complex things should be possible." - A.K.dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) (05/19/89)
: Now, unfortunately, you can't ro-page the text (code) segment : directly from an Amiga executable due to the relocation information : [...] : :I don't see that that necessarily holds. Isn't the relocation :information just a list of the points in the file where you need to :add the base address where loaded? I don't see why you couldn't have :the MMU map the memory to start at 0 [as far as the program/processor :is concerned] and ignore the relocation information entirely. Now, Won't work, if only that every hunk is relative to 0 before relocation info is added. UNIX machines can usually direct-page-in from the text portion of the executable because there is no relocation info in the executable... it's all been completely resolved. :a single large address space, but I don't see that it necessarily :CAN'T be done. : :Let it be a challenge. :-) : :Deven I always like a challenge, but asking the paging system to munge around with the data that gets paged in dependant on the executable format is not something anybody is going to do... you gain nothing. -Matt
limonce@pilot.njin.net (Tom Limoncelli) (05/19/89)
Maybe this is too weird. Maybe there aren't many spare bits left in
the MEM_* flags (I don't have an Amiga here at work so I can't check
if it's a word or a long word) but why not have MEM_PHYSICAL *and*
MEM_VIRTUAL?
Obviously, it would be an error to include both and if you don't
include either it would default to MEM_PHYSICAL. Then C-A could
publish that when they release version x.y (2.0?) that the default
would be switch to MEM_VIRTUAL. This would give publishers time to
update their programs.
Maybe the switch doesn't have to be an a particular release, it could
be in preferences. You would only have to have it set to MEM_PHYSICAL
until you've received the last update on the last piece of your
software that doesn't support the "virtual standard". All that
publishers would have to do for the update is include MEM_PHYSICAL
xor MEM_VIRTUAL in every Alloc().
I think a good time for this would be in the 1.4 release. Since
Commodore is warning publishers that they may have to send "1.4
updates" to their users (because 1.4 may break some things) they might
as well be asked to update their Alloc() calls. They could write an
AWK script to just make them all MEM_PHYSICAL if they are lazy!
Whaddya think?
--
Tom Limoncelli -- tlimonce@drunivac.Bitnet -- limonce@pilot.njin.net
Drew University -- Box 1060, Madison, NJ -- 201-408-5389
Standard Disclaimer: I am not the mouth-piece of Drew Universityu555917504ea@minnie.ucdavis.edu (0040;0000001655;0;340;141;) (05/20/89)
Come on you guys, this is silly! Who cares whether VIRTUAL_MEM or PHYSICAL_MEM is the default? Just have it be user selectable, ala Fastmemfirst and nofastmem. That way, you can run a program, and if it doesn't like virtual memory, you can reboot and select VirtualMemLast or whatever... just my $.000002 -Bruce Rogers (currently u555917504ea@minnie.ucdavis.edu) _______________________________________________________________________________ Just a man whose circumstances went beyond his control....
peter@sugar.hackercorp.com (Peter da Silva) (05/20/89)
In article <8905160628.AA06371@jade.berkeley.edu>, 451061@UOTTAWA.BITNET (Valentin Pepelea) writes: > But this is another prime example of what should not be done. It is much > preferable to use a reserved fixed partition of a hard drive for speed. Files > are great when you wish to store and access data sequentially. But when fixed > size blocks (pages) have to be accessed randomly, going through the file system > will slow down the access time by a factor of ten, optimistically. So precalculate all the offsets to all the blocks in the file and store them in a table. Just use the file system to allocate the file. Even better, isn't it about time AmigaDOS borrowed a leaf from UNIX' book and included raw disk device drivers? RDF0: would be DF0: as a single 880K file. RHD0: would be HD0:, similarly allocated. Then you could have a partition, RHD3:, as your initial swap file. When you run out of space, add HD2:s/swapfile. -- Peter "Have you hugged your wolf today" da Silva `-_-' ...texbell!sugar!peter, or peter@sugar.hackercorp.com 'U`
peter@sugar.hackercorp.com (Peter da Silva) (05/20/89)
In article <8905170500.AA25917@jade.berkeley.edu>, 451061@UOTTAWA.BITNET (Valentin Pepelea) writes: > Unix is slow, Unix is big, Unix is cumbersome. UNIX is a program-O/S interface. The latest bunch of swiss-army-knife implementations are pretty big, but that's not a useful way to define UNIX. Once upon a time I was using a UNIX system with less horsepower and less RAM than an Amiga, shared with 8 other people. Tell me how small and compact AmigaOS is. The semantic difference between AmigaOS and UNIX is very small. And apart from fork(), all the advantages are on the side of UNIX. -- Peter "Have you hugged your wolf today" da Silva `-_-' ...texbell!sugar!peter, or peter@sugar.hackercorp.com 'U`
dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) (05/20/89)
:Come on you guys, this is silly! Who cares whether VIRTUAL_MEM or :PHYSICAL_MEM is the default? Just have it be user selectable, :ala Fastmemfirst and nofastmem. That way, you can run a program, :and if it doesn't like virtual memory, you can reboot and select :VirtualMemLast or whatever... : : just my $.000002 : :-Bruce Rogers (currently u555917504ea@minnie.ucdavis.edu) I think everyone is agreed that that would be even worse! You can't use select something like that. But I agree... which flag is used is just a minor convenience and the one that provides the most compatibility is the one that should be used (gee, guess which one that is? hint: V) -Matt
farren@well.UUCP (Mike Farren) (05/21/89)
limonce@pilot.njin.net (Tom Limoncelli) writes: >Maybe this is too weird. Maybe there aren't many spare bits left in >the MEM_* flags (I don't have an Amiga here at work so I can't check >if it's a word or a long word) but why not have MEM_PHYSICAL *and* >MEM_VIRTUAL? Or, why not have the default value used by AllocMem et al be definable? Like so: 1. Add a new protection bit - 'v'. (Or other mechanism - something older apps will not have set). 2. If 'v' flag is set, AllocMem assumes MEMF_VIRTUAL as default. If 'v' flag is not set, AllocMem assumes MEMF_PHYSICAL as default. Older programs don't have 'v' flag set, don't crash. Newer programs do, run fine. Also, non-VM systems can allocate all memory as physical. I'm assuming that this is naive - it's certainly off-the-cuff. Do you see problems? -- Mike Farren uucp: well!farren
deven@rpi.edu (Deven Corzine) (05/21/89)
In article <8905191401.AA15519@postgres.Berkeley.EDU> dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) writes:
Won't work, if only that every hunk is relative to 0 before relocation
info is added. UNIX machines can usually direct-page-in from the text portion
of the executable because there is no relocation info in the executable...
it's all been completely resolved.
Duh. Forgot that. Well, it could work if EVERYTHING were in a single
hunk... :-)
I always like a challenge, but asking the paging system to munge
around with the data that gets paged in dependant on the executable format
is not something anybody is going to do... you gain nothing.
Clearly. So, the solution would be to create a new executable format
from which you COULD ro-page the executable. Since the AmigaDOS
Standard Object File Format is insufficient for this purpose...
Deven
--
shadow@[128.113.10.2] <shadow@pawl.rpi.edu> Deven T. Corzine (518) 272-5847
shadow@[128.113.10.201] <shadow@acm.rpi.edu> 2346 15th St. Pi-Rho America
deven@rpitsmts.bitnet <userfxb6@rpitsmts> Troy, NY 12180-2306 <<tionen>>
"Simple things should be simple and complex things should be possible." - A.K.new@udel.EDU (Darren New) (05/22/89)
In article <11756@well.UUCP> farren@well.UUCP (Mike Farren) writes: >1. Add a new protection bit - 'v'. (Or other mechanism - something > >I'm assuming that this is naive - it's certainly off-the-cuff. Do you >see problems? Sure. What about code that wasn't loaded from a file? How about libraries, fonts, device drivers, etc? Where are you going to keep this default info? If you keep it in the task structure, then libraries will wind up allocating (say) virtual when they can only work with physical or vica versa. If you have a device driver, doesn't the I/O-Quick flag allow the possibility of the request completing in the callers task rather than the device-driver's task?What about printer drivers, which have different parts loaded from different files? If you can handle all of these problems, you probably don't need to hook anything to the files to start with. Goodness, you would think this was Unix-land, where every process has an associated file... :-) :-) -- Darren
dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) (05/22/89)
:In article <11756@well.UUCP> farren@well.UUCP (Mike Farren) writes:
:>1. Add a new protection bit - 'v'. (Or other mechanism - something
:
:>I'm assuming that this is naive - it's certainly off-the-cuff. Do you
:>see problems?
It is not a good idea to make whether the executable works or not
dependant on protection bits which are often NOT transfered when you transfer
files via modem or even copy them.
-Mattmcr@julie.UUCP (Michael Richardson) (05/22/89)
> (B) Open(file,1006) and write nothing yet. Assume you will be > able to extend the file as needed to 20MB, map 20MB worth of > * my favorite, medium difficulty. One would eventually want > to add a ftruncate() command to the filesystem to do this > right though. Mine too. However, going through the file system isn't the hottest idea ever invented. The blocks that the file occupies (and we are talking FFS here) would be placed in a table, and raw accesses to the disk could be made. This would, however, be an optimisation. There is no reason to reinvent the network disk (nd) and then get rid of it again. Masscomp RTU essentially does this by using a special file type called a "contiguous" file. (Because the blocks are all contiguous, i.e. in a row) [This type is there for dumping high speed incoming data to] > File fragmentation is not a problem.. the user can put the > file on a little-used or unused partition, or make a partition > specifically to hold the VM-file (but it is his choice!). But the size of the swap is still limited by the size of the partition. If the partition is unused, better to go direct, and how many users have "little-used" partitions that are not either 98.7% full, read-only or both... Still, user's choice is good. > The one advantage is that on system crash the OS can raw-write > VM to the swap partition without worrying about destroying > files (cause there are none). Using a file for VM means that > on system crash you cannot dump core. I don't we want to loose > the functionality of files for paging just so we can dump > system core. A contiguous file or block table gets you both. -- :!mcr!: Michael Richardson Amiga v--------+ UUCP: uunet!attcan!lsuc!nrcaer!julie!mcr | INTERNET mcr@doe.carleton.ca Fido: Michael Richardson @ 1:163/109.10<--+ Alter @ 7:483/109.10
karl@sugar.hackercorp.com (Karl Lehenbauer) (05/23/89)
In article <3834@sugar.hackercorp.com>, peter@sugar.hackercorp.com (Peter da Silva) writes: > The semantic difference between AmigaOS and UNIX is very small. And apart > from fork(), all the advantages are on the side of UNIX. ...except for realtime response. There is no standard Unix solution. There may be a few other things the Amiga arguably does better, as well. The way drivers are implemented on the Amiga may be better...I am thinking the Amiga mechanism may provide greater opportunities for concurrency than current monolithic kernel implementations of Unix. The driver programming environment is friendlier, too (more task-like...what a surprise ;-) -- Amiga drivers can sleep and stuff without hullabalutions. -- -- uunet!sugar!karl | "Woof!" -- Free Usenet BBS (713) 438-5018
karl@sugar.hackercorp.com (Karl Lehenbauer) (05/23/89)
I think VIRTUAL is an attribute that should be settable for tasks' code segments. (The means of defining working space size and such have yet to be defined. Perhaps it should be inherited from the CLI? Yeah, and it could be in the icon info for workbench launches.) It has been mentioned that MEMF_VIRTUAL requests must default to physical memory on machines without MMUs and that the existence of machines without MMUs is a compelling argument for why requests for virtual memory should be explicit. I am in wholehearted agreement. I think also that virtual memory under AmigaDOS is something that should mostly be explicitly under the control of the programmer. Many AmigaDOS tasks are highly realtime (games, music programs, *anything* that twiddles the display in an animated/copper list manner) and will simply totally screw up if they take page faults. But even a heavily realtime program could use VM judiciously if it was under explicit control (some requests are for virtual memory, others are not) of the program and the programmer had carefully considered what could safely be virtual. Mapped files would about always be OK as one has to do disk I/O for disk files anyway. (It goes without saying that only new programs could make use of this, but I'll say it anyway to prevent a followup that says it.) Also you need really excellent disk performance to do serious VM; that is, to take more than a few page faults a second and not thrash. Many MMU-equipped systems will fall down in this area, including mine. Finally, I think VM will be of limited usefulness to most people. While OS/2 weenies blather that they can have four gigabyte virtual tasks, they don't have the storage or the application to make use of it. Likewise, the Amiga is readily and reasonably affordably expandable to many megabytes and there are few programs that have need of that much. (I bet most people who want VM already have at least 3 MB of RAM.) I am not saying that VM will not be useful or that it should not be done, quite the contrary I think it is an interesting exercise, practically mandated by the existence of the 68851 on the A2620 (and on-chip MMU on the 68030s), merely that most Amigas will not have MMUs and fast-enough disks to see its widespread use. Bummer. You can't do mapped files without an MMU. That means mapped files would be not work with non-MMU Amigas. OK, you *could* do it if you didn't demand page the file in on non-MMU machines but rather copied the whole thing in on first open and then wrote the whole thing out on last close and maybe at checkpoint intervals or whatever. It's sick, I know. -- -- uunet!sugar!karl | "Woof!" -- Free Usenet BBS (713) 438-5018
peter@sugar.hackercorp.com (Peter da Silva) (05/23/89)
In article <3855@sugar.hackercorp.com>, karl@sugar.hackercorp.com (Karl Lehenbauer) writes: > In article <3834@sugar.hackercorp.com>, peter@sugar.hackercorp.com (Peter da Silva) writes: > > The semantic difference between AmigaOS and UNIX is very small. And apart > > from fork(), all the advantages are on the side of UNIX. > ...except for realtime response. There is no standard Unix solution. Well, when we get to talking about functionality of the systems there are lots of other differences... but I was mainly thinking about the semantics of the normal program interface. -- Peter "Have you hugged your wolf today" da Silva `-_-' ...texbell!sugar!peter, or peter@sugar.hackercorp.com 'U`
peter@sugar.hackercorp.com (Peter da Silva) (05/23/89)
In article <3856@sugar.hackercorp.com>, karl@sugar.hackercorp.com (Karl Lehenbauer) writes: > Bummer. You can't do mapped files without an MMU. That means mapped files > would be not work with non-MMU Amigas. OK, you *could* do it if you didn't > demand page the file in on non-MMU machines but rather copied the whole > thing in on first open and then wrote the whole thing out on last close and > maybe at checkpoint intervals or whatever. It's sick, I know. Actually VMS allows you to specify this sort of mapped file as well as the more normal sort, so I guess there MUST be some use for it. I mean, DEC wouldn't put something in nobody would ever use, now would they? -- Peter "Have you hugged your wolf today" da Silva `-_-' ...texbell!sugar!peter, or peter@sugar.hackercorp.com 'U`
scotth@sgi.com (Scott Henry) (05/23/89)
The reason I would like VM doesn't really have to do with demand-paging
from disk, it has to do with de-fragmenting physical memory! Especially
programs like SB-Prolog, which like to allocate huge contiguous blocks of
memory. I have had many occasions when I haven't rebooted for several
days, but have to because I want to run a program that needs/wants a
larger contiguous chunk of memory than I have left.
--
---------------------
Scott Henry <scotth@sgi.com>
#include <std_disclaimer.h>mph@behemoth.phx.mcd.mot.com (Mark Huth) (05/24/89)
In article <8905200449.AA20664@postgres.Berkeley.EDU> dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) writes: >:Come on you guys, this is silly! Who cares whether VIRTUAL_MEM or >:PHYSICAL_MEM is the default? Just have it be user selectable, ... > I think everyone is agreed that that would be even worse! You can't >use select something like that. But I agree... which flag is used is just Uh, I assume you me the royal everyone, meaning HM Dillon. I certainly do not agree that the user shouldn't be allowed to select the default memory allocation type. I think that most programs will work either way, provided that all chip memory allocations are presumed to be physical allocations. It seems to me, that data which the custom chips access directly must all be in chip memory, and of course, it cannot move once the pointers have been specified. Other DMA device drivers will require modifications to lock the virtual buffer pages into memory and to translate the pointers to physical addresses for the controller. Once the DMA is finished, the virtual view of memory may be used. Careful selection of the transition between virtual and physical space may eliminate additional state information requirements. That is, have the task context translate the address to physical via an OS trap somewhere in the task side of the IO calls, rather than forcing the driver task to do it. These problems arise from the fact that the 68851 is inserted to only allow the 68020 to be a logical master, all other masters are physical. The '030 allows no choice in the matter, since the MMU is buried on the chip. Logical bus masters are nice, but the Amiga architecture would require some serious mods to support it. Of course, then you could have lots of chip ram! Mark Huth
dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) (05/26/89)
I want to page to an arbitrary AmigaDOG device. For example, an NFS running under DNet (heh heh, just kidding :-)) While we all sneer at AmigaDOS I would like to point out that at the moment we are doing things on a whim that take other people (.. other computers) months and years to accomplish. We blithly talk about "oh, an NFS here, RAM disk there" as if such things were easy to integrate into the system. Which they are on the Amiga. Whatever VM goodies get implemented, one should remember that we have a good thing here in DOS even if we like to call it more colorful names. It took me no more than 11 hours to get a pretty-much-working NFS under DNet.... to the point where I can now 1> CD into a remote directory on a UNIX machine at berkeley and run an Amiga compatible binary from it (takes a while to load over 2400 baud, but it works!). -Matt
limonce@pilot.njin.net (Tom Limoncelli) (05/26/89)
You may not get this before you go away but I thought it might be
worth writing. Have you considered writing a serial.device-style
interface of DNet?
With this new "standard" [huh? I had planned all my serial software
this way from day one!] of devicename/unit-number you could "open"
dnet.device unit 0 thru 10 for an error free terminal session to your
local Unix box using anything (like ATalk III+) and use it's
emulations.
It might be a simple way to get Tektronics within DNet.
Just a thought I had last night, enjoy your trip!
-Tom
--
Tom Limoncelli -- tlimonce@drunivac.Bitnet -- limonce@pilot.njin.net
Drew University -- Box 1060, Madison, NJ -- 201-408-5389
Standard Disclaimer: I am not the mouth-piece of Drew Universitychk@dciem.dciem.dnd.ca (C. Harald Koch) (05/26/89)
In article <8905252017.AA29210@postgres.Berkeley.EDU> dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) writes: > It took me no more than 11 hours to get a pretty-much-working NFS >under DNet.... to the point where I can now 1> CD into a remote directory on >a UNIX machine at berkeley and run an Amiga compatible binary from it (takes >a while to load over 2400 baud, but it works!). This mainly says that you are an AmigaDOS guru. We all knew that already. :-) You have written several AmigaDOS device handlers already. You know from *hard won* experience where all the pitfalls are. You also have a huge library of example code and support routines to draw from. I once sat down to try to modify your example ram-disk handler to create a DNet RFS. I fairly quickly gave up because I didn't know enough about the deep magic inside AmigaDOS and filesystem handlers. A year of hacking later, I probably know enough to get something working. Sure, Amiga makes it easier than ever before to extend the OS. But "easier than ever before" != "easy". -- Grandpa Charnock's Law: | C. Harald Koch NTT Systems, Inc., Toronto, Ontario You never really learn | chk@gpu.utcs.utoronto.ca (long-term address) to swear until you learn | chk@zorac.dciem.dnd.ca (my current job) to drive. | chk@chkent.UUCP (my AMIGA at home)
karl@sugar.hackercorp.com (Karl Lehenbauer) (05/27/89)
In article <3859@sugar.hackercorp.com>, peter@sugar.hackercorp.com (Peter da Silva) writes: > In article <3856@sugar.hackercorp.com>, karl@sugar.hackercorp.com (Karl Lehenbauer) writes: > > Bummer. You can't do mapped files without an MMU. That means mapped files > > would be not work with non-MMU Amigas. OK, you *could* do it if you didn't > > demand page the file in on non-MMU machines but rather copied the whole > > thing in on first open and then wrote the whole thing out on last close and > > maybe at checkpoint intervals or whatever. It's sick, I know. > Actually VMS allows you to specify this sort of mapped file as well as the > more normal sort, so I guess there MUST be some use for it. Now that you mention it, this was known as "permanent common" on Plato. It consisted of data space you could write into, and you could set it up to checkpoint periodically as well as the write-on-close business. It was pretty cool. The first line of your program could be "number_of_access++", or the Tutor (ick) equivalent, and as long as you had initialized it properly, it would count number of entries to your game, or whatever. I would use this for some things if it was available on the Ami, like a records section, etc. What, another AllocMem memory type? It's not that hard to read it in and write it out explicitly anyway. Any RSX-11M/plus people out there remember when DEC added this write-out-on- close capability to RSX and didn't tell anybody? Our software relied on being able to get cold images of the commons from the load file, and did its own checkpointing. Needless to say it, broke in a big way, and was kind of hard to find... -- -- uunet!sugar!karl | "Woof!" -- free Usenet access: (713) 438-5018
dbk@teroach.UUCP (Dave Kinzer) (06/03/89)
Just saw someone over in the Macincrash group complaining that he believed the upcoming virtual memory was going to wear out his hard disk faster. There's no pleasing everybody, is there? ;-) | // GOATS - Gladly Offering All Their Support Dave Kinzer (602)897-3085| | // >> In Hell you need 4Mb to Multitask! << uunet!mcdphx!teroach!dbk | | \X/ #define policy_maker(name) (name->salary > 3 * dave.salary) |