collinge@uvicctr.UUCP (Doug Collinge) (04/23/88)
Someone said that OS9 68k is capable of "preloading" commands; that is, loading some programs into memory and executing them there when invoked rather than loading them from disk. This seems to be yet another excellent idea incorporated in OS9 that no-one else seems to have thought of. I have always hated RAMdisks on the conceptual ground that any program loaded from RAMdisk occupies twice the memory that it needs - half for the executing image and half for the copy in the RAMdisk. (Yes, of course I use one...) Temporary files in RAMdisk are also stupid (the filesystem should handle this need in its buffering scheme) but less so, since only the data in the buffers is duplicated. Preloading commands eliminates the major stupidity and seems extremely easy to implement in any shell. Which brings me to my question: How difficult would it be to hack this into Gulaam? Perhaps pm would be kind enough to consider this in a future release. Or maybe he has a hard disk and doesn't care, in which case he might allow someone else to put it in for the rest of us. Maybe someone could point out this note to him if he doesn't see it. I love Gulaam and use it all the time. -- Doug Collinge School of Music, University of Victoria, PO Box 1700, Victoria, B.C., Canada, V8W 2Y2 collinge@uvunix.BITNET decvax!uw-beaver!uvicctr!collinge ubc-vision!uvicctr!collinge
jsp@sp7040.UUCP (John Peters) (04/27/88)
In article <390@uvicctr.UUCP>, collinge@uvicctr.UUCP (Doug Collinge) writes: > Someone said that OS9 68k is capable of "preloading" commands; that is, > loading some programs into memory and executing them there when invoked > rather than loading them from disk. This seems to be yet another > excellent idea incorporated in OS9 that no-one else seems to have > thought of. I have always hated RAMdisks on the conceptual ground that > any program loaded from RAMdisk occupies twice the memory that it needs > - half for the executing image and half for the copy in the RAMdisk. > (Yes, of course I use one...) Temporary files in RAMdisk are also > stupid (the filesystem should handle this need in its buffering scheme) > but less so, since only the data in the buffers is duplicated. > Preloading commands eliminates the major stupidity and seems extremely easy > to implement in any shell. Which brings me to my question: > > How difficult would it be to hack this into Gulaam? Perhaps pm would > be kind enough to consider this in a future release. Or maybe he has a > hard disk and doesn't care, in which case he might allow someone else > to put it in for the rest of us. Maybe someone could point out this > note to him if he doesn't see it. > > I love Gulaam and use it all the time. This would not be an easy hack. This is not a good idea on any system that does not swap or page. What happens when you run out of memory. Remember when people thought 2k bytes would never be all used up. Well I have already run into memory problems on accation (on the 1040ST) with MT-shell, a 100k ram disk, 30k of print spool buffer and acash installed when the c (MWC that is) tried to compile a large source file. What if the editor had been left in memory. No thanks. As for systems that provide the capability to leave executable text (UNIX terminology) as memory resident you just have to look at any real operating system. On UNIX if the sticky bit is set on a file then it stays resident after the first time it is involked. This is typacle of programs such as vi, ls, pwd and others that are often executed. On the VMS operating system this capabilty has been taken quite a bit further. As part of its real time facilitys, the user can make a system call that tells the O.S. to lock it in memory or increase the total available memory for that program (non pagable that is). You can even tell it whit segments of memory you can swap and which it can't. This is not a new concept but it will take a system that is much more sophisticated that TOS can ever dream of being. However, I expect that it is implemented in Idris. The redeming factor for the ST is the emminent release of Minix for it. There are allready ports of it for the PC that are swapping and paging kernels and I can see it implemented for the ST as a locigal extension. Then such thing will be possible to the average St user. Since Minix does not use the ROM's at any time after the boot cycle, the ST with it's more standardized hardware should provide a platform for Minix that will make it easy to port applications to. Mybe we can soon have the "sticky bit" on it also. I realize that I have rambled somewhat. That is the result of being in my last quarter of school and working too much at the same time. The ST is a great hardware platform that has suffered from a lack of operating system. Most of the question and grips I've seen on the net has stemed from this cause. I don't agree with Ataris marketing, but I realize that the only way to change things is for the users to do it themselves. That is why I like the idea of Minix. Basically for free you get a real operating system and the ability to extend it (i.e. Source Code). This will be an environment that will allow porting of much of the public domain unix software without major rewrite. Try doing something with TOS's file system, what a drag. Well enough of building for Minix. Hopefully by mid summer I'll see large numbers of requests for a "comp.os.minix.st" group and we can continue this discussion there. -- Johnnie -- What do you mean the last backup was in 1984?
wes@obie.UUCP (Barnacle Wes) (04/28/88)
In article <390@uvicctr.UUCP>, collinge@uvicctr.UUCP (Doug Collinge) writes: > Someone said that OS9 68k is capable of "preloading" commands; that is, > loading some programs into memory and executing them there when invoked > rather than loading them from disk. This seems to be yet another > excellent idea incorporated in OS9 that no-one else seems to have > thought of. Yes, that is one of the many good features of OS9. Another is pre-loaded modules. You can have "external" modules on OS9 - when your program calls such a module, the OS looks to see if it is in memory. If not, it searches for it in the current execution directory, and loads it. The OS manages the module tables all by itself. You can, of course, pre-load modules. Nice feature. > How difficult would it be to hack this into Gulaam? Probably VERY difficult. TOS is not setup to handle pre-loaded programs; whenever a program exits (via the Pterm or Pterm0 sys calls), the OS unloads all of the memory allocated to the program, including the text segment. This would be difficult to hack around. -- /\ - "Against Stupidity, - {backbones}! /\/\ . /\ - The Gods Themselves - utah-cs!uplherc! / \/ \/\/ \ - Contend in Vain." - sp7040!obie! / U i n T e c h \ - Schiller - wes
hase@netmbx.UUCP (Hartmut Semken) (04/29/88)
In article <390@uvicctr.UUCP> collinge@uvicctr.UUCP (Doug Collinge) writes: >Someone said that OS9 68k is capable of "preloading" commands; that is, >loading some programs into memory and executing them there when invoked >rather than loading them from disk. This seems to be yet another >excellent idea incorporated in OS9 that no-one else seems to have >thought of. OS/9 uses "modules" in RAM. If you envoke a program, it is loaded to the RAM as a module and gets execuded. The "load" system call loads a module from disk and it stays there. If you envoke a module already in RAM is is just executed, not loaded: thats all. Remember: OS/9 is no virtual memory system. hase -- Hartmut Semken, Lupsteiner Weg 67, 1000 Berlin 37 hase@netmbx.UUCP I think, you may be right in what I think you're thinking. (Douglas Adams)
leo@philmds.UUCP (Leo de Wit) (05/02/88)
In article <188@obie.UUCP> wes@obie.UUCP (Barnacle Wes) writes: >In article <390@uvicctr.UUCP>, collinge@uvicctr.UUCP (Doug Collinge) writes: >> Someone said that OS9 68k is capable of "preloading" commands; that is, >> loading some programs into memory and executing them there when invoked >> rather than loading them from disk. This seems to be yet another >> excellent idea incorporated in OS9 that no-one else seems to have >> thought of. > >Yes, that is one of the many good features of OS9. Another is >pre-loaded modules. You can have "external" modules on OS9 - when > [...stuff deleted...] >itself. You can, of course, pre-load modules. Nice feature. > >> How difficult would it be to hack this into Gulaam? > >Probably VERY difficult. TOS is not setup to handle pre-loaded programs; > [...stuff deleted...] Never used gemdos call 0x4b (Exec) with a mode of 4? You can have preloaded programs on the Atari ST as well, and it works quite nicely. I used it in a find command and in a shell I programmed for the Atari (yes, the Unix find command!), for example: find .. -type d ! -name . ! -name .. -exec ls -l {} \; (For those not familiar with find, this one finds all directories beneath the current directory's parent (excluding . and .. names) and does a ls -l on each one of them). Each time loading ls worked fine on the ramdisk I used to use, but once I started working on a hard disk, it becomes really slooooooooooowwww! Then I decided to preload the ls command (with gemdos(0x4b,3,"ls",0,0)). The gemdos function returns the basepage address of the loaded program. Now 'find', each time it has to 'ls', copies the argument string into the basepage; then it runs 'ls' by gemdos(0x4b,4,0,0,0) (I don't remember whether these are the correct values for the parameters). It is now (nearly) as fast as a builtin function! There are however a few problems (and I have not solved all of them): 1) When the parent program changes directories, the child stays in the directory it started in (the current directory and disk are saved somewhere on the basepage, start+55?). 2) There is also a problem with the memory; the loaded child obtains, as usual the rest of the memory available (in addresses basepage+4 to basepage+7 the address of the 'next free location' or something like that is found: 0xF8000 on my 520 ST+). Thus there is a problem with multiple preloaded childs or childs requesting additional memory. I've overcome this (sort of) by using setblock (gemdos 0x4a) and adjusting the 'next_free_location' ptrs on the childs basepages. It's better now, but still buggy (I don't know all about Gemdos memory management, I'm finding out now). 3) There could be problems as well with file descriptors (they're also on the basepage). 4) Initialized static and extern variables do not regain their initial value after a run. This is in consequence of the 'once-only' load. It could be overcome by a) always explicit initializing (by the child process), or b) make a copy of the initialized data space (by the parent process) and use this to re-initiate from the next time (more expensive, but not breaking existing code). If you manage to overcome these difficulties, it should not be very hard to code this preloading into a shell (I did it, having still these little problems ...). The shell maintains a list of preloaded programs, their pathname and their fat number and handles loading/executing depending on memory available, programs preloaded etc. Preloaded programs are almost as fast as functions built into the shell. If anybody has suggestions about the memory management part, or questions 'about the real code', or want examples, please RESPOND! A commented disassembly of the gemdos part of the O.S. would also be great .... leo.
wes@obie.UUCP (Barnacle Wes) (05/04/88)
In article <474@philmds.UUCP>, leo@philmds.UUCP (Leo de Wit) writes: > [...] > Then I decided to preload the ls command (with gemdos(0x4b,3,"ls",0,0)). > The gemdos function returns the basepage address of the loaded program. > Now 'find', each time it has to 'ls', copies the argument string into the > basepage; then it runs 'ls' by gemdos(0x4b,4,0,0,0) (I don't remember whether > these are the correct values for the parameters). It is now (nearly) as fast > as a builtin function! > > There are however a few problems (and I have not solved all of them): One of the problems is that GEMDOS will release all of the memory allocated to program, including that gotten in the original load/Mshrink process, when the program exits via Pterm or Pterm0. I don't know if the Pexec(,4,,,) call re-allocates that memory or not. You might want to experiment by loading a program, then finding the size of memory in the parent program. Exec the program, then again find the size of memory in the parent. If it goes up, DOS has freed the memory consumed by the child program. You might also want to make the child program find the free memory size and print it, too. If it goes up after the first exec, down during the second exec, and up after the second exec (get that?), the DOS is re-allocating the memory when you do Pexec(,4,,,). BTW, you find the "free memory size" is Malloc(-1L). It returns a (long) size of memory in bytes. -- /\ - "Against Stupidity, - {backbones}! /\/\ . /\ - The Gods Themselves - utah-cs!uplherc! / \/ \/\/ \ - Contend in Vain." - sp7040!obie! / U i n T e c h \ - Schiller - wes
fred@pnet01.cts.com (Fred Brooks) (05/05/88)
>Never used gemdos call 0x4b (Exec) with a mode of 4? >You can have preloaded programs on the Atari ST as well, and it works quite >nicely. I used it in a find command and in a shell I programmed for the Atari >(yes, the Unix find command!), for example: >find .. -type d ! -name . ! -name .. -exec ls -l {} \; >(For those not familiar with find, this one finds all directories beneath >the current directory's parent (excluding . and .. names) and does a ls -l >on each one of them). > >Each time loading ls worked fine on the ramdisk I used to use, but once I >started working on a hard disk, it becomes really slooooooooooowwww! >Then I decided to preload the ls command (with gemdos(0x4b,3,"ls",0,0)). >The gemdos function returns the basepage address of the loaded program. >Now 'find', each time it has to 'ls', copies the argument string into the >basepage; then it runs 'ls' by gemdos(0x4b,4,0,0,0) (I don't remember whether >these are the correct values for the parameters). It is now (nearly) as fast >as a builtin function! > >There are however a few problems (and I have not solved all of them): >1) When the parent program changes directories, the child stays in the directory >it started in (the current directory and disk are saved somewhere on the >basepage, start+55?). >2) There is also a problem with the memory; the loaded child obtains, as usual >the rest of the memory available (in addresses basepage+4 to basepage+7 the >address of the 'next free location' or something like that is found: 0xF8000 on >my 520 ST+). Thus there is a problem with multiple preloaded childs or childs >requesting additional memory. I've overcome this (sort of) by using setblock >(gemdos 0x4a) and adjusting the 'next_free_location' ptrs on the childs >basepages. It's better now, but still buggy (I don't know all about Gemdos >memory management, I'm finding out now). >3) There could be problems as well with file descriptors (they're also on the >basepage). >4) Initialized static and extern variables do not regain their initial value >after a run. This is in consequence of the 'once-only' load. It could be >overcome by a) always explicit initializing (by the child process), or >b) make a copy of the initialized data space (by the parent process) and use >this to re-initiate from the next time (more expensive, but not breaking >existing code). > >If you manage to overcome these difficulties, it should not be very hard to >code this preloading into a shell (I did it, having still these little problems >...). The shell maintains a list of preloaded programs, their pathname and >their fat number and handles loading/executing depending on memory available, >programs preloaded etc. Preloaded programs are almost as fast as functions >built into the shell. > >If anybody has suggestions about the memory management part, or questions >'about the real code', or want examples, please RESPOND! A commented disassembly >of the gemdos part of the O.S. would also be great .... > > leo. Is your shell PD or can you post it. I'm looking for a complete unix bourne or korn shell source so i can add the multitasking part with my MX2 kernel. UUCP: {cbosgd hplabs!hp-sdd sdcsvax nosc}!crash!pnet01!fred ARPA: crash!pnet01!fred@nosc.mil INET: fred@pnet01.cts.com