mcr@julie.UUCP (Michael Richardson) (01/23/89)
I was intending to port (my custom) a lisp interpreter now running on a SUN to the
Amiga this month. The project has been put off for awhile because I realise
I can't do it without a hard disk. But a couple of questions:
I currently get (4K) blocks aligned on 4K boundaries from SunOS using
memalign (a variation on malloc). The SUN malloc likes to store the size
of the block in the word preceeding the block, and I would waste
4094 bytes of space every time I allocated a block. Until I write/find
a better malloc, I kludge things and allocate 4094 bytes instead.
(I used to just advance brk() by about 500K to make my arena, but
realising I was going to port it, I changed that recently.)
I use the middle 8 bits
uuuu uuuu uuuu tttt tttt oooo oooo oooo
u - upper bits
t - page number (used to index into a table to determine object type)
o - page offsets
to determine the type of the page, so I need to allocate the pages
on 2^12=4K boundaries.
I have discovered that the Manx malloc is a fake - is uses a 40K
arena which is allocated at run time regardless of how much malloc'ing
you intend to do. (Do any of the library functions call malloc? I'd like
to stay away from them if possible)
The normal AllocMem is fine, I DO know the block size, but it wont
guarantee a 4K boundary. I have thought about AllocMem'ing 8K, taking the
middle 4K chunk and returning the rest, but I am not sure if this will
work. Does AllocMem keep any information of blocks that are in use? If
so, can I do things in a revision-independant way? Or do I have to
walk to memory lists myself? (Worse solution.)
Thanks!
--
:!mcr!:
Michael Richardson Amiga
v--------+
UUCP: uunet!attcan!lsuc!nrcaer!julie!mcr | INTERNET mcr@doe.carleton.ca
Fido: Michael Richardson @ 1:163/109.10<--+ Alter @ 7:483/109.10jesup@cbmvax.UUCP (Randell Jesup) (01/26/89)
In article <0164.AA0164@julie> mcr@julie.UUCP (Michael Richardson) writes: > I currently get (4K) blocks aligned on 4K boundaries from SunOS using >memalign (a variation on malloc). The SUN malloc likes to store the size >of the block in the word preceeding the block, and I would waste >4094 bytes of space every time I allocated a block. Until I write/find >a better malloc, I kludge things and allocate 4094 bytes instead. >(I used to just advance brk() by about 500K to make my arena, but >realising I was going to port it, I changed that recently.) >I use the middle 8 bits > uuuu uuuu uuuu tttt tttt oooo oooo oooo > u - upper bits > t - page number (used to index into a table to determine object type) > o - page offsets > > to determine the type of the page, so I need to allocate the pages >on 2^12=4K boundaries. > > I have discovered that the Manx malloc is a fake - is uses a 40K >arena which is allocated at run time regardless of how much malloc'ing >you intend to do. (Do any of the library functions call malloc? I'd like >to stay away from them if possible) ICK! I'd advise considering Lattice 5.0: optimizer, source debugger, REAL malloc, etc. > The normal AllocMem is fine, I DO know the block size, but it wont >guarantee a 4K boundary. I have thought about AllocMem'ing 8K, taking the >middle 4K chunk and returning the rest, but I am not sure if this will >work. Does AllocMem keep any information of blocks that are in use? If >so, can I do things in a revision-independant way? Or do I have to >walk to memory lists myself? (Worse solution.) You could try this: AllocMem 8K, get a pointer to your 4K block, then FreeMem the part before it and after it. This is legal, though not usually encouraged (see the memory section of the EXEC RKM, it's discussed there). You could also grab several at once, minimising fragmentation. -- Randell Jesup, Commodore Engineering {uunet|rutgers|allegra}!cbmvax!jesup
ditto@cbmvax.UUCP (Michael "Ford" Ditto) (01/26/89)
In article <0164.AA0164@julie> mcr@julie.UUCP (Michael Richardson) writes: > The normal AllocMem is fine, I DO know the block size, but it wont >guarantee a 4K boundary. I have thought about AllocMem'ing 8K, taking the >middle 4K chunk and returning the rest, but I am not sure if this will >work. Yes, you can return parts of an allocated block, but it is important to know that the system will always round the address down to a multiple of MEM_BLOCKSIZE and the size is similarly rounded up. In your case this probably fits in will with what you are doing. Since MEM_BLOCKSIZE (in <exec/memory.h>) is a constant (8), it presumably will not change in an incompatible way in future releases. -- -=] Ford [=- "The number of Unix installations (In Real Life: Mike Ditto) has grown to 10, with more expected." ford@kenobi.cts.com - The Unix Programmer's Manual, ...!sdcsvax!crash!elgar!ford 2nd Edition, June, 1972. ditto@cbmvax.commodore.com
w-colinp@microsoft.UUCP (Colin Plumb) (01/26/89)
Yes, you can FreeMem() only part of an AllocMem()'d chunk. I kind of wish there was a better way to do this, though... Allocating lots and freeing a couple of bits you don't need is a great way to fragment memory. -- -Colin (uunet!microsof!w-colinp)
jbwaters@bsu-cs.UUCP (J. Brian Waters) (01/27/89)
In article <0164.AA0164@julie>, mcr@julie.UUCP (Michael Richardson) writes: > > I have discovered that the Manx malloc is a fake - is uses a 40K > arena which is allocated at run time regardless of how much malloc'ing How did you determine this? The source to Manx's malloc shows it calling lmalloc which in turn calls AllocMem with the size of your request plus room for a tag to allow the clean up routine to free it on exit. -- Brian Waters <backbone>!{iuvax|pur-ee}!bsu-cs!jbwaters uunet!---/
mcr@julie.UUCP (Michael Richardson) (01/27/89)
> The normal AllocMem is fine, I DO know the block size, but it wont >guarantee a 4K boundary. I have thought about AllocMem'ing 8K, taking the >middle 4K chunk and returning the rest, but I am not sure if this will >work. Does AllocMem keep any information of blocks that are in use? If >so, can I do things in a revision-independant way? Or do I have to >walk to memory lists myself? (Worse solution.) The good news is that Dave Thomas sent me a message to tell me that AllocMem does no tracking of in use blocks. AllocMem'ing 8K will work! Wow! _Personal_ replies! 8-) :-) 8^)... -- :!mcr!: Michael Richardson Amiga v--------+ UUCP: uunet!attcan!lsuc!nrcaer!julie!mcr | INTERNET mcr@doe.carleton.ca Fido: Michael Richardson @ 1:163/109.10<--+ Alter @ 7:483/109.10 -- :!mcr!: Michael Richardson Amiga v--------+ UUCP: uunet!attcan!lsuc!nrcaer!julie!mcr | INTERNET mcr@doe.carleton.ca Fido: Michael Richardson @ 1:163/109.10<--+ Alter @ 7:483/109.10
scotth@harlie.SGI.COM (Scott Henry) (01/27/89)
From article <5471@bsu-cs.UUCP>, by jbwaters@bsu-cs.UUCP (J. Brian Waters): > In article <0164.AA0164@julie>, mcr@julie.UUCP (Michael Richardson) writes: >> >> I have discovered that the Manx malloc is a fake - is uses a 40K >> arena which is allocated at run time regardless of how much malloc'ing > > How did you determine this? The source to Manx's malloc shows it calling > lmalloc which in turn calls AllocMem with the size of your request plus room > for a tag to allow the clean up routine to free it on exit. > > -- > Brian Waters <backbone>!{iuvax|pur-ee}!bsu-cs!jbwaters > uunet!---/ I turns out that Manx has TWO malloc()s... the "normal" one that (eventually) calls AllocMem() each time, and one in heapmem.o which works out of (defaults to) 40KB chunks (this one includes a realloc()). If you don't link in heapmem.o you get indirect AllocMem() calls. BTW, I don't understand why Manx only includes a realloc() in heapmem.o. -- Scott Henry <scotth@harlie.sgi.com> #include <std_disclaimer.h>
jim@b11.INGR.COM (Jim Levie ) (01/27/89)
In article <0164.AA0164@julie> mcr@julie.UUCP (Michael Richardson) writes: > ... stuff deleted ... > > I have discovered that the Manx malloc is a fake - is uses a 40K >arena which is allocated at run time regardless of how much malloc'ing >you intend to do. (Do any of the library functions call malloc? I'd like >to stay away from them if possible) I don't know how you determined this, but a quick peek at the library source says that each malloc(size) actually calls lmalloc((unsigned long)size) which in turn ultimately does an "AllocMem(size+sizeof(struct mem), 0L)", where struct mem is declared as: struct mem { struct mem *next; long size; } The mem struct is of course necessary for exit() cleanup and so forth. Amazingly handy those library sources!! > > :!mcr!: > Michael Richardson Amiga > v--------+ > UUCP: uunet!attcan!lsuc!nrcaer!julie!mcr | INTERNET mcr@doe.carleton.ca > Fido: Michael Richardson @ 1:163/109.10<--+ Alter @ 7:483/109.10 -- =+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ Jim Levie REMTECH Inc Huntsville, Al The opinions expressed above are just that. Ph. (205) 536-8581 email: uunet!ingr!b11!jim
higgin@cbmvax.UUCP (Paul Higginbottom MKT) (01/30/89)
In article <0164.AA0164@julie> mcr@julie.UUCP (Michael Richardson) writes: > I have discovered that the Manx malloc is a fake - is uses a 40K >arena which is allocated at run time regardless of how much malloc'ing >you intend to do. That limied malloc is through heapmem.o. If you don't link with heapmem.o you use a malloc from the library which does use AllocMem(). Paul.
mcr@julie.UUCP (Michael Richardson) (01/31/89)
>In article <0164.AA0164@julie>, mcr@julie.UUCP (Michael Richardson) writes: >> >> I have discovered that the Manx malloc is a fake - is uses a 40K >> arena which is allocated at run time regardless of how much malloc'ing > >How did you determine this? The source to Manx's malloc shows it calling >lmalloc which in turn calls AllocMem with the size of your request plus room >for a tag to allow the clean up routine to free it on exit. > >-- >Brian Waters <backbone>!{iuvax|pur-ee}!bsu-cs!jbwaters > uunet!---/ (I also read the other reply in case you are wondering) Page 11-50 of my badly ordered, misindexed, and half missing Manx manual says (in reference to malloc): "It allocates a buffer from the first large enough free block that it encounters. If this search fails, it calls sbrk to get more memory for use by these functions." Page 11-11 (sbrk): "When first called, sbrk gets a block of memory by calling the Amiga function AllocMem. The default size of this block is 40K bytes." ... "ERRORS If an sbrk request would make the sbrk pointer go past the end of sbrk's block of memory, sbrk will return -1 as its value, without modyfying its pointer." BTW: Wouldn't be surprised if my manual is wrong... Regardless, there isn't a proper memalign function. (nor even an inefficient one. - I'll go the AllocMem/FreeMem route - maybe with bigger blocks, I don't know - my program will eventually FreeMem all the blocks it allocates after a garbage collection and reallocate more memory afterward. This might slowly compact the heap if nothing else is busy doing the same.) Also: My manual is missing the fexec functions... (exec() 'See Also's them) Will this work? freopen("in","r",stdin); (==NULL - error...) error=fexec("myprog","myprog","myarg",0); if(error) ... Thanks! -- :!mcr!: Michael Richardson Amiga v--------+ UUCP: uunet!attcan!lsuc!nrcaer!julie!mcr | INTERNET mcr@doe.carleton.ca Fido: Michael Richardson @ 1:163/109.10<--+ Alter @ 7:483/109.10
cks@ziebmef.uucp (Chris Siebenmann) (02/01/89)
In article <25616@sgi.SGI.COM> scotth@harlie.SGI.COM (Scott Henry) writes: ... >It turns out that Manx has TWO malloc()s... the "normal" one that (eventually) >calls AllocMem() each time, and one in heapmem.o which works out of (defaults >to) 40KB chunks (this one includes a realloc()). If you don't link in >heapmem.o you get indirect AllocMem() calls. BTW, I don't understand why >Manx only includes a realloc() in heapmem.o. Unfortunately, on Unix systems, the sequence free(p) ... operations not involving memory allocation ... q = realloc(p,size) will work, and worse still, some programs rely on this (it's even documented in the manual entry for realloc() on BSD and some/all System V systems). Manx's normal free() calls FreeMem(), making it impossible for realloc() to later reliably recover the contents of that memory. The heapmem.o routines allocate memory out of a fixed-size block that is never FreeMem()'d, and can thus guarantee Unix realloc() semantics. The good news is that few programs actually use this property of realloc(), and that a realloc() without this property that works with the standard malloc() is fairly easy to write. If your program does use this property of realloc(), I urge you to rewrite it; you'll be able to use Amiga memory much more effectively. -- "Though you may disappear, you're not forgotten here And I will say to you, I will do what I can do" Chris Siebenmann uunet!{utgpu!moore,attcan!telly}!ziebmef!cks cks@ziebmef.UUCP or .....!utgpu!{,ontmoh!,ncrcan!brambo!}cks
hugh@censor.UUCP (Hugh Gamble) (02/02/89)
In article <0164.AA0164@julie>, mcr@julie.UUCP (Michael Richardson) writes: > > I was intending to port (my custom) a lisp interpreter now running on a SUN to the > Amiga this month. The project has been put off for awhile because I realise > I can't do it without a hard disk. But a couple of questions: [questions, really the whole message, deleted] > :!mcr!: > Michael Richardson Amiga > v--------+ > UUCP: uunet!attcan!lsuc!nrcaer!julie!mcr | INTERNET mcr@doe.carleton.ca > Fido: Michael Richardson @ 1:163/109.10<--+ Alter @ 7:483/109.10 My background project for the year is trying to port Kyoto Common LISP to the Amiga. My target minimum configuration will be a 3Meg machine with HD. The porting will be done on a 5Meg '020 machine. If in the course of porting your lisp you develop some general tools or techniques for SunOS to AmigaDOS porting, I'm sure it would be appreciated if you post them to the net. -- Hugh D. Gamble (416) 581-4354 {lsuc, utzoo}!censor!hugh (Std. Disclaimers) I don't want to live in a beer commercial, I just want to play with some of the girls from one. :^)
toebes@sas.UUCP (John Toebes) (02/08/89)
In article <1989Feb1.001209.27677@ziebmef.uucp> cks@ziebmef.UUCP (Chris Siebenmann) writes: >In article <25616@sgi.SGI.COM> scotth@harlie.SGI.COM (Scott Henry) writes: >>It turns out that Manx has TWO malloc()s... > Unfortunately, on Unix systems, the sequence > > free(p) > ... operations not involving memory allocation ... > q = realloc(p,size) > >will work, and worse still, some programs rely on this (it's even >documented in the manual entry for realloc() on BSD and some/all >System V systems). > > The good news is that few programs actually use this property of >realloc(), and that a realloc() without this property that works with >the standard malloc() is fairly easy to write... >Chris Siebenmann The bad news is that programs that rely upon this behavior are actually very difficult to detect. Too many UN*X programs rely upon this well defined and almost universally supported feature. With the Lattice compiler, we have gon to great lengths to support this. As you point out, it is indeed pretty easy to write but somewhat harder to get right as there are several subtle points to the interaction. In fact, this also works: while(p!= NULL) { free(p); p = p->next; } And is required to work for any UN*X compatible memory manager. If you think about it, this is actually a useful feature (but not a practice that I would recommend) that can make coding easier when you are just hacking out a fast solution. I don't recommend using it but did want to point out that you can expect this to work on any UN*X compatible system (otherwise you may spend a lot of time chasing some very obscure bugs). /*---------------------All standard Disclaimers apply---------------------*/ /*----Working for but not officially representing SAS or Lattice Inc.-----*/ /*----John A. Toebes, VIII usenet:...!mcnc!rti!sas!toebes-----*/ /*------------------------------------------------------------------------*/
crash@jc3b21.UUCP (Frank J. Edwards) (02/09/89)
In article <1989Feb1.001209.27677@ziebmef.uucp> cks@ziebmef.UUCP (Chris Siebenmann) writes: >>In article <25616@sgi.SGI.COM> scotth@harlie.SGI.COM (Scott Henry) writes: >> >> free(p) >> ... operations not involving memory allocation ... >> q = realloc(p,size) >> >>will work, ... >>Chris Siebenmann > [some deleted...] Too many UN*X programs rely upon this well > defined and almost universally supported feature. With the Lattice compiler, > we have gon to great lengths to support this. As you point out, it is indeed > pretty easy to write but somewhat harder to get right as there are several > subtle points to the interaction. In fact, this also works: > while(p!= NULL) > { > free(p); > p = p->next; > } > And is required to work for any UN*X compatible memory manager. If you think > about it, this is actually a useful feature (but not a practice that I would > recommend) that can make coding easier when you are just hacking out a fast > solution. I don't recommend using it but did want to point out that you > can expect this to work on any UN*X compatible system (otherwise you may spend > a lot of time chasing some very obscure bugs). Ahem, ... I'm probably going to be shot for this, but: only expect that behaviour on a BSD-like UN*X system. As I understand it, BSD does not return free'd memory (as in the sbrk() system call) back to the system. That's why the reference will still work. (Of course, on a BSD machine you can allocate 4MB of RAM then free 3MB while the operating system doesn't actually make that 3MB available to other tasks... Wonder what database managers do about that?!) System V machines, however, actually remove those memory pages from the tasks allocation tables -- the memory is GONE! At least from the standpoint of your task, it is. I don't know of any software written for System V which uses this "feature". (But then again I'm somewhat of a hermit ;-) This does not preclude the possibility of an srbk() call which does not free up an entire page, however. Say a machine uses a page size of 4K. That entire page belongs to the task until an sbrk() occurs which specifies an address logically less than the beginning of said page. Well, I've stuck my neck out -- does anyone know differently??? Frank "Crash" Edwards "No one knows the trouble I've seen..."
ditto@cbmvax.UUCP (Michael "Ford" Ditto) (02/10/89)
Time to consider following-up to comp.unix.wizards ... In article <567@jc3b21.UUCP> crash@jc3b21.UUCP (Frank J. Edwards) writes: >> while(p!= NULL) { free(p); p = p->next; } >Ahem, ... I'm probably going to be shot for this, but: only expect that >behaviour on a BSD-like UN*X system. As I understand it, BSD does not >return free'd memory (as in the sbrk() system call) back to the system. I don't know of any malloc() implementation for Unix that gives back freed memory. It is just assumed that either the program will malloc it again or that those pages will be paged out by the OS. This is almost always a reasonable thing to do. (A Unix program which is permanently freeing a large chunk of memory probably should have gotten it via sbrk() in the first place, and can attempt to free it in the same way. This similarly applies to Amiga programs with respect to malloc() vs. AllocMem(), especially considering that there is very little overhead involved in AllocMem compared to a Unix system call.) >System V machines, however, actually remove those memory pages from >the tasks allocation tables -- the memory is GONE! This is not true with the supplied malloc libraries. Such a library could be written, but I doubt it would be of much use, for reasons that are not really related to comp.sys.amiga.* subject matter. -- -=] Ford [=- "The number of Unix installations (In Real Life: Mike Ditto) has grown to 10, with more expected." ford@kenobi.cts.com - The Unix Programmer's Manual, ...!sdcsvax!crash!elgar!ford 2nd Edition, June, 1972. ditto@cbmvax.commodore.com
dillon@POSTGRES.BERKELEY.EDU (Matt Dillon) (02/10/89)
:Ahem, ... I'm probably going to be shot for this, but: only expect that :behaviour on a BSD-like UN*X system. As I understand it, BSD does not :return free'd memory (as in the sbrk() system call) back to the system. :That's why the reference will still work. (Of course, on a BSD machine :you can allocate 4MB of RAM then free 3MB while the operating system :doesn't actually make that 3MB available to other tasks... Wonder what :database managers do about that?!) You are right and you are wrong. BSD 4.2/4.3 (and most UNIX systems) has .. TADA! Demand-Paged- Virtual-Memory. That 3MB that was freed takes up space on the SWAP device, but not necessarily in physical memory. In fact, unless you actually use what you malloc() (for large malloc'd chunks) in a UNIX system, it will NEVER be brought into physical memory. Unused or infrequently used portions of allocated, stack, & text space does not take up physical memory ... what little it does take up will be paged out most of the time... faster if the system is running out of physical memory to play with. Needless to say, most systems have a LOT of swap ... in the hundreds of megabytes or even Gig(gle)bytes. >Well, I've stuck my neck out -- does anyone know differently??? > >Frank "Crash" Edwards >"No one knows the trouble I've seen..." -Matt