karlm@epsilon.tih.no (Karl Martin Lund) (05/02/91)
I've had some problems while using the free() function in SAS C version 5.10. It doesn't seem to free the memory to the system before the program exits. It does, however, free the memory in a way such that the same program can use it again when malloc is called. Has anybody any ideas of whats's the problem, or is it made that way to minimize problems with memory fragmetation?
I am doing a lot of big, temporary memory allocations which I want to free to the system after use. Does anyone know a way to do this in a ANSI compatible way?
Thanks for any help! __
/ /
KMLUND __ / /
\ \/ /
\/_/ictoryjseymour@medar.com (James Seymour) (05/04/91)
In article <1991May02.143148.22732@edb.tih.no> karlm@epsilon.tih.no (Karl Martin Lund) writes: > >I've had some problems while using the free() function in SAS C version >5.10. It doesn't seem to free the memory to the system before the program >exits. It does, however, free the memory in a way such that the same >program can use it again when malloc is called. Has anybody any ideas of >whats's the problem, or is it made that way to minimize problems with >memory fragmetation? That is normal behaviour. When memory is free'd it is returned to a local memory pool (frequently referred to as an arena) for later re-use by the same process (program). Adjacent free blocks are merged (this is referred to as coalescing) so that any subsequent larger requests can be satisfied from the local memory pool. >I am doing a lot of big, temporary memory allocations which I want to >free to the system after use. Does anyone know a way to do this in a >ANSI compatible way? No. There is no ANSI compatable (or even *remotely* portable) way to return un-used memory to the system [other than process termination :-)]. Such a beast would be highly operating system and compiler dependent. The problems you are encountering are not unique to either the SAS package or the Amiga environment. The same set of "problems" exist on nearly every system I've worked with (or, at least, those I can remember), other than the real-time ones. (Thank the powers that be and the good engineers at Motorola and Commodore Amiga that you aren't limited by some inane 640k limit!) >Thanks for any help! __ > / / >KMLUND __ / / > \ \/ / > \/_/ictory Not much help, but you're welcome anyway... -- Jim Seymour | Medar, Inc. ...!uunet!medar!jseymour | 38700 Grand River Ave. jseymour@medar.com | Farmington Hills, MI. 48331 CIS: 72730,1166 GEnie: jseymour | FAX: (313)477-8897
mwm@pa.dec.com (Mike (My Watch Has Windows) Meyer) (05/04/91)
In article <1991May03.225751.26999@medar.com> jseymour@medar.com (James Seymour) writes:
No. There is no ANSI compatable (or even *remotely* portable) way to
return un-used memory to the system [other than process termination :-)].
Such a beast would be highly operating system and compiler dependent.
Process termination might do the trick - after the setup, save state,
start your second half, then exit.
It's pretty striaghtforward to write a couple of routines that map
malloc/free to AllocMem/FreeMem, which might be acceptable in this
case. It'll give you the behavior wanted for the large hunks, anyway.
If you're doing lots of frees/allocs later on, you'll take a
performance hit.
<mike
--
Es brillig war. Die schlichte Toven Mike Meyer
Wirrten und wimmelten in Waben; mwm@pa.dec.com
Und aller-mumsige Burggoven decwrl!mwm
Die mohmem Rath' ausgraben.chopps@ro-chp.UUCP (Chris Hopps) (05/05/91)
In article <1991May02.143148.22732@edb.tih.no> karlm@epsilon.tih.no (Karl Martin Lund) writes: > >I've had some problems while using the free() function in SAS C version 5.10. >It doesn't seem to free the memory to the system before the program exits. >It does, however, free the memory in a way such that the same program can use >it again when malloc is called. Has anybody any ideas of whats's the problem, >or is it made that way to minimize problems with memory fragmetation? yes. the free function releases memory into a pool for re-use, for compatibility with some unix systems it doesn't return it to the system until exit. >I am doing a lot of big, temporary memory allocations which I want to free to >the system after use. Does anyone know a way to do this in a ANSI compatible >way? By ANSI def. you can only (m|c|re)alloc 64k blocks and less. RTFM :^) SAS/C Volume 2, Page L19-22. >Thanks for any help! __ > / / >KMLUND __ / / > \ \/ / > \/_/ictory Chris... -- ------------------------ The Royal Oak Chophouse Chris Hopps Royal Oak, Michigan ....umich!wsu-cs!ro-chp!chopps cs.wayne.edu!ro-chp!chopps
dillon@overload.Berkeley.CA.US (Matthew Dillon) (05/05/91)
In article <MWM.91May3182036@raven.pa.dec.com> mwm@pa.dec.com (Mike (My Watch Has Windows) Meyer) writes: >In article <1991May03.225751.26999@medar.com> jseymour@medar.com (James Seymour) writes: > No. There is no ANSI compatable (or even *remotely* portable) way to >... > >It's pretty striaghtforward to write a couple of routines that map >malloc/free to AllocMem/FreeMem, which might be acceptable in this >case. It'll give you the behavior wanted for the large hunks, anyway. >If you're doing lots of frees/allocs later on, you'll take a >performance hit. > > <mike Actually, AllocMem & FreeMem are *faster* than SAS/C's malloc & free. The only loose is that your system is more susceptible to fragmentation. The only other thing I notice sometimes is that on program exit it may take a little while longer (2 seconds instead of 1) to free the memory if the program has made a few thousand allocations. DICE currently uses AllocMem() & FreeMem() for malloc(). I will probably end up using Allocate & Deallocate in a private memory pool for small allocations, but leave larger ones to AllocMem() & FreeMem(). -Matt >-- >Es brillig war. Die schlichte Toven Mike Meyer >Wirrten und wimmelten in Waben; mwm@pa.dec.com >Und aller-mumsige Burggoven decwrl!mwm >Die mohmem Rath' ausgraben. -- Matthew Dillon dillon@Overload.Berkeley.CA.US 891 Regal Rd. uunet.uu.net!overload!dillon Berkeley, Ca. 94708 USA
davewt@NCoast.ORG (David Wright) (05/06/91)
In article <1991May02.143148.22732@edb.tih.no> karlm@epsilon.tih.no (Karl Martin Lund) writes: >I've had some problems while using the free() function in SAS C version 5.10. >It doesn't seem to free the memory to the system before the program exits. >It does, however, free the memory in a way such that the same program can use >it again when malloc is called. Has anybody any ideas of whats's the problem, >or is it made that way to minimize problems with memory fragmetation? >I am doing a lot of big, temporary memory allocations which I want to free to >the system after use. Does anyone know a way to do this in a ANSI compatible >way? You really should be using the Amiga-specific calls to do this, AllocMem() and FreeMem(). These will do what you want, and are needed if you want to specify flags such as MEMF_CHIP, MEMF_PUBLIC, MEMF_CLEAR, etc. the m/alloc() and free() functions are just provided for ANSI compliance and compatibility with Unix C compilers, and will hold onto the memory until program exit assuming you might want it again (under ANSI C you are supposed to be able to alloc a block of memory, free it, and alloc it again without losing the contents, and some programs make use of this, so it is required for the SAS/C functions to work similarly). Dave
mwm@pa.dec.com (Mike (My Watch Has Windows) Meyer) (05/07/91)
In article <dillon.7247@overload.Berkeley.CA.US> dillon@overload.Berkeley.CA.US (Matthew Dillon) writes: >If you're doing lots of frees/allocs later on, you'll take a >performance hit. > > <mike Actually, AllocMem & FreeMem are *faster* than SAS/C's malloc & free. The only loose is that your system is more susceptible to fragmentation. Matt, I did some tests on 1.3, doing lots of small mallocs/frees vs. doing lots of AllocMem/FreeMems. Malloc/free was _much_ faster. The critical condition is that you're reusing space, and not getting new space from the system. If you're not reusing the space, then AllocMem & FreeMem will be faster. If you're going to do a bunch of allocations, then a bunch of frees, a single AllocEntry (I think that's the name I want) call is probably fastest. <mike -- Cheeseburger in paradise Mike Meyer Making the best of every virtue and vice mwm@pa.dec.com Worth every damn bit of sacrifice decwrl!mwm To get a cheeseburger in paradise
mjl@alison.at (Martin J. Laubach) (05/07/91)
In article <dillon.7247@overload.Berkeley.CA.US>, Matthew Dillon writes: | Actually, AllocMem & FreeMem are *faster* than SAS/C's | malloc & free. Nevertheless AllocMem() is quite slow when you are doing more than a few allocations. I was able to speed a program of mine doing about 14000 small allocs from something like two minutes to about 30 seconds by using a simple pool manager. Thus: Use AllocMem() as long as you like, but when doing lots of similar small allocs, use a custom pool routine. mjl
dillon@overload.Berkeley.CA.US (Matthew Dillon) (05/07/91)
In article <chopps.0579@ro-chp.UUCP> chopps@ro-chp.UUCP (Chris Hopps) writes: >In article <1991May02.143148.22732@edb.tih.no> karlm@epsilon.tih.no (Karl Martin Lund) writes: >> >>.. >>the system after use. Does anyone know a way to do this in a ANSI compatible >>way? > >By ANSI def. you can only (m|c|re)alloc 64k blocks and less. > >RTFM :^) >SAS/C Volume 2, Page L19-22. Under ANSI malloc taks a size_t ... Maybe on an IBM-PC compiler you are limited to 64K, but certainly not on Amiga and UNIX systems. -Matt >-- > ------------------------ > The Royal Oak Chophouse Chris Hopps > Royal Oak, Michigan ....umich!wsu-cs!ro-chp!chopps > cs.wayne.edu!ro-chp!chopps -Matt -- Matthew Dillon dillon@Overload.Berkeley.CA.US 891 Regal Rd. uunet.uu.net!overload!dillon Berkeley, Ca. 94708 USA
jseymour@medar.com (James Seymour) (05/07/91)
In article <chopps.0579@ro-chp.UUCP> chopps@ro-chp.UUCP (Chris Hopps) writes: > >By ANSI def. you can only (m|c|re)alloc 64k blocks and less. > >RTFM :^) >SAS/C Volume 2, Page L19-22. > Actually it seems this is incorrect. I made the assumption that the docs in the SAS (Lattice, actually) manual were accurate, but when I could find *no* supporting docs for the 64k "limit" anywhere (incl. a copy of the ANSI draft standard and K&R 2nd Ed.) I asked SAS Tech. Support about it and they agreed that this item was in error. I suspect that the 64k thing came about because Lattice (who *used* to be responsible for the Amiga product as well) also makes a compiler for MeSsy-DOS and there *would* be a 64k limit on anything except (I think) large and huge model programs. Why Lattic referred to it as an "ANSI Standard" is anyones guess. Can anyone else comment further on this? Did I miss something? Is there *really* an ANSI-imposed 64k malloc limit? This hardly seems reasonable to me. Heck, if I wanted 64k limits, I coulda bought a MeSsy-DOS box (shudder). > >Chris... >-- -- Jim Seymour | Medar, Inc. ...!uunet!medar!jseymour | 38700 Grand River Ave. jseymour@medar.com | Farmington Hills, MI. 48331 CIS: 72730,1166 GEnie: jseymour | FAX: (313)477-8897
dillon@overload.Berkeley.CA.US (Matthew Dillon) (05/08/91)
In article <MWM.91May6120013@raven.pa.dec.com> mwm@pa.dec.com (Mike (My Watch Has Windows) Meyer) writes: >In article <dillon.7247@overload.Berkeley.CA.US> dillon@overload.Berkeley.CA.US (Matthew Dillon) writes: > >If you're doing lots of frees/allocs later on, you'll take a > >performance hit. > > > > <mike > > Actually, AllocMem & FreeMem are *faster* than SAS/C's > malloc & free. The only loose is that your system is more > susceptible to fragmentation. > >Matt, > >I did some tests on 1.3, doing lots of small mallocs/frees vs. doing >lots of AllocMem/FreeMems. Malloc/free was _much_ faster. The critical >condition is that you're reusing space, and not getting new space from >the system. If you're not reusing the space, then AllocMem & FreeMem >will be faster. > >If you're going to do a bunch of allocations, then a bunch of frees, a >single AllocEntry (I think that's the name I want) call is probably >fastest. > > <mike In the experiments I had made, AllocMem() was faster, it was FreeMem() that was slower. At least under 2.0 ... I suppose the speed depends on the overall fragmentation of system memory, something that does not effect Allocate() on a private memory pool. As far as allocating lots of like-sized structures, I don't use malloc() *or* AllocMem(). That is, I use one of the two to allocate a large block, and then allocate the like-sized structures out of the block manually. In this case freeing is accomplished by simply adding the structure to a freelist. The result goes an order of magnitude faster than malloc()/free() OR AllocMem()/FreeMem(). -Matt >-- >Cheeseburger in paradise Mike Meyer >Making the best of every virtue and vice mwm@pa.dec.com >Worth every damn bit of sacrifice decwrl!mwm >To get a cheeseburger in paradise -- Matthew Dillon dillon@Overload.Berkeley.CA.US 891 Regal Rd. uunet.uu.net!overload!dillon Berkeley, Ca. 94708 USA
chopps@ro-chp.UUCP (Chris Hopps) (05/08/91)
>In article <chopps.0579@ro-chp.UUCP> chopps@ro-chp.UUCP (Chris Hopps) writes: >>In article <1991May02.143148.22732@edb.tih.no> karlm@epsilon.tih.no (Karl Martin Lund) writes: >> >>By ANSI def. you can only (m|c|re)alloc 64k blocks and less. > > Under ANSI malloc taks a size_t ... Maybe on an IBM-PC compiler you > are limited to 64K, but certainly not on Amiga and UNIX systems. > > -Matt Well SAS/C says the above, this doesn't mean its true though, I just took it as such. Page L20, volume II "By ANSI definition, calloc, malloc, and realloc can only allocate 64K at a time." But then I went to K&R second edition (certainly not exhaustive) and they said nothing about it. Just to be picky... yes malloc takes the type size_t as its only argument, but size_t is of type unsigned int, thus on the amiga, if you are using 16 bit ints the maximum is 65535 (64k) of course if you are using SAS/C defualt, int is defined as 32 bit, which is a much larger number, something to be aware of though. On this same point ANSI demands that type unsigned int's max value be at least 65535, but guarentees no more, thus if the strive is for portable code (why else would you use malloc :^) ) then 64k is a good assumption. Chris... -- ------------------------ The Royal Oak Chophouse Chris Hopps Royal Oak, Michigan ....umich!wsu-cs!ro-chp!chopps cs.wayne.edu!ro-chp!chopps
allbery@NCoast.ORG (Brandon S. Allbery KB8JRR/AA) (05/09/91)
As quoted from <1991May07.164814.2538@medar.com> by jseymour@medar.com (James Seymour): +--------------- | In article <chopps.0579@ro-chp.UUCP> chopps@ro-chp.UUCP (Chris Hopps) writes: | >By ANSI def. you can only (m|c|re)alloc 64k blocks and less. | >RTFM :^) | >SAS/C Volume 2, Page L19-22. | | *really* an ANSI-imposed 64k malloc limit? This hardly seems reasonable | to me. Heck, if I wanted 64k limits, I coulda bought a MeSsy-DOS box | (shudder). +--------------- ANSI is in the business of codifying the maximum capabilities that are common to every system they can find and calling it a "standard"... 64K may well be it, thanks to that bloody Intel-based hack you mentioned. :-( ++Brandon -- Me: Brandon S. Allbery Ham: KB8JRR/AA 10m,6m,2m,220,440,1.2 Internet: allbery@NCoast.ORG (restricted HF at present) Delphi: ALLBERY AMPR: kb8jrr.AmPR.ORG [44.70.4.88] uunet!usenet.ins.cwru.edu!ncoast!allbery KB8JRR @ WA8BXN.OH
dillon@overload.Berkeley.CA.US (Matthew Dillon) (05/10/91)
In article <chopps.0593@ro-chp.UUCP> chopps@ro-chp.UUCP (Chris Hopps) writes: > >Just to be picky... yes malloc takes the type size_t as its only argument, but >size_t is of type unsigned int, thus on the amiga, if you are using 16 bit >ints the maximum is 65535 (64k) of course if you are using SAS/C defualt, int >is defined as 32 bit, which is a much larger number, something to be aware of >though. > >On this same point ANSI demands that type unsigned int's max value be at least >65535, but guarentees no more, thus if the strive is for portable code (why >else would you use malloc :^) ) then 64k is a good assumption. size_t is not necessarily an unsigned int, else they would have simply specified an unsigned int. size_t exists specifically so that 16 bit int compilers can typedef it to something *other* than an unsigned int. Frankly, in my opinion, any compiler that uses 16 bit ints for things like file seek positions, read()/write() size parameter, malloc(), and the like, are broken. Most IBM-PC compilers, such as TurboC, declare size_t is unsigned, i.e. 16 bit unsigned integer, and do exactly the above (except for seek positions). Of course, these compilers do not really support 32 bit ints well anyway, mainly because processors <386 don't really support 32 bit ints. For example, if you use a 32 bit integer as an index for an array, Turbo C will silently *IGNORE* the high 16 bits. Oops. I have found that I can declare 'short's in SAS/C and DICE and even though these are 32 bit int compilers, they do a pretty good job optimizing operations to stay within 16 bits when possible. If one is worried about efficiency and speed, one can use registered arguments which gets around basic differences in stacking (e.g. always stacking 32 bit arguments). The last time I used 16 bit integers was 5 years ago. >Chris... >-- > ------------------------ > The Royal Oak Chophouse Chris Hopps > Royal Oak, Michigan ....umich!wsu-cs!ro-chp!chopps > cs.wayne.edu!ro-chp!chopps -Matt -- Matthew Dillon dillon@Overload.Berkeley.CA.US 891 Regal Rd. uunet.uu.net!overload!dillon Berkeley, Ca. 94708 USA
comeau@ditka.Chicago.COM (Greg Comeau) (05/10/91)
In article <1991May07.164814.2538@medar.com> jseymour@medar.com (James Seymour) writes: >Can anyone else comment further on this? Did I miss something? Is there >*really* an ANSI-imposed 64k malloc limit? This hardly seems reasonable >to me. Heck, if I wanted 64k limits, I coulda bought a MeSsy-DOS box >(shudder). There is *NO* such ANSI limit. What ANSI tells us is: void *malloc(size_t size) "The ``malloc'' function allocates space for an object whose size is specified by ``size'' and whose value is indeterminate." That says it all. If you want more ammo, investigating size_t: "``size_t'' is the unsigned integral type of the result of the ``sizeof'' operator". (BTW, size_t is also "the type of integer required to hold the maximum size of an array"). The bottom line? It's implementation dependent. - Greg -- Comeau Computing, 91-34 120th Street, Richmond Hill, NY, 11418 Producers of Comeau C++ 2.1 Here:attmail.com!csanta!comeau / BIX:comeau / CIS:72331,3421 Voice:718-945-0009 / Fax:718-441-2310
jseymour@medar.com (James Seymour) (05/12/91)
[This thread is leading in a direction not appropriate to the present newsgroup IMHO, so note followup-to] In article <1991May8.235041.17893@NCoast.ORG> allbery@ncoast.ORG (Brandon S. Allbery KB8JRR/AA) writes: |As quoted from <1991May07.164814.2538@medar.com> by jseymour@medar.com (James Seymour): |+--------------- || In article <chopps.0579@ro-chp.UUCP> chopps@ro-chp.UUCP (Chris Hopps) writes: || >By ANSI def. you can only (m|c|re)alloc 64k blocks and less. || >RTFM :^) || >SAS/C Volume 2, Page L19-22. || || *really* an ANSI-imposed 64k malloc limit? This hardly seems reasonable || to me. Heck, if I wanted 64k limits, I coulda bought a MeSsy-DOS box || (shudder). |+--------------- | |ANSI is in the business of codifying the maximum capabilities that are common |to every system they can find and calling it a "standard"... 64K may well be |it, thanks to that bloody Intel-based hack you mentioned. :-( | Heh... This is similar to the argument made by my manager (as to why 64k may really be an ANSI standard). I argued against it - allowing as how restricting the rest of the world to the limitations of a particular chip- set (the Intel one in particular) was unreasonable. [ ranting, raving, & rambling follows...] This is why I bought an Amiga and why I go out of my way to avoid the MS-DOS boxes and MS-DOS-oriented projects at work. After programming for *real* operating systems (UNIX/Xenix, various real-time, and AmigaDOS), I simply detest MS-DOS (and just about anything related to it or ported from it). Perhaps I've developed an unreasonable aversion to anything even closely related to MS-DOS, IBM-PCs and their clones, etc., but I'd almost rather program for CP/M. At least it doesn't pretend to be something it isn't. I try to write portable code, and stick with ANSI compatable stuff when possible, but I've decided to ignore MS-DOS when writing code because I don't care to place limitations on what I can do based on MS-DOS limitations and Intel iAPX-86 (previous to '386 protected mode) capabilities (or lack thereof). |++Brandon |-- -- Jim Seymour | Medar, Inc. ...!uunet!medar!jseymour | 38700 Grand River Ave. jseymour@medar.com | Farmington Hills, MI. 48331 CIS: 72730,1166 GEnie: jseymour | FAX: (313)477-8897