weis@netmbx.UUCP (Dietmar Weis) (04/28/91)
Free(2) frees memory in a running process, but no other process can use it then. We have seen this with vmstat(8) and a test program which [cm]allocs in 1K steps and frees in 1K steps: Allocating memory increases the avm column and decreases the fre column. -ok- Freeing memory decreases avm but does NOT increase fre. -not ok- This is fatal I would say, because we have a process which allocates 7 MB (don't ask me why) and then frees them later on. But this doesn't succeed. Four such processes and system perfomance goes down because of paging and swapping. Or is free(2) designed to free memory on terminating the process? What is the algorithm, what is the clue ? Thank you very much Dietmar -- weis@netmbx.UUCP | Dietmar Weis DONOP CONSULT GmbH Voice: 030/884 28 54-0 | Uhlandstrasse 179/180 Fax: 030/882 55 29 | D - 1000 Berlin 12
mjr@hussar.dco.dec.com (Marcus J. Ranum) (04/29/91)
weis@netmbx.UUCP (Dietmar Weis) writes: >Free(2) frees memory in a running process, but no other process can use it >then. I assume you mean free(3) - actually, the library routine. malloc and free manage a memory pool, and request memory as it's needed from the underlying operating system (using brk/sbrk). Most of the implementations of free that are out there do not actually return the memory to the system but instead place it on an internal list where it can be re-allocated in chunks during subsequent calls to malloc. So, what happens, is that your process will grow to a steady state but will always be as large as the largest it got. >Allocating memory increases the avm column and decreases the fre column. This is because the internal pool used by malloc/free will run out of spare memory, and more gets requested from UNIX, to re-fill the pool. >Freeing memory decreases avm but does NOT increase fre. This is because the memory is placed back into the pool, and is never returned to UNIX. >This is fatal I would say, because we have a process which allocates 7 MB >(don't ask me why) and then frees them later on. If you can redesign the process so it doesn't do that, it'd be a big win. If you can't, you do have the choice of using sbrk to increase the process size, and brk to shrink it again when you're done. This means you'd get to re-implement some of the functionality of malloc, which is decidedly un-fun [though there is a good discussion of an allocator in K&R V2.0, pp 185-189]. mjr.
martelli@cadlab.sublink.ORG (Alex Martelli) (04/29/91)
weis@netmbx.UUCP (Dietmar Weis) writes:
...
:Free(2) frees memory in a running process, but no other process can use it
:then.
free(2) ? Try "man 2 free" and you'll get "No entry for free in section 2 of
the manual"... it's actually free(3), that is, a SUBROUTINE, *NOT* a system
call, and it's built on top of brk(2) [or sbrk(2), same manual entry:-].
That's not nitpicking - it may be the key to a solution for your problem: you
can rewrite the malloc/realloc/free routine-set, using brk(2) not just to get
more memory when needed, but also to give it back to the system when possible.
You can start with the memory-allocator example in Kernighan and Ritchie's
"The C Programming Language", or with any of several more advanced packages,
such as Conor Cahill's "Debugging-Malloc", Roberto Buratti's "High speed
memory allocator", Larry Wall's own malloc() (from the PERL package), etc
etc, all posted at one time or another on various sources groups.
--
Alex Martelli - CAD.LAB s.p.a., v. Stalingrado 53, Bologna, Italia
Email: (work:) martelli@cadlab.sublink.org, (home:) alex@am.sublink.org
Phone: (work:) ++39 (51) 371099, (home:) ++39 (51) 250434;
Fax: ++39 (51) 366964 (work only), Fidonet: 332/401.3 (home only).