[comp.unix.ultrix] Memory ``leaks'' in DEC servers

joel@pandora.pa.dec.com (Joel McCormack) (03/07/90)

I have received several replies about how to reproduce ``leaks'' in the
server, for which I am quite grateful.  Using some of the examples
provided, I can sure get some surprisingly large amounts of memory used,
especially using PostScript.  For example, I've run my server up to
about 10.5 megabytes.  Yuck.

In addition, there is evidently a real leak in PostScript that is known
about.  Unfortunately for you all, I haven't found any other, unknown
memory leak.  Allow me to clarify a few points:

(1) There is no way to give memory back to the kernel once it is
allocated.  So killing off all of your applications will not reduce the
total amount of memory consumed by the server.  It should reduce the
amount of memory needed to be resident.

(2) Running some new application may require more memory from the server
than it already has allocated, and you may find this number to be
unacceptably large.  This is usually caused by fragmentation.  While I
would like to improve general memory usage, this is a separate problem
from a memory leak.  Note that no X server that I know of uses a
compacting garbage collector, so fragmentation is bound to be a problem
to some extent in all servers.

(3) A memory leak means that you can cycle through some series of
actions, and the server will grow infinitely.  If you cannot give me
such an example, you are not demonstrating a leak.

(4) Using PostScript evidently uses up a lot of memory.  The people who
implemented the PostScript extension are looking into this.  In
addition, there is an actual leak in PostScript code in the UWS 2.2
server: Adobe thought we'd free the memory, we thought they would.

(5) Backing store uses up A LOT of memory.  Backing store allocates a
pixmap as large as your window, at 8 bits per pixel.  It is quite easy
to use up several megabytes this way.  In general, don't use backing
store unless it is quite expensive to repaint a window.

(6) Some incorrect clients can allocate resources in the server and not
free them.  The server can't reclaim any of the associated storage for
use by other clients until the offending client terminates.  The
specific program mentioned was the awm window manager.  This is
particularly bad, because you don't normally terminate a window manager.

So keep trying for a leak.  Also, if anyone has noticed something that
doesn't use PostScript or backing store, but uses a lot more memory than
it used to under earlier servers, I'm interested.  The server really
shouldn't be using any more storage for boring old X stuff than it used to.

I also mentioned to the appropriate people about how some of you would
like new servers with bug fixes made available on gatekeeper.  While I
think this is a fine idea given the way the product release cycle groans
along, they get all paranoid about bugs coming in on lots of different
versions of the server, or bugs coming in for other software that is
really caused by new buggy servers, etc.

- Joel McCormack (decwrl!joel, joel@decwrl.dec.com)

rusty@garnet.berkeley.edu (rusty wright) (03/08/90)

In article <2973@bacchus.dec.com> joel@pandora.pa.dec.com (Joel McCormack) writes:

   From: joel@pandora.pa.dec.com (Joel McCormack)
   Newsgroups: comp.unix.ultrix
   Subject: Memory ``leaks'' in DEC servers
   Date: 7 Mar 90 01:15:45 GMT
   Organization: DEC Western Research Laboratory

   (1) There is no way to give memory back to the kernel once it is
   allocated.  So killing off all of your applications will not reduce the
   total amount of memory consumed by the server.  It should reduce the
   amount of memory needed to be resident.

This isn't true, a program can give memory back to the kernel once it
is allocated; do "man brk".  The problem is in the implementation of
malloc, realloc, and free.  Notice in the Ultrix man page for malloc,
et. al. that for free it says

	The argument to free is a pointer to a block previously
	allocated by malloc. This space is made available for further
	allocation, but its contents are left undisturbed.

I don't know why they do this, but I think it has something to do with
realloc or old code that tries to do what realloc does using free and
malloc.

Presumably one could write a different free, that really frees the
memory and returns it to the kernel via brk.  Version 19 of gnu emacs
is supposed to have something that does this; emacs also has a problem
of growing without bound since it keeps the entire file in memory.
--

--------------------------------------
	rusty c. wright
	rusty@violet.berkeley.edu ucbvax!violet!rusty

bph@buengc.BU.EDU (Blair P. Houghton) (03/08/90)

In article <RUSTY.90Mar7105635@garnet.berkeley.edu> rusty@garnet.berkeley.edu (rusty wright) writes:
>In article <2973@bacchus.dec.com> joel@pandora.pa.dec.com (Joel McCormack) writes:
>   (1) There is no way to give memory back to the kernel once it is
>   allocated.  So killing off all of your applications will not reduce the
>   total amount of memory consumed by the server.  It should reduce the
>   amount of memory needed to be resident.
>
>This isn't true, a program can give memory back to the kernel once it
>is allocated; do "man brk".

While it is true that you can use brk(addr) to return data
space to the system, you have to be sure that you don't
need anything above addr before you do it.  Joel said he
didn't know of any X servers that do garbage collection and
compaction, so what you end up with are long stretches of
mostly free memory with a few valid bytes hanging around.
You can only brk() back down to the highest of these.

The question is, why aren't there any garbage collectors in
X servers?

				--Blair
				  "Dare I crosspost to comp.windows.x?
				   No, I durn't..."

michaud@decvax.dec.com (Jeff Michaud) (03/09/90)

> Presumably one could write a different free, that really frees the
> memory and returns it to the kernel via brk.  Version 19 of gnu emacs
> is supposed to have something that does this; emacs also has a problem
> of growing without bound since it keeps the entire file in memory.

	It's not that easy.  As Joel said, you also would need to
	do memory compaction.  You can't just use brk(2) because
	that sets the upper address for the data segment, and if
	there is any data still in use after the chunk of memory
	being free(3)'ed, someone's going to lose when they try
	to access the now non-existant memory.

	In order to easily do memory compaction, malloc would have
	to return a pointer to a pointer to the block of memory, and
	all references to the block of memory would have have to
	reference the block of memory through the indirect pointer.
	This way malloc can keep the indirect pointers table in
	the low part of the data segment, the rest of the heap
	can be compressed w/out anyone but malloc knowing.
	I believe this is how the Amiga and/or the Mac's memory
	allocators work?  Unfortunutly though, all of your application
	and all the libraries you use would have to use the new
	style memory allocator, or else if something else is using
	sbrk(2) to extend the data segment, then the new memory allocator
	can't safely reduce the data segment size.  Wouldn't it
	be nice it you could have holes in your data segment?
	(using shared memory is the closest you can get)

/--------------------------------------------------------------\
|Jeff Michaud    michaud@decwrl.dec.com  michaud@decvax.dec.com|
|DECnet-ULTRIX   #include <standard/disclaimer.h>              |
\--------------------------------------------------------------/