joel@pandora.pa.dec.com (Joel McCormack) (03/07/90)
I have received several replies about how to reproduce ``leaks'' in the server, for which I am quite grateful. Using some of the examples provided, I can sure get some surprisingly large amounts of memory used, especially using PostScript. For example, I've run my server up to about 10.5 megabytes. Yuck. In addition, there is evidently a real leak in PostScript that is known about. Unfortunately for you all, I haven't found any other, unknown memory leak. Allow me to clarify a few points: (1) There is no way to give memory back to the kernel once it is allocated. So killing off all of your applications will not reduce the total amount of memory consumed by the server. It should reduce the amount of memory needed to be resident. (2) Running some new application may require more memory from the server than it already has allocated, and you may find this number to be unacceptably large. This is usually caused by fragmentation. While I would like to improve general memory usage, this is a separate problem from a memory leak. Note that no X server that I know of uses a compacting garbage collector, so fragmentation is bound to be a problem to some extent in all servers. (3) A memory leak means that you can cycle through some series of actions, and the server will grow infinitely. If you cannot give me such an example, you are not demonstrating a leak. (4) Using PostScript evidently uses up a lot of memory. The people who implemented the PostScript extension are looking into this. In addition, there is an actual leak in PostScript code in the UWS 2.2 server: Adobe thought we'd free the memory, we thought they would. (5) Backing store uses up A LOT of memory. Backing store allocates a pixmap as large as your window, at 8 bits per pixel. It is quite easy to use up several megabytes this way. In general, don't use backing store unless it is quite expensive to repaint a window. (6) Some incorrect clients can allocate resources in the server and not free them. The server can't reclaim any of the associated storage for use by other clients until the offending client terminates. The specific program mentioned was the awm window manager. This is particularly bad, because you don't normally terminate a window manager. So keep trying for a leak. Also, if anyone has noticed something that doesn't use PostScript or backing store, but uses a lot more memory than it used to under earlier servers, I'm interested. The server really shouldn't be using any more storage for boring old X stuff than it used to. I also mentioned to the appropriate people about how some of you would like new servers with bug fixes made available on gatekeeper. While I think this is a fine idea given the way the product release cycle groans along, they get all paranoid about bugs coming in on lots of different versions of the server, or bugs coming in for other software that is really caused by new buggy servers, etc. - Joel McCormack (decwrl!joel, joel@decwrl.dec.com)
rusty@garnet.berkeley.edu (rusty wright) (03/08/90)
In article <2973@bacchus.dec.com> joel@pandora.pa.dec.com (Joel McCormack) writes:
From: joel@pandora.pa.dec.com (Joel McCormack)
Newsgroups: comp.unix.ultrix
Subject: Memory ``leaks'' in DEC servers
Date: 7 Mar 90 01:15:45 GMT
Organization: DEC Western Research Laboratory
(1) There is no way to give memory back to the kernel once it is
allocated. So killing off all of your applications will not reduce the
total amount of memory consumed by the server. It should reduce the
amount of memory needed to be resident.
This isn't true, a program can give memory back to the kernel once it
is allocated; do "man brk". The problem is in the implementation of
malloc, realloc, and free. Notice in the Ultrix man page for malloc,
et. al. that for free it says
The argument to free is a pointer to a block previously
allocated by malloc. This space is made available for further
allocation, but its contents are left undisturbed.
I don't know why they do this, but I think it has something to do with
realloc or old code that tries to do what realloc does using free and
malloc.
Presumably one could write a different free, that really frees the
memory and returns it to the kernel via brk. Version 19 of gnu emacs
is supposed to have something that does this; emacs also has a problem
of growing without bound since it keeps the entire file in memory.
--
--------------------------------------
rusty c. wright
rusty@violet.berkeley.edu ucbvax!violet!rusty
bph@buengc.BU.EDU (Blair P. Houghton) (03/08/90)
In article <RUSTY.90Mar7105635@garnet.berkeley.edu> rusty@garnet.berkeley.edu (rusty wright) writes: >In article <2973@bacchus.dec.com> joel@pandora.pa.dec.com (Joel McCormack) writes: > (1) There is no way to give memory back to the kernel once it is > allocated. So killing off all of your applications will not reduce the > total amount of memory consumed by the server. It should reduce the > amount of memory needed to be resident. > >This isn't true, a program can give memory back to the kernel once it >is allocated; do "man brk". While it is true that you can use brk(addr) to return data space to the system, you have to be sure that you don't need anything above addr before you do it. Joel said he didn't know of any X servers that do garbage collection and compaction, so what you end up with are long stretches of mostly free memory with a few valid bytes hanging around. You can only brk() back down to the highest of these. The question is, why aren't there any garbage collectors in X servers? --Blair "Dare I crosspost to comp.windows.x? No, I durn't..."
michaud@decvax.dec.com (Jeff Michaud) (03/09/90)
> Presumably one could write a different free, that really frees the > memory and returns it to the kernel via brk. Version 19 of gnu emacs > is supposed to have something that does this; emacs also has a problem > of growing without bound since it keeps the entire file in memory. It's not that easy. As Joel said, you also would need to do memory compaction. You can't just use brk(2) because that sets the upper address for the data segment, and if there is any data still in use after the chunk of memory being free(3)'ed, someone's going to lose when they try to access the now non-existant memory. In order to easily do memory compaction, malloc would have to return a pointer to a pointer to the block of memory, and all references to the block of memory would have have to reference the block of memory through the indirect pointer. This way malloc can keep the indirect pointers table in the low part of the data segment, the rest of the heap can be compressed w/out anyone but malloc knowing. I believe this is how the Amiga and/or the Mac's memory allocators work? Unfortunutly though, all of your application and all the libraries you use would have to use the new style memory allocator, or else if something else is using sbrk(2) to extend the data segment, then the new memory allocator can't safely reduce the data segment size. Wouldn't it be nice it you could have holes in your data segment? (using shared memory is the closest you can get) /--------------------------------------------------------------\ |Jeff Michaud michaud@decwrl.dec.com michaud@decvax.dec.com| |DECnet-ULTRIX #include <standard/disclaimer.h> | \--------------------------------------------------------------/