feighery@cyclone.mitre.org (P. D. Feighery) (06/29/91)
Hello, I was testing out the kernel environment on our risc 6k and ran into an interesting feature. I'm hoping someone could shed some light it. I wanted to make sure the IBM did garbage collection in the kernel environment, so I wrote a little program that would xmalloc memory from the kernel_heap. The program simply xmalloced until the xmalloc failed. Then I xfreed bits and pieces from various areas and tried to xmalloc the sum of the pieces I xfreed. Well, I had to xmalloc 230 Meg until the xmalloc failed. What I think is happening is that the IBM is paging back and forth to the regions on our disk which haven't been allocated to a particular partition. Is this true? Pat Feighery feighery@fluky.mitre.org
madd@world.std.com (jim frost) (06/30/91)
feighery@cyclone.mitre.org (P. D. Feighery) writes: >The program simply xmalloced until the xmalloc failed. Then I xfreed bits and >pieces from various areas and tried to xmalloc the sum of the pieces I xfreed. >Well, I had to xmalloc 230 Meg until the xmalloc failed. What I think is >happening is that the IBM is paging back and forth to the regions on our >disk which haven't been allocated to a particular partition. Is this true? No. Both xmalloc and malloc allow overcommit on the RS/6000. This means that you can allocate more space than you actually have swap available for and the allocation will succeed (up to a full segment, which is 256Mb, which is probably why it failed at 230Mb). The pages will not actually be allocated until they have been touched. If you touch enough pages to exceed available space, SIGDANGER will be sent to all processes. Processes are supposed to free whatever memory they can when this happens. If no one frees space, or not enough space is freed, in a reasonable amount of time (I'm not sure what "reasonable" is but it's not very long) then the kernel picks a process and kills it with SIGKILL. It appears that it kills the largest process that is not catching SIGDANGER. On my system that's usually either the linker or the X server. It's pretty annoying when it's the X server. It's been argued as to whether or not it was a good idea to allow overcommit. In my opinion it would have been nice to supply a different malloc interface that allowed overcommit (for those applications which really want it) rather than breaking nearly all current applications. Alas it's done and we have to live with it. If you retry your test but write a byte or a word every 1k (or some other number less than or equal to the page size of the machine -- I can't remember how big the pages are anymore) inside every allocated area, I believe you will see that SIGDANGER happens long before you hit the 256Mb mark (assuming that you don't have 200+Mb of swap -- but few people do). I'm not sure what will happen to the kernel if you're allocating in kernel space but it'll be readily apparent to user processes. Hope this sheds light on things. jim
benson@odi.com (Benson I. Margulies) (06/30/91)
horsefeathers. xmalloc of the paged heap is just using your paging region, whatever size it is. its paged. -- Benson I. Margulies