njk@diku.dk (Niels J|rgen Kruse) (08/12/90)
jdm5548@diamond.tamu.edu (James Darrell McCauley) writes: >Later, I read in a README file for a package called ZPLOT: > ... This is NOT a bug in zplot, > it is a known bug in Sun's system memory allocation algorithm > that arises from calls to malloc() and that's where it crashes > or misallocates memory. Sun says they will be fixing that in > their upcoming SunOS 4.1 release. SunOS 4.1 was installed here last week. Malloc is still flakey, but at least it dumps core less often. >Does anyone know about any bug in Sun's malloc()? Should I take my >problem to the >Sun newsgroup? I would really like to hear from ANYONE who can provide any >insight. This is the output from a benchmark for dynamic storage allocators, when linked with the SunOS malloc and with the data size limit set to 1000. It was run on a (otherwise idle) SS1. The benchmark is intended to test performance in applications, where steady state is never reached, with continuously changing requested size distribution and changing lifetime expectations. To compare storage allocators, i use a larger data size limit and average results from many runs with different seeds. Comments added manually are enclosed in [[ ]]. -------output begins Marathon [[ the name of the benchmark]] ; seed = 1. Average, maximum allocating length is 54.672, 2232. 2000 allocations, 1600 deallocations. usr+sys per call : 25.0 uSec sys per call : 2.8 uSec Average, maximum allocating length is 172.224, 13576. 4170 allocations, 3750 deallocations. usr+sys per call : 27.8 uSec sys per call : 6.3 uSec Average, maximum allocating length is 227.318, 8300. 6698 allocations, 6406 deallocations. usr+sys per call : 38.2 uSec sys per call : 12.2 uSec Average, maximum allocating length is 298.756, 29800. Stuck on allocation of 116 bytes. 4970 allocations, 3120 deallocations. usr+sys per call : 27.2 uSec sys per call : 1.2 uSec Total amount allocated when stuck = 909168 bytes. Total number allocated when stuck = 2979 blocks. Total number of timed calls when stuck = 32714. [[ This ]] Freelist table of contents = 3 * 112 bytes, 10 * 104 bytes, 6 * 96 bytes, 33 * 88 bytes, 5 * 80 bytes, 6 * 72 bytes, 13 * 64 bytes, 12 * 56 bytes, 8 * 48 bytes, 28 * 40 bytes, 12 * 32 bytes, 32 * 24 bytes, 22 * 16 bytes, 1 * 8 bytes. #blocks = 191, #bytes = 10208, average free block = 53.4 bytes. 0.125, 0.25, 0.375, 0.5, 0.75 percentiles = 32, 40, 64, 80, 88. [[ is printed using free_toc(3), which i posted to comp.sources.unix a while back. (It has not appeared yet). It is a 98%+ portable routine for printing the freelist table of contents for whatever dynamic storage allocator is being used. It has been tested with a large number of different storage allocators on various machines. To be exact, 10 different, not counting the SunOS one. ]] [[ This is where the benchmark used to dump core with the SunOS 4.0.3 malloc. So far everything looks fine. ]] Everything deallocated. usr+sys per call : 57.1 uSec sys per call : 23.5 uSec [[ Freeing seems to be hugely inefficient. My guess is that the large amount of sys time indicates, that free tries to reverse sbrk memory whenever there is free memory at the end of the heap. This should not affect the output of free_toc, as it pushes the heapsize to the limit and shows the additional memory as part of the freelist. ]] Freelist table of contents = 1 * 95672 bytes, 1 * 35904 bytes, 1 * 30376 bytes, 1 * 25864 bytes, 1 * 16928 bytes, 1 * 16672 bytes, 1 * 14800 bytes, 1 * 14176 bytes, 1 * 12976 bytes, 1 * 12072 bytes, [[ Zillions of numbers deleted for brevity. ]] 33 * 120 bytes, 3 * 112 bytes, 1 * 104 bytes, 2 * 96 bytes, 5 * 88 bytes, 1 * 80 bytes, 2 * 48 bytes, 2 * 40 bytes, 1 * 32 bytes, 1 * 24 bytes, 1 * 16 bytes. #blocks = 392, #bytes = 742704, average free block = 1894.7 bytes. 0.125, 0.25, 0.375, 0.5, 0.75 percentiles = 1008, 1784, 2648, 4040, 25864. [[ This is very wrong. When everything has been deallocated, there should only be one big free block, merged from all blocks. Also, the amount of free memory should be at least as much as has been freed, but it isn't. The number of free blocks is much smaller than what has been freed and some of the free blocks are larger than any ever allocated, so some blocks must have been merged. But where did the rest of the memory go? ]] free_toc bombed. [[ This is what the benchmark prints in frustration, when it can't compute a meaningfull number for memory utilization. ]] Total usr+sys = 1200 mSec, sys = 300 mSec, number of timed calls = 35693. Total usr+sys per call : 33.62 uSec. -------output ends I have not observed problems with the SunOS malloc, as long as no malloc call has failed (ie. returned null). Most programs, which just bail out as fast as possible when memory runs out, should therefore be safe. It is a bit disappointing that Sun can't produce a bugfree malloc. Oh well, what can you expect from commercial quality code. Fortunately, i don't have to use it, as i have my own storage allocator. Apart from the bugs, the SunOS malloc is a damned good performer, ie. it is almost as good as mine. I wonder what algorithm is being used. For comparison, i show output from the same benchmark, when testing my own storage allocator. I have no further comments, so readers not interested can hit 'n' now. ********* output begins Marathon; seed = 1. Average, maximum allocating length is 54.672, 2232. 2000 allocations, 1600 deallocations. usr+sys per call : 16.7 uSec sys per call : 5.6 uSec Average, maximum allocating length is 172.224, 13576. 4170 allocations, 3750 deallocations. usr+sys per call : 22.7 uSec sys per call : 6.3 uSec Average, maximum allocating length is 227.318, 8300. 6698 allocations, 6406 deallocations. usr+sys per call : 36.6 uSec sys per call : 8.4 uSec Average, maximum allocating length is 298.756, 29800. Stuck on allocation of 376 bytes. 5113 allocations, 3171 deallocations. usr+sys per call : 24.1 uSec sys per call : 4.8 uSec Total amount allocated when stuck = 923168 bytes. Total number allocated when stuck = 3071 blocks. Total number of timed calls when stuck = 32908. Freelist table of contents = 1 * 368 bytes, 1 * 116 bytes, 3 * 112 bytes, 6 * 108 bytes, 2 * 104 bytes, 3 * 100 bytes, 7 * 96 bytes, 38 * 92 bytes, 8 * 84 bytes, 5 * 80 bytes, 1 * 76 bytes, 3 * 72 bytes, 2 * 68 bytes, 5 * 64 bytes, 9 * 60 bytes, 1 * 56 bytes, 17 * 52 bytes, 10 * 48 bytes, 4 * 44 bytes, 21 * 40 bytes, 11 * 36 bytes, 8 * 32 bytes, 19 * 28 bytes, 7 * 24 bytes, 6 * 20 bytes. #blocks = 198, #bytes = 12412, average free block = 62.7 bytes. 0.125, 0.25, 0.375, 0.5, 0.75 percentiles = 40, 52, 64, 84, 92. Everything deallocated. usr+sys per call : 9.8 uSec sys per call : 0.0 uSec Freelist table of contents = 1 * 949904 bytes. #blocks = 1, #bytes = 949904, average free block = 949904.0 bytes. 0.125, 0.25, 0.375, 0.5, 0.75 percentiles = 949904, 949904, 949904, 949904, 949904. Fraction allocated when stuck : 97.1854%. Total usr+sys = 950 mSec, sys = 220 mSec, number of timed calls = 35979. Total usr+sys per call : 26.4043 uSec. ********* output ends -- Niels J|rgen Kruse DIKU Graduate njk@diku.dk