guy@auspex.auspex.com (Guy Harris) (11/25/89)
After the fix I posted earlier is applied so that the new Andrew "malloc" detects that it's being built for a SPARC and aligns blocks on 8-byte boundaries, nothing seems to work; applications drop core shortly after they're started up. It happens while they're doing static loads; part of the data structure pointed to by ClassList is overwritten by some data from an "index" file (possibly "/usr/andrew/dlib/atk/index"). The overwritten part was OK before the explosion (I'd stuck in code to call "class_DumpAllClassInfo to "stdout" at the end of "class_EnterInfo"). If I put the old version of "imalloc.h" back, so that it doesn't notice it's being built for a SPARC, this doesn't happen (although, of course, "malloc"ed blocks don't get put on 8-byte boundaries, so "malloc"ed blocks,including commons from ".do" files, that contain "double"s cause bus errors). I haven't investigated any further.
wjh+@ANDREW.CMU.EDU (Fred Hansen) (11/28/89)
Excerpts from mail: 25-Nov-89 Andrew "malloc" on SPARC do.. Guy Harris@uunet.uu.net (905) > After the fix I posted earlier is applied so that the new Andrew > "malloc" detects that it's being built for a SPARC and aligns blocks on > 8-byte boundaries, nothing seems to work; Oops. A couple of problems with boundary alignment padding still lurked. I fixed them and cleaned up a few little things. The diffs are below, should anyone care to experiment. The testing has been more thorough this time, though I have not tested it on a Sparc. Notice that the test for machine type has been moved to the Imakefile. If it is a Sparc, the symbol WORD is defined to have value 8. Otherwise the value 4 is defined within imalloc.h. With this version using -DMSTATS and MALLOC_DEBUG_ENV, the malloc information can be accessed from ATK applications by typing control-X, \200, m for statistics control-X, \200, t for plumber data (\200 is the Insert key on an RT. I don't know how to generate it on other workstations. If you want to rebind the functions to control-X,m,s and control-X,m,t, you can put the following in .atkinit: addkey im-print-malloc-statistics ^Xms view addkey im-print-malloc-table ^Xmt view The plumber data is excellent for finding core leaks; for each place in the code where a call to malloc exists, the plumber data shows the number of blocks existing, their total size, the range of sizes, and the range of sequence numbers. A call may be the source of a coreleak if it has a large total size, a large number of blocks, and a long range of sequence numbers. Fred Hansen diff -c andrew/overhead/malloc/Imakefile ./Imakefile *** andrew/overhead/malloc/Imakefile Mon Nov 6 01:16:36 1989 --- ./Imakefile Tue Nov 28 08:27:15 1989 *************** *** 5,10 **** --- 5,16 ---- DependTarget() + #ifdef sys_sun4_40 + DEFINES = -DWORD=8 -DMSTATS + #else + DEFINES = -DMSTATS + #endif + #ifdef ANDREW_MALLOC_ENV NormalObjectRule() LibraryTarget(libmalloc.a, malloc.o plumber.o) diff -c andrew/overhead/malloc/imalloc.h ./imalloc.h *** andrew/overhead/malloc/imalloc.h Mon Nov 6 01:16:31 1989 --- ./imalloc.h Mon Nov 27 17:04:43 1989 *************** *** 28,43 **** #ifndef _MALLOCITC_ #define _MALLOCITC_ #define INT long ! /* structs need 8 byte alignment on SPARC */ ! #ifdef sys_sun4_40 ! #define WORD 8 ! #else /* sys_sun4_40 */ #define WORD 4 ! #endif /* sys_sun4_40 */ #define SIZEOFINT 4 #define SIZEOFCHARSTAR 4 #define SEGGRAIN 4096 /* granularity for sbrk requests (in bytes) */ #if WORD % SIZEOFINT WORD must be a multiple of SIZEOFINT #endif --- 28,52 ---- #ifndef _MALLOCITC_ #define _MALLOCITC_ + #ifndef INT #define INT long ! #endif /* INT */ ! ! #ifndef WORD ! /* for SPARC the Makefile has "-DWORD=8" */ #define WORD 4 ! #endif /* WORD */ ! ! #ifndef SIZEOFINT #define SIZEOFINT 4 + #endif /* SIZEOFINT */ + + #ifndef SIZEOFCHARSTAR #define SIZEOFCHARSTAR 4 + #endif /* SIZEOFCHARSTAR */ + #define SEGGRAIN 4096 /* granularity for sbrk requests (in bytes) */ + #if WORD % SIZEOFINT WORD must be a multiple of SIZEOFINT #endif *************** *** 63,70 **** #ifndef IDENTIFY - /* the two low order bits of the Size fields are used for ACTIVE and PREACTIVE */ - #if SIZEOFINT % WORD #define PADHEADER \ int padding[(WORD - SIZEOFINT%WORD) / SIZEOFINT]; --- 72,77 ---- *************** *** 75,80 **** --- 82,89 ---- struct hdr { PADHEADER int Size; /* header for active blocks; Size includes the header */ + /* the two low order bits of the Size fields + are used for ACTIVE and PREACTIVE */ }; struct freehdr { PADHEADER *************** *** 98,105 **** #if (SIZEOFCHARSTAR + 2*SIZEOFINT) % WORD #define PADHEADER \ ! int padding[(WORD-(SIZEOFCHARSTAR+2* SIZEOFINT)%WORD) ! / SIZEOFINT]; #else #define PADHEADER #endif --- 107,113 ---- #if (SIZEOFCHARSTAR + 2*SIZEOFINT) % WORD #define PADHEADER \ ! int padding[(WORD-(SIZEOFCHARSTAR+2* SIZEOFINT)%WORD) / SIZEOFINT]; #else #define PADHEADER #endif *************** *** 109,114 **** --- 117,124 ---- char *caller; int seqno; int Size; + /* the two low order bits of the Size fields + are used for ACTIVE and PREACTIVE */ }; struct freehdr { PADHEADER diff -c andrew/overhead/malloc/malloc.ci ./malloc.ci *** andrew/overhead/malloc/malloc.ci Mon Nov 20 08:03:21 1989 --- ./malloc.ci Tue Nov 28 08:27:49 1989 *************** *** 130,135 **** --- 130,137 ---- static struct arenastate A = {0, 0, 0, 0, 0, 0, 0, 0}; + static short BadSbrkBound = 0; /* kludge for error check "c3" */ + extern char *sbrk(); static struct freehdr *addarena (); *************** *** 376,393 **** /* addarena */ /* create a new or extended arena. Two adjacent arenas will coallesce. */ ! /* In a new arena segment, The first three words are a freehdr with ! Size indicating all of block except last four words; its Active bit is false and its PreActive bit is true (so no coalesce off front ! will be tried); Next and Prev both point to a dummy free element ! in last four words. The dummy in the last four words of the segment ! has Active true (so preceding block will not coalesce with it) and PreActive set depending on the preceding block (initially false); the Size field is zero; Next and Prev both point to the free element at the beginning of the segment. The Front field ! in the last word of the segment points not to the dummy ! free element at the end, but to the beginning of the segment ! (so CheckAllocs can find segment.) The argument min gives the space needed. Return value is NULL or a pointer to a big enough block. --- 378,396 ---- /* addarena */ /* create a new or extended arena. Two adjacent arenas will coallesce. */ ! /* In a new arena segment: ! The last EPSILON bytes are an arena control block. ! The first three words are a freehdr with ! Size indicating all of block except last EPSILON bytes; its Active bit is false and its PreActive bit is true (so no coalesce off front ! will be tried); Next and Prev both point to the arena control block. ! The arena control block is a freehdr with ! Active true (so preceding block will not coalesce with it) and PreActive set depending on the preceding block (initially false); the Size field is zero; Next and Prev both point to the free element at the beginning of the segment. The Front field ! in the last word of the segment points not to the arena control block, ! but to the beginning of the segment (so CheckAllocs can find it.) The argument min gives the space needed. Return value is NULL or a pointer to a big enough block. *************** *** 421,426 **** --- 424,431 ---- if ((x=(INT)new % WORD)) { new = (struct freehdr *)((INT)new+WORD-x); segbytes -= WORD; + segbytes -= segbytes%WORD; + BadSbrkBound = 1; } trlr = (struct freehdr *)((INT)new+segbytes-EPSILON); new->Size = setbits(segbytes - EPSILON, PREACTIVE); *************** *** 644,651 **** register struct freehdr *h = (struct freehdr *)(((char *)ap) - sizeof(struct hdr)); struct freehdr *f = NEXTBLOCK(h); unsigned siz; /* total size needed */ ! unsigned nw; /* desired number of words */ ! register unsigned onw; /* existing number of words */ char *msg; /* reason for failure */ if (A.InProgress++) { --- 649,656 ---- register struct freehdr *h = (struct freehdr *)(((char *)ap) - sizeof(struct hdr)); struct freehdr *f = NEXTBLOCK(h); unsigned siz; /* total size needed */ ! unsigned newsz; /* number of bytes in new area */ ! unsigned oldsz; /* number of bytes in old area */ char *msg; /* reason for failure */ if (A.InProgress++) { *************** *** 694,723 **** a free operation to be skipped ) */ A.InProgress --; ! nw = (siz - sizeof(struct hdr))/WORD; ! onw = (clearbits(h->Size)-sizeof(struct hdr))/WORD; ! if (nw<=onw) { /* is big enough; can we release part? */ ! if (onw-nw>EPSILON/WORD) { h->Size = setbits(siz, ACTIVE | testbit(h->Size, PREACTIVE)); f = NEXTBLOCK(h); ! f->Size = setbits((onw-nw)*WORD, ! ACTIVE | PREACTIVE); A.QueuedToFree[A.NQueued++] = ((struct hdr *)f) + 1; } } else { /* malloc a new one and copy */ ! register INT *s, *t; ! s = (INT *)ap; ap = (struct hdr *)malloc(nbytes); /* ap pts to data, not hdr */ if (ap==NULL) {msg = "rx5"; goto nope;} ! A.QueuedToFree[A.NQueued++] = (struct hdr *)s; ! t = (INT *)ap; ! while(onw-->0) ! *t++ = *s++; } if (CheckLevel >= 4) ferr2 (". . . to size %d at 0x%lx\n", nbytes, ap); --- 699,724 ---- a free operation to be skipped ) */ A.InProgress --; ! newsz = (siz - sizeof(struct hdr)); ! oldsz = (clearbits(h->Size)-sizeof(struct hdr)); ! if (newsz <= oldsz) { /* is big enough; can we release part? */ ! if (oldsz-newsz>EPSILON) { h->Size = setbits(siz, ACTIVE | testbit(h->Size, PREACTIVE)); f = NEXTBLOCK(h); ! f->Size = setbits(oldsz-newsz, ACTIVE | PREACTIVE); A.QueuedToFree[A.NQueued++] = ((struct hdr *)f) + 1; } } else { /* malloc a new one and copy */ ! struct hdr *oldap = ap; ap = (struct hdr *)malloc(nbytes); /* ap pts to data, not hdr */ if (ap==NULL) {msg = "rx5"; goto nope;} ! A.QueuedToFree[A.NQueued++] = oldap; ! bcopy((char *)oldap, (char *)ap, oldsz); } if (CheckLevel >= 4) ferr2 (". . . to size %d at 0x%lx\n", nbytes, ap); *************** *** 750,756 **** { if (testbit(f->Size,ACTIVE)) { /* had better be a segment trailer */ ! register struct freehdr *t = ((struct segtrlr *)f)->Front; return ((((long)f)&(WORD-1)) == 0 && f->Next->Prev == f && f->Prev->Next == f --- 751,758 ---- { if (testbit(f->Size,ACTIVE)) { /* had better be a segment trailer */ ! register struct freehdr *t ! = PREVFRONT((struct hdr *)(((char *)f)+EPSILON)); return ((((long)f)&(WORD-1)) == 0 && f->Next->Prev == f && f->Prev->Next == f *************** *** 857,869 **** ASSERT("c1", t->Next->Prev==t); if (testbit(t->Size, ACTIVE)) { /* this is a segment trlr */ ! struct freetrlr *f ! = (struct freetrlr *)(t+1); ASSERT("c2", clearbits(t->Size)==0); /* Segment must be a multiple of SEGGRAIN bytes: */ ! ASSERT("c3", ( (int)(f+1)-(int)(f->Front) ) /* \ */ %SEGGRAIN==0); ! nfree -= (CheckSegment(m, f->Front, t)); } else ASSERT("c4", t==PREVFRONT(NEXTBLOCK(t))); --- 859,871 ---- ASSERT("c1", t->Next->Prev==t); if (testbit(t->Size, ACTIVE)) { /* this is a segment trlr */ ! struct hdr *nextblk ! = (struct hdr *)(((char *)t)+EPSILON); ASSERT("c2", clearbits(t->Size)==0); /* Segment must be a multiple of SEGGRAIN bytes: */ ! ASSERT("c3", BadSbrkBound | ( ((INT)(nextblk))-(INT)PREVFRONT(nextblk)) %SEGGRAIN==0); ! nfree -= (CheckSegment(m, PREVFRONT(nextblk), t)); } else ASSERT("c4", t==PREVFRONT(NEXTBLOCK(t))); diff -c andrew/overhead/malloc/plumber.ci ./plumber.ci *** andrew/overhead/malloc/plumber.ci Mon Nov 20 08:03:26 1989 --- ./plumber.ci Mon Nov 27 17:13:31 1989 *************** *** 50,67 **** register struct freehdr *t, *f; char *foo = malloc(1); /* to Flush the Free list */ - /* scan free list to find segment trailers and scan each segment */ CheckAllocs("plumber start"); data = NULL; ! t = A->arenaend; ! f = ((struct freetrlr *)(t+1))->Front; ! /* scan the last arena segment for active blocks */ ! for (; f<t; f = (struct freehdr *)((char *)f ! + clearbits(f->Size))) ! if (testbit(f->Size, ACTIVE) ! && f->seqno<nextseq) ! storedata(f, &data); /* output data from tree */ fprintf(outf, "%10s%10s%10s%20s%20s\n\n", "caller", "#blocks", "tot size", "size range ", "seq# range "); --- 50,73 ---- register struct freehdr *t, *f; char *foo = malloc(1); /* to Flush the Free list */ CheckAllocs("plumber start"); + /* scan free list to find segment trailers and scan each segment */ data = NULL; ! t = A->allocp; ! do { ! if (testbit(t->Size, ACTIVE)) { ! /* this is a segment trlr; scan segment for active blocks */ ! f = PREVFRONT((struct hdr *)(((char *)t)+EPSILON)); ! for (; f<t; f = (struct freehdr *)((char *)f ! + clearbits(f->Size))) ! if (testbit(f->Size, ACTIVE) ! && f->seqno<nextseq) ! storedata(f, &data); ! } ! t = t->Next; ! } while (t!=A->allocp); ! /* output data from tree */ fprintf(outf, "%10s%10s%10s%20s%20s\n\n", "caller", "#blocks", "tot size", "size range ", "seq# range "); *************** *** 77,82 **** --- 83,89 ---- register struct callerdata *td = *d; if (td==NULL) { *d = td = (struct callerdata *)malloc(sizeof(struct callerdata)); + if (td == NULL) return; /* no more room to store data */ td->caller = p->caller; td->nblks = 1; td->loseq = td->hiseq = p->seqno;
bader+@ANDREW.CMU.EDU (Miles Bader) (11/29/89)
Fred Hansen <wjh+@andrew.cmu.edu> writes: > With this version using -DMSTATS and MALLOC_DEBUG_ENV, the malloc > information can be accessed from ATK applications by typing > control-X, \200, m for statistics > control-X, \200, t for plumber data > (\200 is the Insert key on an RT. I don't know how to generate it on > other workstations. If you want to rebind the functions to > control-X,m,s and control-X,m,t, you can put the following in .atkinit: Actually, \200 is really \000-- All key-bindings have the high bit stripped in the keymap package (otherwise, there would be no way to specify \000, it being the c string delimiter). So ^@ or ^SPACE will probably work on most keyboards. -Miles
nsb@THUMPER.BELLCORE.COM (Nathaniel Borenstein) (11/30/89)
Is there any chance that the ITC could produce a patch #8 to incorporate the recent SPARC-related fixes, especially to malloc? That would be kind of useful... Thanks. -- Nathaniel
mss+@ANDREW.CMU.EDU (Mark Sherman) (11/30/89)
There is almost no chance that the ITC would generate a patch 8. These changes are being merged into the (one-true) source that now exists at MIT. At the end of next week, we will reintegrate all of the various changes made at MIT back here at CMU, and probably make these bits available as the next release of Andrew via some anonymous FTP mechanism (as well as through the X tape, which is now scheduled to be publically available the first week of January). -Mark