mjr@decuac.dec.com (Marcus J. Ranum) (01/05/91)
Submitted-by: mjr@decuac.dec.com (Marcus J. Ranum) Posting-number: Volume 16, Issue 19 Archive-name: mnemosyne/part01 This is a set of tools designed to help find memory leaks in programs, and to locate memory-hogging functions. It's implemented as a wrapper library that goes around malloc/free/etc, and an include file that "intercepts" calls to malloc/free/etc and makes them call the wrappers. Thus, you can get extensive memory profiling and leak detection by just adding one #include directive at the top of your file and recompiling/linking. Marcus J. Ranum mjr@decuac.dec.com -----cut-----cut-----cut-----cut-----cut-----cut-----cut-----cut----- #! /bin/sh # This is a shell archive. Remove anything before this line, then unpack # it by saving it into a file and typing "sh file". To overwrite existing # files, type "sh file -c". You can also feed this as standard input via # unshar, or by typing "sh <file", e.g.. If this archive is complete, you # will see the following message at the end: # "End of shell archive." # Contents: mnemosyne mnemosyne/README mnemosyne/Makefile # mnemosyne/mnemalyse.c mnemosyne/mnemconf.h mnemosyne/mnemosyne.c # mnemosyne/mnemosyne.h mnemosyne/mtest.c mnemosyne/mnemalyse.1l # mnemosyne/mnemosyne.3l # Wrapped by mjr@hussar.dco.dec.com on Fri Dec 28 16:31:21 1990 PATH=/bin:/usr/bin:/usr/ucb ; export PATH if test ! -d mnemosyne ; then echo shar: Creating directory \"mnemosyne\" mkdir mnemosyne fi if test -f mnemosyne/README -a "${1}" != "-c" ; then echo shar: Will not over-write existing file \"mnemosyne/README\" else echo shar: Extracting \"mnemosyne/README\" \(1935 characters\) sed "s/^X//" >mnemosyne/README <<'END_OF_mnemosyne/README' X X This is a set of tools designed to help find memory leaks in Xprograms, and to locate memory-hogging functions. It's implemented as Xa wrapper library that goes around malloc/free/etc, and an include file Xthat "intercepts" calls to malloc/free/etc and makes them call the Xwrappers. Thus, you can get extensive memory profiling and leak Xdetection by just adding one #include directive at the top of your Xfile and recompiling/linking. X X Unlike some similar tools I've seen in the past, this makes Xsure that it keeps its on-disk data current, so that if the program Xis crashed or interrupted, the results still have some validity. The Xon-disk data is as compacted as I could make it, to give a chance of Xthis being useable in debugging big memory pigs. It adds some cost Xin performance and memory size (since it keeps its own in-memory Xsymbol tables) but since it's only a debugging tool, I think the Xcost is worth the benefit. This library can also be used to track Xonly allocations in a single module, or set of modules, and doesn't Xinterfere with calls to the "real" malloc() that are made in other Xlibrary routines. X X Every effort has been made to ensure that the code is Xportable and won't interfere with running code - it should just Xplug in or out. The biggest hindrances are forward declarations of Xmalloc() [which the preprocessor gleefully turns into syntax errors Xfor you] and structure elements named "free". The code has been Xtested under Ultrix on DEC Risc and VAX systems, and under SunOS Xon a Motorola platform. Please send patches, suggestions, etc, Xto the author, who will probably not have time to do anything with Xthem. X XCompiling and building: X You may wish to edit the Makefile and glance at mnemconf.h, Xthen simply type "make". "make mtest" will build a simple test program Xthat will give you an idea of how things work. "make runmtest" will Xrun the test and do analysis on it. X XMarcus J. Ranum Xmjr@decuac.dec.com END_OF_mnemosyne/README if test 1935 -ne `wc -c <mnemosyne/README`; then echo shar: \"mnemosyne/README\" unpacked with wrong size! fi # end of overwriting check fi if test -f mnemosyne/Makefile -a "${1}" != "-c" ; then echo shar: Will not over-write existing file \"mnemosyne/Makefile\" else echo shar: Extracting \"mnemosyne/Makefile\" \(971 characters\) sed "s/^X//" >mnemosyne/Makefile <<'END_OF_mnemosyne/Makefile' X# X# Makefile for the Mnemosyne memory allocation tracker. X# X# Marcus J. Ranum, 1990 X# X#Options: X# define MALLOC_IS_VOIDSTAR if your system's malloc is declared as a (void *) X# otherwise, it is assumed to be a (char *). a "mall_t" is typedeffed in X# mnemconf.h and mnemosyne.h to implement this. XOPTNS= -DMALLOC_IS_VOIDSTAR X#OPTNS= X X#compiler flags XCFLAGS= -O $(OPTNS) X X#loader flags XLDFLGS= -s X XHDRS= mnemosyne.h mnemconf.h X Xall: mnemalyse libmnem.a X Xmnemalyse: mnemalyse.o X cc $(LDFLGS) -o $@ mnemalyse.o X Xlibmnem.a: mnemosyne.o X ar rcv $@ mnemosyne.o X ranlib $@ X Xmtest: mtest.o libmnem.a X cc $(LDFLGS) -o $@ mtest.o libmnem.a X Xrunmtest: all mtest X @echo "running memory waster" X mtest X @echo "press return for symbol list"; read ff X @cat mnem.syms X @echo "press return for waste analysis"; read ff X mnemalyse X Xclean: X rm -f mtest core *.o X Xclobber: clean X rm -f libmnem.a mnemalyse X X Xmnemosyne.o: Makefile mnemosyne.c $(HDRS) Xmnemalyse.o: Makefile mnemalyse.c $(HDRS) END_OF_mnemosyne/Makefile if test 971 -ne `wc -c <mnemosyne/Makefile`; then echo shar: \"mnemosyne/Makefile\" unpacked with wrong size! fi chmod +x mnemosyne/Makefile # end of overwriting check fi if test -f mnemosyne/mnemalyse.c -a "${1}" != "-c" ; then echo shar: Will not over-write existing file \"mnemosyne/mnemalyse.c\" else echo shar: Extracting \"mnemosyne/mnemalyse.c\" \(5078 characters\) sed "s/^X//" >mnemosyne/mnemalyse.c <<'END_OF_mnemosyne/mnemalyse.c' X/************************************************************************ X * * X * Copyright (c) 1985 by * X * Digital Equipment Corporation, Maynard, MA * X * All rights reserved. * X * * X * The information in this software is subject to change without * X * notice and should not be construed as a commitment by Digital * X * Equipment Corporation. * X * * X * Digital assumes no responsibility for the use or reliability * X * of its software on equipment which is not supplied by Digital. * X * * X * Redistribution and use in source and binary forms are permitted * X * provided that the above copyright notice and this paragraph are * X * duplicated in all such forms and that any documentation, * X * advertising materials, and other materials related to such * X * distribution and use acknowledge that the software was developed * X * by Digital Equipment Corporation. The name of Digital Equipment * X * Corporation may not be used to endorse or promote products derived * X * from this software without specific prior written permission. * X * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR * X * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED * X * WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE.* X * Do not take internally. In case of accidental ingestion, contact * X * your physician immediately. * X * * X ************************************************************************/ X X/* DO NOT INCLUDE "mnemosyne.h" !!! */ X#include <stdio.h> X#include <ctype.h> X#include <sys/types.h> X#include <sys/file.h> X Xstatic char rcsid[] = "$Header: mnemalyse.c 1.1 90/12/25 mjr Rel $"; X X#include "mnemconf.h" X Xextern char *index(); X X/* Xpost-processor to interpret memory allocation maps and search for Xpointers that were allocated but never freed. X X Marcus J. Ranum, 1990. (mjr@decuac.dec.com) X*/ X X X/* Xsimple and braindead, read in the ".lines" file, and store it in a Xtable by number. then read the pointer map, and crossref any unfreed Xpointers. simple as dereferencing NULL... X Xthis could use some cleaning and buffing, but it's damn effective as Xit is. again, fancier symbol table routines would make this faster, Xbut who gives a damn? it only has to be faster than finding memory Xleaks by hand... X*/ X Xstruct xsym { X char *dat; X int lnum; X int map; X struct xsym *nxt; X}; X X X Xmain() X{ X register struct xsym *sp; X register struct xsym *zp; X struct ptr p; X struct xsym *shash[HASHSIZ]; X char inbuf[BUFSIZ]; X FILE *lp; X int fd; X X /* statistics */ X int ptrcnt = 0; X int ptrbad = 0; X int ptrlos = 0; X X /* to chop up lines */ X char *cpmap; X char *cpcalls; X char *cpave; X char *cplnum; X char *cpfnam; X X for(fd = 0; fd < HASHSIZ; fd++) X shash[fd] = (struct xsym *)0; X X if((lp = fopen(LINESFILE,"r")) == (FILE *)0) { X perror(LINESFILE); X exit(1); X } X X if((fd = open(PTRFILE,O_RDONLY|O_RDWR)) < 0) { X perror(PTRFILE); X exit(1); X } X X /* this is ugly, but I refuse to trust !@(#&U!@#&! sscanf() */ X while((cpmap = fgets(inbuf,sizeof(inbuf),lp)) != (char *)0) { X if(inbuf[0] == '#') X continue; X X sp = (struct xsym *)malloc(sizeof(struct xsym)); X if(sp == (struct xsym *)0) { X perror("malloc"); X exit(1); X } X sp->lnum = sp->map = 0; X X if((cpcalls = index(cpmap,'\t')) != (char *)0) X *cpcalls++ = '\0'; X X if((cpave = index(cpcalls,'\t')) != (char *)0) X *cpave++ = '\0'; X X if((cplnum = index(cpave,'\t')) != (char *)0) X *cplnum++ = '\0'; X X if((cpfnam = index(cplnum,'\t')) != (char *)0) X *cpfnam++ = '\0'; X X /* setup symbol */ X sp->map = atoi(cpmap); X X if(cplnum == (char *)0) X sp->lnum = -1; X else X sp->lnum = atoi(cplnum); X X if(cpfnam != (char *)0) { X char *x; X if((x = index(cpfnam,'\n')) != (char *)0) X *x = '\0'; X X sp->dat = malloc((unsigned)(strlen(cpfnam) + 1)); X if(sp->dat == (char *)0) { X perror("malloc"); X exit(1); X } X (void)strcpy(sp->dat,cpfnam); X } else X sp->dat = "unknown"; X X /* check to make sure it is not already in table */ X zp = shash[sp->map % HASHSIZ]; X while(zp != (struct xsym *)0) { X if(zp->map == sp->map) { X (void)fprintf(stderr, X "mnemalyse: duplicate map entry ignored"); X (void)fprintf(stderr, X " (point at both %s and %s)\n",sp->dat,zp->dat); X (void)free(sp); X X /* can't free dat - may not be malloced! */ X sp = (struct xsym *)0; X break; X } X zp = zp->nxt; X } X X /* shrug, link it in */ X if(sp != (struct xsym *)0) { X sp->nxt = shash[sp->map % HASHSIZ]; X shash[sp->map % HASHSIZ] = sp; X } X } X (void)fclose(lp); X X while(read(fd,(char *)&(p.dsk),sizeof(p.dsk)) == sizeof(p.dsk)) { X X /* if the pointer was not deallocated, note it */ X if(p.dsk.siz != 0) { X zp = shash[p.dsk.smap % HASHSIZ]; X while(zp != (struct xsym *)0) { X if(zp->map == p.dsk.smap) { X printf("%d bytes missing %s line:%d\n", X p.dsk.siz,zp->dat,zp->lnum); X } X zp = zp->nxt; X } X ptrbad++; X ptrlos += p.dsk.siz; X } X ptrcnt++; X } X X printf("%d pointers, %d lost totalling %d bytes\n", X ptrcnt,ptrbad,ptrlos); X exit(0); X} END_OF_mnemosyne/mnemalyse.c if test 5078 -ne `wc -c <mnemosyne/mnemalyse.c`; then echo shar: \"mnemosyne/mnemalyse.c\" unpacked with wrong size! fi chmod +x mnemosyne/mnemalyse.c # end of overwriting check fi if test -f mnemosyne/mnemconf.h -a "${1}" != "-c" ; then echo shar: Will not over-write existing file \"mnemosyne/mnemconf.h\" else echo shar: Extracting \"mnemosyne/mnemconf.h\" \(2890 characters\) sed "s/^X//" >mnemosyne/mnemconf.h <<'END_OF_mnemosyne/mnemconf.h' X/************************************************************************ X * * X * Copyright (c) 1985 by * X * Digital Equipment Corporation, Maynard, MA * X * All rights reserved. * X * * X * The information in this software is subject to change without * X * notice and should not be construed as a commitment by Digital * X * Equipment Corporation. * X * * X * Digital assumes no responsibility for the use or reliability * X * of its software on equipment which is not supplied by Digital. * X * * X * Redistribution and use in source and binary forms are permitted * X * provided that the above copyright notice and this paragraph are * X * duplicated in all such forms and that any documentation, * X * advertising materials, and other materials related to such * X * distribution and use acknowledge that the software was developed * X * by Digital Equipment Corporation. The name of Digital Equipment * X * Corporation may not be used to endorse or promote products derived * X * from this software without specific prior written permission. * X * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR * X * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED * X * WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE.* X * Do not take internally. In case of accidental ingestion, contact * X * your physician immediately. * X * * X ************************************************************************/ X X#ifndef _INCL_MNEMCONF_H X X/* X$Header: mnemconf.h 1.1 90/12/25 mjr Rel $ X*/ X X/* Xsite specific and shared internal data structures used by mnemosyne. Xthe only data structure that may need to be shared is the struct ptr, Xwhich is defined herein. X X Marcus J. Ranum, 1990. (mjr@decuac.dec.com) X*/ X X X X/* if your machine has malloc and all declared as a (void *) not a (char *) */ X#ifdef MALLOC_IS_VOIDSTAR Xtypedef void *mall_t; X#else Xtypedef char *mall_t; X#endif X X X/* size of internal hash tables - don't go wild - this is slow anyhow */ X#define HASHSIZ 127 X X X/* names of files to write */ X#define LINESFILE "mnem.syms" X#define PTRFILE "mnem.dat" X X Xextern mall_t malloc(); Xextern mall_t realloc(); Xextern mall_t calloc(); Xextern void free(); X X X/* Xstorage for a pointer map entry - the only data structure we share Xa whole mess of these get written to mnem.dat as calls to malloc and Xwhatnot are made. the distinction between an *allocated* pointer and Xand unallocated one is that 'siz' is 0 in freed ptrs. this is used Xby the post-processor to look for memory leaks. X*/ Xstruct ptr { X mall_t ptr; /* pointer to allocated memory */ X int map; /* this pointer's map # */ X struct ptr *next; X X /* only part that gets written to the disk */ X struct { X unsigned siz; /* size allocated (or 0) */ X int smap; /* symbol map # */ X } dsk; X}; X X#define _INCL_MNEMCONF_H X#endif END_OF_mnemosyne/mnemconf.h if test 2890 -ne `wc -c <mnemosyne/mnemconf.h`; then echo shar: \"mnemosyne/mnemconf.h\" unpacked with wrong size! fi chmod +x mnemosyne/mnemconf.h # end of overwriting check fi if test -f mnemosyne/mnemosyne.c -a "${1}" != "-c" ; then echo shar: Will not over-write existing file \"mnemosyne/mnemosyne.c\" else echo shar: Extracting \"mnemosyne/mnemosyne.c\" \(13742 characters\) sed "s/^X//" >mnemosyne/mnemosyne.c <<'END_OF_mnemosyne/mnemosyne.c' X/************************************************************************ X * * X * Copyright (c) 1985 by * X * Digital Equipment Corporation, Maynard, MA * X * All rights reserved. * X * * X * The information in this software is subject to change without * X * notice and should not be construed as a commitment by Digital * X * Equipment Corporation. * X * * X * Digital assumes no responsibility for the use or reliability * X * of its software on equipment which is not supplied by Digital. * X * * X * Redistribution and use in source and binary forms are permitted * X * provided that the above copyright notice and this paragraph are * X * duplicated in all such forms and that any documentation, * X * advertising materials, and other materials related to such * X * distribution and use acknowledge that the software was developed * X * by Digital Equipment Corporation. The name of Digital Equipment * X * Corporation may not be used to endorse or promote products derived * X * from this software without specific prior written permission. * X * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR * X * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED * X * WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE.* X * Do not take internally. In case of accidental ingestion, contact * X * your physician immediately. * X * * X ************************************************************************/ X X/* DO NOT INCLUDE "mnemosyne.h" !!! */ X#include <stdio.h> X#include <sys/types.h> X#include <sys/file.h> X X/* shared stuff - and decl of struct ptr */ X#include "mnemconf.h" X Xstatic char rcsid[] = "$Header: mnemosyne.c 1.1 90/12/25 mjr Rel $"; X X X/* Xmalloc() realloc() and family wrappers - these functions manage a set Xof data files that are updated whenever a pointer is allocated or freed, Xas well as gathering stats about dynamic memory use and leakage. X X Marcus J. Ranum, 1990. (mjr@decuac.dec.com) X*/ X X X/* Xthere is some egregious use of globals, void functions, and whatnot Xin this code. it is mostly due to the constraint that nothing must Xbe visible to the outside, and that the code must be structurally Xsimple. error checking is pitched out the window in spots, since an Xerror in the mnemosyne code should not affect whatever is being Xinstrumented if at all possible. this means no errors, no signals, Xnothing. (this message brought to you by my ego, in case you think XI don't know how to write better code than this) :) X Xmjr, hacking on Christmas, 1990. X*/ X X#define REC_UNINIT 000 X#define REC_INITTED 001 X#define REC_ERR 002 X#define REC_ON 010 X#define REC_ONOFF 020 Xstatic int rec_state = REC_UNINIT; X X/* Xthis method of storing the symbol maps is not the most efficient, but Xthen that's not important here, since all we're trying to do is find Xmemory leaks. if you choose to improve the symbol management to use Xbucketed hash tables, please contact the author and the code will be Xupdated :) - besides, since we do file I/O every time you malloc or Xfree something, there's no way in hell this is going to set any new Xrecords for speed. X*/ X X X/* storage for code/line # entry */ Xstruct sym { X char *labl; X int lineno; X int mapno; X int mallcnt; X float avsiz; X struct sym *next; X}; X X X X/* static symbol map */ Xstatic struct { X FILE *fp; X FILE *log; X int fd; X X long nalloc; /* count of allocations */ X long nrlloc; /* count of re-allocations */ X long nfree; /* count of frees */ X long nbfree; /* count of bad frees */ X long ninuse; /* known allocated memory in use */ X float avgsiz; /* average malloc size */ X X /* one entry per pointer returned by malloc */ X int pmap; /* current next ptr map to alloc */ X struct ptr *phash[HASHSIZ]; X X /* one entry per line of code that calls malloc/realloc, etc */ X int lmap; /* current next line map to alloc */ X struct sym *shash[HASHSIZ]; /* hash access */ X} map; X X X X X/* print a locale record with checks for closed log file */ Xstatic void Xploc(lab,lin,siz) Xchar *lab; Xint lin; Xint siz; X{ X if(map.log == (FILE *)0) X return; X if(lab != (char *)0) X (void)fprintf(map.log," \"%s\"",lab); X else X (void)fprintf(map.log," unknown"); X if(lin != -1) X (void)fprintf(map.log," line:%d",lin); X if(siz != -1) X (void)fprintf(map.log," size:%d",siz); X} X X X X X/* print a symbol map entry with checks for closed log file */ Xstatic void Xpsym(s) Xstruct sym *s; X{ X if(map.log == (FILE *)0) X return; X (void)fprintf(map.log," \"%s\"",s->labl); X if(s->lineno != -1) X (void)fprintf(map.log," line:%d",s->lineno); X} X X X X X/* output a warning message with checks for closed log file */ Xstatic void Xpmsg(s) Xchar *s; X{ X if(map.log == (FILE *)0) X return; X (void)fprintf(map.log,"%s",s); X} X X X X X/* save an entry to the .lines file */ Xstatic void Xsavesym(s) Xstruct sym *s; X{ X if(map.fp == (FILE *)0) X return; X X (void)fprintf(map.fp,"%d\t%d\t%.1f\t%d\t%s\n", X s->mapno,s->mallcnt,s->avsiz,s->lineno,s->labl); X} X X X X X/* save an entry in the pointer map file */ Xstatic void Xsaveptr(p) Xregister struct ptr *p; X{ X if(lseek(map.fd,(off_t)(p->map * sizeof(p->dsk)),0) != X (off_t)(p->map * sizeof(p->dsk))) { X pmsg("mnemosyne: cannot seek in pointer map file\n"); X rec_state |= REC_ERR; X return; X } X X if(write(map.fd,(char *)&(p->dsk),sizeof(p->dsk)) != sizeof(p->dsk)) { X pmsg("mnemosyne: cannot write in pointer map file\n"); X rec_state |= REC_ERR; X return; X } X} X X X X X/* initialize everything - symbol tables, files, and maps */ Xstatic void Xinitmap() X{ X register int xp; X X if(rec_state & REC_INITTED) X return; X X if((map.fp = fopen(LINESFILE,"w")) == (FILE *)0) X return; X if((map.fd = open(PTRFILE,O_RDWR|O_CREAT|O_TRUNC,0600)) < 0) { X (void)fclose(map.fp); X return; X } X X map.log = stderr; X map.lmap = map.pmap = 0; X map.nalloc = map.nrlloc = map.nfree = map.nbfree = 0L; X map.ninuse = 0L; X map.avgsiz = 0.0; X X for(xp = 0; xp < HASHSIZ; xp++) { X map.phash[xp] = (struct ptr *)0; X map.shash[xp] = (struct sym *)0; X } X X rec_state = REC_INITTED | REC_ON; X} X X X X X/* set logging to a FILE * */ Xvoid Xmnem_setlog(fp) XFILE *fp; X{ X map.log = fp; X} X X X X X/* return state of the recorder */ Xint Xmnem_recording() X{ X return((rec_state & REC_ON) && !(rec_state & REC_ERR)); X} X X X X X/* turn on or off recording */ Xint Xmnem_setrecording(val) Xint val; X{ X if(val) X rec_state |= REC_ON; X else X rec_state &= ~REC_ON; X X if(map.fp != (FILE *)0) X (void)fflush(map.fp); X X rec_state |= REC_ONOFF; X return(0); X} X X X X X/* lookup a pointer record - search pointer hash table */ Xstatic struct ptr * Xlookupptr(ptr) Xmall_t ptr; X{ X register struct ptr *p; X X /* this probably give simply terrible hash performance */ X p = map.phash[(unsigned long)ptr % HASHSIZ]; X while(p != (struct ptr *)0) { X if(ptr == p->ptr) X return(p); X p = p->next; X } X return((struct ptr *)0); X} X X X X X/* X * polynomial conversion ignoring overflows X * [this seems to work remarkably well, in fact better X * then the ndbm hash function. Replace at your own risk] X * use: 65599 nice. X * 65587 even better. X * author: oz@nexus.yorku.ca X */ Xstatic unsigned int Xdbm_hash(str) Xregister char *str; X{ X register unsigned int n = 0; X X while(*str != '\0') X n = *str++ + 65599 * n; X return(n); X} X X X X X/* lookup a line/source entry by name (search hash table) */ Xstatic struct sym * Xlookupsymbyname(nam,lin) Xchar *nam; Xint lin; X{ X register struct sym *s; X char *p = nam; X X if(p == (char *)0) X p = "unknown"; X X s = map.shash[(dbm_hash(p) + lin) % HASHSIZ]; X while(s != (struct sym *)0) { X if(!strcmp(s->labl,nam) && s->lineno == lin) X return(s); X s = s->next; X } X X return((struct sym *)0); X} X X X X X/* lookup a line/source entry by number (exhaustively search hash table) */ Xstatic struct sym * Xlookupsymbynum(num) Xint num; X{ X register struct sym *s; X register int x; X X for(x = 0; x < HASHSIZ; x++) { X s = map.shash[x]; X while(s != (struct sym *)0) { X if(s->mapno == num) X return(s); X s = s->next; X } X } X return((struct sym *)0); X} X X X X/* stuff a pointer's value in the pointer map table */ Xstatic void Xstoreptr(ptr,siz,lab,lin) Xmall_t ptr; Xint siz; Xchar *lab; Xint lin; X{ X register struct ptr *p; X register struct sym *s; X int hv; X X /* X is there is no existing symbol entry for this line of code... X we must needs make one - and painful it is... X */ X if((s = lookupsymbyname(lab,lin)) == (struct sym *)0) { X s = (struct sym *)malloc(sizeof(struct sym)); X if(s == (struct sym *)0) { X pmsg("mnemosyne: cannot allocate sym entry\n"); X rec_state |= REC_ERR; X return; X } X X /* X this is funky - since we know the label is (?) X compiled-in, we can just keep a pointer to it, X rather than copying our own version of it. X */ X if(lab != (char *)0) X s->labl = lab; X else X s->labl = "unknown"; X X s->mapno = map.lmap++; X X /* add sym to hash table */ X s->next = map.shash[hv = ((dbm_hash(s->labl) + lin) % HASHSIZ)]; X map.shash[hv] = s; X X s->lineno = lin; X s->mallcnt = 1; X s->avsiz = siz; X savesym(s); X } else { X /* found an already defined symbol. store some averages */ X s->avsiz = ((s->avsiz * s->mallcnt) + siz) / (s->mallcnt + 1); X (s->mallcnt)++; X } X X p = lookupptr(ptr); X if(p != (struct ptr *)0 && p->dsk.siz != 0) { X struct sym *x; X X pmsg("pointer re-allocated without being freed"); X ploc(lab,lin,(int)siz); X if((x = lookupsymbynum(p->dsk.smap)) != (struct sym *)0) { X pmsg(" last allocated "); X psym(x); X } X pmsg("\n"); X } X X /* heavy sigh. no entry for this pointer. make one. */ X if(p == (struct ptr *)0) { X p = (struct ptr *)malloc(sizeof(struct ptr)); X if(p == (struct ptr *)0) { X pmsg("mnemosyne: cannot expand pointer table\n"); X rec_state |= REC_ERR; X return; X } X X /* link it in */ X p->next = map.phash[(unsigned long)ptr % HASHSIZ]; X map.phash[(unsigned long)ptr % HASHSIZ] = p; X } X X /* if we get to here (hazoo! hazaa!) both 's' and 'p' are OK */ X p->ptr = ptr; X p->dsk.siz = siz; X p->dsk.smap = s->mapno; X p->map = map.pmap++; X X /* store the size */ X map.ninuse += siz; X X saveptr(p); X} X X X X X/* Xmark a pointer as now being free. note that a 1 is returned IF Xthe actual value should NOT be really passed to free() X*/ Xstatic int Xfreeptr(ptr,lab,lin) Xmall_t ptr; Xchar *lab; Xint lin; X{ X register struct ptr *p; X X p = lookupptr(ptr); X if(p == (struct ptr *)0) { X pmsg("pointer freed that was never allocated"); X ploc(lab,lin,-1); X pmsg("\n"); X return(1); X } X X if(p != (struct ptr *)0 && p->dsk.siz == 0) { X struct sym *x; X X pmsg("pointer re-freed when already free"); X ploc(lab,lin,-1); X if((x = lookupsymbynum(p->dsk.smap)) != (struct sym *)0) { X pmsg(" last allocated:"); X psym(x); X } X pmsg("\n"); X return(1); X } X X /* get some free */ X map.ninuse -= p->dsk.siz; X X /* write in the map that it is free */ X p->dsk.siz = 0; X saveptr(p); X X return(0); X} X X X X X/* pretend we are malloc() */ Xmall_t Xmnem_malloc(siz,lab,lin) Xunsigned siz; Xchar *lab; Xint lin; X{ X mall_t ret; X X if(!(rec_state & REC_INITTED)) X initmap(); X X if((ret = malloc(siz)) == (mall_t)0) { X pmsg("malloc returned null pointer at"); X ploc(lab,lin,(int)siz); X pmsg("\n"); X return(ret); X } X X if((rec_state & REC_ON) && !(rec_state & REC_ERR)) X storeptr(ret,(int)siz,lab,lin); X X map.avgsiz = ((map.avgsiz * map.nalloc) + siz) / (map.nalloc + 1); X map.nalloc++; X return(ret); X} X X X X X/* pretend we are calloc() */ Xmall_t Xmnem_calloc(cnt,siz,lab,lin) Xunsigned cnt; Xunsigned siz; Xchar *lab; Xint lin; X{ X mall_t ret; X X if(!(rec_state & REC_INITTED)) X initmap(); X X if((ret = calloc(cnt,siz)) == (mall_t)0) { X pmsg("calloc returned null pointer at"); X ploc(lab,lin,(int)(siz * cnt)); X pmsg("\n"); X return(ret); X } X X if((rec_state & REC_ON) && !(rec_state & REC_ERR)) X storeptr(ret,(int)(cnt * siz),lab,lin); X X map.avgsiz = ((map.avgsiz * map.nalloc) + siz) / (map.nalloc + 1); X map.nalloc++; X return(ret); X} X X X X X/* pretend we are realloc() */ Xmall_t Xmnem_realloc(ptr,siz,lab,lin) Xmall_t ptr; Xunsigned siz; Xchar *lab; Xint lin; X{ X mall_t ret; X X if(!(rec_state & REC_INITTED)) X initmap(); X X if((ret = realloc(ptr,siz)) == (mall_t)0) { X pmsg("realloc returned null pointer at"); X ploc(lab,lin,(int)siz); X pmsg("\n"); X return(ret); X } X X if((rec_state & REC_ON) && !(rec_state & REC_ERR)) { X if(!freeptr(ptr,lab,lin)) X storeptr(ret,(int)siz,lab,lin); X } X X map.nrlloc++; X return(ret); X} X X X X X X/* pretend we are free() */ Xvoid Xmnem_free(ptr,lab,lin) Xmall_t ptr; Xchar *lab; Xint lin; X{ X if(!(rec_state & REC_INITTED)) X initmap(); X X if((rec_state & REC_ON) && !(rec_state & REC_ERR)) X if(freeptr(ptr,lab,lin) == 0) { X (void)free(ptr); X map.nfree++; X } else X map.nbfree++; X} X X X X X/* dump everything we know about nothing in particular */ Xint Xmnem_writestats() X{ X register struct sym *s; X register int x; X X if(map.fp == (FILE *)0) X return(-1); X X (void)fseek(map.fp,0L,0); X X /* dump our life's story */ X (void)fprintf(map.fp,"#total allocations:%ld\n",map.nalloc); X (void)fprintf(map.fp,"#total re-allocations:%ld\n",map.nrlloc); X (void)fprintf(map.fp,"#total frees:%ld\n",map.nfree); X X if(map.nbfree != 0L) X (void)fprintf(map.fp,"#bad/dup frees:%ld\n",map.nbfree); X X (void)fprintf(map.fp,"#total allocated never freed:%ld\n",map.ninuse); X X (void)fprintf(map.fp,"#average size of allocations:%.1f\n",map.avgsiz); X X /* note if we detected an internal error */ X if(rec_state & REC_ERR) X (void)fprintf(map.fp, X "#(figures likely inaccurate due to error)\n"); X X /* note if the system was on all the time ? */ X if(!(rec_state & REC_ON) || (rec_state & REC_ONOFF)) X (void)fprintf(map.fp, X "#(figures likely inaccurate as recording was off)\n"); X X /* write the legend */ X (void)fprintf(map.fp,"#map#\tcalls\tave\tline#\tfile\n"); X X for(x = 0; x < HASHSIZ; x++) { X s = map.shash[x]; X while(s != (struct sym *)0) { X savesym(s); X s = s->next; X } X } X X (void)fflush(map.fp); X return(0); X} END_OF_mnemosyne/mnemosyne.c if test 13742 -ne `wc -c <mnemosyne/mnemosyne.c`; then echo shar: \"mnemosyne/mnemosyne.c\" unpacked with wrong size! fi chmod +x mnemosyne/mnemosyne.c # end of overwriting check fi if test -f mnemosyne/mnemosyne.h -a "${1}" != "-c" ; then echo shar: Will not over-write existing file \"mnemosyne/mnemosyne.h\" else echo shar: Extracting \"mnemosyne/mnemosyne.h\" \(2593 characters\) sed "s/^X//" >mnemosyne/mnemosyne.h <<'END_OF_mnemosyne/mnemosyne.h' X/************************************************************************ X * * X * Copyright (c) 1985 by * X * Digital Equipment Corporation, Maynard, MA * X * All rights reserved. * X * * X * The information in this software is subject to change without * X * notice and should not be construed as a commitment by Digital * X * Equipment Corporation. * X * * X * Digital assumes no responsibility for the use or reliability * X * of its software on equipment which is not supplied by Digital. * X * * X * Redistribution and use in source and binary forms are permitted * X * provided that the above copyright notice and this paragraph are * X * duplicated in all such forms and that any documentation, * X * advertising materials, and other materials related to such * X * distribution and use acknowledge that the software was developed * X * by Digital Equipment Corporation. The name of Digital Equipment * X * Corporation may not be used to endorse or promote products derived * X * from this software without specific prior written permission. * X * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR * X * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED * X * WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE.* X * Do not take internally. In case of accidental ingestion, contact * X * your physician immediately. * X * * X ************************************************************************/ X X#ifndef _INCL_MNEMOSYNE_H X X/* X$Header: nemosyne.h 1.1 90/12/25 mjr Rel $ X*/ X X X/* Xmain include file for the mnemosyne memory allocation tracker. this file Xprovides some pre-processor fakes for malloc(), realloc() and family, Xas well as forward declarations for the mnemosyne functions. X X Marcus J. Ranum, 1990. (mjr@decuac.dec.com) X*/ X X X/* these disguise mnemosyne calls as calls to malloc and family */ X#ifndef NOFAKEMALLOC X#define malloc(siz) mnem_malloc(siz,__FILE__,__LINE__) X#define calloc(siz,cnt) mnem_calloc(siz,cnt,__FILE__,__LINE__) X#define realloc(ptr,siz) mnem_realloc(ptr,siz,__FILE__,__LINE__) X#define free(ptr) mnem_free(ptr,__FILE__,__LINE__) X#endif X X X#ifdef MALLOC_IS_VOIDSTAR Xtypedef void *mall_t; X#else Xtypedef char *mall_t; X#endif X Xextern mall_t mnem_malloc(); Xextern mall_t mnem_calloc(); Xextern mall_t mnem_realloc(); Xextern void mnem_free(); X X/* some internal functions and oddimentia */ Xextern int mnem_recording(); Xextern int mnem_setrecording(); Xextern void mnem_setlog(); Xextern int mnem_writestats(); X X#define _INCL_MNEMOSYNE_H X#endif END_OF_mnemosyne/mnemosyne.h if test 2593 -ne `wc -c <mnemosyne/mnemosyne.h`; then echo shar: \"mnemosyne/mnemosyne.h\" unpacked with wrong size! fi chmod +x mnemosyne/mnemosyne.h # end of overwriting check fi if test -f mnemosyne/mtest.c -a "${1}" != "-c" ; then echo shar: Will not over-write existing file \"mnemosyne/mtest.c\" else echo shar: Extracting \"mnemosyne/mtest.c\" \(647 characters\) sed "s/^X//" >mnemosyne/mtest.c <<'END_OF_mnemosyne/mtest.c' X#include "mnemosyne.h" X Xstatic char rcsid[] = "$Header: mtest.c 1.1 90/12/25 mjr Rel $"; X X/* Xtest harness/demo of mnemosyne library. deliberately leaks memory on the Xfloor, double frees, frees unallocated memory, etc. X X Marcus J. Ranum, 1990. (mjr@decuac.dec.com) X*/ X X Xmain() X{ X char *d = "foobar"; X char *xx; X int x; X X xx = malloc(922); X xx = malloc(123); X X /* huh ? */ X xx = malloc(-9); X X /* waste some memory */ X for(x = 1; x < 8; x++) X xx = malloc(x); X X /* free something we don't own */ X free(d); X X /* double free something */ X free(xx); X free(xx); X X /* not necessary - this triggers a better printout of statistics */ X mnem_writestats(); X} END_OF_mnemosyne/mtest.c if test 647 -ne `wc -c <mnemosyne/mtest.c`; then echo shar: \"mnemosyne/mtest.c\" unpacked with wrong size! fi chmod +x mnemosyne/mtest.c # end of overwriting check fi if test -f mnemosyne/mnemalyse.1l -a "${1}" != "-c" ; then echo shar: Will not over-write existing file \"mnemosyne/mnemalyse.1l\" else echo shar: Extracting \"mnemosyne/mnemalyse.1l\" \(964 characters\) sed "s/^X//" >mnemosyne/mnemalyse.1l <<'END_OF_mnemosyne/mnemalyse.1l' X.\" $Header: mnemalyse.1l 1.1 90/12/28 mjr Rel $ X.TH mnemalyse 1 X.SH Name Xmnemalyse \- analyse dynamic memory allocation statistics X.SH Syntax X.B mnemalyse X.IP XThe X.I mnemalyse Xprogram interprets mnemosyne pointer and symbol databases and prints Xstatistics about dynamically allocated data that was not freed. The Xprogram expects to find the files X.I "mnem.syms" Xand X.I "mnem.dat" Xin the current working directory. There are no options and the output Xis self-explanatory: X.EX Xhussar-> mnemalyse X922 bytes missing mtest.c line:19 X123 bytes missing mtest.c line:20 X1 bytes missing mtest.c line:27 X2 bytes missing mtest.c line:27 X3 bytes missing mtest.c line:27 X4 bytes missing mtest.c line:27 X5 bytes missing mtest.c line:27 X6 bytes missing mtest.c line:27 X9 pointers, 8 lost totalling 1066 bytes X.EE X.SH Files X.DT Xmnem.syms Source code line number map X.br Xmnem.dat Pointer allocation map X.SH See Also Xmnemosyne(3l) X.SH Author XMarcus J. Ranum (mjr@decuac.dec.com) END_OF_mnemosyne/mnemalyse.1l if test 964 -ne `wc -c <mnemosyne/mnemalyse.1l`; then echo shar: \"mnemosyne/mnemalyse.1l\" unpacked with wrong size! fi chmod +x mnemosyne/mnemalyse.1l # end of overwriting check fi if test -f mnemosyne/mnemosyne.3l -a "${1}" != "-c" ; then echo shar: Will not over-write existing file \"mnemosyne/mnemosyne.3l\" else echo shar: Extracting \"mnemosyne/mnemosyne.3l\" \(6278 characters\) sed "s/^X//" >mnemosyne/mnemosyne.3l <<'END_OF_mnemosyne/mnemosyne.3l' X.\" $Header: mnemosyne.3l 1.1 90/12/28 mjr Rel $ X.TH mnemosyne 3l X.SH Name Xmnemosyne: mnem_malloc mnem_free, mnem_realloc, mnem_calloc \- memory allocator Xwith allocation tracing and statistics X.SH Syntax X.sp X#include "mnemosyne.h" X.nf X.B mall_t mnem_malloc(size, lab, lin) X.B unsigned size; X.B char *lab; X.B int lin; X.PP X.B mnem_free(ptr, lab, lin) X.B mall_t ptr; X.B char *lab; X.B int lin; X.PP X.B mall_t mnem_realloc(ptr, size, lab, lin) X.B mall_t ptr; X.B unsigned size; X.B char *lab; X.B int lin; X.PP X.B mall_t mnem_calloc(nelem, elsize, lab, lin) X.B unsigned nelem, elsize; X.B char *lab; X.B int lin; X.fi X.SH Description XThe X.I mnemosyne Xlibrary acts as a "wrapper" around the X.I malloc Xfamily of memory allocation routines, and gathers statistics about Xwhere allocations occur in source code for later analysis. The include Xfile X.I mnemosyne.h Xcontains C-preprocessor directives that "intercept" calls to X.I malloc Xand redirect them to the equivalent X.I mnemosyne Xfunction. As part of this process, additional data is passed the X.I mnemosyne Xfunction, namely the line number where the allocation occurred, Xand the file in which it occurred. A typical application will never Xappear to directly call the X.I mnemosyne Xfunctions, but will simply make calls to "malloc" which are Xintercepted. This approach has the advantage of simplicity and Xportability, with the disadvantage that it is incapable of Xchecking dynamic memory use of library routines that cannot Xbe recompiled with the "intercepted" routines. X.PP X.I mnemosyne Xis intended as a tool for helping to locate memory leaks and Xmemory-profligate routines with a minimal amount of disturbance to Xsource code. As such it can be fooled, but it still offers a Xgreat deal of flexibility for its purpose. X.SH "How it Works" X.PP XWhenever a data pointer is allocated or freed through the Xwrapper routines, the label (the name of the source code file) Xand line number at which the action occurred are searched for Xand possibly added to an internal symbol table. The symbol Xtable is also written to a file named X.I mnem.syms Xwhich is an ascii list of active points in the source code. XEach active pointer in memory (returned from X.I malloc, X.I realloc, Xor X.I calloc, Xis associated with the symbol where its allocation occurred, Xand information about its size is stored in a packed binary Xdatabase in a file named X.I mnem.dat. XThe pointer map is updated every time a pointer is allocated Xor deallocated, which is a perfomance loss, but ensures that Xthe pointer map is current in case the program is terminated Xunexpectedly. It is these two files that X.I mnemalyse Xreads to determine what allocated pointers were never deallocated Xproperly. X.PP XWhenever a pointer is allocated, the symbol table entry for Xthe line of code performing the allocation is updated with Xsome possibly useful statistics about the number of allocations Xthat occurred at that point in the code, average size, etc. XThis information can be dumped to the X.I mnem.syms Xfile with a call to X.I mnem_writestats() Xotherwise the information is not included in the symbol file. XAn example of the symbol statistics resembles: X.EX X#total allocations:9 X#total re-allocations:0 X#total frees:1 X#bad/dup frees:2 X#total allocated never freed:1066 X#average size of allocations:119.2 X#map# calls ave line# file X0 1 922.0 19 mtest.c X1 1 123.0 20 mtest.c X2 7 4.0 27 mtest.c X.EE X.PP XThe internal symbol table management routines and pointer map Xmanagement routines will cause a program using these functions Xto both consume more memory and run longer, but this should not Xpose a problem, since X.I mnemosyne Xis a debugging aid, not a runtime support library. X.SH "Mnemosyne functions" X.PP XSome functions are provided to allow manipulation of the recorder Xand statistics: X.PP X.B "int mnem_recording()" X.br XThis function returns nonzero if the memory-allocation recorder Xis running. X.PP X.B "int mnem_setrecording(value)" X.B "int value;" X.br XIf the value provided is nonzero, the recorder is turned on. If it Xis zero, the recorder is turned off. X.PP X.B "void mnem_setlog(fp)" X.B "FILE *fp;" X.br XSets the runtime error log file pointer to the specified one. The Xdefault is the standard error. If a null pointer is given, runtime Xwarning messages are discarded. X.PP X.B "int mnem_writestats()" X.br XCauses the X.I mnem.syms Xfile to be re-written with extended and updated statistics information. XThis is not necessary for the memory-leak analyser X.I mnemalyse Xto function, but it provides up-to-date information about number of Xallocation calls made, sizes, etc. The function returns nonzero in Xthe event of an error. X.PP X.SH Diagnostics XThe X.I mnemosyne Xfunctions return what their counterpart in the X.I malloc Xlibrary return. X.PP XAdditional diagnostics for detected exceptions may be written to Xthe log file as they occur. Such exceptions include re-freeing an Xalready freed pointer, a call to X.I malloc Xreturning a null pointer, and attempts to free data that was not Xknown to be allocated. For example: X.EX Xmalloc returned null pointer at "mtest.c" line:23 size:-9 Xpointer freed that was never allocated "mtest.c" line:30 Xpointer re-freed when already free "mtest.c" line:34 last allocated: X"mtest.c" line:27 X.EE X.PP XWhenever an internal error occurrs, recording is turned off. This Xwill throw some statistics into doubt. X.SH Restrictions X.PP XLibrary routines that return allocated data to user programs X(such as X.I scandir(3) Xwill cause warnings and trouble when attempts are made to free the Xdata through the "wrapper" routine. If someone wants to hack up a Xversion of the X.I malloc Xlibrary that will cooperate with X.I mnemosyne, Xfeel free to. X.PP XOne problem with using the c-preprocessor to "intercept" free() and Xmalloc() calls is that forward declarations of these in your source Xcode will cause syntax errors. This is unavoidable. Additionally, Xproblems may result if structure elements are named "free" or "malloc", Xwhich is perfectly valid C, but the preprocessor can't cope. A Xfrontend like C++'s X.I cfront Xcan be used, if such is available. X.SH Files X.DT Xmnem.syms Source code line number map X.br Xmnem.dat Pointer allocation map X.SH See Also Xmnemalyse(1l) X.SH Author XMarcus J. Ranum (mjr@decuac.dec.com) END_OF_mnemosyne/mnemosyne.3l if test 6278 -ne `wc -c <mnemosyne/mnemosyne.3l`; then echo shar: \"mnemosyne/mnemosyne.3l\" unpacked with wrong size! fi chmod +x mnemosyne/mnemosyne.3l # end of overwriting check fi echo shar: End of shell archive. exit 0 exit 0 # Just in case... -- Kent Landfield INTERNET: kent@sparky.IMD.Sterling.COM Sterling Software, IMD UUCP: uunet!sparky!kent Phone: (402) 291-8300 FAX: (402) 291-4362 Please send comp.sources.misc-related mail to kent@uunet.uu.net.