padpowell@wateng.UUCP (PAD Powell) (07/19/84)
Thanks to people who replied to my requests on how to do dynamic loading of functions A LA lisp. The following handy set of procedures might make it easier for the next person. Set_program_name, Load_file - support dynamic loading of functions SYNOPSIS struct symbol_references{ char *name; (pointer to entry point name) int (*function)(); (call address) }; Set_program_name( str ) char *str; Load_file( objects, entries, library ) char **objects; struct symbol_references *symbols; char *library; DESCRIPTION The Dynamic Load routines support dynamic loading of func- tions into an already executing program. Set_program_name is called with the name of the currently executing binary file. The PATH environment variable is searched to deter- mine the possible executing files. If PATH was not pro- vided, a default search path is used. If a candidate binary file is not found, then a nonzero value is returned. Load_files is passed a pointer to a list of structures con- taining pointers to strings and corresponding entry point values, a pointer to a list of object of library file names, and a string containing loader options. The structure list is terminated with a structure with a zero entry, as is the object file list (see EXAMPLE below). The UNIX loader ld(1) is invoked, and will search the supplied object files for global entry points of the specified functions or variables. If the load is successful, the entry point values are placed in the structure. The library string can be used to pass additional parameters to the loader, such as a default set of libraries to be searched. EXAMPLE char *objects = { "movies.o", "library.a", 0 }; struct symbol_references look_for[] = { {_good}, {_bad}, {_ugly}, {(char *)0} }; main( argv, argc ) char **argv; int argc; { if( Set_program_name( argv[0] ) == 0 && Load_file(objects, lookfor, "-lmath -ltermlib") == 0 ){ (*look_for[0].function)(); (*look_for[1].function)(); (*look_for[2].function)(); } } Another method of using this would be do have a symbol references table in one of the object files, and to request loading of the table. It is possible to generate the table from a program, by using the C compiler, and to then load the output. If this method is chosen, note that the C com- piler should be run in a writeable directory, like /tmp. SEE ALSO ld(1), lisp(1), cc(1) DIAGNOSTICS Set_program_name and Load_file return 0 upon success, non- zero otherwise. Diagnostic messages are output on stderr using the stdio package. BUGS You must be careful in how you specify your program name. If you are doing peculiar things, you should provide the full path name as the argv[0] value. The entry point names must be specified in ld(2) recogniz- able form. For example, the C compiler will prefix an underscore to external variable names. The F77 compiler will prefix and append underscores. Thus, the variable var will be _var if in a C object file, and _var_ if in a F77 object file. The Load_file routine constructs a command using a fixed size string buffer. If you have lots of entry points, this will overflow, and you will get an error message. One solu- tion to this problem is to create an entry point table as part of an object file. Another solution is to modify the internal function that checks and finds entry points in the symbol table. The code that finds the values of entry points is very crude. It has been partially speeded up to minimize the number of reads that must be done, but will be very slow if a large number of entry points are provided. Again, this may be solved by adding an object file with an entry point table, and determining that table address. *** cut here, and run this shell script with sh *** : Run this shell script with "sh" not "csh" PATH=:/bin:/usr/bin:/usr/ucb export PATH all=FALSE if [ $1x = -ax ]; then all=TRUE fi /bin/echo 'Extracting dynamic_load.3l' sed 's/^X//' <<'//go.sysin dd *' >dynamic_load.3l X.TH "DYNAMIC LOAD" 3L "19 July 1984" X.UC 4 X.SH NAME Set_program_name, Load_file \- support dynamic loading of functions X.SH SYNOPSIS X.PP X.ft B X.nf X.in +.5i X.ti -.5i struct symbol_references{ char *name; (pointer to entry point name) int (*function)(); (call address) }; X.ti -.5i Set_program_name( str ) char *str; X.ti -.5i Load_file( objects, entries, library ) char **objects; struct symbol_references *symbols; char *library; X.ft P X.fi X.in -.5i X.SH DESCRIPTION The X.I "Dynamic Load" routines support dynamic loading of functions into an already executing program. X.I "Set_program_name" is called with the name of the currently executing binary file. The X.B PATH environment variable is searched to determine the possible executing files. If X.B PATH was not provided, a default search path is used. If a candidate binary file is not found, then a nonzero value is returned. X.PP X.I Load_files is passed a pointer to a list of structures containing pointers to strings and corresponding entry point values, a pointer to a list of object of library file names, and a string containing loader options. The structure list is terminated with a structure with a zero entry, as is the object file list (see EXAMPLE below). The UNIX loader X.IR ld (1) is invoked, and will search the supplied object files for global entry points of the specified functions or variables. If the load is successful, the entry point values are placed in the structure. X.PP The library string can be used to pass additional parameters to the loader, such as a default set of libraries to be searched. X.SH EXAMPLE X.PP X.nf char *objects = { "movies.o", "library.a", 0 }; struct symbol_references look_for[] = { {_good}, {_bad}, {_ugly}, {(char *)0} }; main( argv, argc ) char **argv; int argc; { if( Set_program_name( argv[0] ) == 0 && Load_file(objects, lookfor, "-lmath -ltermlib") == 0 ){ (*look_for[0].function)(); (*look_for[1].function)(); (*look_for[2].function)(); } } X.fi X.PP Another method of using this would be do have a symbol references table in one of the object files, and to request loading of the table. It is possible to generate the table from a program, by using the C compiler, and to then load the output. If this method is chosen, note that the C compiler should be run in a writeable directory, like /tmp. X.SH "SEE ALSO" ld(1), lisp(1), cc(1) X.SH DIAGNOSTICS X.I Set_program_name and X.I Load_file return 0 upon success, non-zero otherwise. Diagnostic messages are output on X.I stderr using the X.I stdio package. X.SH BUGS X.PP You must be careful in how you specify your program name. If you are doing peculiar things, you should provide the full path name as the X.I argv[0] value. X.PP The entry point names must be specified in X.IR ld (2) recognizable form. For example, the C compiler will prefix an underscore to external variable names. The F77 compiler will prefix and append underscores. Thus, the variable X.I var will be X.I _var if in a C object file, and X.I _var_ if in a F77 object file. X.PP The X.I Load_file routine constructs a command using a fixed size string buffer. If you have lots of entry points, this will overflow, and you will get an error message. One solution to this problem is to create an entry point table as part of an object file. Another solution is to modify the internal function that checks and finds entry points in the symbol table. X.PP The code that finds the values of entry points is very crude. It has been partially speeded up to minimize the number of reads that must be done, but will be very slow if a large number of entry points are provided. Again, this may be solved by adding an object file with an entry point table, and determining that table address. //go.sysin dd * made=TRUE if [ $made = TRUE ]; then /bin/chmod 644 dynamic_load.3l /bin/echo -n ' '; /bin/ls -ld dynamic_load.3l fi /bin/echo 'Extracting Makefile' sed 's/^X//' <<'//go.sysin dd *' >Makefile CFLAGS = -g all: load test load: main.o dynamic_load.o cc -g -o load main.o dynamic_load.o test: test.o load -o test.o -e _function -e _test_function //go.sysin dd * made=TRUE if [ $made = TRUE ]; then /bin/chmod 644 Makefile /bin/echo -n ' '; /bin/ls -ld Makefile fi /bin/echo 'Extracting dynamic_load.c' sed 's/^X//' <<'//go.sysin dd *' >dynamic_load.c X/* Run time loading of functions. - Basic technique. The loader ld(1) -A option specifies incremental loading. The output of the loader will be a file which can be read into an already executing program. The symbol table of the already executing program must be specifed. The currently executing file is searched for using the entries in the $PATH environment variable. When it is found, the loader is invoked with ln -N Do not make text portion readable or sharable -x no local symbols in symbol table -A <executing file> used to construct new symbol table -T <text end> specifies the text segment origin. -e <name> name of an entry point -o <filename> output file name <files> object files to be used. <library> libraries to be searched, etc. -lc default library Support functions: struct symbol_references{ char *name; (pointer to entry point name) int (*function)(); (call address) }; Set_program_name( str ) Must be called first to set up the program text file name. This is used in the loading process. Load_file( objects, entries, library ) char **objects; struct symbol_references *symbols; char *library; The "objects" points to a list of object file names terminated with a null (char *)0. For example, a static definition char *objects[] = {"lib.a", "usr_lib.b", 0 }; Note: currently, a maximum of 10 object files can be specified. This limit is set by the compile time variable MAXOBJECTS. The "symbols" points to a list of symbol_references structures. Each structure has the name of a function, and a function address field which is set to the name of the function. struct symbol_references symbols[] = { { _ugly }, { _really }, { (char *)0 } }; NOTE: this is acceptable C, missing field initializers are supposed to be acceptable. The "library" should be null (char *)0, or a string containing loader library information. For example: entries = "-lmath -ltermlib" Note that this can also be used to specify a list of object fields, set up "undefined symbol information" to force loading from libraries ("-u _ugly mylib.a -lmath"). IMPLEMENTATION: The functions use the stdio package, and write error messages on STDOUT. If an error is detected, a 1 is returned. The PATHS environment variable is used to determine the executing function. If it is not set, then a default set of paths is used. The sbrk() system call is used to allocate memory. This should have no effect on a properly designed memory allocation package, such as malloc(3). The external entry point names must be specified in the form that the loader needs. For example, the C name "ugly" would have to be specified as "_ugly", and the F77 name "ugly" as "_ugly_". */ #include <sys/param.h> #include <sys/stat.h> #include <sys/file.h> #include <stdio.h> #include <a.out.h> char Program_name[ MAXPATHLEN ]; /* saves the name of the program */ char Symbol_name[ MAXPATHLEN ]; /* name of the new object file */ extern char *Find_text_file(), *Strend(), /* append to end of string */ *strcpy(), *sys_errlist[], *mktemp(); extern int errno; /* system errors */ extern caddr_t sbrk(); /* memory allocation */ struct symbol_references{ char *name; int (*function)(); }; X/* Set_program_name( str ) Checks the file name for execute perms, and then saves the name. */ Set_program_name( str ) char *str; { (void)strcpy( Program_name, str ); if( (str = Find_text_file( Program_name )) == 0 ){ fprintf( stderr, "program name %s not found\n" ); } return( (str == 0) ); } X/* Find_text_file( str ) Find the pathname of the indicated executing file. 1. Get the $PATH environment varible. If none, use a set of default paths. 2. Check each directory in turn, by copying the directory name, then appending the file name. Uses the small Ok_to_exec(str) function */ char * Find_text_file( str ) char *str; { char *path, *directory, *getenv(), *strcpy(); char fullname[ MAXPATHLEN ]; if( (path = getenv("PATH")) == 0 ){ path = ":/usr/ucb:/bin:/usr/bin"; /* default */ } while( *path ){ if( *path == ':' || str[0] == '/' || str[0] == '.'){ if( Ok_to_exec( str ) ){ return( str ); } } else { /* copy directory name */ for( directory = fullname; *path && *path != ':'; *directory++ = *path++ ); if( *path ) ++path; /* skip : */ /* form full path name */ if( strlen( fullname ) + strlen(str)+2 > MAXPATHLEN ){ fprintf(stderr, "Find_text_file: file name too long" ); return( (char *)0 ); } *directory++ = '/'; *directory = 0; (void)strcpy( directory, str ); if( Ok_to_exec( fullname ) ){ return( strcpy( str, fullname ) ); } } } return( (char *)0 ); } X/* Ok_to_exec( str ) check perms for execution. */ Ok_to_exec( str ) char *str; { struct stat stbuff; return( 0 == stat(str, &stbuff) /* stat it */ && (stbuff.st_mode&S_IFMT) == S_IFREG /* check for file */ && 0 == access(str,X_OK) /* and can be executed */ ); } X/* Load_file( objects, entries, library ) NOTE: the number of object files is limited to 10 in this implementation. 1. generate a command for the loader. NOTE: this uses the execv function. 2. execute the loader command. 3. get the entry points */ #define round(x,s) ((((x)-1) & ~((s)-1)) + (s)) int Load_file( objects, entries, library ) char **objects; struct symbol_references *entries; char *library; { char *file; /* temp variable */ char cmd_str[2*MAXPATHLEN + 2048], *cmd, *cmdend; /* command string for loader */ static char *file_proto = "/tmp/llXXXXXX"; /* tmp name */ int readsize, /* total amount of object to read */ totalsize, /* total memory needed */ n, diff; /* working variables */ caddr_t end; /* current end of memory */ int fcb; /* used to read the object file */ struct exec header; /* a.out header */ struct symbol_references *entry; /* single pointer */ /* create a temp file to hold output from loader */ (void) strcpy( Symbol_name, mktemp(file_proto) ); /* force end of memory to be aligned to a 1024 byte boundary */ /* note: this restriction is applied by the loader */ end = sbrk( 0 ); /* find the currend end */ n = (int)end; diff = round(n,1024)-n; /* get the difference */ # ifdef DEBUG printf( "end %x, diff %x\n", end, diff ); # endif DEBUG if( diff != 0 ){ /* use sbrk to round up */ end = sbrk( diff ); if( (int)end <= 0 ){ fprintf( stderr, "sbrk failed: %s\n", sys_errlist[errno] ); return( 1 ); } end = sbrk( 0 ); } # ifdef DEBUG printf( "sbrk sets end to be 0x%x\n", end ); # endif DEBUG /* make up a command */ cmd = cmd_str; cmdend = cmd + sizeof( cmd_str ); /* find the loader name */ file = Find_text_file( "ld" ); if( file == 0 ){ fprintf( stderr, "cannot find loader\n" ); return( 1 ); } /* set up the first part of the command string */ sprintf( cmd, "%s -N -x -A %s -T %x -o %s ", file, /* loader */ Program_name, /* program text file */ end, /* end of memory */ Symbol_name /* executable file */ ); # ifdef DEBUG printf( "command: %s\n", cmd ); # endif DEBUG /* set cmd to end of string */ cmd = cmd + strlen( cmd ); /* now add the entry points */ entry = entries; if( entry->name == 0 ){ fprintf( stderr,"missing entry name" ); return( 1 ); } cmd = Strend( cmd, "-e", cmdend ); if( cmd == 0 ){ fprintf( stderr, "too many entries\n" ); } cmd = Strend( cmd, entry->name, cmdend ); if( cmd == 0 ){ fprintf( stderr, "too many entries\n" ); } ++entry; /* set up the rest */ for( ; entry->name; ++entry ){ cmd = Strend( cmd, "-u", cmdend ); if( cmd == 0 ){ fprintf( stderr, "too many entries\n" ); } cmd = Strend( cmd, entry->name, cmdend ); if( cmd == 0 ){ fprintf( stderr, "too many entries\n" ); } } /* now add the object files */ for( ; *objects; ++objects ){ cmd = Strend( cmd, *objects, cmdend ); if( cmd == 0 ){ fprintf( stderr, "too many objects\n" ); } } /* now add the library */ if( library && *library ){ cmd = Strend( cmd, library, cmdend ); if( cmd == 0 ){ fprintf( stderr, "library too long\n" ); } } /* now add the defaults */ cmd = Strend( cmd, "-lc", cmdend ); if( cmd == 0 ){ fprintf( stderr, "total loader command too long\n" ); } # ifdef DEBUG printf( "loader command %s\n", cmd_str ); # endif DEBUG if( (n = system( cmd_str )) != 0 ){ (void)unlink( Symbol_name ); fprintf( stderr, "load of objects and entries failed\n"); return( 1 ); } # ifdef DEBUG printf( "load was successful\n" ); # endif DEBUG /* now try and read the information from the symbol table */ if( (fcb = open( Symbol_name, O_RDONLY)) < 0){ fprintf( stderr, "cannot open temp file %s: %s\n", Symbol_name, sys_errlist[errno] ); return( 1 ); } # ifdef DEBUG printf( "output file opened successfully\n" ); # endif DEBUG /* read the a.out header and find out how much room to allocate */ if(sizeof(header)!=read(fcb,(char *)&header,sizeof( header ))){ fprintf( stderr, "cannot read header from temp file %s\n", Symbol_name ); return( 1 ); } /* calculate sizes */ readsize = round(header.a_text, 4) + round(header.a_data, 4); totalsize = readsize + header.a_bss; totalsize = round( totalsize, 512 ); /* round up a bit */ # ifdef DEBUG printf( "read header: a_text %d, a_data %d, a_entry 0x%x\n", header.a_text, header.a_data, header.a_entry); printf( "readsize %d, totalsize %d\n", readsize, totalsize ); # endif DEBUG /* allocate more memory, using sbrk */ end = sbrk( totalsize ); # ifdef DEBUG printf( "sbrk(0x%x) returns 0x%x\n", totalsize, end ); # endif DEBUG if( (int)end <= 0 ){ fprintf(stderr, "sbrk failed to allocate: %s\n", sys_errlist[errno] ); return( 1 ); } # ifdef DEBUG printf( "end is now 0x%x\n", end ); # endif DEBUG /* now read in the functions */ if(readsize != read(fcb,(char *)end,readsize)){ fprintf( stderr, "cannot read %s: %s\n", Symbol_name, sys_errlist[errno] ); return( 1 ); } /* set the first entry up to the header value */ entry = entries; entry->function = (int (*)())header.a_entry; ++entry; if( entry->name ){ if( Find_symbol_names( entry, fcb, header ) ){ fprintf( stderr, "failed to find all symbols\n" ); return( 1 ); } } (void)close(fcb); return( 0 ); } X/* Append a string and a space, check for overflow */ char * Strend( str, append, last ) char *str, *append, *last; { while( str < last && *append ){ *str++ = *append++; } if( *append ){ return( (char *)0 ); } if( str+1 < last ){ *str++ = ' '; *str = 0; } else { return( (char *)0 ); } return( str ); } X/* Look through the symbol table and find entries */ #define NAMES 100 /* number of entries to read at a time */ Find_symbol_names( entries, fcb, header ) struct symbol_references *entries; /* list of entries */ int fcb; /* symbol file fcb */ struct exec header; /* symbol file header */ { int symbols, strings; /* offsets to symbols and strings */ int c, i, n; /* temp variable */ int next_symbol; /* next symbol */ struct nlist name[NAMES], *nl; /* entry for a symbol */ char *str_val; /* string */ int str_offset; /* offset into string tables */ int str_start, str_end; /* start and end location of str */ char str_buff[MAXPATHLEN]; /* should be nice and big */ char *s; /* temp variable */ symbols = N_SYMOFF(header); /* offset to symbols */ strings = N_STROFF(header); /* offset to strings */ str_start = str_end = 0; # ifdef DEBUG printf( "symbols %d, strings %d\n", symbols, strings ); # endif DEBUG for( next_symbol = symbols; next_symbol < strings; ){ /* read in the necessary entries */ if( lseek( fcb, next_symbol, 0 ) == -1 ){ fprintf( stderr,"lseek on symbol file failed: %s\n", sys_errlist[errno] ); return( 1 ); } if( sizeof(name) != (n = read(fcb,(char *)name,sizeof(name)))){ if( n <= 0 ){ fprintf( stderr, "read on symbol file failed: %s\n", sys_errlist[errno] ); } } n = n / sizeof( struct nlist ); # ifdef DEBUG printf( "read %d symbols\n",n ); # endif DEBUG for( i=0; i < n && next_symbol < strings; ++i ){ next_symbol += sizeof( struct nlist ); nl = name+i; # ifdef DEBUG /* print the information */ printf( "n_un.n_strx %d, n_type %d, n_desc %d, n_value %d\n", nl->n_un.n_strx, nl->n_type, nl->n_desc, nl->n_value); # endif DEBUG if( (nl->n_type & N_EXT) ){ # ifdef DEBUG printf( "external symbol\n" ); # endif DEBUG /* seek to the string location */ readnext: str_offset = strings + nl->n_un.n_strx; # ifdef DEBUG printf( "string offset %d\n", str_offset); # endif DEBUG /* get string in buffer */ /* NOTE: this is the first time I have ever found a use for a "goto"; I suppose I could have made the following a function. */ while( str_offset < str_start || str_offset > str_end ){ /* seek to position and read */ if( lseek(fcb, str_offset, 0 ) == -1){ fprintf( stderr, "lseek on symbol file failed: %s\n", sys_errlist[errno] ); return( 1 ); } if( (c = read(fcb, str_buff, sizeof( str_buff ) )) <= 0 ){ fprintf( stderr, "read on symbol file failed: %s\n", sys_errlist[errno] ); return( 1 ); } str_start = str_offset; str_end = str_start + c; } str_val = s = str_buff + (str_offset - str_start); while( *s && str_offset < str_end ){ ++s; ++str_offset; } if( *s ){ str_offset = strings + nl->n_un.n_strx; if( str_offset != str_start ){ str_end = 0; /* force read */ goto readnext; } else { fprintf( stderr, "symbol too long\n" ); return( 1 ); } } # ifdef DEBUG printf( "str_val %s\n", str_val ); # endif DEBUG if( Check_entry_for_value( str_val, (unsigned)nl->n_value,entries) == 0 ){ return(0); } } } } return( 0 ); } X/* Check_entry_for_value( str, val, entries ) char *str: string name unsigned val: value for it struct symbol_references *entries: list of entries 1. for each un-set entry, check to see if string matches. 2. if it does, then you set the function value. */ int Check_entry_for_value( str, val, entries ) char *str; unsigned val; struct symbol_references *entries; { struct symbol_references *entry; int found; found = 0; for( entry = entries; entry->name; ++entry ){ if( entry->function == 0 ){ found = 1; if( strcmp( str, entry->name ) == 0 ){ entry->function = (int (*)())val; # ifdef DEBUG printf( "update %s with %d\n", entry->name, entry->function ); # endif DEBUG return( found ); } } } return( found ); } //go.sysin dd * made=TRUE if [ $made = TRUE ]; then /bin/chmod 644 dynamic_load.c /bin/echo -n ' '; /bin/ls -ld dynamic_load.c fi /bin/echo 'Extracting main.c' sed 's/^X//' <<'//go.sysin dd *' >main.c X/* Test function for the dynamic loader */ #include <stdio.h> char *Program; struct symbol_references{ char *name; int (*function)(); }; main( argc, argv) int argc; char **argv; { char *objects[100], **obj; struct symbol_references refs[100], *ref; char *parm; /* get the name of the object file, and an ref point */ printf( "argc %d, argv[0] %s\n", argc, argv[0] ); Program = argv[0]; if( argc < 2 ){ Use_msg(); } ++argv; --argc; ref = refs; obj = objects; while( argc-- > 0 ){ parm = *argv++; printf( "parm %d, %s\n", argc, parm ); if( *parm != '-' ){ Use_msg(); } switch( *++parm ){ case 'o': *obj++ = *argv++; printf( "Object %s\n", argv[-1] ); argc--; break; case 'e': (ref++)->name = *argv++; printf( "Entry %s\n", argv[-1] ); argc--; break; default: Use_msg(); break; } } /* now find the name of the currently executing file */ if( Set_program_name( Program ) ){ fprintf( stderr, "cannot find program file %s\n", Program ); exit(1); } if( Load_file( objects, refs, "" ) ){ fprintf( stderr, "cannot load objects\n" ); exit( 1 ); } for( ref = refs; ref->name; ++ref ){ printf( "ref point for %s is %d\n", ref->name, ref->function ); } for( ref = refs; ref->name; ++ref ){ printf( "calling %s\n", ref->name ); (*ref->function)(); } return( 0 ); } Use_msg() { fprintf( stderr, "%s: -e entry_point -o object\n", Program ); exit(1); } //go.sysin dd * made=TRUE if [ $made = TRUE ]; then /bin/chmod 644 main.c /bin/echo -n ' '; /bin/ls -ld main.c fi /bin/echo 'Extracting test.c' sed 's/^X//' <<'//go.sysin dd *' >test.c X/* test file */ #include <stdio.h> function() { printf( "Hello, world\n" ); } test_function() { printf( "Hi, I am your friendly loadable function\n" ); } //go.sysin dd * made=TRUE if [ $made = TRUE ]; then /bin/chmod 644 test.c /bin/echo -n ' '; /bin/ls -ld test.c fi