jac@yoko.rutgers.edu (Jonathan A. Chandross) (05/02/91)
Submitted-by: Andy McFadden (fadden@cory.berkeley.edu) Posting-number: Volume 1, Source:38 Archive-name: archive/unix/nulib/part07.10 Architecture: UNIX Version-number: 3.03 =Benchmarks -Some (old) benchmarks for NuLib: - -Uncompressing Moria GS (from 320K to 577K, ShrinkIt Type I LZW): - -Program Run / Machine Used Run Time CPU Time ----------------------------------------------- -------------- -------------- -NuLib v2.1 (APW C), running on a //gs 12 min -ShrinkIt v3.0.1, running on a //gs (HD) 2 min 03 sec -ShrinkIt v3.0.1, running on a //gs (RAM disk) 1 min 21 sec -GS/ShrinkIt v1.0.2 (HD) 1 min 30 sec -NuLib v2.2 on a lightly loaded VAX 11/750 1 min 33 sec 1 min 28.4 sec -NuLib v2.2 on a lightly loaded VAX 11/785 38 sec 34.0 sec -NuLib v2.2 on a lightly loaded Sun 3/50 17 sec 13.0 sec -NuLib v2.2 on an empty Apollo workstation 12 sec 11.5 sec -NuLib v2.2 on an empty DEC 3100 9 sec 2.6 sec -NuLib v2.2 on an empty Sun 4 7 sec 2.2 sec - -"Run Time" is total time from start to finish. "CPU Time" is CPU time -actually used under a multiprogrammed OS (in this case, UNIX). - -The hard drive (HD) used on the //gs is an AI InnerDrive 20. The results -obtained from running on a RAM disk could almost be considered the CPU -time, but I've decided to keep them separate. All versions of NuLib -running on UNIX systems used the optimization features of the C compiler. - -Doing an uncompressed extract can produce remarkable results... like -running to completion in less than one second on the Apollo (local disk). - -Interestingly enough, 16-bit UNIX uncompress works faster than NuLib does. -The reason for this is that the compression code in NuLib actually makes -two passes, one to undo the LZW compression, and the other to undo -non-repeat compression. Was it worth it do to both kinds? For Moria it -was; even though "uncompress" ran faster, ShrinkIt's 12-bit LZW packed -the file smaller than compress's 16-bit LZW. - -No benchmarks for compress yet. Be patient. - -********** - -Raw data for benchmarks - -Times on the //gs were from start to finish of compression, timed with a -stop watch. - -All tests were performed in verbose mode (i.e., the filename was printed -as the file was extracted, etc). This is how I would expect it to be -run under "normal" conditions. It does add to the overhead, which -you can pull out by looking at the data below. - -/bin/csh time format appears to be (the man page is most unhelpful): -user_time system_time real_time cpu% ??+??k ??+??io page_faults+??w - -user_time is the time actually spent grinding along. This is what appears - in the "CPU time" column. -system_time is for I/O (disk accesses, writing to terminal). -real_time is how long a stopwatch would show (user_time + system_time + - overhead starting/stopping NuLib + time spent running other processes - on a multiprogrammed OS like UNIX). This appears in the "Run Time" col. -cpu% appears to be some kind of estimate of how much time was spent doing - useful work. -If you don't know what a page fault is, don't worry about it. - -Both cory and holden used the same copy of NuLib (well not the *same* copy, -but they're identical), so any differences in speed can be attributed -to the processor. - -***** holden.berkeley.edu, a VAX 11/750 ***** -Script started on Sun Mar 25 11:52:52 1990 ---- reading aliases...done -holden 1 c184-cz% ls -s tmp/nulib - 68 tmp/nulib* -holden 2 c184-cz% uptime - 11:52am up 26 days, 3:59, 4 users, load average: 0.31, 0.17, 0.13 -holden 3 c184-cz% time tmp/nulib xv moria.shk -Extracting 'MORIA' (data)...unshrinking (I)...-\|/-done. -88.4u 3.3s 1:33 98% 83+252k 49+89io 9pf+0w -holden 4 c184-cz% exit -holden 5 c184-cz% -script done on Sun Mar 25 11:55:34 1990 - -***** cory.berkeley.edu, a VAX 11/785 ***** -Script started on Sun Mar 25 11:50:07 1990 ---- reading aliases...done -cory 1 foo% ls -s ../bin/nulib - 68 ../bin/nulib* -cory 2 foo% uptime - 11:50am up 2 days, 2:04, 6 users, load average: 0.55, 0.39, 0.27 -cory 3 foo% time nulib xv moria.shk -Extracting 'MORIA' (data)...unshrinking (I)...-\|/-done. -34.0u 1.6s 0:37 95% 51+244k 46+82io 5pf+0w -cory 4 foo% exit -cory 5 foo% -script done on Sun Mar 25 11:51:29 1990 - -***** a Sun 3/50 ***** -Script started on Sat Apr 21 17:00:43 1990 -ux1:nulib% ls -s nulib - 64 nulib* -ux1:nulib% uptime - 5:01pm up 2 days, 10:06, 6 users, load average: 1.84, 1.66, 1.51 -ux1:nulib% time nulib xv moria.shk -Extracting 'MORIA' (data)...unshrinking (I)...-\|/-done. -13.0u 0.6s 0:17 76% 0+168k 1+78io 0pf+0w -ux1:nulib% exit -ux1:nulib% -script done on Sat Apr 21 17:02:33 1990 - -***** avalanche.berkeley.edu, an Apollo DN3500 ***** -Script started Sat Apr 21 17:04:53 1990 ---- reading aliases...done -avalanche 1 nulib% ls -s nulib - 85 nulib* -avalanche 2 nulib% uptime - 5:05pm up 1 day, 1:50, 3 users, load average: 0.06, 0.16, 0.00 -avalanche 3 nulib% time nulib xv ../dwn/moria.shk -Extracting 'MORIA' (data)...unshrinking (I)...-\|/-done. -11.5u 2.6s 0:16 86% 0+0k 577+0io 0pf+0w -avalanche 6 nulib% exit -avalanche 7 nulib% -Script done Sat Apr 21 17:06:59 1990 - -***** a DEC 3100 ***** -Script started on Sat Apr 21 16:18:10 1990 -vampire:nulib% ls -s nulib - 88 nulib* -vampire:nulib% uptime - 4:18pm up 30 days, 5 hrs, 2 users, load average: 0.57, 0.25, 0.00 -vampire:nulib% time nulib xv moria.shk -Extracting 'MORIA' (data)...unshrinking (I)...-\|/-done. -2.6u 0.5s 0:09 32% 163+404k 71+71io 3pf+0w -vampire:nulib% exit -vampire:nulib% -script done on Sat Apr 21 16:19:40 1990 - -***** a Sun 4 ***** -Script started on Sat Apr 21 16:23:47 1990 -ux5:nulib% ls -s nulib - 64 nulib* -ux5:nulib% uptime - 4:23pm up 2 days, 8:50, 7 users, load average: 0.15, 0.24, 0.03 -ux5:nulib% time nulib xv moria.shk -Extracting 'MORIA' (data)...unshrinking (I)...-\|/-done. -2.2u 0.2s 0:07 34% 0+368k 0+71io 0pf+0w -ux5:nulib% exit -ux5:nulib% -script done on Sat Apr 21 16:25:20 1990 - =nuread.h -/* - * nuread.h - linked list structures used for holding NuFX header data, - * and structure definitions for archive innards - * - * (this will be included by all source files which access NuFX archives) - * - * NuLib v3.0 February 1991 Freeware (distribute, don't sell) - * By Andy McFadden (fadden@cory.berkeley.edu) - */ - - -/* The NuFX master format version we output, and the maximum we can extract */ -#define OURMVERS 2 -#define MAXMVERS 2 - -/* The NuFX record format version we output, and the maximum we can extract */ -#define OURVERS 0 -#define MAXVERS 3 - -/* "NuFile" in alternating ASCII */ -static onebyt MasterID[7] = { 0x4e, 0xf5, 0x46, 0xe9, 0x6c, 0xe5, 0x0 }; - -/* "NuFX" in alternating ASCII */ -static onebyt RecordID[5] = { 0x4e, 0xf5, 0x46, 0xd8, 0x0 }; - - -/* - * Structure definitions for NuFX innards - */ - -/* master header block */ -typedef struct { - onebyt ID[6]; - twobyt master_crc; - fourbyt total_records; - Time arc_create_when; - Time arc_mod_when; - twobyt master_version; - onebyt reserved1[8]; - fourbyt master_eof; - onebyt reserved2[6]; -} MHblock; -#define MHsize 48 /* this should not change */ - -/* record header block */ -typedef struct { - onebyt ID[4]; - twobyt header_crc; - twobyt attrib_count; - twobyt version_number; - twobyt total_threads; - twobyt reserved1; - twobyt file_sys_id; - onebyt file_sys_info; - onebyt reserved2; - fourbyt access; - fourbyt file_type; - fourbyt extra_type; - twobyt storage_type; - Time create_when; - Time mod_when; - Time archive_when; - twobyt option_size; - /* future expansion here... */ -} RHblock; -#define RHsize 58 /* sizeof(RHblock) should work, but might not */ -#define ATTSIZE 64 /* default attrib_count when creating new */ - -/* - * This buffer must be able to contain three things (not all at once): - * - The master header block (size = MHsize) - * - The record header block (size = RHsize) - * - Attributes not specified in the RHblock (attrib_count - RHsize - 2) - * - * Currently, it only needs to be 64 bytes. Since it is allocated as local - * storage only once during execution, making it reasonably large should - * not cause any problems in performance but will make the program stable - * if it encounters an archive with a drastically expanded RHblock. - */ -#define RECBUFSIZ 256 - -/* thread record */ -typedef struct { - twobyt thread_class; - twobyt thread_format; - twobyt thread_kind; - twobyt thread_crc; - fourbyt thread_eof; - fourbyt comp_thread_eof; -} THblock; -#define THsize 16 /* this should not change */ - - -/* - * Definitions for the linked lists - * A linked list of Record headers, with linked lists of Threads attached - */ - -/* thread nodes */ -typedef struct TNode_s { - THblock *THptr; /* points to thread info */ - long fileposn; /* absolute position of this thread in the file */ - struct TNode_s *TNext; /* points to next thread node */ -} TNode; - -/* record nodes */ -typedef struct RNode_s { - RHblock *RHptr; /* points to the record header block */ - char *filename; /* filename of record */ - twobyt filename_length; /* length of filename (as stored in record hdr) */ - twobyt real_fn_length; /* length of filename (actual) */ - TNode *TNodePtr; /* points to first thread node */ - fourbyt unc_len; /* total uncompressed length of all threads */ - fourbyt comp_len; /* total compressed length of all threads */ - struct RNode_s *RNext; /* points to next record node */ -} RNode; - -/* head of list */ -typedef struct { - char *arc_name; /* filename of archive */ - MHblock *MHptr; /* points to master header */ - RNode *RNodePtr; /* points to first record node */ - long nextposn; /* abs. position in file to put next record (for ADD) */ -} ListHdr; - - -/* - * function declarations - */ - -extern ListHdr *NuRead(); /* read archive info into memory */ -extern void NuTest(); /* archive integrity check */ -extern twobyt CalcCRC(); /* calculate a CRC on a range of bytes */ -extern char *PrintDate(); /* print a date from a (Time *) struct */ -extern void BCopy(); /* copy bytes: *src, *dest, num, order? */ -extern void HiSwap(); /* swap bytes (maybe): *ptr, src_index, dst_index */ =nudefs.h -/* - * nudefs.h - system-dependent typdefs and #defines for all NuLib files, - * plus extern declarations for global variables (allocated in - * numain.c). - * - * IMPORTANT: This file must be first on the list of #includes, since some - * include files will be processed based on these #defines - * - * NuLib v3.0 February 1991 Freeware (distribute, don't sell) - * By Andy McFadden (fadden@cory.berkeley.edu) - */ - - -/* SYSTEM DEPENDENCIES */ -typedef unsigned char onebyt; -typedef unsigned short twobyt; -typedef unsigned long fourbyt; - -/* byte ordering; TRUE if high byte is first (68xxx), else FALSE (65xxx) */ -extern int HiLo; /* actually part of numain.c */ - -/* Setup for Apple //gs APW */ -/* [ "APW" is automatically defined by the compiler ] */ - -/* Setup for MS-DOS machines (80xxx based: IBM PC) */ -/* [ "MSDOS" is defined by the compiler [MS C5.1 anyway] ] */ - -/* Setup for other microcomputers (Macintosh, Amiga) */ -/* ??? */ - -/* Setup for AIX */ -/* #define SYSV */ -/* #define BSD_INCLUDES */ - -/* Setup for BSD UNIX systems */ -#define UNIX -#define BSD43 -/* #define VAX */ - -/* Setup for XENIX/386 */ -/* NOTE: if you get error messages about readdir() and opendir() during */ -/* linking, remove the leading '#' from the line "#CLIBS= -lx" in */ -/* "Makefile" */ -/* #define XENIX386 */ -/* #define UNIX */ -/* #define SYSV */ -/* #define NO_RENAME */ /* no rename() call in C library */ - -/* Setup for AOS/VS @ DG */ -/* #define UNIX */ -/* #define DATAGENERAL */ -/* #define AOSVS */ - -/* Setup for other UNIX systems */ -/* #define UNIX */ -/* #define SYSV (or whatever) */ - -/* use table lookups to get CRCs */ -#define CRC_TAB - -/* - * Some global defs - */ - -/* errno wasn't defined in <errno.h> on some systems... */ -#ifndef MSDOS -extern int errno; -#endif - -/* Maximum file name length that we intend to handle (watch stack size!) */ -#define MAXFILENAME 1024 - -/* file operations */ -#define S_ABS 0 /* seek absolute */ -#define S_REL 1 /* seek relative */ -#define S_END 2 /* seek from end */ - -#ifdef UNIX /* stuff for open() */ -# define WPERMS 0644 /* read/write for owner, read only for others */ -# define FREAD_STR "r" -# define FWRITE_STR "w" -# define O_BINARY 0 /* for non-UNIX open(); easier than #ifdefs */ -#else -# ifdef APW -# define WPERMS 0666 /* read/write for all; this may not work for some */ -# define FREAD_STR "rb" -# define FWRITE_STR "wb" -# endif -# ifdef MSDOS -# define S_IREAD 0000400 /* read permission, owner */ -# define S_IWRITE 0000200 /* write permission, owner */ -# define WPERMS S_IREAD | S_IWRITE -# define FREAD_STR "rb" -# define FWRITE_STR "wb" -# endif -# ifndef WPERMS /* other system */ -# define WPERMS 0666 /* +PORT+ */ -# define FREAD_STR "rb" -# define FWRITE_STR "wb" -# endif -#endif /*UNIX*/ - -/* Time structure; same as TimeRec from misctool.h */ -/* one-byte entries should not have alignment problems... */ -typedef struct { - onebyt second; - onebyt minute; - onebyt hour; - onebyt year; - onebyt day; - onebyt month; - onebyt extra; - onebyt weekDay; -} Time; - - -/* - * global to entire program - */ -extern int HiLo; /* byte ordering; FALSE on low-first (65816) */ -extern int verbose; /* BOOLEAN: print verbose? */ -extern int interact; /* BOOLEAN: interactive when overwriting? */ -extern int dopack; /* BOOLEAN: do we want to pack/unpack? */ -extern int doSubdir; /* BOOLEAN: expand subdirectories? */ -extern int doExpand; /* BOOLEAN: expand archived filenames? */ -extern int doMessages; /* BOOLEAN: do comments instead of data? */ -extern int transfrom; /* how to do CR<->LF translation? (-1 = none) */ -extern int transto; -extern int packMethod; /* how to pack a file (thread_format) */ -extern fourbyt defFileType; /* default file type */ -extern fourbyt defAuxType; /* default aux type */ -extern onebyt *pakbuf; /* used by compression routines; created once to */ - /* eliminate overhead involved in malloc()ing a 64K buffer */ -extern char *prgName; /* for errors; don't like argv[0] */ - =crc.h -/* - * crc.h - tables for fast computation of 16-bit CRCS - * - * NuLib v3.0 February 1991 Freeware (distribute, don't sell) - * By Andy McFadden (fadden@cory.berkeley.edu) - */ - -/* - * updcrc macro derived from article Copyright (C) 1986 Stephen Satchell. - * NOTE: First srgument must be in range 0 to 255. - * Second argument is referenced twice. - * - * Programmers may incorporate any or all code into their programs, - * giving proper credit within the source. Publication of the - * source routines is permitted so long as proper credit is given - * to Stephen Satchell, Satchell Evaluations and Chuck Forsberg, - * Omen Technology. - */ - - -/*#define updcrc(cp, crc) ( crctab[((crc >> 8) & 255)] ^ (crc << 8) ^ cp)*/ -#define updcrc(cp, crc) ( (crctab[((crc >> 8) & 0xFF) ^ cp] ^ (crc << 8)) & 0xFFFF) - - -/* crctab calculated by Mark G. Mendel, Network Systems Corporation */ -static unsigned short crctab[256] = { - 0x0000, 0x1021, 0x2042, 0x3063, 0x4084, 0x50a5, 0x60c6, 0x70e7, - 0x8108, 0x9129, 0xa14a, 0xb16b, 0xc18c, 0xd1ad, 0xe1ce, 0xf1ef, - 0x1231, 0x0210, 0x3273, 0x2252, 0x52b5, 0x4294, 0x72f7, 0x62d6, - 0x9339, 0x8318, 0xb37b, 0xa35a, 0xd3bd, 0xc39c, 0xf3ff, 0xe3de, - 0x2462, 0x3443, 0x0420, 0x1401, 0x64e6, 0x74c7, 0x44a4, 0x5485, - 0xa56a, 0xb54b, 0x8528, 0x9509, 0xe5ee, 0xf5cf, 0xc5ac, 0xd58d, - 0x3653, 0x2672, 0x1611, 0x0630, 0x76d7, 0x66f6, 0x5695, 0x46b4, - 0xb75b, 0xa77a, 0x9719, 0x8738, 0xf7df, 0xe7fe, 0xd79d, 0xc7bc, - 0x48c4, 0x58e5, 0x6886, 0x78a7, 0x0840, 0x1861, 0x2802, 0x3823, - 0xc9cc, 0xd9ed, 0xe98e, 0xf9af, 0x8948, 0x9969, 0xa90a, 0xb92b, - 0x5af5, 0x4ad4, 0x7ab7, 0x6a96, 0x1a71, 0x0a50, 0x3a33, 0x2a12, - 0xdbfd, 0xcbdc, 0xfbbf, 0xeb9e, 0x9b79, 0x8b58, 0xbb3b, 0xab1a, - 0x6ca6, 0x7c87, 0x4ce4, 0x5cc5, 0x2c22, 0x3c03, 0x0c60, 0x1c41, - 0xedae, 0xfd8f, 0xcdec, 0xddcd, 0xad2a, 0xbd0b, 0x8d68, 0x9d49, - 0x7e97, 0x6eb6, 0x5ed5, 0x4ef4, 0x3e13, 0x2e32, 0x1e51, 0x0e70, - 0xff9f, 0xefbe, 0xdfdd, 0xcffc, 0xbf1b, 0xaf3a, 0x9f59, 0x8f78, - 0x9188, 0x81a9, 0xb1ca, 0xa1eb, 0xd10c, 0xc12d, 0xf14e, 0xe16f, - 0x1080, 0x00a1, 0x30c2, 0x20e3, 0x5004, 0x4025, 0x7046, 0x6067, - 0x83b9, 0x9398, 0xa3fb, 0xb3da, 0xc33d, 0xd31c, 0xe37f, 0xf35e, - 0x02b1, 0x1290, 0x22f3, 0x32d2, 0x4235, 0x5214, 0x6277, 0x7256, - 0xb5ea, 0xa5cb, 0x95a8, 0x8589, 0xf56e, 0xe54f, 0xd52c, 0xc50d, - 0x34e2, 0x24c3, 0x14a0, 0x0481, 0x7466, 0x6447, 0x5424, 0x4405, - 0xa7db, 0xb7fa, 0x8799, 0x97b8, 0xe75f, 0xf77e, 0xc71d, 0xd73c, - 0x26d3, 0x36f2, 0x0691, 0x16b0, 0x6657, 0x7676, 0x4615, 0x5634, - 0xd94c, 0xc96d, 0xf90e, 0xe92f, 0x99c8, 0x89e9, 0xb98a, 0xa9ab, - 0x5844, 0x4865, 0x7806, 0x6827, 0x18c0, 0x08e1, 0x3882, 0x28a3, - 0xcb7d, 0xdb5c, 0xeb3f, 0xfb1e, 0x8bf9, 0x9bd8, 0xabbb, 0xbb9a, - 0x4a75, 0x5a54, 0x6a37, 0x7a16, 0x0af1, 0x1ad0, 0x2ab3, 0x3a92, - 0xfd2e, 0xed0f, 0xdd6c, 0xcd4d, 0xbdaa, 0xad8b, 0x9de8, 0x8dc9, - 0x7c26, 0x6c07, 0x5c64, 0x4c45, 0x3ca2, 0x2c83, 0x1ce0, 0x0cc1, - 0xef1f, 0xff3e, 0xcf5d, 0xdf7c, 0xaf9b, 0xbfba, 0x8fd9, 0x9ff8, - 0x6e17, 0x7e36, 0x4e55, 0x5e74, 0x2e93, 0x3eb2, 0x0ed1, 0x1ef0 -}; - =nucompfn.h -/* - * nucompfn.h - function declarations for nucomp.c - * - * NuLib v3.0 February 1991 Freeware (distribute, don't sell) - * By Andy McFadden (fadden@cory.berkeley.edu) - */ - -extern int u_compress(), - u_decompress(); - -/* COMPRESS.FNS global function declarations */ -/* this should be compatible with any type of declaration for external - functions. See compress.h for explaination */ -#ifdef NPROTO -extern void Usage(); -extern int check_error(); -extern char *name_index(); -extern char *get_program_name(); -#ifdef NO_STRCHR -extern char *strchr(); -#endif -#ifdef NO_STRRCHR -extern char *strrchr(); -#endif -#ifdef NO_REVSEARCH -extern char *strrpbrk(); -#endif -extern int is_z_name(); -extern int cl_block(); -extern int make_z_name(); -extern void unmake_z_name(); -extern void compress(); -extern void putcode(); -extern void decompress(); -extern CODE getcode(); -extern void writeerr(); -extern void copystat(); -#ifndef NOSIGNAL -extern int foreground(); -extern SIGTYPE onintr(); -extern SIGTYPE oops(); -#endif -extern void prratio(); -extern void version(); -#ifdef NEARHEAP -extern ALLOCTYPE *emalloc(); -extern void efree(); -#else -extern ALLOCTYPE FAR *emalloc(); -extern void efree(); -#endif -extern int alloc_tables(); -extern void init_tables(); -extern int nextcode(); -#else -extern void Usage(int); -extern int check_error(void); -extern char *name_index(char *); -extern int cl_block(void); -extern char *get_program_name(char *); -extern int is_z_name(char *); -extern int make_z_name(char *); -extern void unmake_z_name(char *); -#ifdef NO_STRCHR -extern char *strchr(char *,int); -#endif -#ifdef NO_STRRCHR -extern char *strrchr(char *,int); -#endif -#ifdef NO_REVSEARCH -extern char *strrpbrk(char *,char *); -#endif -extern void compress(void); -extern void putcode(CODE,int); -extern void decompress(void); -extern CODE getcode(void); -extern void writeerr(void); -extern void copystat(char *,char *); -#ifndef NOSIGNAL -extern int foreground(void); -extern SIGTYPE onintr(void); -extern SIGTYPE oops(void); -#endif -extern void prratio(FILE *,long,long); -extern void version(void); -#ifdef NEARHEAP -extern ALLOCTYPE *emalloc(unsigned int,int); -extern void efree(ALLOCTYPE *); -#else -extern ALLOCTYPE FAR *emalloc(unsigned int,int); -extern void efree(ALLOCTYPE FAR *); -#endif -extern int alloc_tables(CODE,HASH); -extern void init_tables(void ); -extern int nextcode(CODE *); -#endif - + END OF ARCHIVE