cssd@CHEVIOT.UUCP (STUDENTS) (03/07/86)
We recently put an inquiry on the net for information about benchmarking UNIX systems, the response to that message brought us much information which we are presently collating. We have, in the meantime, built up a 'refer' data-base of the bibliography of UNIX specific references which we now offer to the net for recommendations and further inclusions. If we have missed off your most prized work, we first apologise and then ask you to get in touch and let us know. We also take this opportunity to thank all those who mailed us and haven't yet received a reply. Jim Lyons for CSSD UUCP ...!ukc!cheviot!cssd ARPA cssd%cheviot.newcastle@ucl-cs.ARPA JANET cssd@newcastle.cheviot --------------------------------------------------------------------- %A J. Feder %T The Evolution of UNIX System Performance %J AT&T Bell Laboratories Technical Journal %V 63.8 %D October 1984 %P 1791 %O The definitive work on UNIX development from the team who built it. A good article giving an historic view on benchmarking in the Bell Labs. Shows the improvement gained over various issues of UNIX and improvement in performance brought about by changing algorithms to suit new hardware. %A D. F. Hinnant %T Benchmarking UNIX systems %J BYTE %V 9.8 %D August 1984 %P 132 %O This is an article entirely about benchmarking UNIX systems. The benchmarks measure performance at three different levels, the system level, the command level and multi-user level. The benchmarks are constructed with the minimum of overhead and this article describes them and the results achieved on various systems. %A A. S. Tanenbaum %T Two programs, many UNIX systems. %J EUUG %V 3.4 %P 24-26 %D Winter 1983 %O A comparison of two benchmarks run on many systems. The article shows the results of two test programs. The two test programs are run on a wide variety of machines from single-user to multi-users. One test program measures CPU/memory speed and the other measures I/O speed. Care must be taken when interpreting the results. Some vendors may have carefully selected comparisons to show how fast a particular machine is. The article actually said that "figures don't lie, but liars figure". %A B. Clark %T UNIX - the horse before the cart %J Computer Systems %V 4/11 %P 85-87 %D November 1984 %O Machine architectures built specifically to run Unix. A description of a machine designed to run Unix, rather than tailoring Unix to suit the hardware. The article discusses hardware features which complement Unix, especially the advantages of using a RISC architecture with many registers for the Unix machine. Some crude performance figures are given of the RISC machine against a VAX 11/780 and a Motorola 68000. %A Bob Critchlow %T Optimising Unix performance %J Computer Systems %V 3/11 %P 73-76 %D November 1983 %O Comparing Intel System 286/310 performances running Unix to other machines. A review of the Intel System 286/310 computers which run many versions of Unix. The results of six types of benchmarks, run on six different manufacturers machines are presented. This article is generally a 'plug' for Intel - the author Bob Critchlow is Intel's technical systems marketing manager. %A Mike Pearce %T Who needs high capacities %J Computer Systems %V 3/11 %I Techpress %C Bromley %D November 1983 %P 4 %O An article which primarily discusses the electro-mechanical features of a range of Winchester disk drives. It makes passing reference to Unix's need for a high performance disk and plots transfer rate against access time for a range of disk systems to show that disk access time is a critical factor. %A M. K. McKusick %A W. N. Joy %A S. J. Leffler %A R. S. Fabry %T A Fast File System for Unix %I Computer Systems Research Group, Dept of EECS, Berkeley, Ca. %J Technical Report #8 %O This paper sets out to describe the differences in file systems between version 4.1 and 4.2. This paper addresses itself to the enhancements and changes made to the Berkeley UNIX version 4.1 to bring it to version 4.2, it discusses changes to block sizes and access algorithms and related subjects. %A L. F. Cabrera %T Benchmarking Unix - A comparative study %B Experimental Computer Performance Evaluation %I North-Holland Publishing Company %P 205-215 %O Compares UNIX performance at the user level, performance is defined as response time against number of processes. Article explains the method and results of the high level benchmarking of three UNIX systems. Three 'typical' jobs were devised and run periodically on each system while that system is loaded with real processes. Response times for these jobs are marked against a number of processes. This is a useful article. It also notes that it would be better to have a load that is reproducible on any system, rather than rely on real loads whose characteristics are subject to variation over the systems. %A E. D. Lazowska %T The Benchmarking, Tuning and Analytic Modelling of VAX/VMS %J Performance Evaluation Review %V 8.3 %D Fall 1979 %C Department of Computer Science, University of Washington, Seattle %P 57-64 %O Benchmarking, tuning and modelling the DEC's VMS executive with a strong emphasise in modelling. This work arises from the performance problems which the author encountered. For example, the response time for simple editing requests takes several minutes; it takes between fifteen minutes to three hours for compilation and the system capacity is four active users. An analytic model was designed for performance prediction. The author has presented a good analytic model as the predictions from the model are very close to the actual results obtained. %T Benchmark report : Datasystem 355 %J Performance Evaluation Review, Sigmetrics %V 9.3 %D Fall 1980 %P 7-30 %O A report of the benchmark carried out on 13 users of DEC's 11/34 based systems. This article is about benchmarking the DEC Datasystem 355. It contains the benchmark process, the benchmark results, details about the system and its peripherals and a summary of user comments. Article looks into how well the equipment performs in specific applications and that performance compares with other computing systems. It also concerns with subjective factors like ease of use, versatility and the support service. The benchmark measures speed and response time against the number of terminals. The response time increases as the number of terminals has increased. %A Robert G. Estell %T Benchmarks and Watermarks %J Performance Evaluation Review, Sigmetrics %V 9.3 %D Fall 1980 %O The article consists of the author's opinions and some points to consider when benchmarking the performance measure of computers. Include some valuable points. %A S. Spanier %T Sun-3 Benchmarks %J Sun Microsystems internal memo %D August 1985 %O A wide range of benchmarks results comparing the SUN-3 machine and other machines like IBM PC, VAX, APOLLO. This article characterises the performance of the SUN-3 machine and provides an aid for people choosing a workstation. Benchmark, in this article, is defined as a test of the performance of one or more elements of a computer system. A wide range of benchmarks were carried out to show how good the SUN-3 performances are as compared with some other machines. %A David Hough %T Weitek 1164/5 Floating Point Accelerators %O Article is about the Floating Point Accelerator (FLA) measured at the SUN. It talks about the performance of SUN-3 Floating Point Accelerator. Suggested to use the Linpack benchmark for measuring performance on scientific and engineering floating point calculations because it measures the performance of hardware and compiler in an indisputable way on a realistic computation, and the Livermore loop benchmarks for larger inner loops. %A T. Lang %A J. P. Newbury %A D. E. P. Watkins %T Experiences in Benchmarking for the Selection of an Interactive System %J Software - Practice and Experience %V 12 %P 531-542 %O Report on how a benchmark was constructed to assist in the selection of a general purpose interactive computing system. The article describes the stages in constructing a benchmark to measure the hardware and software performances on a number of computer systems. The problems the authors encountered and their experiences gained in this exercise are also described in the paper. Provides some assistance to anyone starting on a benchmarking exercise for the first time. %A B.A. Wichmann %A H.J. Curnow %T A Synthetic Benchmark: Whetstone %J Computer Journal %V 19 No: 1 %D February 1976 %P 43 %O Presents the background to the Whetstone benchmark and how it was constructed originally to measure the performance of the ALGOL compiler. The article describes the stages in constructing a benchmark to measure the performances of a High Level Language, such as ALGOL on a number of computer systems. The benchmark takes a number of commonly used statements and checks their performance (timed events). The benchmark has also been translated in other High Level Languages. There are a set of four performance measures that are used. Regarded as a 'standard' test for any benchmark suite of programs. %A R.P. Weicker %T Dhrystone: A Synthetic systems programming benchmark %J Communications of the ACM %V 27 No: 10 %D October 1984. %P 1013 %O Presents the background to the Dhrystone benchmark and how it was constructed to measure the performance of executing different types of High Level Language programming statements. The article describes the stages in constructing a benchmark to measure the performances of a High Level Language on a number of computer systems. This is basically very similar to the Whetstone benchmark, but perhaps more widely used as a measure of performance. The benchmark takes a number of commonly used statements (a mix of statements) then 'times' them to evaluate their performance. The actual benchmark can be run in two modes, with and without the use of 'extra' registers at execution time. It has been run on many machines using the UNIX operating system. Regarded as a 'standard' test for any benchmark suite of programs. %A P. Marvit. %A M. Nair. %T Benchmark Confessions. %B The UNIX System - Encyclopaedia. %I Yates Ventures. %D 1984. %P 63. %O Presents the whole concept of benchmarking UNIX systems and what factors need to be considered beforehand and sets guidelines on the evaluation of results. The article describes the philosophy and practice behind the whole concept of benchmarking UNIX systems. It states that benchmarking UNIX is, at best, problematic, and at worst, a gross distortion of reality. It points out the loop holes that can easily be entered by misinterpreting the results obtained and warns against these by setting guidelines on how to use the results. This methodology would be very useful in our own work when we come to analyse our results. It also points out the h/w and s/w considerations to be taken into account when constructing your benchmark. %A P. Marvit %T A UNIX Benchmark Blitzkrieg %B The UNIX System - Encyclopaedia %I Yates Ventures %D 1984 %P 69 %O Presents a set of six different benchmarks for UNIX systems. The article looks into the benchmark tests and briefly comments on the result obtained, including the 'Whetstone' benchmark. It shows a table of results obtained on several machines. But otherwise it is just listings of the benchmarks themselves.