[comp.software-eng] New Book Announcement: The Art of Computer Systems Performance Analysis

jain@manage.enet.dec.com (Raj Jain, DEC, 550 King st, Littleton, MA, 01460, 508-486-7642, Fax:486-5279) (05/05/91)

              The Art of Computer Systems Performance Analysis:
  Techniques for Experimental Design, Measurement, Simulation, and Modeling
                                 By Raj Jain

    John Wiley & Sons, New York, ISBN 0471-50336-3, 720 pages, April 1991.

This book emphasizes simple-to-use techniques that can be applied by computer
system designers, managers, marketing professionals, buyers, analysts, and
others who need to compare alternative systems (computers, processors,
devices, networks, databases, operating systems, languages, algorithms, or
applications) or manage such projects.

The 720-page volume covers virtually every aspect of systems evaluation from
performance specification, capacity planning, and monitoring systems in use,
to summarizing measured data, designing experiments, simulating future
designs, and modeling the effect of proposed changes.

The book reveals how to avoid common mistakes, and shows how to protect
oneself from misleading analyses presented by others.  Given a set of
performance data on two or more systems, it is sometimes possible to
manipulate the data, benchmarks, analysis, metrics, or data presentation such
that either system can be shown to outperform the other!

More than 150 examples and case studies cover topics such as evaluation of
microprocessors, RISC processors, remote procedure calls, the UNIX operating
system, garbage collection algorithms, interconnection networks, local area
networks, text-formatting programs, cache and scheduler design issues and
more.

It has been selected as the *main* selection by the NewBridge Library of
Computer and Information Sciences (LCIS) bookclub.

                             EXPERTS' OPINIONS...

"At last, a welcome and needed text for computer professionals who require
practical, ready-to-apply techniques for performance analysis.  Highly
recommended!"
            --- Professor Leonard Kleinrock, Univ of California at Los Angeles

"The Art of Computer Systems Performance Analysis is an extraordinary book.
It has a practical, problem-oriented style which appeals immediately to
engineers engaged in real-world design and analysis."
                                          --- Vint Cerf, Chairman, ACM SIGCOMM

"An entirely refreshing text which has just the right mixture of theory and
real world practice.  The book is ideal for both classroom instruction and
self study."
    --- Professor Raymond L. Pickholtz, President, IEEE Communications Society

"An extraordinarily comprehensive treatment of both theoretical and practical
issues."
            --- Dr. Jeffrey P. Buzen, Internationally known performance expert

"It is the most thorough book available to date."
                  --- Professor Erol Gelenbe, Universite Rene Descartes, Paris

"This is an unusual object, a textbook that one wants to sit down and peruse.
The prose is clear and fluent, but more important, it is witty."
                                             --- Allison Mankin, in Simulation

"The frequent use of case studies was effective and sometimes entertaining."
                    --- Craig Partridge, Editor, Computer Communication Review

"It is extremely comprehensive -- I am hard pressed to think of a performance
evaluation technique that isn't discussed here.  The emphasis is on practical
techniques that could be used without a great deal of mathematical
sophistication."
                      ---Professor David Finkel, Performance Evaluation Review

"I found that the pace of the material was very even, which makes it a
particularly suitable text from which to teach."
  --- Professor Jon Crowcroft, Univ College London, in Computer Communications

                  A LIST OF INTERESTING "DON'T MISS" TOPICS

o  How to show a better performance for your system without any changes
   (Beware of ratio games played by your competitors)
   (Performance games people play) [p 130-131, 146, 165-174]
o  Four rat holes to avoid in performance presentations
   (Four easy ways to stall anybody's performance presentation) [p 162]
o  Twenty six ways to stall your competitor's presentation
   (How to kill your competitors' performance presentations) [p 161-162]
o  Six mistakes to avoid in preparing charts for presentations [p 144-146]
o  Six tricks to watch out for in your competitors' presentation graphics [p
   146-150]
o  Twenty two mistakes to avoid in your performance analyses [p 14-22]
o  Ten problems in computer systems capacity planning [p 125-127]
o  Twelve common mistakes in benchmarking computer systems [p 127-130]
o  Seven benchmarking games to watchout [p 130-131]
o  Thirty four ways to improve your programs' performance [p 114-116]
o  Twenty six guidelines for good graphics in your presentations [p 141-143]
o  Eleven mistakes to avoid in empirical modeling [p 266-269]
o  Six common mistakes in experimental design [p 278-279]
o  Eight mistakes that may lead to incorrect simulation results [p 394-395]
o  Seven reasons why most simulations fail [p 395-397]
o  Six guidelines for selecting seeds for random number generators [p 453-455]
o  Six myths about random-number generators [p 455-458]
o  Thirteen system behaviors that are difficult to analyze using queueing
   models [p 620-622]

                            OTHER TOPICS DISCUSSED

I  AN OVERVIEW OF PERFORMANCE EVALUATION
o  When to measure, simulate, or model?
o  How to select the right performance criteria?
o  How to specify performance requirements?

II MEASUREMENT TECHNIQUES AND TOOLS
o  Which benchmarks are commonly used in the industry?
o  How to design the right workload for your system?
o  How to monitor your distributed systems?
o  How to use accounting logs to determine the workload of your system?
o  How to plan and manage capacity required for your computer installation?

III PROBABILITY THEORY AND STATISTICS
o  How to summarize measurements with a single number?
o  How to report variability?
o  How much confidence can you put on data with a large variability?
o  How many measurements do you need to compare two systems?
o  How to compare systems using multiple benchmarks?

IV EXPERIMENTAL DESIGN AND ANALYSIS
o  How many experiments do you really need?
o  How to get the most information with the minimum number of experiments?
o  Is one system really better than another?  Or is it the effect of very
   different benchmarks?
o  How to isolate measurement errors?
o  How to check if a model is adequate?

Part V:  SIMULATION
o  What language should you use for a simulation?
o  How to verify and validate a simulation model?
o  How long to run a simulation?
o  How to generate random numbers and how to select seeds?
o  What probability distributions should you use?

Part VI:  QUEUEING MODELS
o  How to use simple queueing models to quickly answer common questions?
o  How to obtain response time, queue lengths, and device utilizations?
o  How to obtain bounds, variance, and other statistics on system performance?
o  How to subdivide a large queueing network model and solve it?

                         FOR INSTRUCTORS AND STUDENTS

Designed to be the main textbook for a first course on performance evaluation,
it provides a comprehensive treatment of all aspects of performance analysis.
You no longer need separate books for measurement, statistics, experimental
design, simulation, and queueing theory.  If you teach a course on computer
systems such as architecture, engineering, networks, or databases, you can use
it as a supporting textbook to cover performance issues.

The chapters are organized so that each one can be presented in 45 minutes,
with time left over to discuss the exercises and solutions in a typical
55-minute class.  This makes the text ideally suited for a one- or
two-semester course.  Practice exercises at the end of each chapter can be
used for homework.  Solutions to the exercises appear at the end of the book.

                              TABLE OF CONTENTS

The book consists of 36 chapters divided into six parts.  The chapter titles
are listed below.

1. AN OVERVIEW OF PERFORMANCE EVALUATION:  Introduction.  Common Mistakes and
   How to Avoid Them.  Selection of Techniques and Metrics.

2. MEASUREMENT TECHNIQUES AND TOOLS:  Types of Workloads.  The Art of Workload
   Selection.  Workload Characterization Techniques.  Monitors.  Program
   Execution Monitors and Accounting Logs.  Capacity Planning and
   Benchmarking.  The Art of Data Presentation.  Ratio Games.

3. PROBABILITY THEORY AND STATISTICS:  Summarizing Measured Data.  Comparing
   Systems Using Sample Data.  Simple Linear Regression Models.  Other
   Regression Models.

4. EXPERIMENTAL DESIGN AND ANALYSIS:  Introduction to Experimental Design.
   2**k Factorial Designs.  (2**k)r Factorial Designs with Replications.
   2**(k-p) Fractional Factorial Designs.  One-Factor Experiments.  Two-Factor
   Full Factorial Design without Replications.  Two-Factor Full Factorial
   Design with Replications.  General Full Factorial Designs with k Factors.

5. SIMULATION:  Introduction to Simulation.  Analysis of Simulation Results.
   Random-Number Generation.  Testing Random-Number Generators.
   Random-Variate Generation.  Commonly Used Distributions.

6. QUEUEING MODELS:  Introduction to Queueing Theory.  Analysis of a Single
   Queue.  Queueing Networks.  Operational Laws.  Mean-Value Analysis and
   Related Techniques.  Convolution Algorithm.  Hierarchical Decomposition of
   Large Queueing Networks.

                               ABOUT THE AUTHOR

With over sixteen years of experience in the field of computer systems
performance, Raj Jain is a Senior Consulting Engineer at Digital Equipment
Corporation.  He received the Ph.D.  degree in Computer Science from Harvard
University and has taught courses on performance analysis at Massachusetts
Institute of Technology.

Author's Address:  Raj Jain, 137 Dutton Road, Sudbury, MA 01776-2804, USA.
Internet:  Jain@Erlang.enet.DEC.Com

                                 BOOK REVIEWS

o  Simulation, Vol.  56, No.  1, January 1991, p.  60.
o  Computer Communications Review, Vol.  21, No.  1, January 1991, p.  13.
o  Performance Evaluation Review, Vol.  18, No.  3, November 1990, p.  21-22.
o  ConneXions, Vol.  5, No.  2, February 1991, p.  22.
o  Computer Communications, Vol.  14, No.  4, May 1991.
o  Computer Marketing Newsletter, Vol.  XIV, No.  12, May 1991, p.  4.

The book (ISBN 0471-50336-3, LCCN:  QA76.9E94J32 1991, 720 pp., April 1991,
$52.95, Hardcover) is published by John Wiley & Sons, 1 Wiley Drive, Somerset,
NJ 08875, Phone: (908)-469-4400, (800)-225-5945 Ext 2499, Fax: (908)-302-2300.
It is available NOW at most technical bookstores in the United States.  It can
also be obtained directly from the publisher.