jain@manage.enet.dec.com (Raj Jain, DEC, 550 King st, Littleton, MA, 01460, 508-486-7642, Fax:486-5279) (05/05/91)
The Art of Computer Systems Performance Analysis:
Techniques for Experimental Design, Measurement, Simulation, and Modeling
By Raj Jain
John Wiley & Sons, New York, ISBN 0471-50336-3, 720 pages, April 1991.
This book emphasizes simple-to-use techniques that can be applied by computer
system designers, managers, marketing professionals, buyers, analysts, and
others who need to compare alternative systems (computers, processors,
devices, networks, databases, operating systems, languages, algorithms, or
applications) or manage such projects.
The 720-page volume covers virtually every aspect of systems evaluation from
performance specification, capacity planning, and monitoring systems in use,
to summarizing measured data, designing experiments, simulating future
designs, and modeling the effect of proposed changes.
The book reveals how to avoid common mistakes, and shows how to protect
oneself from misleading analyses presented by others. Given a set of
performance data on two or more systems, it is sometimes possible to
manipulate the data, benchmarks, analysis, metrics, or data presentation such
that either system can be shown to outperform the other!
More than 150 examples and case studies cover topics such as evaluation of
microprocessors, RISC processors, remote procedure calls, the UNIX operating
system, garbage collection algorithms, interconnection networks, local area
networks, text-formatting programs, cache and scheduler design issues and
more.
It has been selected as the *main* selection by the NewBridge Library of
Computer and Information Sciences (LCIS) bookclub.
EXPERTS' OPINIONS...
"At last, a welcome and needed text for computer professionals who require
practical, ready-to-apply techniques for performance analysis. Highly
recommended!"
--- Professor Leonard Kleinrock, Univ of California at Los Angeles
"The Art of Computer Systems Performance Analysis is an extraordinary book.
It has a practical, problem-oriented style which appeals immediately to
engineers engaged in real-world design and analysis."
--- Vint Cerf, Chairman, ACM SIGCOMM
"An entirely refreshing text which has just the right mixture of theory and
real world practice. The book is ideal for both classroom instruction and
self study."
--- Professor Raymond L. Pickholtz, President, IEEE Communications Society
"An extraordinarily comprehensive treatment of both theoretical and practical
issues."
--- Dr. Jeffrey P. Buzen, Internationally known performance expert
"It is the most thorough book available to date."
--- Professor Erol Gelenbe, Universite Rene Descartes, Paris
"This is an unusual object, a textbook that one wants to sit down and peruse.
The prose is clear and fluent, but more important, it is witty."
--- Allison Mankin, in Simulation
"The frequent use of case studies was effective and sometimes entertaining."
--- Craig Partridge, Editor, Computer Communication Review
"It is extremely comprehensive -- I am hard pressed to think of a performance
evaluation technique that isn't discussed here. The emphasis is on practical
techniques that could be used without a great deal of mathematical
sophistication."
---Professor David Finkel, Performance Evaluation Review
"I found that the pace of the material was very even, which makes it a
particularly suitable text from which to teach."
--- Professor Jon Crowcroft, Univ College London, in Computer Communications
A LIST OF INTERESTING "DON'T MISS" TOPICS
o How to show a better performance for your system without any changes
(Beware of ratio games played by your competitors)
(Performance games people play) [p 130-131, 146, 165-174]
o Four rat holes to avoid in performance presentations
(Four easy ways to stall anybody's performance presentation) [p 162]
o Twenty six ways to stall your competitor's presentation
(How to kill your competitors' performance presentations) [p 161-162]
o Six mistakes to avoid in preparing charts for presentations [p 144-146]
o Six tricks to watch out for in your competitors' presentation graphics [p
146-150]
o Twenty two mistakes to avoid in your performance analyses [p 14-22]
o Ten problems in computer systems capacity planning [p 125-127]
o Twelve common mistakes in benchmarking computer systems [p 127-130]
o Seven benchmarking games to watchout [p 130-131]
o Thirty four ways to improve your programs' performance [p 114-116]
o Twenty six guidelines for good graphics in your presentations [p 141-143]
o Eleven mistakes to avoid in empirical modeling [p 266-269]
o Six common mistakes in experimental design [p 278-279]
o Eight mistakes that may lead to incorrect simulation results [p 394-395]
o Seven reasons why most simulations fail [p 395-397]
o Six guidelines for selecting seeds for random number generators [p 453-455]
o Six myths about random-number generators [p 455-458]
o Thirteen system behaviors that are difficult to analyze using queueing
models [p 620-622]
OTHER TOPICS DISCUSSED
I AN OVERVIEW OF PERFORMANCE EVALUATION
o When to measure, simulate, or model?
o How to select the right performance criteria?
o How to specify performance requirements?
II MEASUREMENT TECHNIQUES AND TOOLS
o Which benchmarks are commonly used in the industry?
o How to design the right workload for your system?
o How to monitor your distributed systems?
o How to use accounting logs to determine the workload of your system?
o How to plan and manage capacity required for your computer installation?
III PROBABILITY THEORY AND STATISTICS
o How to summarize measurements with a single number?
o How to report variability?
o How much confidence can you put on data with a large variability?
o How many measurements do you need to compare two systems?
o How to compare systems using multiple benchmarks?
IV EXPERIMENTAL DESIGN AND ANALYSIS
o How many experiments do you really need?
o How to get the most information with the minimum number of experiments?
o Is one system really better than another? Or is it the effect of very
different benchmarks?
o How to isolate measurement errors?
o How to check if a model is adequate?
Part V: SIMULATION
o What language should you use for a simulation?
o How to verify and validate a simulation model?
o How long to run a simulation?
o How to generate random numbers and how to select seeds?
o What probability distributions should you use?
Part VI: QUEUEING MODELS
o How to use simple queueing models to quickly answer common questions?
o How to obtain response time, queue lengths, and device utilizations?
o How to obtain bounds, variance, and other statistics on system performance?
o How to subdivide a large queueing network model and solve it?
FOR INSTRUCTORS AND STUDENTS
Designed to be the main textbook for a first course on performance evaluation,
it provides a comprehensive treatment of all aspects of performance analysis.
You no longer need separate books for measurement, statistics, experimental
design, simulation, and queueing theory. If you teach a course on computer
systems such as architecture, engineering, networks, or databases, you can use
it as a supporting textbook to cover performance issues.
The chapters are organized so that each one can be presented in 45 minutes,
with time left over to discuss the exercises and solutions in a typical
55-minute class. This makes the text ideally suited for a one- or
two-semester course. Practice exercises at the end of each chapter can be
used for homework. Solutions to the exercises appear at the end of the book.
TABLE OF CONTENTS
The book consists of 36 chapters divided into six parts. The chapter titles
are listed below.
1. AN OVERVIEW OF PERFORMANCE EVALUATION: Introduction. Common Mistakes and
How to Avoid Them. Selection of Techniques and Metrics.
2. MEASUREMENT TECHNIQUES AND TOOLS: Types of Workloads. The Art of Workload
Selection. Workload Characterization Techniques. Monitors. Program
Execution Monitors and Accounting Logs. Capacity Planning and
Benchmarking. The Art of Data Presentation. Ratio Games.
3. PROBABILITY THEORY AND STATISTICS: Summarizing Measured Data. Comparing
Systems Using Sample Data. Simple Linear Regression Models. Other
Regression Models.
4. EXPERIMENTAL DESIGN AND ANALYSIS: Introduction to Experimental Design.
2**k Factorial Designs. (2**k)r Factorial Designs with Replications.
2**(k-p) Fractional Factorial Designs. One-Factor Experiments. Two-Factor
Full Factorial Design without Replications. Two-Factor Full Factorial
Design with Replications. General Full Factorial Designs with k Factors.
5. SIMULATION: Introduction to Simulation. Analysis of Simulation Results.
Random-Number Generation. Testing Random-Number Generators.
Random-Variate Generation. Commonly Used Distributions.
6. QUEUEING MODELS: Introduction to Queueing Theory. Analysis of a Single
Queue. Queueing Networks. Operational Laws. Mean-Value Analysis and
Related Techniques. Convolution Algorithm. Hierarchical Decomposition of
Large Queueing Networks.
ABOUT THE AUTHOR
With over sixteen years of experience in the field of computer systems
performance, Raj Jain is a Senior Consulting Engineer at Digital Equipment
Corporation. He received the Ph.D. degree in Computer Science from Harvard
University and has taught courses on performance analysis at Massachusetts
Institute of Technology.
Author's Address: Raj Jain, 137 Dutton Road, Sudbury, MA 01776-2804, USA.
Internet: Jain@Erlang.enet.DEC.Com
BOOK REVIEWS
o Simulation, Vol. 56, No. 1, January 1991, p. 60.
o Computer Communications Review, Vol. 21, No. 1, January 1991, p. 13.
o Performance Evaluation Review, Vol. 18, No. 3, November 1990, p. 21-22.
o ConneXions, Vol. 5, No. 2, February 1991, p. 22.
o Computer Communications, Vol. 14, No. 4, May 1991.
o Computer Marketing Newsletter, Vol. XIV, No. 12, May 1991, p. 4.
The book (ISBN 0471-50336-3, LCCN: QA76.9E94J32 1991, 720 pp., April 1991,
$52.95, Hardcover) is published by John Wiley & Sons, 1 Wiley Drive, Somerset,
NJ 08875, Phone: (908)-469-4400, (800)-225-5945 Ext 2499, Fax: (908)-302-2300.
It is available NOW at most technical bookstores in the United States. It can
also be obtained directly from the publisher.tar@math.ksu.edu (Tim Ramsey) (05/06/91)
jain@manage.enet.dec.com writes: > The Art of Computer Systems Performance Analysis: > Techniques for Experimental Design, Measurement, Simulation, and Modeling > By Raj Jain Holy blowing your own horn, Batman! Please keep the commercial ads in the appropriate places. comp.newprod or biz.* are appropriate. comp.misc is not. -- Tim Ramsey/system administrator/tar@math.ksu.edu/(913) 532-6750/2-7004 (FAX) Department of Mathematics, Kansas State University, Manhattan KS 66506-2602
lstowell@pyrnova.pyramid.com (Lon Stowell) (05/07/91)
In article <22502@shlump.lkg.dec.com> jain@manage.enet.dec.com (Raj Jain, DEC, 550 King st, Littleton, MA, 01460, 508-486-7642, Fax:486-5279) writes: > > The Art of Computer Systems Performance Analysis: > Techniques for Experimental Design, Measurement, Simulation, and Modeling > By Raj Jain > > John Wiley & Sons, New York, ISBN 0471-50336-3, 720 pages, April 1991. > You know, I believe that there are actually a FEW boards on the net that this posting didn't make its way onto? At first I didn't get the whole picture until I noticed that the author was the article poster as well. While pride in your work is understandable....does concern for netters disk space enter any where?
baihelp@venus.tamu.edu (Ignatow, Bryan A.) (05/07/91)
In article <154597@pyramid.pyramid.com>, lstowell@pyrnova.pyramid.com (Lon Stowell) writes... >In article <22502@shlump.lkg.dec.com> jain@manage.enet.dec.com (Raj Jain, DEC, 550 King st, Littleton, MA, 01460, 508-486-7642, Fax:486-5279) writes: >> >> The Art of Computer Systems Performance Analysis: >> Techniques for Experimental Design, Measurement, Simulation, and Modeling >> By Raj Jain >> >> John Wiley & Sons, New York, ISBN 0471-50336-3, 720 pages, April 1991. >> > .. > While pride in your work is understandable....does concern > for netters disk space enter any where? It is nice to see an announcment for a book on computer systems performance. Having taken a class on the subject, I know that finding material on this subject is not always the easiest. I agree that pushing your own work for your own profit is not an admirable quality, but that is overshadowed by the synopsis, book highlights and outline provided in the message, which I would not be able to find out, unless I invested the money to buy the book and read it. But then how often is a person likely to buy a 720 page book to see if it is worth anything or is yet another piece of trash on the subject. (by the way, I find many of the subjects detailed in the overview and outline to be of sufficient interest to buy this particular book) Bryan A. Ignatow Graduate Student, Computer Science Texas A&M University The above comments are not to be taken as a statement made by my employeers, department, college, or university.
rothstei@mcs.kent.edu (Michael Rothstein) (05/07/91)
In article <15935@helios.TAMU.EDU> baihelp@venus.tamu.edu writes: >In article <154597@pyramid.pyramid.com>, lstowell@pyrnova.pyramid.com (Lon Stowell) writes... >>In article <22502@shlump.lkg.dec.com> jain@manage.enet.dec.com (Raj Jain, DEC, 550 King st, Littleton, MA, 01460, 508-486-7642, Fax:486-5279) writes: >>> >>> The Art of Computer Systems Performance Analysis: >>> Techniques for Experimental Design, Measurement, Simulation, and Modeling >>> By Raj Jain >>> >>> John Wiley & Sons, New York, ISBN 0471-50336-3, 720 pages, April 1991. >>> >> >.. >> While pride in your work is understandable....does concern >> for netters disk space enter any where? > >It is nice to see an announcment for a book on computer systems performance. >Having taken a class on the subject, I know that finding material on this >subject is not always the easiest. > >I agree that pushing your own work for your own profit is not an admirable >quality, but that is overshadowed by ... (some drooling deleted) The issue here is that this guy re-POSTED the same (long) article to just about every group on the net! He didn't even cross-post!!! That is a serious nettiquette violation, IMO. It is a waste of bandwidth which translates into a waste of $$ for those netters who get to pay for their feed on a prorated basis, not to mention the duplication of information which probably clogged a few news servers. IMO very inconsiderate behavior! -- Michael Rothstein (Kent State U)| If cars want to kill themselves, (rothstei@cs.kent.edu) | that's their problem: what I can't | understand is why they keep doing it (std. disclaimer) | with people inside. (Mafalda (Quino))
dprrhb@inetg1.ARCO.COM (Reginald H. Beardsley) (05/08/91)
In article <15935@helios.TAMU.EDU>, baihelp@venus.tamu.edu (Ignatow, Bryan A.) writes: > > It is nice to see an announcment for a book on computer systems performance. > Having taken a class on the subject, I know that finding material on this > subject is not always the easiest. > > I agree that pushing your own work for your own profit is not an admirable > quality, but that is overshadowed by the synopsis, book highlights and outline > provided in the message, which I would not be able to find out, unless I > invested the money to buy the book and read it. But then how often is a person > likely to buy a 720 page book to see if it is worth anything or is yet another > piece of trash on the subject. (by the way, I find many of the subjects > detailed in the overview and outline to be of sufficient interest to buy this > particular book) > > Bryan A. Ignatow > Graduate Student, Computer Science > Texas A&M University > I concur! At first I was a bit put out by the posting by Mr. Jain. However, when I read it over I was glad that he did so. It IS difficult to find material on the subject. I also will look for the book and probably buy a copy. This is a fine line to tread, but in the case of one time only announcements of books on esoteric subjects, not unreasonable. (MS-DOS and You, by Hack Writer would not be acceptable in my view) -- Reginald H. Beardsley ARCO Information Services Plano, TX 75075 Phone: (214)-754-6785 Internet: dprrhb@arco.com
chaplin@keinstr.uucp (chaplin) (05/09/91)
In article <154597@pyramid.pyramid.com> lstowell@pyrnova.pyramid.com (Lon Stowell) writes: >In article <22502@shlump.lkg.dec.com> jain@manage.enet.dec.com (Raj Jain, DEC, 550 King st, Littleton, MA, 01460, 508-486-7642, Fax:486-5279) writes: >> >> The Art of Computer Systems Performance Analysis: >> Techniques for Experimental Design, Measurement, Simulation, and Modeling >> By Raj Jain >> >> John Wiley & Sons, New York, ISBN 0471-50336-3, 720 pages, April 1991. >> > > You know, I believe that there are actually a FEW boards on > the net that this posting didn't make its way onto? > > At first I didn't get the whole picture until I noticed that > the author was the article poster as well. > > While pride in your work is understandable....does concern > for netters disk space enter any where? My theory: he's generating case studies on "Computer Systems Performance" for a future book. This is one of his "Techniques for Experimental [...] Measurement." -- Roger Chaplin / Instruments Division Engineering / uunet!keinstr!chaplin CI$: 76307,3506 / voice: (216) 498-2815 / FAX: (216) 248-6168 "In the last analysis the customer is the independent auditor. In the merciless light of real use, every flaw will show." - Frederick P. Brooks, Jr.
danup@runx.oz.au (Daniel Upthegrove) (05/09/91)
>In article <22502@shlump.lkg.dec.com> jain@manage.enet.dec.com (Raj Jain, DEC, 550 King st, Littleton, MA, 01460, 508-486-7642, Fax:486-5279) writes: >> >> The Art of Computer Systems Performance Analysis: >> Techniques for Experimental Design, Measurement, Simulation, and Modeling >> By Raj Jain >> >> John Wiley & Sons, New York, ISBN 0471-50336-3, 720 pages, April 1991. >> > .. I also found the description and synopsis to be good enough to forgive the author for flogging his own product and will be ordering the book. Is there a news group dedicated to technical book & publication reviews? Should there be? I subscribe to ACM's Computing Reviews, and a newsgroup of this type would be of great benefit. (I apologise in advance for just throwing the question into the air with the hope that some poor soul will take it and run. Aside from dumping the news, I'm not on the network enough (or familiar with the procedures) to even consider trying to establish or mediate a news group.) _____________________________________________________________________ How do you get a Daniel & Inez Upthegrove crocodile's attention? _ -_|\ a.k.a. "the Grovers" / \ danup@runxtsa.runx.oz.au Give it a Yank! \_.--._/ <---- Gosford NSW Australia v
cosell@bbn.com (Bernie Cosell) (05/13/91)
danup@runx.oz.au (Daniel Upthegrove) writes: }Is there a news group dedicated to technical book & publication reviews? Oh, you mean like alt.books.technical? /b\