argv@island.uu.net (Dan Heller) (03/29/89)
Submitted-by: Claus Gittinger <unido!sinix!claus> Posting-number: Volume 3, Issue 61 Archive-name: xbench/part04 #! /bin/sh # This is a shell archive. Remove anything before this line, then unpack # it by saving it into a file and typing "sh file". To overwrite existing # files, type "sh file -c". You can also feed this as standard input via # unshar, or by typing "sh <file", e.g.. If this archive is complete, you # will see the following message at the end: # "End of archive 4 (of 4)." # Contents: xbench.doc # Wrapped by x11@babel on Thu Mar 23 14:08:04 1989 PATH=/bin:/usr/bin:/usr/ucb ; export PATH if test -f 'xbench.doc' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'xbench.doc'\" else echo shar: Extracting \"'xbench.doc'\" \(14481 characters\) sed "s/^X//" >'xbench.doc' <<'END_OF_FILE' X.. @(#)xbench.doc 1.7 89/03/22 X.RP X.TL xbench - a set of benchmark tests for X servers X.AF "" X.AU C. Gittinger X.AI Siemens AG Munich Dep. D ST SP 4 X.NH 1 How to run xbench X.LP XFor a first test, you should run: X.nf X.sp X \fBxbench -ask -sync -timegoal 5 -nruns 1\fP X.fi X.sp to find out which tests work correctly on your machine (look carefully at the output of the dashed-line test, the arc tests and the tile/stipple rectangle fill tests). X.br If your server doesn't perform an operation correctly (or crashes), it should be excluded from the test. It wouldn't be fair, to compare a server doing dashes with another one which draws solid lines instead. X.sp X.I Xbench will ask you for some information about the client/server machine - these are for documentation only and will appear in the output listing. You may just enter <return> here if you're not going to publish the benchmark - if you send it to some newsgroup it is recommended to describe the server and client configuration here. (for example: no FPU, graphic accelerator, 8mb main memory ...). X.sp You can also put everything into a script-file and feed this onto X.I xbench. The distribution contains a prototype-script called "script.run" - change this according to your needs. X.br Then start the real X.I xbench run by calling: X.nf X.sp X \fBxbench -ask < script.run > xbench.out\fP X.fi X.sp On some machines (those which draw nothing when screen-saver is active) the screen saver should be turned off before: X.sp X.nf X \fBxset s off\fP X.fi X.sp If your server turns off the cursor while drawing so that you get different performance data depending in the position of the cursor, you should probably leave the cursor in the draw area - since this is the way you usually work with the system; or run the test twice (first run with cursor in the upper left, X2nd run with cursor in the lower right). X.NH 1 XXbench tests X.LP XEach test in xbench has associated with it a X.I level number. If you run X.I xbench without a level argument, only the level 1 tests are performed. X.sp XFor comparison of different server implementations, the level 1 tests give you enough information, while the higher levels are mostly useful for server tuners. X.LP The following chapters describe the individual tests. The tables include the test-names, the level and a short description of what is done. X.br The last 2 cols of the tables are the reference value and the weight for the X.I xstone rating - this will be described in more detail in the next chapter. X.KS X.NH 2 Solid lines X.LP The X.B line* benchmark draws horizontal, vertical and diagonal lines of different length: X.TS X.box, tab(!); c | c | c | c | c | c l | c | r | r | r | r. name!level!description!ref.value!weight X= line2!2!vectors of len 2!-!0 line10!1!len 10!2616!300 line100!1!len 100!1631!400 line400!1!len 400!644!300 line500!2!len 500!-!0 X_ X ! ! ! ! 1000 X.TE X.KE X.KS X.NH 3 Lines more detail X.LP Horizontal, vertical and diagonal lines can be tested separately with: X.TS X.box, tab(!); c | c | c | c | c | c l | c | r | r | r | r. name!level!description!ref.value!weight X= hline2!3!horizontal vectors of len 2!-!0 hline10!2!len 10!-!0 hline100!2!len 100!-!0 hline400!2!len 400!-!0 hline500!3!len 500!-!0 X= vline2!3!vertical vectors of len 2!-!0 vline10!2!len 10!-!0 vline100!2!len 100!-!0 vline400!2!len 400!-!0 vline500!3!len 500!-!0 X= sline2!3!sloped (diagonal) vectors of len 2!-!0 sline10!2!len 10!-!0 sline100!2!len 100!-!0 sline400!2!len 400!-!0 sline500!3!len 500!-!0 X= X.TE X.KE X.KS X.NH 2 Dashed lines X.LP The X.B dline* benchmark draws dashed lines. The dashes are [4,4]-dashes meaning 4 dots on, 4 dots off. X.TS X.box, tab(!); c | c | c | c | c | c l | c | r | r | r | r. name!level!description!ref.value!weight X= dline2!2!dashed vectors of len 2!-!0 dline10!1!len 10!819!50 dline100!1!len 100!172!150 dline400!1!len 400!49!100 dline500!2!len 500!-!0 X_ X ! ! ! ! 300 X.TE X.KE X.KS X.NH 2 Wide lines X.LP The X.B wline* benchmark draws wide lines. The linewidth is 5. X.TS X.box, tab(!); c | c | c | c | c | c l | c | r | r | r | r. name!level!description!ref.value!weight X= wline2!2!wide vectors of len 2!-!0 wline10!1!len 10!78!50 wline100!1!len 100!19!100 wline400!1!len 400!6!50 wline500!2!len 500!-!0 X_ X ! ! ! ! 200 X.TE X.KE X.KS X.NH 2 Rectangles X.LP The X.B rects* benchmark draws rectangles. X.TS X.box, tab(!); c | c | c | c | c | c l | c | r | r | r | r. name!level!description!ref.value!weight X= rects2!2!2x2 rectangles!-!0 rects10!1!10x10!1115!150 rects100!1!100x100!797!250 rects400!1!400x400!414!150 rects500!2!500x500!-!0 X_ X ! ! ! ! 550 X.TE X.KE X.KS X.NH 2 XFilled rectangles X.LP The X.B fillrects* benchmark draws solid filled rectangles. X.TS X.box, tab(!); c | c | c | c | c | c l | c | r | r | r | r. name!level!description!ref.value!weight X= fillrects2!2!2x2 rectangles!-!0 fillrects10!1!10x10!1245!350 fillrects100!1!100x100!694!450 fillrects400!1!400x400!157!350 fillrects500!2!500x500!-!0 X_ X ! ! ! ! 1150 X.TE X.KE X.KS X.NH 2 Tiled rectangles X.LP The X.B tiledrects* benchmark draws tiled rectangles. The tile used is a 32x32 tile (which is most common). X.TS X.box, tab(!); c | c | c | c | c | c l | c | r | r | r | r. name!level!description!ref.value!weight X= tiledrects2!2!2x2 rectangles!-!0 tiledrects10!1!10x10!1387!150 tiledrects100!1!100x100!323!250 tiledrects400!1!400x400!48!150 tiledrects500!2!500x500!-!0 X_ X ! ! ! ! 550 X.TE X.KE X.KS X.NH 2 Stippled rectangles X.LP The X.B stippledrects* benchmark draws stippled rectangles. The stipple is a 32x32 stipple. X.TS X.box, tab(!); c | c | c | c | c | c l | c | r | r | r | r. name!level!description!ref.value!weight X= stippledrects2!2!2x2 rectangles!-!0 stippleddrects10!1!10x10!835!50 stippleddrects100!1!100x100!153!100 stippleddrects400!1!400x400!35.6!50 stippleddrects500!2!500x500!-!0 X_ X ! ! ! ! 200 X.TE X.KE X.KS X.NH 2 Inverting rectangles X.LP The X.B invrects* benchmark inverts rectangles on the screen. X.TS X.box, tab(!); c | c | c | c | c | c l | c | r | r | r | r. name!level!description!ref.value!weight X= invrects2!2!2x2 rectangle invert!-!-!0 invrects10!1!10x10!3440!100 invrects100!1!100x100!938!150 invrects400!1!400x400!134.5!100 invrects500!2!500x500!-!0 X_ X ! ! ! ! 350 X.TE X.KE X.KS X.NH 2 Arcs X.LP The X.B arc* benchmark draws arcs of varying angle. The angle-range is 5 to 360 degrees. X.TS X.box, tab(!); c | c | c | c | c | c l | c | r | r | r | r. name!level!description!ref.value!weight X= arcs2!2!2x2 box!-!0 arcs10!1!10x10!5.4!50 arcs100!1!100x100!1.2!100 arcs400!1!400x400!0.53!50 arcs500!2!500x500!-!0 X_ X ! ! ! ! 200 X.TE X.KE X.KS X.NH 2 XFilled arcs X.LP The X.B filledarc* benchmark draws filled arcs of varying angle. The angle-range is 5 to 360 degrees. X.TS X.box, tab(!); c | c | c | c | c | c l | c | r | r | r | r. name!level!description!ref.value!weight X= filledarcs2!2!2x2 box!-!0 filledarcs10!1!10x10!25.3!50 filledarcs100!1!100x100!6.0!100 filledarcs400!1!400x400!2.7!50 filledarcs500!2!500x500!-!0 X_ X ! ! ! ! 200 X.TE X.KE X.KS X.NH 2 XFilled polygons X.LP The X.B filledpoly* benchmark draws a filled polygon with 5 points. X.TS X.box, tab(!); c | c | c | c | c | c l | c | r | r | r | r. name!level!description!ref.value!weight X= filledpoly10!2!vector len 10!-!0 filledpoly100!1!len 100!100!200 X_ X ! ! ! ! 200 X.TE X.KE X.KS X.NH 2 BitBlt's X.LP A number of different bitblt tests are available: X.br Screen to Screen copy, its special case with srcX == dstX X(i.e. scrolling), pixmap to screen copy using XCopyArea and bitmap to screen using XCopyPlane (which is more often used on color systems). For monochrome systems, pixmap to screen and bitmap to screen is the same operation. The scroll test copies a 640x400 window; it has been added since the server may have a special-tuned function for this and perform scrolling faster than general bitblt's. X.TS X.box, tab(!); c | c | c | c | c | c l | c | r | r | r | r. name!level!description!ref.value!weight X= screencopy2!2!screen->screen copy 2x2 rectangle!-!0 screencopy10!1!10x10!226!50 screencopy100!1!100x100!95!150 screencopy400!1!400x400!17.8!100 screencopy500!2!500x500!-!0 X= scroll!1!screen->screen (640x400) scroll!68!1000 X= pixmapcopy2!3!pixmap->screen copy 2x2 pixmap!-!0 pixmapcopy10!2!10x10 pixmap!-!0 pixmapcopy100!2!100x100 pixmap!-!0 pixmapcopy400!2!400x400 pixmap!-!0 pixmapcopy500!3!500x500 pixmap!-!0 X_ bitmapcopy2!2!bitmap->screen copyplane 2x2 bitmap!-!0 bitmapcopy10!1!10x10 bitmap!303!50 bitmapcopy100!1!100x100 bitmap!68.3!100 bitmapcopy400!1!400x400 bitmap!10.7!50 bitmapcopy500!2!500x500 bitmap!-!0 X_ X ! ! ! ! 1500 X.TE X.KE X.KS X.NH 2 Drawing characters X.LP The X.B imagestring* benchmark draws a string. The font most likely to be found on every machine (all releases) is "fixed". The other fonts are propably not on every machine. X.TS X.box, tab(!); c | c | c | c | c | c l | c | r | r | r | r. name!level!description!ref.value!weight X= imagestring:fixed!1!characters font=fixed!5636!3000 imagestring:6x12!2!font=6x12!-!0 imagestring:8x13!2!font=8x13!-!0 imagestring:9x15!2!font=9x15!-!0 X_ X ! ! ! ! 3000 X.TE X.KE X.KS X.NH 2 Complex test X.LP All of the above tests are low level tests, measuring mainly the performance of the graphics engine/ddx code. To get a feeling of overall server performance when dealing with windows, graphic contexts etc., this tests creates a window, creates a GC, clears an area, draws some text, scrolls the window and finally destroys the window. X.TS X.box, tab(!); c | c | c | c | c | c l | c | r | r | r | r. name!level!description!ref.value!weight X= complex1!1!create/draw/destroy!18.6!600 X_ X ! ! ! ! 600 X.TE X.KE X.KS X.NH 1 Output X.LP The output of X.B xbench consists of: X.NH 2 Absolute performance data X.LP The absolute run time in seconds and the number of runs per second are printed. Some tests (for example fillrects) also output the number of pixels drawn per second. X.NH 1 Statistics and xStones X.LP To compare different servers performances, the absolute performance data (as output by X.I xbench) can be processed by some awk-scripts to compute: X.NH 2 Relative performance X.LP The ratio of the tested servers performance with the performance of an unloaded monochrome Sun3/50 running UNTUNED (i.e. original MIT code) X11R3 under OS 3.4. X.sp This machine has been choosen, since it is: X.sp X.nf X well known. X available at our site. X it uses no graphics co-processor and plain MIT code. X.fi X.sp The X.I ref.values in the previous tables are the numbers obtained by running the benchmarks on this reference machine. (BTW: thanks to my brother Juergen for running the bench on his machine for me) X.NH 2 Subjective performance X.LP XFor every group of tests the time it takes to execute it is compared with the time it takes on the reference machine. The individual tests are weighted in this computation according to to X.I weight values in the tables listed above. X.LP The weights have been scaled to give a monochrome sun3/50 running untuned MIT release 3 code an xstone rating of 10000 X(so that percentages are easy to compute). X.LP The weights are based on our experience on what is used often. Since most of the time you're dealing with text its weight is very high. XXCopyPlane from bitmap to screen is used more often than XXCopyArea from pixmap to screen (on a monocrome server they are equivalent). Solid lines/rectangles are used more often than dashed, tiled and stippled ones. Tiled rectangles are drawn more often than stippled ones (root window background). X.LP If you look at the weights in the tables above, you will notice that text performance counts 30%, bitblts 15% (with scroll alone 10%), solid lines + rectangles 15% and filled + tile-filled rectangles 17%. X.br Since arcs, dashed- and wide lines are not very often drawn, their weight together is only about 10%. X.LP Of course, depending on your applications, the rating could differ from your personal feeling. If you are running a CAD application doing only line-drawing, the line weight is too small compared to text/bitblt rating. If you are only working with xterm, the scroll/character weight are too small compared to line rating (and so on ....). X.LP The final output consists of the individual X.I stone ratings for the line-, fill-, blit-, text-, arc- and complex tests and the total X.I xStone rating. X.br XFor a server with an X.I xStone rating of 5000 you can expect that a graphical mix with 30% text, X15% lines etc. takes twice the time to execute as it takes on the reference machine. X.br If you are doing mostly text, look at the X.I textStone ratings; if you are doing mostly line-drawing, look at the X.I lineStones. X.LP All benchmarks excluded (with -not option) will be counted in the final X.I xstone rating according to the rating of the other benchmarks - so that a server showing double sun3 performance in all other tests will get this rating also for the missing tests. This number is of course wrong - but how should a non-working server be compared to a working one ? X.LP The scripts to generate the ratings are in the "scripts"-directory. To use them, create a subdirectory "results", put all your ".out"-files into this directory and call: X.sp X.nf X make summary.ms detail.ms X.fi X.sp This will create 2 files named "summary.ms" and "detail.ms" which can be printed with: X.sp X.nf X tbl summary.ms detail.ms | troff -Txxx -ms X or X tbl summary.ms detail.ms | nroff -ms | your_printer X.fi X.KE X.KS X.NH 1 XErrors X.LP As X.I xbench measures times in 1 second deltas, therefore within each run, the error (with timegoal=10) is about 10%. Bigger timegoals lead to a smaller error but also increased runtime. Also care should be taken, that the machine under test is load-free; if possible disable daemons (like rwhod). If testing via ethernet the measured time also depends on how fast the client-machine (the one on which X.I xbench runs) can send the requests. If possible run the bench on different client-machines and take the best result. Run the bench at times X(lunch time or at night) when your ethernet-load is low. X.NH 1 XFinal Note X.LP I'd better not included the weighting stuff - I can see the flames on the net .... X.br Before you send me any mail on xbench please think about 2 things: X.sp X.nf X How fair/exact are other *stone tests ? X.sp X What could/should be done better and why ? X.fi X.sp If there are any suggestions, enhancements I will add them into xbench and release a new version of it. X.br XFor now - X.sp X.sp X\fB- Have fun\fP X.sp Claus Gittinger X.br Siemens Munich X.br Dep D ST Sp 4 X.br D-8000 Munich/Neuperlach X.br Otto-Hahn-Ring X.br West Germany X.sp X(..!unido!sinix!claus) X.KE END_OF_FILE if test 14481 -ne `wc -c <'xbench.doc'`; then echo shar: \"'xbench.doc'\" unpacked with wrong size! fi # end of 'xbench.doc' fi echo shar: End of archive 4 \(of 4\). cp /dev/null ark4isdone MISSING="" for I in 1 2 3 4 ; do if test ! -f ark${I}isdone ; then MISSING="${MISSING} ${I}" fi done if test "${MISSING}" = "" ; then echo You have unpacked all 4 archives. rm -f ark[1-9]isdone else echo You still need to unpack the following archives: echo " " ${MISSING} fi ## End of shell archive. exit 0