Kevin_Crowston@XV.MIT.EDU (02/09/88)
Does anyone have or know where to find machine readable sources for the Common Lisp benchmarks that were published in (I think) Gabriel's book? Alternately, has anyone run them on a Dandelion running Lyric? Also, what are people's impressions of the programmes as a way to compare machines? Reasonable? Unbiased? Useful? or not? Kevin Crowston MIT Sloan School of Management
ralph@lasso.UUCP (Ralph P. Sobek) (02/12/88)
| From: inria!XV.MIT.EDU!Kevin_Crowston | | ... Also, what | are people's impressions of the programmes as a way to compare machines? | Reasonable? Unbiased? Useful? or not? | | Kevin Crowston | MIT Sloan School of Management | Normally, XEROX has on one of their floppies most of Gabriel's benchmarks in Common Lisp! I have not tried them, yet! What are they worth? Well, that depends on *lots* of things. Most (Some?) of these benchmarks test some precise functionality of the lisp system, e.g., access to global variables, two-dimensional arrays, procedure stacking, etc. Unfortunately, to compare machines on *those* precise functionalities may be valid as a comparison exactly for the user whose application uses precisely those functionalities!! User programs make use of a weighted sum of such functionalities plus *other* functionalities. Also, it is not sure what Gabriel's benchmarks as a group measure. There exist other benchmarks (such as "Long Loop") which measure differently. Furthermore, the benchmarks not only reflect on *certain* capacities of a machine, but also on the machine's compiler and it's compiler writer group. Next is the problem of passing the benchs. Under what conditions? With or without declarations? Garbage Collection activated or not? Benchmarks run in what kind of virtual memory mix and run how often? What's the mean execution time? Etc. Finally, the best bench is to pass one's own application on a number of machines. The problem here, is that *all* machines permit tuning of the application in order to increase its performance -- and that requires time!! Ralph P. Sobek | UUCP: uunet!mcvax!inria!lasso!ralph, or | ralph@lasso.uucp LAAS-CNRS | Internet: ralph@lasso.laas.fr, or 7, avenue du Colonel-Roche | ralph%lasso.laas.fr@uunet.UU.NET F-31077 Toulouse Cedex, FRANCE | ARPA: sobek@shadow.Berkeley.EDU (forwarded\ +(33) 61-33-62-66 | BITNET/EARN: SOBEK@FRMOP11 \ to UUCP ) = = = = = = = = = = = = = = = = = = = = = = = = = = =