ravi@crackle.amd.com (Ravi Chitturi) (09/26/89)
I would like to start a discussion on the topic of "Benchmarking RISC/CISC machines for embedded control". Those of you who attended 1989 Microprocessor forum would agree that currently available benchmarks provided by most chip vendors are very ambiguous at the least and are of little practical use to the customers per se. As things stand, it is hard to distinguish the truth from marketing hype. I would suggest creation of application specific benchmarks e.g.,benchmarking programs for embedded control. Currently, all chip makers can pick and choose only the programs that run well on their machines. This makes every chip look equally good, but with different set of benchmarking programs, and thus confuses the customer. It would be nice to see standard benchmarking programs for embedded controllers. We may have to create 2 or 3 sets of benchmarking programs to cover the entire embedded control spectrum adequately. ......Ravi P.S. Disclaimer : Above mentioned comments and opinions are solely mine and not AMD's.
scarter@gryphon.COM (Scott Carter) (09/29/89)
In article <27483@amdcad.AMD.COM> ravi@crackle.amd.com (Ravi Chitturi) writes: > > I would like to start a discussion on the topic of "Benchmarking RISC/CISC machines for > embedded control". Those of you who attended 1989 Microprocessor forum would agree that > currently available benchmarks provided by most chip vendors are very ambiguous at the least > and are of little practical use to the customers per se. As things stand, it is hard to > distinguish the truth from marketing hype. > > I would suggest creation of application specific benchmarks e.g.,benchmarking programs for > embedded control. Currently, all chip makers can pick and choose only the programs that run > well on their machines. This makes every chip look equally good, but with different set of > benchmarking programs, and thus confuses the customer. It would be nice to see standard > benchmarking programs for embedded controllers. We may have to create 2 or 3 sets of benchmarking > programs to cover the entire embedded control spectrum adequately. > We've been battling this, primary application areas avionics, weapon control, missile guidance, etc. Not so much your high-volume application, but still respectable $$ :). One major problem is that any non-trivial benchmark typically takes weeks to simulate. This assumes that you can manage to find a non-trivial benchmark with no (or removable) I/O or OS dependencies. Trying to simulate the I/O has proved an insurmountable obstacle so far (given budget constraints, etc.). I'm not sure that one can make up an embedded system benchmark that is necessarily any better a predictor than some seemingly vanilla benchmark, e.g. our missile guidance benchmarks correlate as well with Doduc as with each other. In another case, a binning and clustering benchmark originally written for the 68020 (Green Hills) made heavy use of shorts in a context where ints would not have been a major memory waster. This wound up hitting a pathology of a certain major RISC vendor's compiler, with a 20+% performance hit. What do you do with that kind of data? (besides bitch to the vendor ...) I'd still love to see something in this area, but I'm not sure that a specific benchmark suite will be that attainable.
ravi@crackle.amd.com (Ravi Chitturi) (09/30/89)
Few netusers have asked me to post the same article in a more readable format, so here it is. I would like to start a discussion on the topic of "Benchmarking RISC/CISC machines for embedded control". Those of you who attended 1989 microprocessor forum would agree that currently available benchmarks provided by most chip vendors are very ambiguous at the least and are of little practical use to the customers per se. As things stand, it is hard to distinguish the truth from marketing hype. I would suggest creation of application specific benchmarks e.g.,benchmarking programs for embedded control. Currently all chip makers can pick and choose only the programs that run well on their machines. This makes every chip look equally good, but with different sets of benchmarking programs, and this confuses the customer. It would be nice to see standard benchmarking programs for embedded controllers. We may have to create 2 or 3 sets of benchmarking programs to cover the entire embedded control spectrum adequately. Ravi DISCLAIMER : Above mentioned comments and opinions are solely mine and not AMD's.