mark@hubcap.clemson.edu (Mark Smotherman) (01/15/91)
Sorry, email bounced.
There are at least two real-time benchmarks available under similar
names:
Realstone from MODCOMP. It is an extension to Whetstone (see below).
[1] Julio Tunon, "A Whetstone-Based Real-Time Benchmark,"
Measurements & Control, October 1987, vol. 21, no. 5, pp. 156-163.
[2] G. Rabbat, B. Furht, and R. Kibler, "Three-Dimensional Computers
and Measuring Their Performance," ACM Computer Architecture News,
vol. 16, no. 3, June 1988, pp. 9-16.
Rhealstone from Dr. Dobb's Journal. It is a suite of six kernels:
1) task switching,
2) high-priority task preemption,
3) interrupt latency,
4) "semaphore shuffle" (i.e. cost of semaphore operations when
they are used to implement mutual exclusion),
5) "deadlock break" (i.e. time for system to notice and raise the
priority of the task that holds a requested critical resource)
6) intertask message latency
[1] R.P. Kar and K. Porter, "Rhealstone: A Real-Time Benchmarking Proposal,"
Dr. Dobb's Journal, February 1989, pp. 14-16,18,22,24.
[2] R.P. Kar, "Implementing the Rhealstone Real-Time Benchmark,"
Dr. Dobb's Journal, April 1990, pp. 46,48,50-51,53-55.
[3] Rhealstone Program Listing, Dr. Dobb's Journal, April 1990, pp. 100-
104. [code is specific to iRMX; article states that source is
available on a disk from DDJ via US mail, on CompuServe DDJ Forum,
and on the DDJ Listing Service (phone number given)]
-----------------------------------
I obtained info on realstone in the summer of 1988 from Nigel Gamble
of MODCOMP.
> Realstone was developed at MODCOMP to measure the performance
> of the CLASSIC range of real-time processors. It measures
> CPU speed as a function of interrupt loading (but not I/O
> processing). ... ([The Tunon] article refers to
> the benchmark as M-Stone, but it is, in fact, the same one that
> MODCOMP calls Realstone.)
>
> However, I believe that a further refinement of the Realstone benchmark
> which measures the interrelationship of CPU speed, interrupt handling
> capability AND I/O throughput is being developed by our special
> systems group. This will be known as the M^3 ("M cubed") benchmark.
> For an account of this benchmarking technique see "Three-dimensional
> computer performance" by Guy Rabbat, Borko Furht and Ron Kibler, in
> Computer (the journal of the Computer Society of the IEEE), July 1988.
> The article introduces the concept of a 3D computer and defines it
> in terms of a three-dimensional performance graph whose axis are
> labelled MIPS-1 (Mega Instructions Per Sec), MIPS-2 (Mega Interrupts
> Per Sec) and MIPS-3 (Mega I/O Per Sec).
> ...
>
> Nigel Gamble uunet!modcomp!nigel
> MODCOMP/AEG
> 1650 W McNab Rd
> PO Box 6099
> Ft Lauderdale
> Fl 33340-6099
--
Mark Smotherman, Comp. Sci. Dept., Clemson University, Clemson, SC 29634
INTERNET: mark@hubcap.clemson.edu UUCP: gatech!hubcap!markshri@ncst.ernet.in (H.Shrikumar) (01/30/91)
In article <1404@gouldfr.UUCP> mmira@gouldfr.UUCP (Michel Mirabella) writes: > I'm looking for the source code of a benchmark called rhealstones >that give rhealstones rating based on a number real time metrics. Sometime back too, I had asked .. but no response. Seems like even Dr. Dobbs, who proposed the benchmark, themselves have forgotten all about it. Anybody know ? Anybody know any RealTime benchmarks ? Rhealstone had some nice concepts,.. 1. One number is not good enough, you must quote a bunch of numbers. 'Cos all real time applications are not very similar. 2. The family includes numbers for context switch time, IPCF rate, etc. and some other "regular" performance figures. I dont remember they proposed a specific way to "average" it into one number, so they did not get into the "how to get one number from several" debate. But they recommended using a weighted average, leaving it to the individual benchmarker to come up with weights appropriate to his requirements. In specifying for a family of numbers, by 1 above, I think they predate SPEC. Talking of SPEC, could SPEC not include some Real Time Programs too ? How about using a parallel make as a process spawn benchmark, a ring of processes (such as the code that discoveres the ledge in the Sun.) for context switch etc. Comments ? -- shrikumar ( shri@ncst.in )
rnovak@mips.COM (Robert E. Novak) (02/05/91)
In article <1270@shakti.ncst.ernet.in> you write: > Talking of SPEC, could SPEC not include some Real Time Programs >too ? How about using a parallel make as a process spawn benchmark, >a ring of processes (such as the code that discoveres the ledge in the Sun.) >for context switch etc. > > Comments ? 1) Whenever possible, SPEC tries to use real applications, not synthetically created benchmarks. 2) To date, SPEC has not yet awarded contracts to write benchmarks on 'spec'. 3) If you have live applications, then please, please, please submit your code to SPEC! If you can't locate anyplace else, please send it to me and I will forward it to the right folks. > >-- shrikumar ( shri@ncst.in ) > > -- Robert E. Novak Mail Stop 5-10, MIPS Computer Systems, Inc. {ames,decwrl,pyramid}!mips!rnovak 950 DeGuigne Drive, Sunnyvale, CA 94086 rnovak@mips.COM (rnovak%mips.COM@ames.arc.nasa.gov) +1 408 524-7183 -- Robert E. Novak Mail Stop 5-10, MIPS Computer Systems, Inc. {ames,decwrl,pyramid}!mips!rnovak 950 DeGuigne Drive, Sunnyvale, CA 94086 rnovak@mips.COM (rnovak%mips.COM@ames.arc.nasa.gov) +1 408 524-7183
eugene@nas.nasa.gov (Eugene N. Miya) (02/05/91)
Hi Bob! Missed you are Reinhold's dinner.
What constitutes a "real" [tm] application?
We should discuss that.
--e. nobuo miya, NASA Ames Research Center, eugene@orville.nas.nasa.gov
{uunet,mailrus,other gateways}!ames!eugene
AMERICA: CHANGE IT OR LOSE IT.warner@arrayb.uucp (Dave Warner) (02/05/91)
In <45540@mips.mips.COM> rnovak@mips.COM (Robert E. Novak) writes: >1) Whenever possible, SPEC tries to use real applications, not >synthetically created benchmarks. >2) To date, SPEC has not yet awarded contracts to write benchmarks on >'spec'. >3) If you have live applications, then please, please, please submit your >code to SPEC! If you can't locate anyplace else, please send it to me and >I will forward it to the right folks. In my experience (IMHE?) benchmarking for embedded real time is quite a bit different than for workstations or other "general purpose" applications. To run a "real" embedded application benchmark I have to totally develop the hardware platform and then run the real code - that defeats the purpose. Note that the performance of the hardware platform is sensitive to much more that just the processing engine. What I want to do (and am currently involved in) is use something to assist is selecting a processor for an embedded application. My approach is to use simulators/emulators and, hopefully, evaluation boards provided by the various vendors as platforms and run "meaningful code fragments", "standard" benchmarks (of which rhealstones would be only one), somewhat pathological cache thrashers and TLB flushers for worst cases, etc... In addition, I am using DineroIII and Tycho for cache simulation, as well as application specific analytic and simulation models. My point is that ALL of these are of necessity "synthetic" in the sense that they are only analogs of my as yet undeveloped real application. If anyone has a better or more comprehensive way to do this please, please let me know. Dave PS to the net: I've also heard that a SPEC subcommittee is developing an I/O benchmark - I'd like to find out more about this. Any pointers? -- _____________________________________________________________________ | Dave Warner | e-mail address: warner@intellistor.com | | Intellistor, Inc. | USmail address: 2402 Clover Basin Dr. | | (303)682-6555 | Longmont, CO 80503 | ---------------------------------------------------------------------