eugene@pioneer.arpa (Eugene N. Miya) (03/04/88)
Attached is a modified copy of the NBS's Measurement Record.
I'm helping to revise it for them and for our (NAS's) interests
in recording information. YES, I know it's a little verbose.
It only records the ancillary information about running on machines.
Similar descriptions for benchmarks will also be made (like modification
level and instruction counts, etc.).
If you have comments on things to add, please send them to me by mail.
Don't bother posting a follow-up.
From the Rock of Ages Home for Retired Hackers:
--eugene miya, NASA Ames Research Center, eugene@ames-aurora.ARPA
"You trust the `reply' command with all those different mailers out there?"
"Send mail, avoid follow-ups. If enough, I'll summarize."
{uunet,hplabs,hao,ihnp4,decwrl,allegra,tektronix}!ames!aurora!eugene
Why in machine readable form?
Several reasons 1) the information can be included in source code header files
[e.g., stdio.h, math.h] A similar techniques is used with the port
library and netlib.
2) make transportability easier.
3) easy for marco processors to use (like m4).
Why in print statements?
1) to keep this data also with the source code.
2) to embed any benchmark output back into source code.
3) keep it near any hardcopy output
<The record starts here (additions left, original indented right):>
V1.1 measrec.wp
NBS COMPUTER MEASUREMENT RESEARCH FACILITY
BENCHMARK MEASUREMENT RECORD
Benchmark:
Machine:
Location:
Date, Time:
Machine Environment
CPU type:
Word size (specify in bits, give number of mantissa and exponent
bits for numeric benchmark and specify base):
Show numbering order (e.g., IBM: 0, 1, 2, ... 31, DEC 31, 32, ... 1, 0)
Machine cycle time (major and minor cycles, specify units):
Memory Size (specify units (byte or word oriented), MBytes, GBytes, etc.):
Min configuration (units) (and increments)
Max configuration (units)
Memory fetch (cycles)
Amount of global or private memory (per processor):
Number of mem. boards & size of each:
Memory interleaving (describe): (factor)
Number of channels [buses] bet. local mem. and CPU/regs.:
Bandwidth of mem. channels (specify units):
Number of processors (number in system and maximum number used):
Type of interconnection between processors (e.g., shared memory,
network):
May the machines be "clustered" or "loosely coupled?"
Bandwidth of an interconnection path or paths (specify unit,
rate, data-carrying capacity and total size of packets if
used):
Width as well as bandwidth
Other (everything else about machine and configuration that could
be relevant to duplicating timing results with the benchmark
used):
Special purpose hardware or processors (Describe)
instruction rate
instruction size
Cache and size:
cache size and cache line size
replacement policy implementation (reconfigurable?)
can you supply a PMS diagram?
can you supply an ISP?
Number of pipes:
Vectors (length, strip mined?)
Special fast memory present and whether used:
Paging device (type and speed, specify units, if none, say so):
I/O devices exercised in benchmark (type, speed, specify units):
What are compatible I/O front ends?
Does this machine stripe data to/from disks?
How does disk striping effect processing rates?
How are additional users accommodated, peripheral controllers, more cpus?
What networking interfaces and standards are supported:
ethernet, proteon, hyperchannel, fddi,
channel to channel, channel protocols, protocol engines?
Software Environment
Operating System and version:
Process scheduling (Round-robin, etc.)
Compiler (name and version):
Do you cross compile?
Vectorizing and optimizing techniques used
Does compiler use an intemediate language? (e.g., Pcode or IF1)
Source language of compiler?
Standard (if any) this compiler claims conformance to:
Do we need to know about a development environment(s)?
Do you have virtual CPU support?
Distributed file system support?
Window systems (X, NEWS)
Network protocols, including DARPA, ISO?
Lisp dialects (along with packages like MACSYMA)
Statistical packages (SPSS, BMDP)
Graphics languages (Commerical DISPLA, DI-3000)
Databases (DBMS Ingres, Oracle)
Distributed editors
Other applications and system utilities, such as system performance tools
Benchmark Conditions
Compiler options used (name them and describe their effects; give
compiler command used):
Compiler messages encountered:
Object module size (specify units):
Load module size (specify units):
Linker/loader options used (name them and describe their effects;
give linker/loader command used);
Number of lines of code required to be modified to compile
successfully:
Number of lines of code modified for successful run:
Tuning Level (as with NAS Kernels: 0, 10, 20, 1000):
Run parameters (special execution time options required to run
benchmark, e.g., amount of storage pre-allocated, etc.,
often specified in job control statements; give run command
used):
Files required (input and output; give sizes, number of records,
how blocked):
Other workload present/absent during run:
Timer type (stopwatch, timers internal to benchmark -- give call
statement or function invocation; describe the value
returned, e.g., whether system time, elapsed job execution
time, etc. and give resolution of timing call):
Number of runs with this benchmark:
System messages encountered (specify whether from loader or run
time, e.g., "DROP FILE EXTENDED TO SIZE 496" on a CDC Cyber
205):
Page faults (if any) recorded:
Problem size (give range, specify units, e.g. "64-bit real arrays
of from 100 to 65535 elements", number of iterations, etc.):