pajari@grads.cs.ubc.ca (George Pajari) (04/12/88)
With all the discussions regarding performance of serial I/O cards (especially 'intelligent' vs 'smart' vs 'dumb' vs 'retarded') it would seem useful if we had a serial I/O benchmark (BAUDSTONE?) which could be used to compare boards. The idea is that such a benchmark would be written and distributed over the net and results would be collected. Obviously the device driver/OS/CPU are significant factors but (a) we've got to start somewhere, and (b) the collected results would list the important variables so that they could be factored out (to some extent) during comparisons. If this has been done already then my apologies and please enlighten me. Otherwise I would like to propose the following design for such a BAUDSTONE. The idea is to solicit comments (either over the net or mailed to me) so that a consensus :-) may be reached, a benchmark written (I'll volunteer if someone else hasn't/doesn't beat me to it), code distributed, and results collected and summarised (me again). Obviously if someone has done this already, please let us know. This is one wheel which I have no desire to reinvent unless necessary. BAUDSTONE A: Output Loading =========================== attempt on 1, 2, 4, and n ports simultaneously (where n = number of ports on the board) 1. Initialize self and wait for common starting signal. 2. Write 100K to the chosen port using 1K writes and write(2) calls (each 1K buffer to contain 20 newlines equally distributed). (Nothing to be connected to the port.) 3. Note time it took to transmit 100K and compute effective baud rate. Communications parameters for output port: 9600 baud 7-bits, even parity, 1 stop bit, OPOST, ONLCR, no delays Simultaneously, run a CPU loop program at low priority to detect host CPU consumption. The results of interest include: - the range of effective baud rates - the average effective baud rate - the % of host CPU consumed BAUDSTONE B: Input/Output Loading (Loopback) ============================================ attempt on 1, 2, 4, and n ports simultaneously (where n = number of ports on the board) (Each port to have a loopback connector, i.e. TXD to RXD on same port). for each port start two processes: PROCESS A --------- 1. Initialize self and wait for common starting signal. 2. Write 100K to the chosen port using 1K writes and write(2) calls (each 1K buffer to contain 20 lines of equal length with five to contain erase characters and five to contain line kill characters at the end of the line). 3. Note time it took to transmit 100K and compute effective baud rate. PROCESS B --------- 1. Initialize self 2. Read from the chosen port using 1K reads and the read(2) call and store the data in a buffer. 3. Note time it took to receive the data and check that the correct canonical processing took place and that no characters were dropped. Communications parameters for output port: 9600 baud 7-bits, even parity, 1 stop bit, OPOST, ONLCR, no delays Communications parameters for input port: 9600 baud 7-bits, even parity, 1 stop bit, INPCK, ISTRIP, IGNCR Note that no flow control is enabled. Simultaneously, run a CPU loop program at low priority to detect host CPU consumption. The results of interest include: - the range of effective baud rates (output & input) - the average effective baud rate (output & input) - number of characters dropped (total, max per line, average) - the % of host CPU consumed ------------------------------------------------------------------------- Comments please! George Pajari
caf@omen.UUCP (Chuck Forsberg WA7KGX) (04/15/88)
In article <1974@ubc-cs.UUCP> pajari@grads.cs.ubc.ca (George Pajari) writes:
A loopback configuration is next to useless for predicting the
ability of a system to receive data at speed from external
sources (other computers).
The limiting factor is often the ability of the system to
accept data at the interrupt level. Such congestion tends to
delay transmission of the next few characters so a loopback
test will pass all characters but a real world application
will encounter missing data.
Chuck Forsberg WA7KGX ...!tektronix!reed!omen!caf
Author of YMODEM, ZMODEM, Professional-YAM, ZCOMM, and DSZ
Omen Technology Inc "The High Reliability Software"
17505-V NW Sauvie IS RD Portland OR 97231 503-621-3406
TeleGodzilla BBS: 621-3746 CIS: 70007,2304 Genie: CAFsl@van-bc.UUCP (pri=-10 Stuart Lynne) (04/15/88)
In article <1974@ubc-cs.UUCP> pajari@grads.cs.ubc.ca (George Pajari) writes: > >With all the discussions regarding performance of serial I/O >cards (especially 'intelligent' vs 'smart' vs 'dumb' vs 'retarded') >it would seem useful if we had a serial I/O benchmark (BAUDSTONE?) >which could be used to compare boards. The idea is that such >Otherwise I would like to propose the following design for such a BAUDSTONE. >The idea is to solicit comments (either over the net or mailed to me) >so that a consensus :-) may be reached, a benchmark written (I'll >volunteer if someone else hasn't/doesn't beat me to it), code distributed, >and results collected and summarised (me again). > >Simultaneously, run a CPU loop program at low priority to detect host CPU >consumption. The program I use is the standard Dhrystone program. I also try and run it a high priority to keep it running by preference. What this does is to allow me to see the amount of time lost to servicing interrupts (by the reduction in dhrystone ratings, eg for my current driver on my callan going to a timer to empty the receive buffer reduces the dhrystone rating from 931 to about 920 due to overhead in the timer call). I can also see the amount of overhead for non-interrupt time be seeing the difference in real time, ie does the dhrystone go from 95% down to 85% or processor usage or 65 to 75 seconds real time. This is a good idea. And the use of standard program such as dhrystone would allow it to easily be implemented. All that needs to be developed is a set of procedures to follow. I also question whether 7bit, even parity is close to the norm of actual use. How about simple straight forward 8bit, no parity. -- {ihnp4!alberta!ubc-vision,uunet}!van-bc!Stuart.Lynne Vancouver,BC,604-937-7532
pajari@grads.cs.ubc.ca (George Pajari) (04/15/88)
In <661@omen.UUCP>, caf@omen.UUCP (Chuck Forsberg WA7KGX) writes: > A loopback configuration is next to useless for predicting the > ability of a system to receive data at speed from external > sources (other computers). I don't agree. First, if the test shows that at 9600 baud (8-N-1) 960 characters per second were being transmitted and received, then you could safely state that the board will accept 9600 baud input regardless of the source. Furthermore, I would claim that if the measured throughput were N characters per second, then one could have reasonable confidence that the system could handle input up to N * 10 baud. (Obviously the key to the BAUDSTONE is to measure the number of characters per second actually received. You are quite correct in as much as a 9600 loop-back test tells you nothing about the ability of the system to accept 9600 baud input. If you measure the actual character throughput, however, you can draw some conclusions.) George Pajari
david@daisy.UUCP (David Schachter) (04/18/88)
Um, can we call it "BPSSTONE", instead of "BAUDSTONE"? A baud != bit per second. Thank you.
caf@omen.UUCP (Chuck Forsberg WA7KGX) (04/19/88)
In article <2063@ubc-cs.UUCP> pajari@grads.cs.ubc.ca (George Pajari) writes: :In <661@omen.UUCP>, caf@omen.UUCP (Chuck Forsberg WA7KGX) writes: : :> A loopback configuration is next to useless for predicting the :> ability of a system to receive data at speed from external :> sources (other computers). : :I don't agree. An ex-boss of mine didn't agree either, and bought a system based on loopback tests, only to discover the real world performance was as I had feared. I loopback test does not necessarily detect throughput bottlenecks caused by interrupt latency from critical regions.