[comp.realtime] Real-Time Harris

steved@hrshcx.csd.harris.com (Steve Daukas) (12/20/90)

Greetings,

My appologies for not responsing to some of your questions
concerning an earlier post of mine... I have been tied-up.

DISCLAIMER:  I do not speak for Harris proper.  I read this group
and respond on my own.  Information given is published and can be 
obtained officially by contacting your local Harris Computer Systems
Division office, or by calling (800) 666-4544 and asking for the
sales and marketing extension (this is in Florida).  I am not a
sales person, just an engineer...  I have no authority from Harris to
speak on Harris' behalf!

I don't remember the specific questions, but they were in
response to some numbers I gave as follows:  5 usec interrupt 
response time and less than 50 usec context switch times.  These
numbers reflect response times for Harris' CX/RT and CX/UX
Unix kernals running on the Night Hawk systems (M68030 and M88100
based).  Keep in mind that CX/UX and CX/RT are only available on
a Harris machine...

Someone asked how this was accomplished.  Well,  I can't give away
any secrets, but I will describe the general concepts:

Essentially,  we've added some capability to the context switcher
that enhances the context switcher's ability to make tasks runnable
for a given run queue (we are talking about multi-processors).  This,
combined with a multi-threaded preemptable kernal, allows many tasks
to be running in the kernal or device driver at one time.  When an
interrupt comes along, the handler can immediately run while other
tasks are still active.  The kernal itself is symmetric, so any task
on any CPU has equal access.  Also, all interrupts are vectorable to any
CPU.  An example for device drivers is that SPL8 doesn't actually do what 
you might think or are used to.

An example real-world test follows:

Using an external edge-triggered interrupt, a VME based DR11 interface, 
and a scope, an experiment was set up to see the delta between the interrupt 
and data available from the DR11.  This delta was much less than 50 usec.  
So, this includes the interrupt response, handler, context switch, and 
processing necessary for the DR11 to start sending data.  The CX/RT kernal 
gives much less than 50 usec (the same number) consistantly, while the CX/UX 
kernal has some gitter (CX/UX is the non-realtime version, hence the 
nondeterministic gitter).

There are many other tests done for specific requirements, as well as the
normal industry standard benchmarks.  Obviously, customer specific info is
not given out.  The other performance data can be obtained by contacting a
local Harris CSD rep.

Sorry I can't respond directly to the questions, I don't have access to 
old posts at my site...

Steve
--

Stephen C. Daukas              | sdaukas@csd.harris.com  
Harris Corporation             | uunet!hcx1!misg!sdaukas  
Computer Systems Division      | (617) 221-1834, (617) 221-1830

     "Old MacDonald had an agricultural real estate tax abatement."

tdonahue@prost.bbn.com (Tim Donahue) (12/27/90)

In article <891@hrshcx.csd.harris.com>, steved@hrshcx (Steve Daukas) writes:
>...
>I don't remember the specific questions, but they were in
>response to some numbers I gave as follows:  5 usec interrupt 
>response time and less than 50 usec context switch times.  These
>numbers reflect response times for Harris' CX/RT and CX/UX
>Unix kernals running on the Night Hawk systems (M68030 and M88100
>based).

The main question was, "What exactly is meant by `interrupt response
time`?"  Since you've repeated the performance numbers here, I presume
you'd still like to stand by them, perhaps with some clarification.  I
think I understand that:

1. The processor is an MC88100 (or is a MC68030?)
2. The processor clock rate is unspecified (although I can find out what it
    is if I call the Marketing Department?)
3. The operating system is CX/RT (or is it CX/UX?)
4. The "interrupt response time" is 5 microseconds.
5. The interval corresponding to the "interrupt response time" begins when 
    the interrupt signal (the 88100 INT pin?) becomes active.

>Someone asked how this was accomplished.  Well,  I can't give away
>any secrets, but I will describe the general concepts:
>
>Essentially,  we've added some capability to the context switcher...
>...An example for device drivers is that SPL8 doesn't actually do what 
>you might think or are used to.

Sounds like Unix (CX/UX) is really running?

>An example real-world test follows:
>
>Using an external edge-triggered interrupt, a VME based DR11 interface, 
>and a scope, an experiment was set up to see the delta between the interrupt 
>and data available from the DR11.  This delta was much less than 50 usec.  
>So, this includes the interrupt response, handler, context switch, and 
>processing necessary for the DR11 to start sending data.  The CX/RT kernal 
>gives much less than 50 usec (the same number) consistantly...

Since there is a "context switch" mentioned in the above, I presume the
task running at the time of the interrupt is preempted, and another task
associated with the DR11 is scheduled.  Is this correct?  If so, how is
the "interrupt response time" separated from the "context switch time?"

I guess I still don't understand:

1. What time elapses between when the INT pin becomes active and the
    execution of the first instruction of an interrupt handler
    installed by the application. (Applications can install
    interrupt handlers?)
2. The state of the 88100 CPU when this instruction executes (i.e., is 
    shadowing enabled and have the pipelines been drained?) 
3. If your 88100 system has external 88200 CMMUs.  If so, are the
    data caches in use?
4. What is shortest interval between the execution of a system call in one
    task and the execution of the first instruction of a second, preempting
    task, made runnable by the execution of the system call (i.e., the 
    user-mode context switch time).
 
>Stephen C. Daukas              | sdaukas@csd.harris.com  
>Harris Corporation             | uunet!hcx1!misg!sdaukas  

Cheers,
Tim

steved@hrshcx.csd.harris.com (Steve Daukas) (02/09/91)

I have been travelling around and have just gotten back to answer this
messgae.  My humble appologies for the delay!

>   Tim Donahue
>>  Steve Daukas

>From: tdonahue@prost.bbn.com (Tim Donahue)
>Newsgroups: comp.realtime
>Message-ID: <61753@bbn.BBN.COM>
>Date: 26 Dec 90 18:08:40 GMT
>References: <891@hrshcx.csd.harris.com>
>Organization: BBN Advanced Computers, Inc.
>Lines: 65
>In-reply-to: steved@hrshcx.csd.harris.com (Steve Daukas)

>The main question was, "What exactly is meant by `interrupt response
>time`?"  Since you've repeated the performance numbers here, I presume
>you'd still like to stand by them, perhaps with some clarification.  I
>think I understand that:

>1. The processor is an MC88100 (or is a MC68030?)

The interrupt response times I gave of less than 5usec are for either
the MC88100 or MC68030.  This time is the latency from time of interrupt
to the first instruction of the ISR.

>2. The processor clock rate is unspecified (although I can find out what it
>   is if I call the Marketing Department?)

The clock rates are 25MHz for both the 68030 and 88100.

>3. The operating system is CX/RT (or is it CX/UX?)

The interrupt latency is the same under CX/UX and CX/RT, as well as the
context switch times.  CX/RT is the "hard Real-Time" Unix Kernal.  This
means that things like user accounting et al are not present.

>4. The "interrupt response time" is 5 microseconds.

Actually less than 5, but Correct. 

>5. The interval corresponding to the "interrupt response time" begins when 
>   the interrupt signal (the 88100 INT pin?) becomes active.

The interval that was timed began with the rising edge of a signal
present on the Edge-Triggered Inteerrupt pin (ETI) located on the back
door of the cabinet.  I'm not a hardware expert so as to how the interrupts
are tied electrically, I can't exactly say.  They are vectorable to any CPU
in the system (up to 8) via a software config at boot time...  By the way,
the kernal is fully multithreaded, preemtable, and symmetric.

>Sounds like Unix (CX/UX) is really running?

Absolutely.  The Unix Kernal (_BOTH_ CX/UX and CX/RT pass SVVS and are SVID)
is always running - its the only kernal we offer.  CX/RT is a somewhat
stripped down version of CX/UX.  We (internally) only talk about one kernal.
The source tree is the same for both CX/UX and CX/RT.  CX/RT has fewer options
selected (or more - depending on your point of view) in the configuration
file.  Having the names UX or RT was a marketing decision.  Back a few years,
real-time Unix was considered a lie by most real-time users...  Marketing
decided that rather than referring to our Unix as having three modes
(CX/UX, CX/RT, and CX/SX), they would present it as seperate products.
(CX/SX is a B1 level secure operating system as defined by the NCSC.  Harris
developed it in conjunction with AT&T).

As an aside, our kernal fully supports the BSD libraries, AT&T, X, TCP/IP
Ethernet, etc..  Its really Unix!

>>An example real-world test follows:
>>
>>Using an external edge-triggered interrupt, a VME based DR11 interface, 
>>and a scope, an experiment was set up to see the delta between the interrupt 
>>and data available from the DR11.  This delta was much less than 50 usec.  
>>So, this includes the interrupt response, handler, context switch, and 
>>processing necessary for the DR11 to start sending data.  The CX/RT kernal 
>>gives much less than 50 usec (the same number) consistantly...

The actual number (marketing says its OK to use this number now) was
27 microseconds...  I want to clarify this based on some of your questions.
You had asked about user-level interrupts.  This is the case here.  When the
int is fired, the kernal vectors the int to the ISR asynchronously (this is a
hard interrupt), saves the floating point registers, changes the system maps, 
etc., etc..

>Since there is a "context switch" mentioned in the above, I presume the
>task running at the time of the interrupt is preempted, and another task
>associated with the DR11 is scheduled.  Is this correct?  If so, how is
>the "interrupt response time" separated from the "context switch time?"

The benchmarking folks have many tools available to measure these times,
both directly and indirectly.  One tool (approved by the US Air Force to
measure system performance) used is our performance monitor.  This tool uses
idel loops, time stamps from the 250 nanosecond system clock, et al,
to directly measure and/or calculate various metrics.  I won't pretend
that I know the exact methods.  All I can say is that it is repeatable, 
excepted by the DoD, and other customers.  If you are _really_ interested,
call me and we can talk about it - I can put you in touch with one of our
benchmark gurus...

As far as the context switch goes, this is the saving of the floating point
registers, the change of the system map, et al.  Its not a User-Level
context switch in the sense of your last question below...  The ISR runs while
the original task sleeps.  All of the normal int handleing stuff takes place
so that the state of the CPU is preserved, however the ISR runs at user level.

>I guess I still don't understand:

>1. What time elapses between when the INT pin becomes active and the
>   execution of the first instruction of an interrupt handler
>   installed by the application. (Applications can install
>   interrupt handlers?)

Yep.  You can even write an entire device driver from user level if
you like.  The elapsed time is ~5 usec.

>2. The state of the 88100 CPU when this instruction executes (i.e., is 
>   shadowing enabled and have the pipelines been drained?) 

The 88100 CPU is running the first instruction of the ISR.  All registers
have been saved, caches cleaned up, local memory taken care of, etc..
I think I need more info to discuss details - I don't want to misinterpret
your questions and then answer incorrectly...  I'm operating from the point of
view of a "software weenie", which means they don't let me play with
screw drivers!  Essentially, the 88100 goes through whatever it would go
through when you run another task...  That task, in this case, is an ISR...

>3. If your 88100 system has external 88200 CMMUs.  If so, are the
>   data caches in use?

Yes.  Up to 128kb per CPU, depending on configuration.

>4. What is shortest interval between the execution of a system call in one
>   task and the execution of the first instruction of a second, preempting
>   task, made runnable by the execution of the system call (i.e., the 
>   user-mode context switch time).

I'll assume this question is being asked to put a line of demarkation
around the previous questions (asking in reverse as it were)...

If you are asking me "how long does it take A to wake-up B", the answer
is ~40 microseconds.  This is in pure user-mode.  User interrupt is alot
faster.  

This time only applies to a single CPU system.  On a multiple CPU system, 
no context switch takes place...  The system call simply causes the other 
task to run on the other CPU.  Now, of course, if there is a task on the 
other CPU or if that task has a higher priority, then we have another situation.
 
>>Stephen C. Daukas              | sdaukas@csd.harris.com  
>>Harris Corporation             | uunet!hcx1!misg!sdaukas  

>Cheers,
>Tim


I hope this begins to clear things up Tim!  If not, or (God Forbid),
I have made matters worse, please call, post or email...

Steve

P.S. - I speek for myself.  What I say is no way represents
       official claims on behalf of Harris.  These comments
       are not valid outside of the context I present them in.
       They are true and correct to the best of my knowledge.
--

Stephen C. Daukas              | sdaukas@csd.harris.com  
Harris Corporation             | uunet!hcx1!misg!sdaukas  
Computer Systems Division      | (617) 221-1834, (617) 221-1830

     "Old MacDonald had an agricultural real estate tax abatement."

tdonahue@bbn.com (Tim Donahue) (02/12/91)

In article <983@hrshcx.csd.harris.com>, steved@hrshcx (Steve Daukas) writes:
>
><much deleted, see prior posting>
>
Thank you for taking the time to find out and reply, Steve.  This sort
of information may be hard to obtain.  To summarize, I'd say your system
has the following specifications:

0. Machine:  88100, 25 MHz, up to 128 Kb cache (? I / ? D)

1. OS:  Harris CX/RT

2. Time from interrupt signal active to first instruction of interrupt
	handler:  < 5 microseconds

3a. Time from interrupt signal active to first instruction of interrupt
	handler which does useful work (i.e., 88100 "thawed", interrupts
	reenabled, necessary context saved, etc.): 27 microseconds
	(Note that this is somewhat different from "interrupt response
	time < 5usec")

3b: Time from user interrupt handler return (executing "jmp r1") to
	interrupted instruction in task or lower-priority ISR: 
	? microseconds

4.  Time required to switch from one user task to another, same CPU:
	(about) 40 microseconds

5.  Time required for task on CPU 1 to ready task on CPU 2: ? microseconds

Note:  All measurements performed under favorable cacheing conditions.

Cheers,
Tim

Timothy P. Donahue
Operating Systems Development Group
BBN Advanced Computers, Inc.
10 Fawcett Street
Cambridge, MA 02138
(617) 873-6000

tdonahue@bbn.com
--