[comp.os.misc] Multi-CPU system task scheduling

walker@ficc.uu.net (Walker Mangum) (11/10/88)

I'm working on a real-time multi-CPU system in which a process can
be scheduled by another process, a timer, or an event in the "most
attractive" CPU.  The actual mechanics of loading and scheduling
the new process work just fine.  The problem is the definition of
"most attractive" CPU.

The goal of "most attractive" scheduling is to get the process to
execute in the CPU that will allow the process to complete as
quickly as possible, and to keep resource utilization among the
CPUs somewhat even.  Once a process is assigned to a CPU, it
remains in that CPU until it completes.  Process scheduling within
CPUs is priority based.

Currently, the "most attractive" CPU is determined by sampling the
load average for all CPUs, and selecting the CPU that has the
lowest load average and has no processes holding for memory
allocation.

THe problem is that the load average is determined once per second,
and occasionally *many* processes are simultaneously scheduled.
This means that, during each second, the current "most attractive"
CPU does not change (unless it becomes low on available memory),
and *all* processes are scheduled in that CPU!  Needless to say,
the results are not as desired....

We're considering adding an arbitrary delta to the load average for
a CPU when a process is scheduled that CPU.  This delta could be
(although not easily) a unique value for each process, with the
proper value for each process determined by system test and
performance evaluation.  This would effectively scatter processes
scheduled simultaneously, avoiding the current overloading problem.

Is there a better way to handle this problem?  I would appreciate
*any* and *all* ideas/answers.

Please to uunet!ficc!walker.

----------------------------------------------------------------------
-- 
Walker Mangum                                  |  Adytum, Incorporated
phone: (713) 333-1509                          |  1100 NASA Road One  
UUCP:  uunet!ficc!walker  (walker@ficc.uu.net) |  Houston, TX  77058
Disclaimer: $#!+ HAPPENS

drac@igloo.UUCP (Bruce Maynard) (11/13/88)

In article <2174@ficc.uu.net>, walker@ficc.uu.net (Walker Mangum) writes:
> 
> I'm working on a real-time multi-CPU system in which a process can
> be scheduled by another process, a timer, or an event in the "most
> attractive" CPU.  The actual mechanics of loading and scheduling
> the new process work just fine.  The problem is the definition of
> "most attractive" CPU.
> 
>i*abbreviated*
 
> Currently, the "most attractive" CPU is determined by sampling the
> load average for all CPUs, and selecting the CPU that has the
> lowest load average and has no processes holding for memory
> allocation.

 *abbreviated*> 
> Is there a better way to handle this problem?  I would appreciate
> *any* and *all* ideas/answers.
> 

   Can you change 'most favored CPU' selection to the one with the longest
idle time? Seems to me that if you use that as the major criterion (and, of
course, change that cpu's idle time to '0' immediately) you may get better 
results; the difference is not immediately obvious, but we use it all the
time on some of our systems, and it works rather well...
There is a possibility, of course, of the system bottlenecking if all CPU's
are 'busy' when a job is ready to be submitted; you can either hold the job,
or THEN pick the CPU with the lowest loading. Personally, I'd think that what
you might want to do is have a 'central' pool of job submissions and then have
each CPU query the pool (say, every second or so) to see if there's a new
process waiting. Then, as each task is entered, it would automatically select
the next job in line (queued by priority and time of submission of course).

     Hope this's some help; if not, Se la guerre...

"If it ain't broke, don't fix it!"        Bruce Maynard
                                          Hanover Park, IL

jms@antares.UUCP (joe smith) (11/17/88)

In article <2174@ficc.uu.net>, walker@ficc.uu.net (Walker Mangum) writes:
> 
> I'm working on a real-time multi-CPU system in which a process can
> be scheduled by another process, a timer, or an event in the "most
> attractive" CPU.  The actual mechanics of loading and scheduling
> the new process work just fine.  The problem is the definition of
> "most attractive" CPU.

This information may not help in your particular problem, but you might
want look at how BiiN is doing it.  Their CPUs have the ability to do
a context switch and pull the next task block off of a queue in a single
instruction.  The scheduler software builds a queue of runnable processes
that the hardware understands.  Whenever a CPU determines that it cannot
run the current process anymore (waiting for a resource, for I/O, or time
quanta expired), it picks the next available process to run.  There is no
need to schedule a job for a particular CPU; they are all identical.

In order for this to work, all I/O controllers have to be accessable to
all CPUs.  All CPUs see the same set of interrupts.  Whichever CPU that
can handle it first takes care of the interrupt.

I doubt that your system has this capability, but I find the concept
intriguing.
-- 
+----------------------------------------------------------------------------+
| TYMNET:JMS@F29  CA:"POPJ P,"  UUCP:{ames|pyramid}oliveb!tymix!antares!jms  |
| INTERNET: (Office-1.ARPA is no more)      PHONE:Joe Smith @ (408)922-6220  |
+----------------------------------------------------------------------------+