[comp.unix.wizards] A very interesting problem -- multiprocessing

richd@sci.UUCP (Rich doherty) (05/01/87)

	I have a program that I have dynamically linked in several
functions.  I want this same program to call these functions, then
send an interrupt to halt the function and return control momentarily
to the main function, with the ability to return to the special function.

In other words, I want:

functionA()
{
/* garunteed infinite loop */
}

functionB()
{
/* another infinite loop */
}

main()
{
Start functionA and suspend it;
Start functionB and suspend it;
while (1)
{
	continue functionA for 10 seconds;
	continue functionB for 10 seconds;
}
}


	I somehow want to do the equivalent of setjmp/longjmp,
except that I want to remember many stack histories, and I do
not want the stack to be unwound without someway to get it back.
Note also that functionA and functionB will call other functions,
and they may both call the same function -- hence I need local variables
and stack calling history preserved.  Again, I don't mind if functionA
and functionB forget who called them, since they will never end.

I could do this by having the program fork itself multiple times,
and then suspend and continue each child copy.  But I want all
these functions to operate on the same global variables, and shared
memory does not exist in BSD4.2 .  I also intend to have several of these
special functions, and forking that many times seems wasteful.
I could make each of these satellite functions be actual programs,
all running under the control of some main program.  If this is
the only way to solve this problem, then I need a suggestion as to
how to rapidly send packets of data between processes (I assume that
packet sending is fairly common, so that this solution should be feasible).

Any comments or advice would be appreciated.
		-- Rich Doherty

thomas%spline.uucp@utah-gr.UUCP (Spencer W. Thomas) (05/04/87)

Funny you should ask: you could do this on V6, through a special
coroutine library.  Unfortunately, this was not very portable
(depended on PDP-11 architecture, I think), so out the window it went
(was it in V7?  I don't remember.)

Anyway, if you want to coroutine a couple of functions with no
arguments (easiest case), you just allocate a "large enough" chunk of
memory for each one's stack, set the stack pointer to the correct end
of it (top on a Vax), and call the function.  (Oh, I should mention
that you need to save the real stack pointer somewhere.)  The
"Suspend" action then saves the co-routine stack pointer in a static
variable, and resumes execution on the real stack.  To continue a
co-routine, you restore its saved stack pointer and do a subroutine
return.  Maybe a little code fragment would help:

    static char * real_sp;  	    	/* Save real stack pointer */
    static int num_coroutines = 0;  	/* None initially */
    static char * save_co_sp[MAX_CORTN];	/* stack pointer storage */
    static int cur_coroutine = -1;    	/* Currently executing coroutine */

    /* Start a co-routine function.  Returns (via coroutinepause) an
     * integer identifier for that function.
     */
    /* All local vars must be register.  Don't want any on stack,
       will screw up return from coroutinepause. */
    coroutinestart( fn, stack_size )
    register void (*fn)();
    register int stack_size;
    {
    	register char * stack;

        if ( cur_coroutine >= 0 )
    	    error( ... ) /* Can't start co-routine from a co-routine */
    	if ( ++num_coroutines == MAX_CORTN )
    	    error( ... ) /* core dump or something */
    	
    	stack = malloc( stack_size );
    	stack += stack_size;	/* Assume stack grows as --sp */
    	cur_coroutine = num_coroutines - 1;
    	/* Now go into some fancy assembly code that saves the
    	   current stack and invokes the co-routine.  Example is
    	   pseudo-vax */
        asm( "mov sp,real_sp" );
    	asm( "mov stack,sp" );
    	(*fn)();	    	/* Should never return, but in case ... */
    	asm( "0" ); 	    	/* Or something else illegal */
    	/* Could do something reasonable here if we had the concept
    	   of a "dead" coroutine, but we don't. */
    }

    /* Called from a coroutine to relinquish control */
    /* Note that this will return to the fn that called
       coroutinestart or coroutineresume.  A return from here occurs
       via coroutineresume.  (Confused yet?) */
     
    coroutinepause()
    {
    	register char * stack;
        register int last_cortn;
    	if ( cur_coroutine < 0 )
    	    return; 	    	/* Paranoid check */
    	/* More assembly to get back to regular stack */
    	asm( "mov sp,stack" );
    	asm( "mov real_sp,sp" );
    	/* Save costack pointer */
    	save_co_sp[cur_coroutine] = stack;
        last_cortn = cur_coroutine;
    	cur_coroutine = -1;
    	return last_cortn;
    }

    /* Called from normal code to resume a coroutine. Also returns
       via coroutinepause.  Returns id.  All locals must be register.
     */
    coroutineresume( id )
    register int id;
    {
    	register char * stack;
    	if ( id < 0 || id >= num_coroutines || cur_coroutine >= 0 )
    	    return id;	    /* Non-existant co-routine, or already active */
    	stack = save_co_sp[id];
    	cur_coroutine = id;
    	asm( "mov sp, real_sp" );
    	asm( "mov stack,sp" );
    	return; /* from call to coroutinepause */
    }

    /* Switch from one coroutine to another */
    coroutineswitch( id )
    register int id;
    {
    	register char * stack;
        /* Sanity check id, ensure co-routine active */
    	if ( id < 0 || id >= num_coroutines || cur_coroutine < 0 )
    	    return id;
    	/* Switch stacks */
    	asm( "mov sp, stack" );
    	save_co_sp[cur_coroutine] = stack;
    	stack = save_co_sp[id];
    	cur_coroutine = id;
    	asm( "mov stack,sp" );
    	return;
    }

Each co-routine should look like this:
    a_co_rtn()
    {
    	coroutinepause();
    	for (;;)
    	{
    	    /* Do some work */
    	    coroutinepause();

    	    /* Do some more work, maybe switch to another */
    	    coroutineswitch( other_id );

    	    /* etc */
    	}
    	/* NOTREACHED */
    }

The initial call to coroutinepause() is for two reasons: it permits
the function id to be recorded somewhere meaningful for the
application, and it gets the coroutine stack pointer set up.

You have to be careful not to exceed your allocated stack space, the
computer will not check it for you!  You really can't expand the stack
once you start, either.  I imagine there is not more than one bug per
line of code here, but the ideas should be right.  There is no doubt
potential for combining common code, and eliminating the special
cases.  You should be able to start a coroutine from another, and so
on.  (E.g., if resumecoroutine is called from a coroutine, just do the
coroutineswitch case.  I wanted to separate function for clarity.)

This code is NOT tested.  Have fun with it.

=Spencer   ({ihnp4,decvax}!utah-cs!thomas, thomas@cs.utah.edu)

libes@nbs-amrf.UUCP (05/06/87)

Comer's XINU provides multitasking in a single process.  If your
coroutines explicitly give up control, you can use setjmp/longjmp.
If you want preemptive scheduling, you must save the registers by
hand with a little assembler, and set up a timer interrupt that
calls the scheduler.

I ported XINU to a 68k running 4.2 and it works great.  Here is
your program rewritten in XINU:

void proc1()
{
	for (;;) fprintf(stderr,"1");
}

void proc2()
{
	for (;;) fprintf(stderr,"2");
}

user_main()
{
	xresume(xcreate(proc1,2000,20,"proc1",0));
	xresume(xcreate(proc2,2000,20,"proc2",0));
}

The args to xcreate (X for Xinu to avoid conflicts with UNIX funcs)
are 1) function, 2) stack size (I use 2000 because fprintf uses a
lot!), 3) priority, 4) function name, 5) number of arguments to
function, 6) optional arguments.  Currently, the quantum size is
compiled into the scheduler, but that could also be passed as an
argument.

I wrote this all up in a paper, and submitted it to USENIX but it
was rejected.  Now it's being studied by the ;login:  folks.  If
they don't like it, perhaps I'll just give up and have it printed
locally as a TR.  I know it's not much, but so many people have
expressed interest in it, I figure it must be worth printing
somewhere.  Any suggestions?

Don Libes                {seismo,mimsy}!nbs-amrf!libes

tpm@datlog.co.uk ( Tim Murnaghan ) (05/06/87)

In article <2017@utah-gr.UUCP> thomas%spline.UUCP@utah-gr.UUCP (Spencer W. Thomas) writes:
>
>
>Anyway, if you want to coroutine a couple of functions with no
>arguments (easiest case), you just allocate a "large enough" chunk of
>memory for each one's stack, set the stack pointer to the correct end
>of it (top on a Vax), and call the function.  (Oh, I should mention
>that you need to save the real stack pointer somewhere.)  The
>"Suspend" action then saves the co-routine stack pointer in a static
>variable, and resumes execution on the real stack.  To continue a
>co-routine, you restore its saved stack pointer and do a subroutine
>return.  Maybe a little code fragment would help:
>....
Of course you don't have to use assembler to do the stack setting, setjmp
remembers everything that you need.
All you have to do is gallop down the stack until you've left enough stack
for the coroutine below and then do your setjmp (we have a routine called
gobble).
Some implementors have qualms about longjmp'ing the wrong way which is a shame,
but at least the asssembler bits can be hidden behind a standard interface
when you write your own.

We do actually use this sort of thing (with a bit more business for
synchronisation ) keeping our co-routines on a list and having a noddy
scheduler kick them off and calling them activities (for we are using the MOD
methodology MASCOT).