ken@gvax.cs.cornell.edu (Ken Birman) (08/18/89)
Mark Wood showed me something that might interest some of you.
He was working with a student who wrote an interactive ISIS program.
Because this program reads both from the console and from ISIS, it
was designed to use isis_input_sig. The basic loop looked something
like this:
iocntl(tty_fdes, .. nonblocking ...)
forever
{
static condition input_avail = (condition)0;
switch(read(tty_fdes, &c, 1))
{
case 1: consume(c); continue;
case 0: printf("Quit.\n"); exit(0);
case -1: if(errno != EWOULDBLOCK) { printf("Quit.\n"); exit(0);}
}
/* got EWOULDBLOCK */
isis_input_sig(tty_fdes, &input_avail, 0);
t_wait(&input_avail);
isis_input_sig(tty_fdes, (condition*)0, 0);
}
The reason for the second call to isis_input_sig() is that the
ISIS manual warns that "if input becomes available and no task
is waiting for it, clib will panic." (Actually, we may change this
to buffer up signals, but if you think about it you will see that
this gets very tricky.) Calls to isis_input_sig are VERY
cheap, so this is a good solution.
Now, if the above is part of a task (run from t_fork, isis_entry,
isis_mainloop, etc) it works correctly. This is because ISIS runs
tasks non-preemtively. That is, between when the isis_input_sig
is first called and when the wait occurs, ISIS itself won't get to
run.
In Mark's student's example, however, the same loop was called
directly from the program's main procedure, after calling isis_init().
That is, isis_mainloop wasn't used. (In this case, because of the
frequent calls to t_wait, isis_accept_events isn't even needed, although
normally it would be called somewhere in the loop).
This resulted in a race condition. Now, ISIS is permitted to run
in parallel with the main task, and in particular it does get to
run briefly just as the t_wait is entered (this has to do with the
way that ISIS maps a "non-isis task" so that it will have a task-table
entry). A result is that the program now panics: ISIS may well notice
input on the tty_fdes before the t_wait actually sets up its caller as
a task and queues it on the condition variable.
Seems a bit grungy? Actually, this is pretty normal for concurrency
with lightweight tasks, and in fact this is why ISIS itself runs
everything non-preemptively. As the example illustrates, people tend
to THINK non-preemptively. If you plan to actually code with
preemptive lightweight tasks, I strongly urge you to read up on the
topic. Most good operating systems texts cover this in depth.
A quick fix solved the problem. The macro ISIS_ENTER() is used to
register the caller as an isis task (it is a no-op when called from
a task). ISIS_EXIT() is used to return to the non-ISIS, preemptive
world. So, the same code never panics if changed to the following:
/* got EWOULDBLOCK */
ISIS_ENTER(); /* Grab mutual exclusion */
isis_input_sig(tty_fdes, &input_avail, 0);
t_wait(&input_avail);
isis_input_sig(tty_fdes, (condition*)0, 0);
ISIS_EXIT(); /* Release mutual exclusion */
I didn't realize that there might be a good reason to document
ISIS_ENTER/EXIT (and ISIS_RETURN(value)). Guess I'll have to
update the manual and add a man page...
Ken