pete@violet.berkeley.edu (Pete Goodeve) (03/30/88)
.....................
... and the Elephant.
----------------
Looking back over the flood, we've had a lot of interesting proposals.
What's most interesting is the difficulty we all seem to be having in
agreeing on what we really mean by "InterProcess Communication". Also,
as Peter da Silva pointed out a while back ("IPC -- what are our goals?"
<798@nuchat.UUCP>) and Lion Kuntz has complained to me in a private message,
we've tended to put forward all these neat ideas without really seeming to
think about how they would be used in practice.
I don't think it's so much that we haven't thought about actual
applications, as that we each have our own picture of what WE want IPC to
do for us and have tended to assume that everyone else sees the same. In
fact the possibilities are vast, and I don't think any one scheme can
really handle it all.
That's why I was so concerned with getting a common-kernel protocol agreed
on. I thought that it should be possible to service more than one IPC
scheme in a single program; ID codes in the message itself would indicate
format and function. This way we could try out alternatives without
committing ourselves prematurely.
For a while I was trying to squeeze a common format into the standard
message structure -- using the ln_Type field in particular -- so that we
could retrofit to existing formats like AREXX and StdIOReq structures. My
reconsidered thoughts are that it is better to keep such incompatible
formats apart with separate ports, if we really want to handle more than
one. Mixing message types is obviously often a lost cause: Stuart's
"method/object" scheme and AREXX, for example, are sufficiently different
in philosophy that you probably could never couple the two. (The
possibility of an "interpreter process" always exists, though.)
On the other hand, I see no real gulf between, say, "method/object" and my
"message broker", or Peter's Browser. We may trip up over the details, but
I think we can at least aim for compatibility.
Anyhow, I thought this would be a good time to give at least my perspective
on the whole field. I'll try to include a few examples, but this is mostly
what I conceive as the spectrum of possibilities.
+++++++++++++++++++++++++++++++
1) Mechanisms for IPC:
1.1) Exec Messages/Ports:
This is what we've all been concentrating on. It certainly appeals
to me as the obvious basis for general IPC, though, as we've
discussed, the mechanisms -- especially port access -- will need
some tidying up. Messages win in speed and flexibility over any
approach that uses the file system for example, or any other
serial-data oriented system. I'm going to stick mostly to messages
in this tract anyway.
1.2) Files -- esp. RAM:
Nothing wrong with passing information this way when it suits.
It's not particularly fast -- though it's often fast enough
compared to human response time, and it's universally available.
And we can always pass a file handle (or lock) in a message (can't
we?). With multiple processes around, you've got to coordinate
access to avoid collisions, so you can't do without messages as
well.
1.3) Pipes:
Again, these shouldn't be ignored where they serve the purpose.
Programs needn't be specially written to use them. They have both
the advantage and disadvantage of being strictly serial streams:
makes things very simple sometimes, but can frustrate you totally
when you want random access. In passing, I like (and use a lot) Ed
Puckett's "Pipe directory" scheme, but I think there are a couple
of extra features that would be useful. Sometime I'll try to get
into that.
1.4) Other filesystem mechanisms:
e.g. ENV:
Again, why not?
1.5) Clipboard:
Now here, I have difficulty understanding how a clipboard improves
on a RAM: file (or a pipe). It's faster, I suppose, but the file
has the big advantage of a name (and you can have more than one at
a time...!). Can anyone enlighten me?
2) IPC Regimes:
I've tried here to arrange the spectrum of applications into some sort
of grouping. How much have I left out?
2.1) Master <-> Slave -- Synchronous:
This is the simplest setup of all -- almost hardly counts as IPC.
Here one program calls on another to perform a particular job, and
waits for it to complete.
An example is VideoScape-3D, which can signal another program that
it has generated a picture; this second program can, for instance,
snap a camera shutter, or capture an IFF image of the screen into a
disk file. VideoScape waits until the other program replies that
it is done before starting on the next frame.
2.2) Master <-> Slave -- Asynchronous:
The same as the previous, except that the master is doing other
things while the slave performs its function. No specific example,
but you could imagine Videoscape drawing into an off-screen buffer
meanwhile.
2.3) Cooperating programs passing data (including commands) directly:
What I think most of us conceive as the core of IPC. The main
difference from (2.2) is the lack of any real dominance hierarchy;
each request passed will involve a Client and a Server, but a
process could be filling both roles at once, possibly with the same
partner process.
Proposals such as "method/object" and the Browser port seem aimed
at this level. Also Peter's "Patch Panel" concept. Anything from
MIDI modules to communicating complexes of Editors, Spreadsheets,
and Desktop Publishers can fall under this heading, but this
doesn't mean that they can all necessarily use the same mechanism:
there may be very different constraints (see below, section 4).
More ideas from people as to specific jobs they'd like to see
implemented in this way would be great!
2.4) "Broadcasting" of data for other processes to pick up if they want:
My own background sparked my desire for this facility. I do a lot
of work on laboratory systems for experiment control and data
collection and processing. I see the Amiga as ideal for this, but
it needs an edge over those other machines we don't mention.
One thing I've found is that the actual interface to the real world
for a particular experiment is pretty standard, but you want to do
different things with the data -- maybe more than one thing at
once. Sometimes processing is too complicated to do real-time, and
about all you can do is record data for later analysis, but more
often you can at least do Q&D chomping on the stuff coming in, so
the experimenter knows what's going on. Now wouldn't it be nice if
you could just drop recording and analysis modules into place?
They would pick up the collected data from the central broadcasting
module, which needn't know or care where its output was going, or
how many different modules were picking it up. (There is actually
already a similar idea for the Mac, I'm afraid, called LabView, but
of course it doesn't have the freedom of multitasking.)
That's not the only application I see for such a "message broker"
facility. MIDI modules could easily be connected this way, and
there must be lots of other occasions where you'd like several
destinations for data emanating from one source. It wouldn't be
quite as fast as direct message passing between processes, because
the messages have to be duplicated: some form of memory block
transfer or sharing scheme would have to go along with it if large
data buffers were to be passed -- you wouldn't want to copy them!
My original version of the scheme was not in fact message-based,
and might still be better.
2.5) Common script command control of multiple processes:
In other words, AREXX (or future competitors). I suppose this is
really a special case of (2.3), but I don't think it's synonymous
with it. It gives you more freedom in melding diverse programs
together, but is likely (let's say certain..) to be a bottleneck
for other IPC applications that need speed.
The best example of this approach to date is probably Tom Rokicki's
Tex/mgemacs link up.
2.6) Multiple scripts -- "hyperamiga"?
An elaboration of (2.5) -- perhaps in the form of a resident
library -- that lets the user build up a network of interlocking
processes and scripts to perform the most complicated tasks he can
imagine. (Is that vague enough to cover everything else? (:-))
This I suspect is close to Ron Minnich's dream system that sparked
this whole affair off.
3) Types of message content:
3.1) Command codes (e.g. longwords)
3.2) Text commands
3.3) Data buffers
3.4) locks, handles, ports etc.
.. and doubtless others you can think of.
Remember that if we're looking to the Next Generation, ALL these items
should be ONLY in "Public" memory (MEMF_PUBLIC), or we'll suddenly find
that all our programs are breaking. [Just like what happened to some
people when systems started getting bigger than 512K...]
4) Special case constraints and considerations:
I think that we have to allow for different IPC applications to be
affected by any of the following. Arbitrarily forbidding any of them
at this stage will probably rebound on us later. (Mostly I'll just
list them briefly for your consideration. A few notes where I think
they're needed.)
4.1) Maximum speed:
MIDI
Serial communications
4.2) Large data blocks:
Bitmaps
4.3) Transfer of data ownership:
Data buffers, file locks etc.
4.4) Remote processes:
Networks
-- restricts kinds of data that can be passed (no handles, locks
etc.)
4.5) Non-existence (or disappearance) of target process or port:
-- Matt Dillon's port-with-use-count is a complete solution to this
problem.
4.6) Priorities of simultaneous processes:
-- if a process never has to wait for I/O, and completes fairly
quickly, it is wasteful to task swap in the middle of it; the
invoker should wait for it to finish.
4.7) Resource sharing and locking:
Disk thrashing (we all know this one, don't we...)
Memory areas may be common to several processes (and the original
creator may not stick around); a common management protocol may be
needed here too.
4.8) Deadlocks:
-- always a danger when you get complex interactions; we should
include mechanisms -- timeouts or perhaps a standard "breakout"
message a user could send -- to break them.
++++++++++++++++++++++++++++++++++++++
OK -- now YOU take the binoculars...
___________________________________________________
|
One of the drawbacks of a renaissance is |
that everyone wants to be a critic. |
|
-- David Brin [New Scientist 18 February 88] |
|
Hey, let's have a renaissance...! | -- Pete Goodeve --
__________________________________________________|jesup@pawl16.pawl.rpi.edu (Randell E. Jesup) (03/31/88)
In article <8139@agate.BERKELEY.EDU> pete@violet.berkeley.edu (Pete Goodeve) writes: > 2.5) Common script command control of multiple processes: > In other words, AREXX (or future competitors). I suppose this is > really a special case of (2.3), but I don't think it's synonymous > with it. It gives you more freedom in melding diverse programs > together, but is likely (let's say certain..) to be a bottleneck > for other IPC applications that need speed. > 2.6) Multiple scripts -- "hyperamiga"? > An elaboration of (2.5) -- perhaps in the form of a resident > library -- that lets the user build up a network of interlocking > processes and scripts to perform the most complicated tasks he can > imagine. (Is that vague enough to cover everything else? (:-)) I think there's some basic misconception here. ARexx IS designed as a resident library. You can have ARexx scripts talking to other ARexx scripts, which are talking to other applications, ad nauseum. It shouldn't be a bottle- neck unless the interpreted language is to slow, in shich case you can add compiled libraries of functions to do all the dirty work. Remember, there can be any number of ARexx processes using the library. > 4.4) Remote processes: > Networks > -- restricts kinds of data that can be passed (no handles, locks > etc.) With appropriately clever design, even this can be done (if there is a way to issue a read request across the network to a handler process on another machine. This process would do the read, then pass the data back across the net. This requires preparing the filehandle for export before passing it, and may require issuing a request for the remote node to make a filehandle for you to associate with the local filehandle. A bit tricky, but certainly possible. // Randell Jesup Lunge Software Development // Dedicated Amiga Programmer 13 Frear Ave, Troy, NY 12180 \\// beowulf!lunge!jesup@steinmetz.UUCP (518) 272-2942 \/ (uunet!steinmetz!beowulf!lunge!jesup) BIX: rjesup (-: The Few, The Proud, The Architects of the RPM40 40MIPS CMOS Micro :-)