shf@well.UUCP (Stuart H. Ferguson) (03/05/88)
IPC - Standards
A proposal
Yeah, let's do it! It's high time that the Amiga had a standard method of
inter-process communication. With products like ARexx and Browser coming
out, we may get stuck with a "de-facto" standard which may not be well
designed, or several "standards" that are mutually incompatable. ("It's
supports ARexx, but does it support Browser?")
Lets get on the ball now and *design* a standard before one gets designed
for us. The potential rewards are enormous, and the costs are just the
effort to get it started and the extra programming effort for everyone to
use it, and that is distributed over a large number of programmers over a
large period of time, so the individual cost is small.
The reward is an integrated environment without equal. An interprocess
communication standard is not just another linear development, but has very
non-linear effects, leveraging each component of the system against the
others. Multi-tasking multiplies the power of individual tools together.
An interprocess communication standard promises to multiply them again.
Ok, ok. Enough hype. You get the idea that I'm really excited by the idea?
Good. I am. Here's a first cut at it ...
------
IPC1 - Interprocess Communication Standard (Proposal)
This describes the format for a standard message block. The format is
general enough to allow for any type of data necessary to be included in the
message, but is also restrictive enough to prevent many errors and to
provide for upward compatability. The basic design is derived from the EA
IFF 85 standard, since that has worked so well and lots of Amiga users
already understand it.
The standard message block has the form:
struct GenericMessage {
struct Message gm_mess;
ID magic_code;
long gm_size;
ID gm_code;
char gm_body [ /* value of gm_size - 4 */ ];
}
where "ID" is a four-byte integer (i.e., a "long"). The possible values
for gm_code are defined the same way as the Chunk ID's for an IFF file,
namely, 4 characters from a certain range of ASCII codes concatenated
together. Trailing spaces are permitted. The "ID" value magic_code shall
be the characters "IPC1" encoded this way, and will be the way for
programs to identify standard message blocks. The value of gm_size is the
size in bytes of the message block starting with the byte AFTER the last
byte of the gm_size field itself (like IFF files).
Gm_code will a 4 character ID code, and the standard definition should
include some preset values for general message types. There may also be
code values that all tasks chosing to support the IPC1 protocol will have
to know how to answer.
The "body" of the message, gm_body, will consist of zero or more "Chunks" in
series, each one starting on the first even byte after the previous one.
Chunks are defined as:
struct Chunk {
ID chunk_id;
long chunk_size;
char chunk_body [ /* value of chunk_size */ ];
}
The meaning of the data in chunk_body depends on the value of chunk_id as
well as the value of gm_code. The data in chunk_body are positionally
defined.
The nested chunk design allows for future expansion so that the data that
can be passed in a message can change without breaking programs that only
understand the unexpanded message. Parsing this structure is almost
trivial, and ignoring unrecognized chunks is as simple as skipping over
them.
Other issues:
This is just a first attempt at attacking just one part of the problem of
defining an interprocess communication standard. There are other issues
that should be addressed:
1. What is the communication model to be? Clients & servers?
Asyncronous free-for-all? Other ideas?
2. What are some important message classes to define as standard? What
message does everyone have to understand?
3. How do the public named message ports figure into all of this,
especially question number 1?
4. Can devices be folded in as a special case, or should they just be
regarded as their own standard?
Also, just hashing out a good idea won't make it a standard. It will
require some software that uses it, as well as some cooperation from the
software people at Commodore. Any ideas about how to go about the "public
relations" part of the standards design?
All too often we get stuck with short-sighted or limiting "de-facto"
standards more or less by accident (the IBM PC is a good example). Let's
not let this happen this time. Thanx for the bandwidth.
--
Stuart Ferguson (shf@well.UUCP)
Action by HAVOC (shf@Solar.Stanford.EDU)ewhac@well.UUCP (Leo 'Bols Ewhac' Schwab) (03/06/88)
In article <5375@well.UUCP> shf@well.UUCP (Stuart H. Ferguson) writes: > >IPC - Standards > >A proposal > >Yeah, let's do it! It's high time that the Amiga had a standard method of >inter-process communication. With products like ARexx and Browser coming >out, we may get stuck with a "de-facto" standard which may not be well >designed, or several "standards" that are mutually incompatable. ("It's >supports ARexx, but does it support Browser?") > I had hoped this controversy would have stayed quiet until I could have thought of an elegant mechanism for this. I have the beginnings of it, but it's not at all completely worked out. >IPC1 - Interprocess Communication Standard (Proposal) > >The basic design is derived from the EA IFF 85 standard, since that has >worked so well and lots of Amiga users already understand it. > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ HAAAAHAHAHAHAHAHAHAHAHAHA....... IFF is one of the most byzantine "things" ever created. It is extremely difficult to parse (if you want to be "truly compatible"), and no one uses the same piece of code to read or write it. Everyone avoids EA's published code, since it's so vile. Stuart then goes on to describe the data exchange format. I've been thinking about this problem for a while, since I've had to address it directly and indirectly a number of times over the past few months. I've been trying to come up with an elegant solution. UNIX achieves IPC (usually) through pipes. One program's stdout is connected to another program's stdin. On UNIX, this works great since, in most cases, all user I/O through the program is through stdin and stdout. On the Amiga, however, this is not the case. User interaction may come through stdin. More often than not, however, it comes through the IDCMP port. No assumptions can be made about what's coming through the IDCMP port, since primary user interaction may be anything from RAWKEY events, to menu events, to gadget events, to morse code tapped in on the mouse button. The only solution to this dichotomy of user interaction methods is to define a new message port. The sole purpose of this port is to accept standardized commands from outside. As I'm led to believe, this is how ARexx does it currently. However, ARexx does not establish a standard command format. I used to believe that the only requirement would be some sort of command port, and document its use. It would be the responsibility of external programs to abide by the established protcol. Taking this method to its logical conclusion, I ended up with a very messy rat's nest of user scripts, making things unreliable. So a free-for-all in terms of command structure won't work. I'm still working out the actual command structure. I'm trying to consider all the things that a programmer or user will want to do, and provide a facility for it. I also want to keep the spec open, in case I left something out, and I also want the spec to seem familiar (i.e. based on an existing Amiga facility) so that a new mess of procedures will not have to be learned. Stuart was trying to address data exchange. This is only a minor part of the whole problem. The primary requirement is getting programs to *do* something outside of normal user interaction. One of these commands could be to dump its data. The format of the data should be the problem of the program asking for it (this is how it is currently). Data format exchange standards should not be worried about just yet. We have to establish a method of communication and "remote control" first. Once we get the remote controls in, then everything will be possible. If Sculpt and VideoScape had remote control ports, the John Foust's Interchange program could be nothing more than an ARexx script. Gimme some time; I'll try and cook something up. Who knows? It may actually be worth something. _-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_ Leo L. Schwab -- The Guy in The Cape ihnp4!ptsfa -\ \_ -_ Recumbent Bikes: dual ---> !{well,unicom}!ewhac O----^o The Only Way To Fly. hplabs / (pronounced "AE-wack") "Work FOR? I don't work FOR anybody! I'm just having fun." -- The Doctor
dillon@CORY.BERKELEY.EDU (Matt Dillon) (03/06/88)
>IPC - Standards > >A proposal > >Yeah, let's do it! It's high time that the Amiga had a standard method of >inter-process communication. With products like ARexx and Browser coming >out, we may get stuck with a "de-facto" standard which may not be well >designed, or several "standards" that are mutually incompatable. ("It's >supports ARexx, but does it support Browser?") I agree, but let's do it right. One reason the clipboard device never caught on (to non-commercial programs) is that it is difficult to use. ARexx is interesting, but also somewhat convoluted and takes too much overhead. It doesn't have as much power as I would like. Whatever we decide on, it must be made relatively easy to use. But we don't simply want a stream oriented IPC. People don't take enough time to think of easy ways to provide maximum power and capability. This is what I envision: -Low per-task memory overhead -Symbolic Asynchronous Remote-Control capability (domain based) e.g. one application can send symbolic commands to another... A data stream, as part of a command, would be a minor item in this system. (DME.file1(unblock block block) DME.file2(bcopy)) (DME.file1(saveas ram:x) TEXTED(File-Open ram:x)) CLI(Delete ram:x)) Also, have a standard format for SARC menu item simulation. e.g. DME(File-Quit) -Macro Expansion capability (which is a superset of what ARexx would give us) e.g. program X gives the driver a symbolic macro which the driver executes, possibly causing remote-control commands be sent back to program X and/or to other applications. -Fully reentrant and stackable remote commands, with infinite loop detection, etc... without taking a huge amount of overhead. e.g. program X sends a Macro to the driver, which sends resolved commands back to program X which just happen to be macros in program X which is expanded again and sent as a Macro to the driver, which sends resolved commands to program's X and Y, program X executing the command, etc.... Also allowing program X to handle such things synchronously or asynchronously (not wait for macro completion before continuing, but not getting confused either). -Error recovery... so things don't freeze up. -Exit recovery. Allow program X to exit at any time without screwing up any macro's in progress (have them gracefully fall to their deaths). i.e. calling SarcClose() clears anything pending. -Standard command format. Ability to specify logical or physical streams (i.e. supply your own functions or use DOS functions, etc...). -Low level IFF decoder for arbitrary IFF streams. Since IFF is structured, the decoder would not have to know specific formats, -Ditto Low level IFF encoder for arbitrary IFF chunks.. Structures the chunks for you. -Run time Library to handle interaction. I could work on something like this after I get DNET out. -Matt
pete@violet.berkeley.edu ( Pete Goodeve ) (03/06/88)
Yeah...! The race is on...! =============================================== shf@well.UUCP (Stuart H. Ferguson): > Ok, ok. Enough hype. You get the idea that I'm really excited by the idea? > Good. I am. Here's a first cut at it ... > > IPC1 - Interprocess Communication Standard (Proposal) > =============================================== dillon@CORY.BERKELEY.EDU (Matt Dillon) > This is what I envision: > ..... > > I could work on something like this after I get DNET out. > =============================================== ewhac@well.UUCP (Leo 'Bols Ewhac' Schwab) > I had hoped this controversy would have stayed quiet until I could > have thought of an elegant mechanism for this. I have the beginnings of it, > but it's not at all completely worked out. > > Gimme some time; I'll try and cook something up. Who knows? It may > actually be worth something. =============================================== That's odd, Leo -- those have been exactly MY feelings for the past two years... And I thought *I* was going to be the one to do it... Anyhow, when Ron Minnich finally brought the subject up, I thought it was about time to get on with it. As I expected, though, each of us is focussing on a different part of the problem (which is an excellent thing, naturally). Stuart and I are more concerned (at the moment anyway) with basic message protocols, while Leo on the other hand says: > Stuart was trying to address data exchange. This is only a minor > part of the whole problem. I disagree. It's the MOST important thing to get right FIRST. I do agree that it shouldn't be very hard, and that there are lots of fascinating avenues to explore in the area of command scripts immediately afterward. I'll stress again, though, that messages are NOT necessarily command lines. See my current posting "IPC (2)..." for examples of the other sort. > ..... The format of the data should be the problem of > the program asking for it (this is how it is currently). ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Exactly. And exactly the problem at the moment. If we are to have our network of interacting small modules, each MUST be able to understand what others are saying, without having to write special cases for them all. > Data format exchange standards should not be worried about just yet. > We have to establish a method of communication and "remote control" first. We seem to disagree again... Try it the other way around. > If Sculpt and VideoScape had remote control ports, the John Foust's > Interchange program could be nothing more than an ARexx script. Dunh Hunh? How do you figure that? --------------- OK...now turning to rip Matt to shreds...(:-)) Actually, aside from the fact that Matt has again focussed on the building rather than the foundations, I don't have much criticism of his suggestions. > But we don't simply want a stream oriented IPC. People don't take enough > time to think of easy ways to provide maximum power and capability. So right. And I think that first sentence is a key point: streams work well a lot of the time, but not always. Messages will always arrive sequentially ('least until we get a truly parallel Amiga), but they should be able to give us full random access of any object -- a paragraph from a text file, a chunk out of a picture, or whatever. > -Symbolic Asynchronous Remote-Control capability (domain based) > e.g. one application can send symbolic commands to another... > A data stream, as part of a command, would be a minor item in > this system. Could you expand on this please, Matt? I assume by "Symbolic" you don't necessarily mean ASCII. And the existence and format of the (binary?) data would be determined by the command. If so, this isn't very different in concept from a standard message format, where the ID -- and possibly subsequent codes -- determines the "command". > Also, have a standard format for SARC menu item simulation. ^^^^ This acronym seems to have slipped by me. Can you translate, please? My general comment on both Matt's and Leo's proposals is that the script handler, macro handler, or whatever you want to call it, should be just another module among the set that a user can hang together under a universal IPC structure. Matt should write one according to his philosophy, and Leo should write his, but a user should be able to pick the one he likes and plug it in with the rest of his favorites. He could even use ARexx if it suited him and obeyed the common message rules. This is pretty much the way it is at the moment with command shells: some people like my Sili(Con:), others like Matt's shell, and there are lots of other possibilities that all work (mostly!) within the same environment, but have different philosophies that suit different people. And that's healthy. --------------- Finally, I'd like to disagree with Stuart a bit too. (Controversy is fun, isn't it!) > This describes the format for a standard message block. ..... > ..... > ....... The basic design is derived from the EA > IFF 85 standard, since that has worked so well and lots of Amiga users ^^^ I think that requiring an IFF type format for all messages is overkill (though I don't share Leo's disdain for the format -- it certainly should be an available option). > struct GenericMessage { > struct Message gm_mess; > ID magic_code; > long gm_size; > ID gm_code; > char gm_body [ /* value of gm_size - 4 */ ]; > } If you compare this with my suggested structure ("IFF (1): Ports & Messages") you'll see that I favor a rather simpler one. I don't really think a "magic code" is needed, because I can't envisage any situation where a "rogue" message would arrive at a port; only applications that knew about the protocol would exist in the shared environment. Also the gm_size slot is superfluous, because there's already a mn_length slot in the message structure. From that point on, our concepts of "code" and "data body" pretty much mesh -- except as I said, I don't think we need to enforce an IFF type structure on the basic message. The structure of the data should be defined only and exactly by the message code. If the code happens to be "FORM" or "LIST" or "CAT", then it IS an IFF (complete with following size word and subsidiary chunks), but a "TEXT" type for example would just be a null-terminated string of characters. I've tried to address some of the other issues that Stuart raises -- communication model and so on -- in my other postings. I'll see what scorn is heaped upon them before I go further in that line. > Also, just hashing out a good idea won't make it a standard. It will > require some software that uses it, as well as some cooperation from the > software people at Commodore. Any ideas about how to go about the "public > relations" part of the standards design? Once again... Right! So how about it CATS? > > All too often we get stuck with short-sighted or limiting "de-facto" > standards more or less by accident (the IBM PC is a good example). Let's > not let this happen this time. Thanx for the bandwidth. We're off to a good start, folks. Let's keep hashing a little while, though, and see if we can raise most of the snags BEFORE we implement. Then we might just end up with something really great! -- Pete --
peter@nuchat.UUCP (Peter da Silva) (03/07/88)
In article <5375@well.UUCP>, shf@well.UUCP (Stuart H. Ferguson) writes: > inter-process communication. With products like ARexx and Browser coming > out, we may get stuck with a "de-facto" standard which may not be well > designed, or several "standards" that are mutually incompatable. ("It's > supports ARexx, but does it support Browser?") I'm insulted. The Browser Message will be of this form: struct BrowserMessage { struct Message b_message; ULONG b_id; /* 'BRWS' */ ULONG b_len; /* sizeof(struct BrowserMessage) - sizeof(struct Message) */ ULONG b_type; /* 'ADD ' or 'DEL ' */ ULONG b_lock; /* directory lock of file */ char *b_name; /* name of file */ }; You loacte the port to send this message with FindPort("BROWSER"); Any objections? -- -- a clone of Peter (have you hugged your wolf today) da Silva `-_-' -- normally ...!hoptoad!academ!uhnix1!sugar!peter U -- Disclaimer: These aren't mere opinions... these are *values*.
pete@violet.berkeley.edu (03/07/88)
peter@nuchat.UUCP (Peter da Silva) writes: | The Browser Message will be of this form: | | struct BrowserMessage { | struct Message b_message; | ULONG b_id; /* 'BRWS' */ | ULONG b_len; /* sizeof(struct BrowserMessage) - sizeof(struct Message) */ | ULONG b_type; /* 'ADD ' or 'DEL ' */ | ULONG b_lock; /* directory lock of file */ | char *b_name; /* name of file */ | }; | | You loacte the port to send this message with FindPort("BROWSER"); | | Any objections? A little teensy one... It looks almost exactly the sort of structure I'm thinking of, except for the b_len field. Is that really needed? You have to fill in the b_message.mn_Length slot anyway (or you should, for consistency with exec conventions). The b_len field doesn't actually conflict with my suggested convention, because the 'BRWS' ID is in the right place -- it just seems superfluous, is all. How about my other point -- that FindPort can be hazardous, if Browser should somehow be shut down between a FindPort and PutMsg? Would you consider adding codes 'HOLD' and 'RLSE' to your 'ADD ' and 'DEL '? (I avoided the word 'LOCK' for probably obvious reasons...) -- Pete --
peter@nuchat.UUCP (Peter da Silva) (03/07/88)
Points:
(1) The magic_id should be just plain 'IPC ' for generic messages.
This is more along the lines of IFF. For messages specific to
a particular application this should be a unique application ID.
(2) The size feild is redundant... it's a copy of a field in the
message structure.
(3) I don't think the byte-stream model is necessary. Messages can
and do contain pointers, locks (which are pointers), strings
(which are pointers), seglists (which are more pointers), and
so on. There's no reason to ignore shared memory. For example:
struct Browser {
struct Message b_message;
ULONG b_unique;
ULONG b_id;
union {
struct {
char *b_fname;
ULONG b_dlock;
} b_file;
struct {
struct Window *b_window;
struct MsgPort *b_port;
} b_win;
struct {
char *b_title;
struct MsgPort *b_port;
} b_menu;
} b_u;
};
#define B_UNIQUE 'BRWS' /* Unique ID for browser */
#define B_ADDFILE 'ADDF' /* I just created this file */
#define B_DELFILE 'DELF' /* I just deleted this file */
#define B_ADDTOOL 'ADDT' /* Add me to the tool menu */
#define B_DELTOOL 'DELT' /* I'm exiting, remove me from the menu */
#define B_ADDWIN 'ADDW' /* Add this window to the "copy" list */
#define B_DELWIN 'DELW' /* I'm about to close this window, delete it */
The message ports might be redundant... I *could* just use the
reply ports for that purpose. I don't think that's quite in keeping
with the spirit of the thing, though.
The semantics of this should be obvious, but if they're not... let
me know. And don't forget to flame :->.
--
-- a clone of Peter (have you hugged your wolf today) da Silva `-_-'
-- normally ...!hoptoad!academ!uhnix1!sugar!peter U
-- Disclaimer: These aren't mere opinions... these are *values*.rokicki@polya.STANFORD.EDU (Tomas G. Rokicki) (03/08/88)
> How about my other point -- that FindPort can be hazardous, if Browser > should somehow be shut down between a FindPort and PutMsg? Would you > consider adding codes 'HOLD' and 'RLSE' to your 'ADD ' and 'DEL '? > (I avoided the word 'LOCK' for probably obvious reasons...) Nope, all code which sends a message to a public port which might be shut down should do a Forbid() ; if (port=FindPort("foobar.baz")) PutMsg(port,msg) ; Permit() ; if (port) /* then message was sent */ Arguments are probably confused, but you get the idea. Hold/Release is okay, except the Browser might need to abort due to some catastrophic problem when a Hold has been put on it. Do the above, and everything's hunky-dory, except you have a FindPort() within Forbid()/Permit() . . . Oh, and give them a longword length if at all possible; I can see no reason to *limit* message length to 64K. It's that kind of thinking that gave MSDOS 640K and TOS 40 folders. -tom
pete@violet.berkeley.edu (03/08/88)
In article <2116@polya.STANFORD.EDU> rokicki@polya.UUCP (Tomas G. Rokicki) writes: >> [had to delete my original comments -- mailer objected to lack of novelty!] > >Nope, all code which sends a message to a public port which might be >shut down should do a > > [shows Forbid/FindPort/Permit sequence -- see "IPC (1):" article] > >Arguments are probably confused, but you get the idea. Hold/Release >is okay, except the Browser might need to abort due to some >catastrophic problem when a Hold has been put on it. Do the above, >and everything's hunky-dory, except you have a FindPort() within >Forbid()/Permit() . . . Tom -- did you read my "IPC (1)..." posting? That F/F/P sequence is just what I'm trying to get away from. For an IPC scheme we must do better than that. (I forgot to mention this part in my note to Peter, but it is in my original posting:) Browser should also be able to send a -- universally recognized -- DISCONNECT message that will tell the caller not to use that port address any more, and part of the standard protocol is that the caller MUST recognize such a message. Remember that we're implicitly requiring that communication be two-way, and the standard procedures we write should always be able to handle this. > >Oh, and give them a longword length if at all possible; I can see >no reason to *limit* message length to 64K. It's that kind of >thinking that gave MSDOS 640K and TOS 40 folders. But we're using an exec Message structure, and this already has a UWORD for the length! We aren't applying an absolute limit here, though: just define a type of message for the job which has a pointer to the data -- who wants to copy that much data into the message structure anyway? -- Pete --
cmcmanis%pepper@Sun.COM (Chuck McManis) (03/09/88)
In article <2116@polya.STANFORD.EDU> (Tomas G. Rokicki) writes: > ... Do the above, > and everything's hunky-dory, except you have a FindPort() within > Forbid()/Permit() . . . Hmmm, the problem everyone is complaining about is the speed of doing a FindPort() which takes to long. Maybe what we need instead is a way of checking to see that a port we originally got was still valid and then encased the destruction of the port and sending to the port in F/P pairs. As an illustration : In the client : /* in the init section */ port = FindPort("Foobar.port"); /* in the loop */ Forbid(); If (IsValidPort(port)) PutMsg(port,msg); Permit(); In the server : /* in the init section */ port = CreatePort("Foobar.port"); /* in the exit code */ Forbid(); MoveMsgList(port,safearea); NewDeletePort(port); /* Marks memory pointed to by port as invalid */ Permit(); ReplyAllMsgs(safearea); exit(0); Note that the NewDeletePort() would scribble on the port in some way to make it appear invalid (nulling out the memory would be sufficient) The the client task knows that port is now invalid and won't send to it. Then the server sends replies to all the messages that were queued. And exits. If all of the clients follow this rule then A) it is fast (IsValidPort could be as simple as port.mn_Type == NT_MESSAGE) and B) it is reasonably safe from race conditions. --Chuck McManis uucp: {anywhere}!sun!cmcmanis BIX: cmcmanis ARPAnet: cmcmanis@sun.com These opinions are my own and no one elses, but you knew that didn't you.
dillon@CORY.BERKELEY.EDU (Matt Dillon) (03/09/88)
:> :>Oh, and give them a longword length if at all possible; I can see :>no reason to *limit* message length to 64K. It's that kind of :>thinking that gave MSDOS 640K and TOS 40 folders. : :But we're using an exec Message structure, and this already has a UWORD :for the length! We aren't applying an absolute limit here, though: :just define a type of message for the job which has a pointer to the :data -- who wants to copy that much data into the message structure :anyway? : -- Pete -- : You guys don't think big enough! Don't use an exec message structure, use an exec IOStdReq request structure (which is headed by an exec message structure). Remember that somebody might want to pass a HUGE buffer (say, a BitMap) which can certainly be more than 64K, and would want to pass it by reference. Half hazardly sticking in thousands of 'special' messages all in different formats is a quick way to get nowhere... because nobody will use the system. -Matt
ali@polya.STANFORD.EDU (Ali Ozer) (03/09/88)
In article <44626@sun.uucp> cmcmanis@sun.UUCP (Chuck McManis) writes: >Hmmm, the problem everyone is complaining about is the speed of doing >a FindPort() which takes to long. Maybe what we need instead is a >way of checking to see that a port we originally got was still valid ... >Note that the NewDeletePort() would scribble on the port in some way to make >it appear invalid (nulling out the memory would be sufficient) >... this rule then A) it is fast (IsValidPort could be as simple >as port.mn_Type == NT_MESSAGE) and B) it is reasonably safe from race >conditions. What if that same chunk of memory gets allocated as a port as soon as it's closed? A low probability, but possible... Or some byte gets put in that makes the chunk look like a port? You might need to check that the port type in NT_MESSAGE, and then also probably check that the name is still the same. That'd still beat searching through all the ports. You'd need IsStillValid (port, name). Ali Ozer, ali@polya.stanford.edu ps. BTW, got just got SDB, and it's neat! A bug that I spent half an hour the day before took 10 seconds to find after I started SDB. There it was: "Divide by zero error". Where I would never have found it. Wow!
rminnich@udel.EDU (Ron Minnich) (03/09/88)
In article <44626@sun.uucp> cmcmanis@sun.UUCP (Chuck McManis) writes: >In article <2116@polya.STANFORD.EDU> (Tomas G. Rokicki) writes: >> ... Do the above, >> and everything's hunky-dory, except you have a FindPort() within >> Forbid()/Permit() . . . > >Hmmm, the problem everyone is complaining about is the speed of doing >a FindPort() which takes to long. Maybe what we need instead is a >way of checking to see that a port we originally got was still valid >and then encased the destruction of the port and sending to the port >in F/P pairs. As an illustration : Or maybe the following: a "reliable ports" add-on. When you want to use a reliable port you register as a user. Result is that a link-count gets incremented. When you are done, you say so, and when link-count goes to zero, then port goes away. This on second reading sounds like what a device-driver does! I am real uncomfortable with the mechanisms proposed so far. Sounds like we really need a 'port broker' that people talk to. Or something. Are you all sure that we can't use something like PIPE: as a (higher level, not really RKM kind of ) port broker? This scribbling into a port just bugs me somehow. Its a start, anyway ... This is fun! -- ron (rminnich@udel.edu)
shf@well.UUCP (Stuart H. Ferguson) (03/09/88)
>peter@nuchat.UUCP (Peter da Silva) says: >In article <5375@well.UUCP>, shf@well.UUCP (Stuart H. Ferguson) writes: >> inter-process communication. With products like ARexx and Browser coming >> out, we may get stuck with a "de-facto" standard which may not be well >> designed, or several "standards" that are mutually incompatable. ("It's >> supports ARexx, but does it support Browser?") > >I'm insulted. >The Browser Message will be of this form: [format deleted] >You loacte the port to send this message with FindPort("BROWSER"); >Any objections? I was actually refering to another message of yours: >From: peter@nuchat.UUCP (Peter da Silva) Newsgroups: comp.sys.amiga Subject: Browser 2.0 idea. Message-ID: <724@nuchat.UUCP> |Anybody have any nifty ideas for what I can do with a public message port |in Browser? ... |Or how about "Hi, I want to be a tool. When you want me to wake up just send |the WBStartup message to this port." |Or how about "Hi, I want to be a tool. Just give me WBStartup messages to |this port when someone drops files in this window." |Anyone interested in becoming integrated? These kinds of messages are the type that are of general interest to many programs in a message-passing, tools-intensive environment such as the one we're envisioning. As such they should probably be part of a standard message protocol so that Browser and other programs can exist in peace and harmony. To some extent, the IPC discussion is a reply to your question about "nifty ideas..." I just picked Browser and Arexx as random examples of the NEED for a standard -- no insult intended. -- Stuart Ferguson (shf@well.UUCP) Action by HAVOC (shf@Solar.Stanford.EDU)
pete@violet.berkeley.edu (03/09/88)
dillon@CORY.BERKELEY.EDU (Matt Dillon) says: > You guys don't think big enough! Don't use an exec message > structure, use an exec IOStdReq request structure (which is headed by > an exec message structure). ...... What in the Guru's name would that buy you? We aren't interested in all the baggage that goes into an IO request. Just a standard basic message structure that will prevent misunderstanding between programs, and will be easy for people to incorporate into their stuff. > ..... Remember that somebody might want to > pass a HUGE buffer (say, a BitMap) which can certainly be more than > 64K, and would want to pass it by reference. I thought that that was exactly what I was suggesting.... Big buffers get pointers rather than being copied. (And of course what is pointed to will have a size slot in some recognized place, probably in the message too.) It's obvious. > Half hazardly sticking in thousands of 'special' messages all > in different formats is a quick way to get nowhere... because nobody > will use the system. Exactly. But exactly NOT what we have proposed. The concept is totally parallel to the IFF philosophy -- but a lot simpler because MOST messages will be very short. We need a mechanism which lets a program know what the message it's just received contains. A common envelope with an ID header in a fixed place makes it very easy for a program to decide whether it knows about that kind of message. Particular ID header codes will be publicly announced with the associated format by the people first developing that kind of data (as Peter da Silva has already done with his 'BRWS' ID). -- Pete --
pete@violet.berkeley.edu (03/09/88)
A couple of people have had suggestions regarding the FindPort Hazard problem: cmcmanis%pepper@Sun.COM (Chuck McManis) proposes: >.... > a FindPort() which takes to long. Maybe what we need instead is a > way of checking to see that a port we originally got was still valid > and then encased the destruction of the port and sending to the port > in F/P pairs. As an illustration : > [.....omitted] > Note that the NewDeletePort() would scribble on the port in some way to make > it appear invalid (nulling out the memory would be sufficient) The the client > task knows that port is now invalid and won't send to it. Then the server > sends replies to all the messages that were queued. And exits. If all of the and from rminnich@udel.EDU (Ron Minnich): > Or maybe the following: > a "reliable ports" add-on. When you want to use a reliable port you > register as a user. Result is that a link-count gets incremented. > When you are done, you say so, and when link-count goes to zero, > then port goes away. >..... > Its a start, anyway ... This is fun! Yeah, ain't it? I like Chuck's proposal best so far. It's big advantage is that unlike both Ron's and my schemes it doesn't need a cooperating client. (And I don't think we want a port to go away when its use count goes to zero; after all, it's initially zero until someone uses it! Anyway, the server might be a long term resident...) My concept of lock and release messages would work for "server abort" conditions if a client always checked ALL messages waiting on its reply port (one of which might be a "bye-bye" from the server) BEFORE sending more to the server port. Alternatively the server whould have to wait for replies to its abort message from all its clients before shutting down. BTW the server about to abort should do a RemPort first of all, to prevent any NEW clients accessing the port by name; the port is still valid to those processes who know its address, so all current conversations can be brought nicely to a close. So my scheme is hazard proof providing all clients obey the rules, but one which forgot to release, or didn't respond properly to an abort, could really screw things up. Chuck's suggestion -- though it does need the client to use the proper calls ( which I assume would be encapsulated in standard procedures as the sequence is always the same) -- avoids the handshake sequence, and reduces the number of messages slightly. My only worry is the remote possibility of, say, the system planting ANOTHER port in exactly the same place, so the ValidPort() call wouldn't catch anything. How about this, though? A valid public port always has a name, therefore it has a specific pointer in its name field. If ValidPort() checks on this, there is essentially no way it can make an error (unless a subsequent program planted a pointer to an identically-addressed data item in the same place!). An aborting server does a RemPort, then zeroes the name pointer to prevent further messages. We're making progress... -- Pete --
pete@violet.berkeley.edu (03/09/88)
[My apologies if this message appears twice anywhere on the net. My first sending seems to have been eaten again ... at least it never appeared here.] A couple of people have had suggestions regarding the FindPort Hazard problem: cmcmanis%pepper@Sun.COM (Chuck McManis) proposes: >.... > a FindPort() which takes to long. Maybe what we need instead is a > way of checking to see that a port we originally got was still valid > and then encased the destruction of the port and sending to the port > in F/P pairs. As an illustration : > [.....omitted] > Note that the NewDeletePort() would scribble on the port in some way to make > it appear invalid (nulling out the memory would be sufficient) The the client > task knows that port is now invalid and won't send to it. Then the server > sends replies to all the messages that were queued. And exits. If all of the and from rminnich@udel.EDU (Ron Minnich): > Or maybe the following: > a "reliable ports" add-on. When you want to use a reliable port you > register as a user. Result is that a link-count gets incremented. > When you are done, you say so, and when link-count goes to zero, > then port goes away. >..... > Its a start, anyway ... This is fun! Yeah, ain't it? I like Chuck's proposal best so far. It's big advantage is that unlike both Ron's and my schemes it doesn't need a cooperating client. (And I don't think we want a port to go away when its use count goes to zero; after all, it's initially zero until someone uses it! Anyway, the server might be a long term resident...) My concept of lock and release messages would work for "server abort" conditions if a client always checked ALL messages waiting on its reply port (one of which might be a "bye-bye" from the server) BEFORE sending more to the server port. Alternatively the server whould have to wait for replies to its abort message from all its clients before shutting down. BTW the server about to abort should do a RemPort first of all, to prevent any NEW clients accessing the port by name; the port is still valid to those processes who know its address, so all current conversations can be brought nicely to a close. So my scheme is hazard proof providing all clients obey the rules, but one which forgot to release, or didn't respond properly to an abort, could really screw things up. Chuck's suggestion -- though it does need the client to use the proper calls ( which I assume would be encapsulated in standard procedures as the sequence is always the same) -- avoids the handshake sequence, and reduces the number of messages slightly. My only worry is the remote possibility of, say, the system planting ANOTHER port in exactly the same place, so the ValidPort() call wouldn't catch anything. How about this, though? A valid public port always has a name, therefore it has a specific pointer in its name field. If ValidPort() checks on this, there is essentially no way it can make an error (unless a subsequent program planted a pointer to an identically-addressed data item in the same place!). An aborting server does a RemPort, then zeroes the name pointer to prevent further messages. Of course ValidPort would need an extra argument (the value of the name pointer -- obtained at the time of the original FindPort) so that it could actually make the check! We're making progress... -- Pete --
pete@violet.berkeley.edu (03/09/88)
[Apologies if this appears twice anywhere... my first sending seems to have been eaten -- at least locally! ] In article <8803081858.AA07450@cory.Berkeley.EDU> dillon@CORY.BERKELEY.EDU (Matt Dillon) writes: > You guys don't think big enough! Don't use an exec message > structure, use an exec IOStdReq request structure (which is headed by > an exec message structure). ...... What in the Guru's name would that buy you? We aren't interested in all the baggage that goes into an IO request. Just a standard basic message structure that will prevent misunderstanding between programs, and will be easy for people to incorporate into their stuff. > ..... Remember that somebody might want to > pass a HUGE buffer (say, a BitMap) which can certainly be more than > 64K, and would want to pass it by reference. I thought that that was exactly what I was suggesting.... Big buffers get pointers rather than being copied. (And of course what is pointed to will have a size slot in some recognized place, probably in the message too.) It's obvious. > Half hazardly sticking in thousands of 'special' messages all > in different formats is a quick way to get nowhere... because nobody > will use the system. Exactly. But exactly NOT what we have proposed. The concept is totally parallel to the IFF philosophy -- but a lot simpler because MOST messages will be very short. We need a mechanism which lets a program know what the message it's just received contains. A common envelope with an ID header in a fixed place makes it very easy for a program to decide whether it knows about that kind of message. Particular ID header codes will be publicly announced with the associated format by the people first developing that kind of data (as Peter da Silva has already done with his 'BRWS' ID). -- Pete --
braun@thumper.bellcore.com (David A. Braun) (03/09/88)
(...) Folks - What is the problem with using a port that is shut down anyway. The send message function (whatever it's called) should just return an error like "opened cupboard - no bone". The "real" problem with the amiga DOS and lots of other PC OS's is the use of addresses as pointers to dynamic resources. The obvious problem is that the resource is freed and the address re-used for something else. GURU time. A better solution is to use UNIQUE handles for resources like message ports. To clients, the value is meaningless. To the owner of the resource, the value may well be an address, but if so the owner must validate the value. I've seen several solutions to this problem (I won't take space here describing my favorite - but if you ask ...). Most have little impact on performance (1 memory reference instruction to de-reference the handle) and great impact on reliability. > >> >>Oh, and give them a longword length if at all possible; I can see >>no reason to *limit* message length to 64K. It's that kind of >>thinking that gave MSDOS 640K and TOS 40 folders. > >But we're using an exec Message structure, and this already has a UWORD >for the length! We aren't applying an absolute limit here, though: >just define a type of message for the job which has a pointer to the >data -- who wants to copy that much data into the message structure >anyway? > -- Pete -- I can think of a whole bunch of reasons for having a 32 bit length field! Especially in the area of resource control - oh that's right, AmigaDOS doesn't have resources! I keep forgetting. dave Braun braun@bellcore.com
dillon@CORY.BERKELEY.EDU (Matt Dillon) (03/10/88)
> >dillon@CORY.BERKELEY.EDU (Matt Dillon) says: > >> You guys don't think big enough! Don't use an exec message >> structure, use an exec IOStdReq request structure (which is headed by >> an exec message structure). ...... > >What in the Guru's name would that buy you? We aren't interested in all >the baggage that goes into an IO request. Just a standard basic message >structure that will prevent misunderstanding between programs, and will be >easy for people to incorporate into their stuff. It buys you generality, that is what it buys you. Think man! It makes everything so @#$!@$ easy if you use a IOStdReq (or similar) structure. Look at all the problems people have had with devices because the designers didn't stick to generic base structures (an IORequest is the base structure, but inadequate. The IOStdReq should have been made the base structure). And now you want to do what?!! An IOStdReq has all the necessary message junk, you can use other fields (like io_Offset) for other purposes, and it's big enough that most commands will be in a STANDARD EASY TO USE format. It is, in fact, a very compact and useful structure. I don't want to have to use ten thousand different custom structures when I use this system! I want to be able to (write/use) a RUN TIME LIBRARY to handle MOST FUNCTIONS SO I DON'T END UP WRITING 5000 LINES OF CODE TO SUPPORT THE G*D*MN SYSTEM! For instance, sending and receiving messages, creating message ports, basic functions, masking of unwanted or unsupported messages, searching for named ports, disconnecting, etc... should all be placed in a RUN-TIME library and NOBODY should make assumptions as to what it does.... just that you now have a port that you can receive messages on (etc...). > -- Pete -- -Matt
kim@amdahl.uts.amdahl.com (Kim DeVaughn) (03/10/88)
In article <8803081858.AA07450@cory.Berkeley.EDU>, dillon@CORY.BERKELEY.EDU (Matt Dillon) writes: > :> > :>Oh, and give them a longword length if at all possible; I can see > :>no reason to *limit* message length to 64K. It's that kind of > :>thinking that gave MSDOS 640K and TOS 40 folders. > : > > You guys don't think big enough! Don't use an exec message > structure, use an exec IOStdReq request structure (which is headed by > an exec message structure). Remember that somebody might want to > pass a HUGE buffer (say, a BitMap) which can certainly be more than > 64K, and would want to pass it by reference. Right. BTW, ARexx allows this already. /kim -- UUCP: kim@amdahl.amdahl.com or: {sun,decwrl,hplabs,pyramid,ihnp4,uunet,oliveb,cbosgd,ames}!amdahl!kim DDD: 408-746-8462 USPS: Amdahl Corp. M/S 249, 1250 E. Arques Av, Sunnyvale, CA 94086 CIS: 76535,25
dillon@CORY.BERKELEY.EDU (Matt Dillon) (03/10/88)
>How about this, though? A valid public port always has a name, therefore >it has a specific pointer in its name field. If ValidPort() checks on >this, there is essentially no way it can make an error (unless a subsequent >program planted a pointer to an identically-addressed data item in the same >place!). An aborting server does a RemPort, then zeroes the name pointer >to prevent further messages. > >We're making progress... > > -- Pete -- The Proper solution of the public-port randezvous problem is to extend the port structure (via allocating a bit more memory than the normal PORT structure size) to include a reference count field. The owner of the port (the one receiving messages) simply creates it with the reference count set to 1. When the owner goes away, it sets the mp_Task field in the port to NULL and decrements the reference count, deleting the port if rcount == 0: When creating the port, the owner would first check to see if it exists with FindPort() under Forbid(), and if so and if no other owner is attached, can simply reconnect: SPECIALPORT { MSGPORT port; long count; }; SPECIALPORT * OwnerOpen(name) char *name; { SPECIALPORT *port; Forbid(); if (port = FindPort(name)) { if (port->port.mp_Task) return(NULL); /* port already has a master */ port->port.mp_Task = FindTask(NULL); } else { port = CreateExtendedPort(name, sizeof(SPECIALPORT)); /* NOTE: CreateExtendedPort allocates a copy of the name * since there is no guarentee it will stay around */ } port->count = 1; Permit(); return(port); } SPECIALPORT * ApplOpen(name) char *name; { SPECIALPORT *port; Forbid(); if ((port = FindPort(name)) && port->port.mp_Task) { ++port->count; Permit(); return(port); } Permit(); return(NULL); } OwnerClose(port) SPECIALPORT *port; { Forbid(); while (ior = GetMsg(port)) { ior->io_Error = -1; ReplyMsg(ior); } port->port.mp_Task = NULL; if (--port->count == 0) DeleteExtendedPort(port, sizeof(SPECIALPORT)); Permit(); } ApplClose(port) SPECIALPORT *port; { Forbid(); if (--port->count == 0) DeleteExtendedPort(port, sizeof(SPECIALPORT)); Permit(); } Anybody intending to send messages to the port, does ApplOpen() ONCE, which incrementing the reference count while Forbid()n. Every time he PutMsg()'s to the port, he checks the mp_Task field, and if NULL AND the message has not been returned (in case the master shutsdown after our PutMsg() but before we check mp_Task), Forbid()s, does a second check while Forbid()n, pulls it off and returns an error: void ApplSendMsg(port,ior) SPECIALPORT *port; IOR *ior; { ior->io_Message.mn_Node.nt_Type = NT_MESSAGE; PutMsg(port, ior) if (port->port.mp_Task == NULL && ior->io_Message.mn_Node.nt_Type != NT_REPLYMSG); Forbid(); if (ior->io_Message.mn_Node.nt_Type != NT_REPLYMSG) { Remove(ior); ior->io_Error = IOERR_NOMASTER; ReplyMsg(ior); } Permit(); } } Note that this works in all timing cases (because nothing bad happens if you PutMsg to a port with a NULL mp_Task field. NOTE THAT THERE IS ESSENTIALLY NO OVERHEAD FOR NORMAL OPERATION. The check inside the Forbid() is needed in case the master (or another master) re-connects to the port. When the application is through with the port, it simply decrements the reference count while Forbid()n, and if 0, removes the port (essentially the same thing the master does when through with the port). -Matt
peter@nuchat.UUCP (Peter da Silva) (03/11/88)
In article ... rokicki@polya.STANFORD.EDU (Tomas G. Rokicki) writes: > Oh, and give them a longword length if at all possible; I can see > no reason to *limit* message length to 64K. It's that kind of > thinking that gave MSDOS 640K and TOS 40 folders. But since a message can contain pointers to other PUBLIC memory, what's the problem? -- -- a clone of Peter (have you hugged your wolf today) da Silva `-_-' -- normally ...!hoptoad!academ!uhnix1!sugar!peter U -- Disclaimer: These aren't mere opinions... these are *values*.
peter@nuchat.UUCP (Peter da Silva) (03/12/88)
In article <1427@louie.udel.EDU>, rminnich@udel.EDU (Ron Minnich) writes: > I am real uncomfortable with the mechanisms proposed so far. Sounds > like we really need a 'port broker' that people talk to. Or something. Damn this net delay :->. I hope you got my port-broker proposal before you get this message. > Are you all sure that we can't use something like PIPE: as a > (higher level, not really RKM kind of ) port > broker? This scribbling into a port just bugs me somehow. It's a bit low level, yes, but it's a lot more efficient than PIPE:, and it doesn't have to be a stream of bytes. You can pass all sorts of things in a message that would be inconvenient to pass in a PIPE. Things like pointers and locks. -- -- a clone of Peter (have you hugged your wolf today) da Silva `-_-' -- normally ...!hoptoad!academ!uhnix1!sugar!peter U -- Disclaimer: These aren't mere opinions... these are *values*.
peter@nuchat.UUCP (Peter da Silva) (03/13/88)
In article <8803091817.AA18212@cory.Berkeley.EDU>, dillon@CORY.BERKELEY.EDU (Matt Dillon) writes: > >dillon@CORY.BERKELEY.EDU (Matt Dillon) says: > >> ... use an exec IOStdReq request structure ... > >What in the Guru's name would that buy you? > It buys you generality, that is what it buys you. IOStdReq buys you Open, Close, Read, and Write. That's fine, but if you're going to do that you might as well make all the message ports device ports and put them in the AmigaDOS namespace. Something I suggested a long time ago and got shot down over. If you're not going to do that... make them look like files, that is... you might as well not force them into the linear space of files. The standard we're working on here has more in common with IFF than AmigaDOS. > An IOStdReq has all the necessary message junk, you can use other > fields (like io_Offset) for other purposes... Exactly. If you want to do more than the common file operations you're going to have to extend or modify it anyway. Since we're mostly talking about structures that go beyond them anyway, why not blow it off? > I don't want to have to use ten thousand > different custom structures when I use this system! You won't have to, any more than you do with IFF. -- -- a clone of Peter (have you hugged your wolf today) da Silva `-_-' -- normally ...!hoptoad!academ!uhnix1!sugar!peter U -- Disclaimer: These aren't mere opinions... these are *values*.
dillon@CORY.BERKELEY.EDU (Matt Dillon) (03/13/88)
:In article <8803091817.AA18212@cory.Berkeley.EDU>, dillon@CORY.BERKELEY.EDU (Matt Dillon) writes: :> >dillon@CORY.BERKELEY.EDU (Matt Dillon) says: :> >> ... use an exec IOStdReq request structure ... :> >What in the Guru's name would that buy you? :> It buys you generality, that is what it buys you. :IOStdReq buys you Open, Close, Read, and Write. It buys you whatever the @$#@! you want. Since when are we limited by Open/Close/Read/Write??? It buys you an effective extensible structure which is intuitive and known by every real Amiga programmer in existance. It inherently uses messages and message ports, and gives full implementation flexibility. >The standard we're working on here has more in common with IFF than >AmigaDOS. So far, I see nothing in the standard you are working on that makes the programmer's job any easier. Didn't anybody *read* the brainstorm I posted for a 'function support library'?????? You think people are going to include 20K of support code in the applications to make this thing work? >Exactly. If you want to do more than the common file operations you're >going to have to extend or modify it anyway. Since we're mostly talking >about structures that go beyond them anyway, why not blow it off? What an attitude. Essentially: "Since I haven't thought about what the programmer is going to have to do to implement my messaging system, I have decided not to think about it". >> I don't want to have to use ten thousand different custom structures >>when I use this system! >You won't have to, any more than you do with IFF. Not the way you're going. This is the problem with some discussions... we often get into arguments, split, and go our own way. Well, so be it. You work on your idea, and I'll work on mine, and we'll see what happens. At least IFF is structured! -Matt
Paul-Andre_Panon@van-bc.UUCP (03/15/88)
This is my first posting to the net so I hope it goes through all right.
I think that some kind of clearing house for setting up IPC is important and
that some alternative to the Forbid(); - Findport(...); - Permit(); sequence
currently in use is important.
First an example: suppose I have my super integrated-package, it does spread-
sheets, database, word processing and communications functions. All these
are really separate modules which pass messages to eachother telling eachother
what's been changed so that the spreadsheet re-calculates because it gets
told by the database that something has been changed ( a la Excel w/ DDE but
on a wider scale). Now that can be hard wired with a limited number of ports
being recognized so that each module only knows about the other modules that
came with the package. What happens if I've forgotten something in my
implementation because of time/money/market/delivery concerns? Maybe my
package only does two-D graphs for instance. It's very hard for somebody
else to come along and provide a good 3-D plotting module as an expansion
(Actually this is a guess, but *I* wouldn't relish doing 1-2-3 add-ons).
Anybody who wanted to do add-on modules would have to take apart your code to
figure out how to patch in and make itself known to the other modules.
On the other hand, if we do have a standard clearing house for different
processes to find out what other processes might be interested in a subject
(Kind of a process dating service :-) ) with some standard categories and
formats such as we have for IFF, it could make things much simpler.
The ideal would be if you could create your own integrated system (but from
a business point of view, not a programming/hypertext POV) whereby you find
the spreadsheet that suits you, your favourite word processor and drawing
programs happily intercommunicating. They wouldn't even have to be from the
same company but you could have Wordperfect, an SQL package, and Excel clones
intercommunicating/updating eachother's data and improving your efficiency.
Having a standard for doing such things would make such programs more quickly
available because you wouldn't have everybody re-inventing the wheel every
time.
So why take a look at it from the business stand-point instead of through the
integrated compiler approach? Because for business integrated software there'll
always be something that somebody wants that isn't in your package. If Pete's
programming service has an add-on that fits the ommission in your program
then you'll sell more copies and that's to your advantage. After all, the
add-on features for 1-2-3 almost certainly have something to do with its
success. Can you say AutoCAD? I knew you could. If you don't care about
improving your tools too much and are satisfied with your current environment,
you should still care about how successful your end product will be (Assuming
you plan on selling software).
Any comments? (I hope I didn't get too preachy but I tend to climb
on soapboxes too easily).
P-A
~~~
--
-- Don't use the reply addresses in my message headers. Instead please use...-+
-- |
-- INTERNET: userpap1%UBC@um.cc.umich.edu |
-- UUCP: {ihnp4,alberta,watmath,uw-beaver,uunet}!ubc-vision!ubc-mts!userpap1
-- BITNET: userpap1@UBCMTSG ------------------------------------------------+peter@nuchat.UUCP (Peter da Silva) (03/16/88)
Calm down, Matt... In article ... dillon@CORY.BERKELEY.EDU (Matt Dillon) writes: > :In article ... dillon@CORY.BERKELEY.EDU (Matt Dillon) writes: > :> >dillon@CORY.BERKELEY.EDU (Matt Dillon) says: > :> >> ... use an exec IOStdReq request structure ... > :> >What in the Guru's name would that buy you? > :> It buys you generality, that is what it buys you. > :IOStdReq buys you Open, Close, Read, and Write. > > It buys you whatever the @$#@! you want. Since when are we > limited by Open/Close/Read/Write??? I didn't say it was limited by Open/Close/Read/Write. Open/Close/Read/Write are the only functions defined as existing on all devices. If we're going to go this way, then my personal preference would be to make the message ports real AmigaDOS devices (using the code that you so very graciously provided for doing this), instead of adding yet another protocol to the overburdened Amiga. There's already DOS devices, Exec devices (some of which could even be used for much of what we're trying to do: pipes and clipboards), REXX ports, IDCMP Ports, and god knows what else. I suggested this and was shot down. > >The standard we're working on here has more in common with IFF than > >AmigaDOS. > > So far, I see nothing in the standard you are working on that > makes the programmer's job any easier. Didn't anybody *read* the > brainstorm I posted for a 'function support library'?????? You > think people are going to include 20K of support code in the applications > to make this thing work? I hardly think it's going to take 20K of support code, since you don't need to parse anything out of the message (at least you don't with my protocol), and since you can safely ignore any messages you don't recognise. > >Exactly. If you want to do more than the common file operations you're > >going to have to extend or modify it anyway. Since we're mostly talking > >about structures that go beyond them anyway, why not blow it off? > > What an attitude. Essentially: "Since I haven't thought about > what the programmer is going to have to do to implement my messaging > system, I have decided not to think about it". Wait a second. That's not a fair nor a valid interpretation of the sort of thing I'm talking about. I never said you shouldn't have a support library, nor have I argued against one. I'm just saying that we should figure out how to make the wheel work before we decide what color to paint it. > >> I don't want to have to use ten thousand different custom structures > >>when I use this system! > >You won't have to, any more than you do with IFF. > > Not the way you're going. This is the problem with some > discussions... we often get into arguments, split, and go our own way. > Well, so be it. You work on your idea, and I'll work on mine, and > we'll see what happens. Well, Pete and I are in pretty good agreement so far. We have our differences but we're pretty close and willing to make concessions. And while we're in the radical ideas department... how about using AmigaDOS for this stuff? A lot of what we're talking about can be handled already by clipboards and pipes. Perhaps we could develop what we have already? > At least IFF is structured! As is my IPC message form. -- -- a clone of Peter (have you hugged your wolf today) da Silva `-_-' -- normally ...!hoptoad!academ!uhnix1!sugar!peter U -- Disclaimer: These aren't mere opinions... these are *values*.
reese@pdnbah.UUCP (Don Reese) (03/23/88)
In article <5377@well.UUCP> ewhac@well.UUCP (Leo 'Bols Ewhac' Schwab) writes: > UNIX achieves IPC (usually) through pipes. One program's stdout is >connected to another program's stdin. On UNIX, this works great since, in >most cases, all user I/O through the program is through stdin and stdout. > What do you consider a pipe? Unix provides many methods of interprocess communications. Here is a list of the more widely used methods: Pipe: A method of passing output from one program (usually standard out but not required) to the input of another program (usually standard input but also not required). General implementation done using a tempory fixed size file. Named Pipe: Simular method of passing output from one program to input of another program but using a predetermined file name for the purpose. Queues: Interprocess communications consisting of messages that are passed between programs across message queues. Involves creation of internal queues within the OS and methods of accessing the queues. Sockets: Bidirectional data stream between two processes simular to a pipe. Within Unix sockets are normally used to communicate between processes that may not be running on the same processor (networked processors usually using a LAN). Semaphores and Shared Memory: Two processes attach to the same memory location and control updates through the setting and checking of flags internal to the OS. I am probably missing other methods, but the point here is that several methods of interprocess communications exist under Unix. Each has it's advantages or disadvantages depending upon the needs of the system being designed. If you are considering adding IPC facilities, consider adding multiple methods. ================================================================================ || Don Reese || Paradyne Corporation || || {codas,usfvax2}!pdn!pdnbah!reese || Mail Stop LF-207 || || Phone: (813) 530-8361 || P.O. Box 2826 || || || Largo, FL 34649-2826 || ================================================================================
ewhac@well.UUCP (Leo 'Bols Ewhac' Schwab) (03/26/88)
In article <2609@pdn.UUCP> reese@pdnbah.UUCP (Don Reese) writes: >In article <5377@well.UUCP> ewhac@well.UUCP (Leo 'Bols Ewhac' Schwab) writes: >> UNIX achieves IPC (usually) through pipes. One program's stdout is >>connected to another program's stdin. On UNIX, this works great since, in >>most cases, all user I/O through the program is through stdin and stdout. >> >What do you consider a pipe? Unix provides many methods of interprocess >communications. Here is a list of the more widely used methods: > Pipe: [ ... ] This is how most *users* accomplish IPC under UNIX. They say: % foo | bar | baz | bletch | grep 'slime-mold' | lpr And it works. Unfortunately, it won't work for the majority of Amiga programs, since their user interfaces are, by and large, not driven by stdin and stdout. They're driven by gadgets, menus, requesters, mouse movements, etc. The console.device does try and translate some of this stuff into escape sequences, but there's just too much stuff here. Ordinary UNIX-style pipes are extraordinarily useful, but they are nowhere near the general solution we're looking for. > Named Pipe: [ ... ] Same problems, but still useful. > Queues: [ ... ] You're getting warm.... > Sockets: [ ... ] Now you're cookin'. Now all we need is a standard of command and data transfer. Not even UNIX has this yet, largely because they've gotten along so well with piping stdin and stdout all over the place for so long. This isn't a drawback; the fact that UNIX only *really* supports ASCII terminals has been a great advantage, making IPC relatively easy for them. However, visually oriented systems will require a more general solution. I wonder how Sun did it with SunTools, if they did it at all? > Semaphores and Shared Memory: [ ... ] The current Exec message and semaphore facilities fulfill this aspect nicely, I feel. _-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_ Leo L. Schwab -- The Guy in The Cape ihnp4!ptsfa -\ \_ -_ Recumbent Bikes: dual ---> !{well,unicom}!ewhac O----^o The Only Way To Fly. hplabs / (pronounced "AE-wack") "Work FOR? I don't work FOR anybody! I'm just having fun." -- The Doctor
cmcmanis%pepper@Sun.COM (Chuck McManis) (03/27/88)
Argh! IPC, such an innocuous TLA (three letter acronym), so many people have such disparate views on just what the hell IPC is that I am suprised you can talk about it at all. Let's set some ground rules, and maybe we can bring 3 meg of mail messages together into a fairly cohesive whole. First some definitions : IPC - InterProcess Communication IPC Facility - A programmatic means whereby two processes may pass and share data. Architecture - A specification of the abilities, and policies of a given facility. Implementation - A specification of the interface too, the code implementation of, the operating system restrictions, and the efficiency of a facility. Now, if you use these definitions, and I will be doing that here, then you will note that there have been a lot of messages on the implementation of an IPC facility, and a few on the architecture of an IPC facility, and a bunch on something entirely different. Now I don't mean to pick on Leo, but his message happened to be here when I had some time to sit down and espouse some opinions. Sorry Leo. What Leo disusses in his message (much of it is included below) it a user interface control system. It is built on *top* of an IPC facility and is *not* in itself an IPC facility. For the Amiga's part it *has* and IPC facility, and they are called message ports. An IPC facility allows two processes to share data period. Message ports do that quite nicely. The difficulty with message ports are that they do *no error checking*. Now that in itself is not a problem, it just makes programming with them a little bit tougher. Note included text (delineated by '>') is Leo's stuff ... > UNIX achieves IPC (usually) through pipes. One program's stdout is > connected to another program's stdin. On UNIX, this works great since, in > most cases, all user I/O through the program is through stdin and stdout. True, UNIX processes can send data through a pipe (which are often implemented as special case sockets). Usually, as in Leo's example, such exchanges are one way. > This is how most *users* accomplish IPC under UNIX. They say: >% foo | bar | baz | bletch | grep 'slime-mold' | lpr Ahh, but users don't accomplish anything, this is how they *use* pipes. And *pipes* use the IPC facility. There is a difference. By the same token, if the person who wrote 'foo' and 'bar' above did not *specifically* program them to be run in this way, then they *break*. This is important too. You see, 'bar', 'baz', 'bletch', and 'grep' in the above example are being used as "filters" and they were *designed* to be filters, otherwise nothing useful would happen. Take the real life unix command line : pepper% ps -aux | awk | lex | yacc | cc -o strange.stuff What happens here? Well you get lots of wild error messages because what 'ps' produces is not meaningful to awk and what awk produces is nothing like what 'lex' expects to see, much less 'yacc', but the output of yacc could be valid input to 'cc' except it doesn't have any valid input at all. So what you get is junk. Garbage in, garbage out. The point of all this is that the IPC facility that pipes are implemented on top of does not constrain the pipes in any way or how you use them, it just copies data from the output of one program into the input of another. This exact same facility is available on the Amiga although the CLI is not set up to let you at it this easily. And any programs that were written to use it, do so. Just like UNIX. > Unfortunately, it won't work for the majority of Amiga programs, since > their user interfaces are, by and large, not driven by > stdin and stdout. They're driven by gadgets, menus, requesters, mouse > movements, etc. The console.device does try and translate some of this > stuff into escape sequences, but there's just too much stuff here. Ordinary > UNIX-style pipes are extraordinarily useful, but they are nowhere near the > general solution we're looking for. Now, if you look closely you will notice it doesn't work for the majority of UNIX programs either. Just those that were designed with pipes in mind. An overall design philosophy in UNIX programming is that it is better to build lots of little tools that can be stacked, than one big tool with a zillion options. The same can be true of Amiga programs if the programmers choose to do things that way. My experience has been that ex-VMS programmers that write UNIX tools for the first time leave this out of their design. That's ok, because they still work, however when they realize the amplification effect they sometimes go back and rewack their code. Amiga programs have both a Stdin() and an IDCMP message port. If you had some way of describing Intuimessages it shouldn't be to tough to design an input handler to feed those to intuition. The results of course are not pretty and look a lot like the 'Journal' program running. But the key here is that this has *nothing* to do with an Interprocess Communication facility, it is a programming style question. >> Sockets: [ ... ] > > Now you're cookin'. Now all we need is a standard of command and > data transfer. Not even UNIX has this yet, largely because they've gotten > along so well with piping stdin and stdout all over the place for so long. > This isn't a drawback; the fact that UNIX only *really* supports ASCII > terminals has been a great advantage, making IPC relatively easy for them. > However, visually oriented systems will require a more general solution. I believe you are confusing the existing 'general' solution with a more constrained solution. All those UNIX programs were designed to deal with sockets, you can design your program to work with aREXX and get the same capabilities in a visually oriented system. Basically, while it would be neat to send your existing copy of Deluxe Paint a command to transfer its current image to your icon generating program, it is not possible. Why? because every program is a unique case, and there is no way to specify all of the useful commands you may wish to give to a program. It is not a bounded problem. The *ONLY* way this would work would be for Dan Silva to *design in* a process interface. Now he will probably do it because he is a nice guy and could see the benefits of doing this. It is even more likely if there is a piece of example code that shows what is expected of his code. But there is no way to telepathically tell his program what you want it to do. > I wonder how Sun did it with SunTools, if they did it at all? Can't do what you want to do with SunView. But you can watch for data from a socket or a pipe if you choose, that's how my Pente program communicates amoung the players, it is designed in. >> Semaphores and Shared Memory: [ ... ] > > The current Exec message and semaphore facilities fulfill this >aspect nicely, I feel. Yes they do don't they. Ok, so on with the new stuff ... What Leo is arguing for is an externally mandated standard for InterPROGRAM communication. And while it would be nice, it is *extremely* difficult to 'add on' after a system has gone through a couple of releases. The best shot you have here is aREXX. Get some example code out there that has a 'black' box interface to it (maybe something you could add into your regular intuition event loop) and then buy programs that use it and shun programs that don't. If Dan Silva releases a version of Deluxe Paint that uses it, every other paint program will follow suit. They already mimic the keyboard equivalents that Dpaint uses. The same is true for word processors/editors. If one uses it and sets the stage then they all will use the same commands. But there is no way you will ever get this thing to talk reliably to existing programs. About the closest you could come would be to use the 'stuff things in the input stream' technique which would make it impossible to use the Amiga while this was going on. AREXX answeres *every* requirement of Leo's and the other proponents of this type of communication except that it isn't free. InterPROCESS communication is another matter. In my opinion, you needn't reinvent the wheel. Just implement a socket library (or streams library) and use the same semantics/architecture as UNIX. The benefits are clear. a) It is a well understood and tested architecture. b) It adds to the attractivness of the Amiga because UNIX code may be more easily ported. c) It gives the programmer the flexibility of determining the type of communication, (datagram/packet). d) The interface supports several underlying network designs, there is no reason that DNET *and* amiga-tcp as well as basic message ports couldn't be used. e) It is well understood by many people and extensible. f) It facilitates moving to local area networks in the future. (say you want to send a REXX message to DPaint on a different machine for instance). Now given that *architecture* you can implement it in anyway your choose, whether it be by a new device or handler or library. That can be decided later. Well, it is sort of a weak finish but it addresses *every* IPC issue that I have seen raised that were architecture issues. --Chuck McManis uucp: {anywhere}!sun!cmcmanis BIX: cmcmanis ARPAnet: cmcmanis@sun.com These opinions are my own and no one elses, but you knew that didn't you.