vkr@osupyr.mps.ohio-state.edu (Vidhyanath K. Rao) (07/04/90)
device in 2.0? Specifically, my question is: Can PIPE be made to return read packets as soon as a newline is encountered [as CON: will do] instead of waiting for the buffer to fill up. [I would like to put my front end on Maple, but it has an input buffer of several hundred bytes; of course, I don't have the source code ;-^]. -- It is the man, not the method, that Nath solves the problem. vkr@osupyr.mps.ohio-state.edu -Poincare. (614)-366-9341
pete@violet.berkeley.edu (Pete Goodeve) (07/06/90)
In posting <1453@osupyr.mps.ohio-state.edu>, Vidhyanath K. Rao (vkr@osupyr.mps.ohio-state.edu) asks: > device in 2.0? Specifically, my question is: Can PIPE be made to [ ^^ Hey.. is the line eater still alive...?! (:-))] > return read packets as soon as a newline is encountered [as CON: will do] > instead of waiting for the buffer to fill up. [I would like to put my > front end on Maple, but it has an input buffer of several hundred bytes; > of course, I don't have the source code ;-^]. Yeah, that IS an annoying feature of PIPE:, isn't it? (Especially as the 1.3 Enhancer manual indicates otherwise...) There's no option on the PIPE: device (even in 2.0) to change its buffering (though I don't see any reason why there COULDN'T be), but as it happens I've just [about?] finished a DOS device that may do exactly what you want. In practice, it behaves -- and is used -- very like a named pipe, but there are a couple of significant differences. For one thing, it does exactly what you need: data is buffered by PACKET, rather than in a fixed buffer, so that the reader is woken up the moment any data arrives. Second, there can be several processes opened for writing to one channel (though only one reader) -- I've thought maybe I should call it a "Funnel". The channel stays open as long as ANY process is connected to it (reader or writer), but by default an EOF (null packet) is sent when the last current writer closes. This can be inhibited if you wish (or ignored -- if you are coding the reader program...). Either end of the channel can be opened first, but any writing done when there is no reader is basically thrown away. (Actually it returns a write error, but it's amazing how many programs ignore this!) This could cause problems if you're expecting pipe-like buffering, so it doesn't entirely replace the PIPE:. So what is this device? Well, [naturally, if you know me!(;-))] it uses IPCPorts (ppipc.library) as the link between writer and reader. Basically, for each packet sent to the writer, it posts an IPCMessage referencing the packet data to its designated port. Data is NOT copied at this point. If a read packet has been posted at the other end, the data (or as much as there is room for) is copied to the read packet's buffer, and this is imediately returned. The write packet is released by the reply of the IPCMessage, which occurs the moment all data has been transferred (which could take several reads). You use it almost exactly as you would a pipe (after you've installed ppipc.library and IPC-Handler, and Mounted the "IP:" device). Like: run type IP:mychan list >IP:mychan Note the bass-ackwards ordering, because I've assumed these commands were given to the same shell for example's sake (and read should start before write). If I DO want to type the data in practice, I usually have a separate shell to do it anyway. [I should say, BTW, for those who know PPIPC, that the handler does use LoadIPCPort, so a Broker can be triggered by opening the write side, and then there would be no danger of losing early data] [Another aside: I was GOING to use 'Search' in my example above -- a practical example, y'know -- but I just discovered it doesn't accept input from IP: OR PIPE:! (Neither device recognizes filesystem type packets, which apparently 'Search' uses somehow. Ah well.] + + + Status? Well, it all seems to work well, but there are a few little things to be taken care of yet. Like docs, for instance. Shouldn't be too long, I hope. -- Pete --
peter@sugar.hackercorp.com (Peter da Silva) (07/06/90)
Pity. What I really want to see is a "tank": you write "things" into it, each read returns a "thing" as written. If there are no writers, it returns EOF only after all the "things" are read. Sort of like a terminal in canonical mode, in UNIX. -- Peter da Silva. `-_-' <peter@sugar.hackercorp.com>.
pete@violet.berkeley.edu (Pete Goodeve) (07/07/90)
Responding to my posting on the "IP:" device "funnel", Peter da Silva <peter@sugar.hackercorp.com> remarks: (message <6003@sugar.hackercorp.com>) > Pity. > What I really want to see is a "tank": you write "things" into it, each read > returns a "thing" as written. If there are no writers, it returns EOF only > after all the "things" are read. Sort of like a terminal in canonical mode, > in UNIX. Yeah -- me too. Another mechanism I want badly is a "tee". But, for both of them, why not? Remember our philosophy of PPIPC... way back when...(:-)) It's purpose is to allow plugging together fairly small modules that combine to do just what you want. It would be possible to do either with some variant on the IP: handler, but as communication is via IPCPorts I'd much rather drop another little module in the chain, as: tank inp outp run list >IP:inp type IP:outp There's no reason one utility couldn't do both jobs, actually. ["Tank-Tee"? -- Nahh, sounds too much like summer clothing...] tank inp out1 out2 out3... The module would simply make copies of each incoming IPCMessage and its data, and post them to each of its output channels, returning the original immediately. You'd have to do something about the problem of absent reader processes, so data wouldn't get lost. Probably the simplest is for 'tank' to set the LOADING flag in each of its target ports (if they aren't served already), so that the posted messages get queued until the server actually arrives. On the other hand you also have to do something to prevent the tank -- i.e. memory -- overflowing! If no server ever showed up, messages could keep getting tacked on to the port. I suggest that when you invoke the program you could set a limit (with some default) on the maximum number of outstanding messages. This would be quite easy for the tank because it expects each message it posts to be replied eventually. It would block its input port (which would also block the sending process if it was via IP:), or could even return an error. You would also have to arrange for it to "scavenge" all messages hanging on "dead" ports when it was finally closed. - - - - There were a couple of things I forgot in the first posting. One has fairly well just been covered: these are IPCPorts, so ANY equipped program -- not just another IP: device -- can communicate with an IP: channel. The format of the IPCMessages has been kept as generic as possible, and of course will be public. (And for those who don't know PPIPC, messages are "self-identifying", so it'd be hard to confuse, except maliciously.) Second, althoguh I stressed that you can have multiple writers, there is also a "exclusive access" option (which probably needs improvement). Finally, the device is unusual in that you can specify both in and out channel names at once. This makes it possible to start up a windowless shell to handle command requests from another process: newshell IP:inchan/outchan I think it may be useful. -- Pete --