shannon%datsun@Sun.COM (Bill Shannon) (09/26/89)
As "everyone knows", MNP and uucp's 'g' protocol just don't get along. So, I decided there must be a better way. What does MNP give you? MNP 5 gives you reliable data transfer (at least from modem to modem if not from program to program) and data compression. Well, uucp's 'g' protocol already gives you reliable data transfer (from program to program) so all that missing is compression. But, we have a pretty good compression program in UNIX (compress 4.0) so why not just use that? So, what I ended up doing is writing a new uucp protocol based on the 'g' protocol but including automatic compression of the data. I call the new protocol 'c'. It will compress the data when sending (and uncompress when receiving) unless the data has already been compressed (it has some heuristics to guess this), and uses the same old 'g' protocol to send the compressed data. To do the compression it just forks the compress program (ya, I could build it into the protocol code, but why bother?). Using this new protocol you just turn off MNP while using the modem for uucp. Of course, MNP is still quite useful when using the modem for tip or slip or things like that. By the way, compress is much better at compressing data than MNP 5. Using my 'c' protocol I can get 1800 characters per second effective transfer rate using a V.32 9600 baud modem. This is for a nice text file, like /etc/termcap. For binary files like /vmunix I get something like 1450 characters per second, which is even better than the 1320 or so I've gotten with Telebits. Needless to say, there are some tradeoffs involved. This scheme takes more cpu time (and more memory) than either a Telebit or MNP modem, and obviously interoperates with nothing at this point. You might not want to run this on a machine with lots of uucp's running simultaneously. If there's interest I can post the diffs (although they're diffs against our version of the SVR3.2 uucp, so they won't be directly applicable to other versions). Bill Shannon
loverso@Xylogics.COM (John Robert LoVerso) (09/27/89)
In an article, Bill Shannon writes: > So, what I ended up doing is writing a new uucp protocol based on > the 'g' protocol but including automatic compression of the data. > I call the new protocol 'c'. ... > Needless to say, there are some tradeoffs involved. This scheme takes > more cpu time (and more memory) than either a Telebit or MNP modem, ... There is one other possible solution to your problem. The `g' protocol, or rather, the underlying Packet Driver Protocol, contain{s,ed} code for negotiating a packet size from any of the set: { 1, 32, 64, 128, 256, 512, 1024, 2048, 4096 } I.e., if you are going to invent (and distribute?) a `g' replacement, be sure to re-allow this size negotitation! My understanding is that the code defaults to a 64 byte packet because of a bug in the `g' protocol in v7 that would cause it to crash on large packet sizes. The solution that people came up was to limit `g' rather than to fix and replace broken UUCPs. Does anyone really use v7 UUCP anymore? The bug has long since been fixed in BSD UUCP, btw; its just that the code to select a packet size other than 64 is gone. I would hope that Telebit's `g'-spoofing can understand the negotiation! (ps: see the _Packet_Driver_Protocol_ paper from Greg Chesson for details.) John -- John Robert LoVerso Xylogics, Inc. 617/272-8140 loverso@Xylogics.COM Annex Terminal Server Development Group
rob@europa.inmos.co.uk (Robin Pickering) (09/27/89)
In article <125285@sun.Eng.Sun.COM> shannon%datsun@Sun.COM (Bill Shannon) writes: >As "everyone knows", MNP and uucp's 'g' protocol just don't get along. >So, I decided there must be a better way. What does MNP give you? >MNP 5 gives you reliable data transfer (at least from modem to modem >if not from program to program) and data compression. Well, uucp's >'g' protocol already gives you reliable data transfer (from program >to program) so all that missing is compression. But, we have a pretty >good compression program in UNIX (compress 4.0) so why not just use >that? Why use up all that expensive CPU to do something the modem will already do for you?. Why not use one of the non error correcting protocols like 'f' or 't' over your MNP modems which will do the error correction (and compression) for you. Rob Pickering, Inmos Limited. -- JANET: ROB@UK.CO.INMOS | Snail: 1000 Aztec West Internet: rob@inmos.com | Almondsbury Path: ukc!inmos!rob or uunet!inmos-c!rob | Bristol BS12 4SQ Phone: +44 454 611517 | UK
clewis@eci386.uucp (Chris Lewis) (09/28/89)
In article <125285@sun.Eng.Sun.COM> shannon%datsun@Sun.COM (Bill Shannon) writes: >As "everyone knows", MNP and uucp's 'g' protocol just don't get along. >So, I decided there must be a better way. What does MNP give you? >MNP 5 gives you reliable data transfer (at least from modem to modem >if not from program to program) and data compression.... >So, what I ended up doing is writing a new uucp protocol based on >the 'g' protocol but including automatic compression of the data. >I call the new protocol 'c'. From the point of view of processor performance, why didn't you simply use (or implement) "f" protocol? Then you could use MNP compression without using any host overhead, and not worry about end-to-end-to-end ACK delays? Not to mention being able to talk to other "f" implementations (eg: stock 4.3 BSD UUCP). Having checksumming ACK's is only of interest when the end-to-end link isn't error free, so if you do have error correction and reliable host-modem links (eg: working hardware flow control), why bother? (except of course for a single one at the end of each file, or perhaps every (baud-dependent) number of kilobytes/number of transmission seconds.). T'would even work if you had to multi-hop thru X.25 lines (ie: telenet supported dialup MNP modems). And, of course, you could *greatly* speed "f" up by eliminating the 7-bit bashing for binary files. I have a tendency to think that UUCP protocol should vary *only* to take into account transmission medium, not additional bells and whistles. As another has pointed out, encryption/compression should really be another layer. Having uucico *automatically* invoke [en|de]cryption and/or [de]compression *prior* to sending or *post* receiving sounds like a better idea than yet another transport protocol. A brute force mechanism would be for uucico to automatically compress anything going outwards, and detecting ".Z" suffixes and attempting to decompress on incoming. -- He's a consultant: | Chris Lewis, Elegant Communications Inc. Lend him your watch | UUCP {uunet!attcan,utzoo}!lsuc!eci386!clewis and he'll tell you the time. | Moderator of the Ferret mailing list. Don Munroe, Cosmic Commander|
les@chinet.chi.il.us (Leslie Mikesell) (09/29/89)
In article <1989Sep27.211826.17398@eci386.uucp> clewis@eci386.UUCP (Chris Lewis) writes: >From the point of view of processor performance, why didn't you simply >use (or implement) "f" protocol? Then you could use MNP compression >without using any host overhead, and not worry about end-to-end-to-end >ACK delays? I though someone had determined that at least on one machine the host overhead of compression was less than the overhead of writing the uncompressed data out the port. This would depend on how much compression you can achieve as well as the specific hardware, of course. Anyone have figures for different machines? >T'would even work if you had to multi-hop thru X.25 lines (ie: telenet >supported dialup MNP modems). And, of course, you could *greatly* speed >"f" up by eliminating the 7-bit bashing for binary files. Is the source for 'T' available? >I have a tendency to think that UUCP protocol should vary *only* to take into >account transmission medium, not additional bells and whistles. As another >has pointed out, encryption/compression should really be another layer. >Having uucico *automatically* invoke [en|de]cryption and/or [de]compression >*prior* to sending or *post* receiving sounds like a better idea than >yet another transport protocol. A brute force mechanism would be for >uucico to automatically compress anything going outwards, and detecting >".Z" suffixes and attempting to decompress on incoming. Sorry, but if I xfer compressed files, I'd prefer for them to stay that way until I want them uncompressed. A better way would be to add a "data" type field to the packet header and do whatever is appropriate. Perhaps when we get STREAMS tty's we can just push an appropriate protocol on both ends of the link and let uucico use the 'e' protocol. The advantage would be that *any* session could have compression/ error correction, not just a single program, and it could be done at the point where it would help the most CPU-wise assuming it really is cheaper to compress than to send out a port. Les Mikesell
peter@ficc.uu.net (Peter da Silva) (09/29/89)
In article <1989Sep27.211826.17398@eci386.uucp>, clewis@eci386.uucp (Chris Lewis) writes: that you don't need to use an erro-correcting protocol with MNP because... > Having checksumming ACK's is only of interest > when the end-to-end link isn't error free, so if you do have error > correction and reliable host-modem links (eg: working hardware flow > control), why bother? Perhaps because MNP doesn't provide an end-to-end error-free link, unless it's implemented in the computer. It's only point-to-point (between the modems) end-to-end you have two non-error-free links: from the computer to the modem, and then from the modem to the computer at the other end. -- Peter da Silva, *NIX support guy @ Ferranti International Controls Corporation. Biz: peter@ficc.uu.net, +1 713 274 5180. Fun: peter@sugar.hackercorp.com. `-_-' "That is not the Usenet tradition, but it's a solidly-entrenched U delusion now." -- brian@ucsd.Edu (Brian Kantor)
shannon%datsun@Sun.COM (Bill Shannon) (10/14/89)
In article <1989Sep27.211826.17398@eci386.uucp>, clewis@eci386.uucp (Chris Lewis) writes: > From the point of view of processor performance, why didn't you simply > use (or implement) "f" protocol? Then you could use MNP compression > without using any host overhead, and not worry about end-to-end-to-end > ACK delays? Not to mention being able to talk to other "f" implementations > (eg: stock 4.3 BSD UUCP). I have 'f'. the problem with 'f' is that it expands the data before handing it to the modem to compress. for simple text, 'f' is not too bad. with MNP 5 I was able to get about 1200 cps. with binaries it's much worse, getting about 700 cps. > Having checksumming ACK's is only of interest > when the end-to-end link isn't error free, so if you do have error > correction and reliable host-modem links (eg: working hardware flow > control), why bother? (except of course for a single one at the end of > each file, or perhaps every (baud-dependent) number of kilobytes/number > of transmission seconds.). the hardware flow control I have is uni-directional; the modem can tell me when to stop but not vice versa. even if it were bi-directional, I'm not convinced there aren't other reasons that the kernel might drop data. this might be rare, but is it rare enough that you want to trust your files to it? > T'would even work if you had to multi-hop thru X.25 lines (ie: telenet > supported dialup MNP modems). And, of course, you could *greatly* speed > "f" up by eliminating the 7-bit bashing for binary files. ah, but then it wouldn't be 'f', would it? > I have a tendency to think that UUCP protocol should vary *only* to take into > account transmission medium, not additional bells and whistles. As another > has pointed out, encryption/compression should really be another layer. if uucp were a real networking system, I'd agree with you. uucp is a hack, long past its prime. this additional hack may give it a little more life, but rather than really making it better we should be building a replacement. > Having uucico *automatically* invoke [en|de]cryption and/or [de]compression > *prior* to sending or *post* receiving sounds like a better idea than > yet another transport protocol. A brute force mechanism would be for > uucico to automatically compress anything going outwards, and detecting > ".Z" suffixes and attempting to decompress on incoming. this would cause it to uncompress things that started out compressed, probably not what most people have in mind.