granvold@tymix.UUCP (Tom Granvold) (09/20/85)
- A couple of weeks ago I posted an article asking how best to transmit files and programs for machines like the Amiga and ST. These are the responces that I received. Any more comments? I am going to ask in net.micro.mac what they use and how well has it worked. I feel like an infidel about to journey into the holy land :-). I will post any interesting responces here. Thanks to all who replied, Tom Granvold ucbvax!allegra!oliveb!tymix!granvold >From: <hplabs!ames!jaw> > >as for compression, use the 4.3 bsd standard "compress". >it beats any huffman coder, and runs on many unix systems. >relevant questions here would be whether amiga lattice c includes the >"standard i/o" library. the mac folks had trouble, i heard, with >their compressor (it wasn't "compress"). > >also need a binary-to-text converter for passing through the unix mailer. >"uuencode" (bsd) is sometimes used, but is not optimal. >others ("btoa" (?)) which have been posted to the net are more efficient >because they expand 6.5 "bits" into 8, rather than the 6-to-8 "sixel" method. > >all this is easy with unix pipes, of course, but someone's got to >mash it all up for whatever os the amiga runs. beat's me why they >didn't just use unix for the underlying o.s. > >From: John McNamee <hplabs!ucbvax!eneevax!jpm@BNL44.ARPA> > >>From: <seismo!ihnp4!uw-beaver!uw-june!rling@maryland> >> >>I vote for bin->hex with a checksum at the end of each record/line. >>Those who have the capability of downloading binary files could convert >>these hex files to binary before downloading and that would halve the >>transfer time. > >I don't see the purpose of the checksum. We can assume that netmail is >transfered corrected (either by UUCP or SMTP/TCP). I think we can also >assume that ST users will have either XMODEM or KERMIT to download the >files from their local Unix site. The protocol will take care of error >checking, so there is no need for it inside the file. > >I would much rather see some binary->ASCII conversion scheme that was >able to get better than a 2 to 1 expansion. I think the Mac people have >a system like that, and isn't there a Usenet semi-standard set of programs >called uuencode/uudecode that does the same thing? >-- >From: Mohorovicic discontinuity <hplabs!ucbvax!eneevax!ken@rochester.ARPA> > >I'd thought I'd add my two cents worth. Disclaimer: I don't own a ST >(yet) but I am getting very excited by the traffic on this mailing >list: > >Choosing a binary to printable ASCII conversion is easy. Besides >uu??code there are a couple of alternatives: atob and btoa, which were >posted along with compress a while back to Usenet; and one I posted a >while back called encode/decode (which gives 23% expansion but uses >division and modulus, so may be slower on some machines). If ultimate >compression is not the goal, uu??code is good enough. > >Typically, binary files contain long runs of values like zeros. It >would be nice to have a format that could compress these runs, like the >new BinHex formats for the Mac do. > >I know little about the Mac format, but could it be adapted? This has >the great advantage that existing software like macget/put could be >reused. > >I venture to suggest this: start posting now in uu??code, there are >enough hooks to add upward compatible compression later. >From: allegra!hplabs!well!wvucenic (Wayne Vucenic) > >In addition to the two concerns you identified, binary files and >data compression, I'd like to add a third concern: How to handle >collections of files (it'd sure be convenient to be able to transfer >a set of files as one "archive" file, rather than one at a time.) > >I don't know much about how this is done for the Macintosh, but looking >at the Mac newsgroup, I notice they use a program called "BinHex" to >convert binary files into printable ascii. I also noticed a discussion >about a new, "better" version of BinHex, which was incompatible with >the existing version. Thus, these ascii files are preceded with a >message like "must be unpacked with BinHex 4.0". Finally, nothing >in the printable ascii format is human readable, so one has to run >the file through BinHex to even determine the file name. > >I've a few thoughts on the format of printable ascii files (I'll refer >to them as "transfer files") that I think might avoid some of the problems >that BinHex seems to have. Perhaps the file could consist of the >following fields: > >1) A "magic number" that identifies this as a transfer file, so that >the program that "unpacks" these files can have some assurance that >its input is really a transfer file. 4 characters (like ")]$#" ) >would probably be sufficient. > >2) The version number of the pack/unpack program that created this >file (so we don't have to say "must use BinHex 4.0") > >3) An optional character string of arbitrary length. This could be >used for a SHORT description of the contents of this file. This is >not the place to put documentation. It is intended to be a "memory >jogger" when you're looking at this file and can't remember what >it is. For example "C source files for bouncing ball demo" > >*** the fields above this line do not become part of the unpacked file *** > >4) Whatever "directory info" AmigaDOS needs. This is really anything that >is not part of the actual "data" content of the file. File name, access >permissions, dates of creation/modification, etc. Since some of this >information is fairly short (like file name), and little space would be >saved by compressing it, I suggest that this field be broken into two >sub fields: > >4a) The "directory info" that would make sense to put in human readable >(i.e., uncompressed) form. File name is an obvious candidate here. If the >file name is not compressed, it will be readable without unpacking the file. >Again, this just makes it a little easier to figure out what the transfer >file contains without taking the time/trouble to unpack it. > >*** the fields above this line are in uncompressed, human readable form *** > >4b) The rest of the "directory info", in compressed form, if appropriate. > >5) The actual contents of the file, converted to ascii and probably compressed > >6) A checksum > >Anyway, these are just a few random thoughts. It would be nice to know >more about how the Macintosh folks do it, and what pitfalls they've >encountered. I agree with you that it'd be nice to get something like >this in place sooner rather than later. > Tom Granvold ucbvax!allegra!oliveb!tymix!granvold
granvold@tymix.UUCP (Tom Granvold) (09/21/85)
_ A couple of weeks ago I posted an article asking how best to transmit files and programs for machines like the Amiga and ST. These are the responces that I received. Any more comments? I am going to ask in net.micro.mac what they use and how well has it worked. I feel like an infidel about to journey into the holy land :-). I will post any interesting responces here. Thanks to all who replied, Tom Granvold ucbvax!allegra!oliveb!tymix!granvold >From: <hplabs!ames!jaw> > >as for compression, use the 4.3 bsd standard "compress". >it beats any huffman coder, and runs on many unix systems. >relevant questions here would be whether amiga lattice c includes the >"standard i/o" library. the mac folks had trouble, i heard, with >their compressor (it wasn't "compress"). > >also need a binary-to-text converter for passing through the unix mailer. >"uuencode" (bsd) is sometimes used, but is not optimal. >others ("btoa" (?)) which have been posted to the net are more efficient >because they expand 6.5 "bits" into 8, rather than the 6-to-8 "sixel" method. > >all this is easy with unix pipes, of course, but someone's got to >mash it all up for whatever os the amiga runs. beat's me why they >didn't just use unix for the underlying o.s. > >From: John McNamee <hplabs!ucbvax!eneevax!jpm@BNL44.ARPA> > >>From: <seismo!ihnp4!uw-beaver!uw-june!rling@maryland> >> >>I vote for bin->hex with a checksum at the end of each record/line. >>Those who have the capability of downloading binary files could convert >>these hex files to binary before downloading and that would halve the >>transfer time. > >I don't see the purpose of the checksum. We can assume that netmail is >transfered corrected (either by UUCP or SMTP/TCP). I think we can also >assume that ST users will have either XMODEM or KERMIT to download the >files from their local Unix site. The protocol will take care of error >checking, so there is no need for it inside the file. > >I would much rather see some binary->ASCII conversion scheme that was >able to get better than a 2 to 1 expansion. I think the Mac people have >a system like that, and isn't there a Usenet semi-standard set of programs >called uuencode/uudecode that does the same thing? >-- >From: Mohorovicic discontinuity <hplabs!ucbvax!eneevax!ken@rochester.ARPA> > >I'd thought I'd add my two cents worth. Disclaimer: I don't own a ST >(yet) but I am getting very excited by the traffic on this mailing >list: > >Choosing a binary to printable ASCII conversion is easy. Besides >uu??code there are a couple of alternatives: atob and btoa, which were >posted along with compress a while back to Usenet; and one I posted a >while back called encode/decode (which gives 23% expansion but uses >division and modulus, so may be slower on some machines). If ultimate >compression is not the goal, uu??code is good enough. > >Typically, binary files contain long runs of values like zeros. It >would be nice to have a format that could compress these runs, like the >new BinHex formats for the Mac do. > >I know little about the Mac format, but could it be adapted? This has >the great advantage that existing software like macget/put could be >reused. > >I venture to suggest this: start posting now in uu??code, there are >enough hooks to add upward compatible compression later. >From: allegra!hplabs!well!wvucenic (Wayne Vucenic) > >In addition to the two concerns you identified, binary files and >data compression, I'd like to add a third concern: How to handle >collections of files (it'd sure be convenient to be able to transfer >a set of files as one "archive" file, rather than one at a time.) > >I don't know much about how this is done for the Macintosh, but looking >at the Mac newsgroup, I notice they use a program called "BinHex" to >convert binary files into printable ascii. I also noticed a discussion >about a new, "better" version of BinHex, which was incompatible with >the existing version. Thus, these ascii files are preceded with a >message like "must be unpacked with BinHex 4.0". Finally, nothing >in the printable ascii format is human readable, so one has to run >the file through BinHex to even determine the file name. > >I've a few thoughts on the format of printable ascii files (I'll refer >to them as "transfer files") that I think might avoid some of the problems >that BinHex seems to have. Perhaps the file could consist of the >following fields: > >1) A "magic number" that identifies this as a transfer file, so that >the program that "unpacks" these files can have some assurance that >its input is really a transfer file. 4 characters (like ")]$#" ) >would probably be sufficient. > >2) The version number of the pack/unpack program that created this >file (so we don't have to say "must use BinHex 4.0") > >3) An optional character string of arbitrary length. This could be >used for a SHORT description of the contents of this file. This is >not the place to put documentation. It is intended to be a "memory >jogger" when you're looking at this file and can't remember what >it is. For example "C source files for bouncing ball demo" > >*** the fields above this line do not become part of the unpacked file *** > >4) Whatever "directory info" AmigaDOS needs. This is really anything that >is not part of the actual "data" content of the file. File name, access >permissions, dates of creation/modification, etc. Since some of this >information is fairly short (like file name), and little space would be >saved by compressing it, I suggest that this field be broken into two >sub fields: > >4a) The "directory info" that would make sense to put in human readable >(i.e., uncompressed) form. File name is an obvious candidate here. If the >file name is not compressed, it will be readable without unpacking the file. >Again, this just makes it a little easier to figure out what the transfer >file contains without taking the time/trouble to unpack it. > >*** the fields above this line are in uncompressed, human readable form *** > >4b) The rest of the "directory info", in compressed form, if appropriate. > >5) The actual contents of the file, converted to ascii and probably compressed > >6) A checksum > >Anyway, these are just a few random thoughts. It would be nice to know >more about how the Macintosh folks do it, and what pitfalls they've >encountered. I agree with you that it'd be nice to get something like >this in place sooner rather than later. > Tom Granvold ucbvax!allegra!oliveb!tymix!granvold
granvold@tymix.UUCP (Tom Granvold) (09/21/85)
_ Sorry that I ended up sending two copies of the same article. Tom Granvold ucbvax!allegra!oliveb!tymix!granvold
peter@graffiti.UUCP (Peter da Silva) (09/26/85)
I'd think that uuencode/uudecode would be fine. I've gotten IBM programs that way, and we know there's a 'C' compiler for the Amiga. I'd also like to suggest that where possible *sources* be posted, in plain text format. They're much more useful than binaries...