mjranum@osiris.UUCP (Marcus Ranum) (02/01/87)
Finally got Dr. T.'S book from the bookstore, and software
is on the way. The first thing that struck me about MINIX is that
it does represent a wonderful opportunity for those of us systems
hacks who can't afford a Sun workstation as a home computer. (I am
regrettably in that position.) I already have plans for various
tweeks, ports, and programs to write when I get the chance and have
my hard disk up and running.
To get to the point: I am an avid reader of comp.sources and
other software related netgroups. One of the things that has on a few
occasions had me frothing at the mouth is the incredible variety of
different formats in which binaries and sources are posted. What I
would like to suggest, since we are starting out afresh here, is that
we establish a MINIX source/binary posting standard. I have often been
bewildered when I get a program I really wanted, and it's in uusquoze
arck'ed format, or some such.
What I suggest is the following: initially (from what I gather
from the book) we don't have an equivalent of uuencode or cpio(1).
We do, however, have tar(1) and shar. What if we all agree that
postings of MINIX material be in of tar format. This would entail,
I realize, making net postings of uuencoded tar archives, uudecoding
them, downloading them, and then dosread(MINIX)ing them before finally
untar(1)ing them. This is a pain in the tuchus for sure, but I feel
it beats the alternative of sending everything in shar format, which
takes more bytes than is needed. The slowest part (I anticipate) is
the downloading part to the Mush-Dos floppy...
Certainly there might be better ways of doing this, and I
encourage someone to develop them. I do feel some discussion of this
is (very) relevant, since we have a potentially lively newsgroup
here, and I feel that other groups (particularly comp.sys.ibm.pc)
could use some form of 'of the cuff' standard...
--mjr;
--
{decuac}!gouldsd!mjranum || {decuac}!osiris!mjranum
"It is better to shred the bugger than to bugger the shredder."
-ancient doltic proverb.gnu@hoptoad.UUCP (02/03/87)
In article <962@osiris.UUCP>, mjranum@osiris.UUCP (Marcus Ranum) writes: > What if we all agree that > postings of MINIX material be in of tar format. This would entail, > I realize, making net postings of uuencoded tar archives, uudecoding > them, downloading them, and then dosread(MINIX)ing them before finally > untar(1)ing them. This is a pain in the tuchus for sure, but I feel > it beats the alternative of sending everything in shar format, which > takes more bytes than is needed. The slowest part (I anticipate) is > the downloading part to the Mush-Dos floppy... No! I wrote the PD tar and I assure you that tar format wastes more bytes than a well done "shar". Tar puts a 512-byte header on the front of each file, and 1024 bytes of zeros on the end of the whole archive. At worst "shar" puts a few hundred bytes around each file, and at best (cat >file <<'EOF file') about 20 bytes. Shar also has the advantage that it is text, so it doesn't need uuencoding (increases the space required). This also means that people can *read* it. I consider this the most significant advantage, actually. Just think -- a communication medium *directly readable by humans*. What won't they think of next... Also remember that many people working with Minix will not be on IBM Feces "downloading to the Mush-Dos floppy". Within a few months I expect that somebody will be running uucp and netnews on Minix systems; and within a year Minix will run on systems that have some hair on their chests and faster I/O than floppies and 1200 baud. Remember, anything that can run on a PDP-11 can probably run in the Minix 64kI+64kD address space, modulo licensing restrictions. Since netnews runs on 11's, it can be ported to Minix. Note that all the netnews software is public domain already (even though recent versions have an invalid copyright notice by Rick Adams: the software was written by many people and contributed by them to the public domain). -- John Gilmore {sun,ptsfa,lll-crg,ihnp4}!hoptoad!gnu gnu@ingres.berkeley.edu Love your country but never trust its government. -- from a hand-painted road sign in central Pennsylvania
henry@utzoo.UUCP (Henry Spencer) (02/04/87)
> ...different formats in which binaries and sources are posted. What I > would like to suggest, since we are starting out afresh here, is that > we establish a MINIX source/binary posting standard... Fine idea. I would like to suggest that the first element of the posting standard should be "thou shalt post thy sources, not thy binaries". Bear in mind that MINIX is already being ported to at least one major machine that is *not* binary compatible with the 8088. > We do, however, have tar(1) and shar. What if we all agree that > postings of MINIX material be in of tar format. ... > ...it beats the alternative of sending everything in shar format, which > takes more bytes than is needed. Come again? Don't you realize that tar takes the file, pads it out to a 512-byte boundary, and then prepends a 512-byte header block? Shar takes significantly *fewer* bytes than tar, and it avoids all the encode/decode nonsense needed to transmit non-text files over a text-oriented news system. Something that did data compression might be nice, but after uuencoding the result to make it text again, the benefits are modest and the hassles of uncrating the stuff are considerable. -- Legalize Henry Spencer @ U of Toronto Zoology freedom! {allegra,ihnp4,decvax,pyramid}!utzoo!henry
henry@utzoo.UUCP (Henry Spencer) (02/04/87)
Incidentally, if one *must* do compression on things to be sent over the
network, please do *NOT* reinvent the wheel one more time with yet another
different compression program. "Compress", published in mod.sources a while
ago, is the thing to use. It's available, it should be possible to run it
under MINIX (it runs on a pdp11 with the same sort of address-space limits),
it does a much better job of compression than Joe Hacker's random kludge,
and best of all it's already a de facto standard.
--
Legalize Henry Spencer @ U of Toronto Zoology
freedom! {allegra,ihnp4,decvax,pyramid}!utzoo!henryapratt@atari.UUCP (02/05/87)
in article <1717@hoptoad.uucp>, gnu@hoptoad.uucp (John Gilmore) says: > > In article <962@osiris.UUCP>, mjranum@osiris.UUCP (Marcus Ranum) writes: >> What if we all agree that >> postings of MINIX material be in of tar format. ... > > Shar ... has the advantage that it is text, so it doesn't need uuencoding > (increases the space required). This also means that people can *read* it. The problem I see with Shar is the absurdly-simple Trojan Horse loophole it opens up. When you turn your machine over to somebody else's command script, AT YOUR OWN COMMAND PROMPT, you're just asking for trouble. A special "unshar" script, which only lets the shar run certain commands and put files in certain directories, would be an appropriate patch to this. Makefiles also introduce this security problem, I realize, but we can only go so far. /----------------------------------------------\ | Opinions expressed above do not necessarily | -- Allan Pratt, Atari Corp. | reflect those of Atari Corp. or anyone else. | ...lll-lcc!atari!apratt \----------------------------------------------/
mjranum@gouldsd.UUCP (02/05/87)
my previous comments re tar(1): > > ...it beats the alternative of sending everything in shar format, which > > takes more bytes than is needed. [Well-deserved flames deleted] Okay, so I blew that one ! Let me take back what I said, etc. At any rate, let's not avoid the point I was making and worry about what is bigger. My point was that some discussion of standards is necessary if we are going to post sources/binaries, etc. More to the point, suppose I make a version of uuclone and am willing to post it for beta-test, but do not want to give out sources ? If it it a potential moneymaker, or the source files are *huge* and many, giving out binaries is a better bet anyhow, no ? I can think of several cases where binaries with config files would take less space on the net. Swapping huge sources is best done with floppy-disk mailers anyway. My stupid comments about tar(1) aside, I stand by my original proposal that we form an off-the-cuff standard: shar wherever possible, tar for binaries, and no "arc" "pack" "squeeze" "much" or "gulp" formats. --mjr; -- "It is better to shred the bugger than to bugger the shredder." -ancient doltic proverb.
mjranum@gouldsd.UUCP (02/05/87)
In article <7620@utzoo.UUCP>, henry@utzoo.UUCP (Henry Spencer) writes: > Incidentally, if one *must* do compression on things to be sent over the > network, please do *NOT* reinvent the wheel one more time with yet another > different compression program. "Compress", published in mod.sources a while > ago, is the thing to use. I agree. Compress is excellent. I submit that if something is so big it has to be compressed, it's better to send it in a floppy mailer unless there is a huge demand. If I recall my sequence right: 1) I take binaries and sources and shar/tar them 2) I compress the output 3) I uuencode it. 4) I post it 5) news takes it, spools it, *COMPRESSES* it, and sends it. 6) news receives it, decompresses it, etc... 7) I receive it 8) I uudecode it 9) I decompress it 10) I unshar it. 11) Now I can have fun. There has *GOT* to be a better way ! Even at the expense of space, maybe we should give some thought to minimizing the # of steps ? It was this possibly erroneous thinking that led me to suggest that we limit ourselves to tar(1), shar, or floppy mailer format. ++mjr; -- "It is better to shred the bugger than to bugger the shredder." -ancient doltic proverb.
bandy@amdcad.UUCP (02/06/87)
In article <552@atari.UUCP> apratt@atari.UUCP (Allan Pratt) writes: >[ blindly unsharing things is dumb ] >[ there should be an "unshar" script to protect from bozoes ] How quickly the young forget their history! There have been a number of unshar programs posted, one of which was written by Michael Mauldin (mlm, aka "Fuzzy" in his Gloria file) of CMU. The date in the man page is 29 Jan 1985. I'm not sure about MLM's, but at least one of the unshar programs was written so that you could unshar (and shar) on your Amiga (or ST or...). >Makefiles also introduce this security problem, I realize, but we can >only go so far. What do you think "make -n" is for? -- Andrew Scott Beals, {lll-crg,decwrl,allegra}!amdcad!bandy +1 408 749 3683
rpw3@amdcad.UUCP (02/06/87)
In article <7620@utzoo.UUCP> henry@utzoo.UUCP (Henry Spencer) writes: +--------------- | Incidentally, if one *must* do compression on things to be sent over the | network, please do *NOT* reinvent the wheel one more time with yet another | different compression program. "Compress", published in mod.sources... +--------------- Some actual results: I had to send the Minix disks over UUCP (*gasp* *gag*) to get them to the target system (a Fortune 32:16 with no AT-compatible floppies), and used "compress -b15" to save line costs (-b15 gives ALMOST as good a compression as the default -b16, at half the memory cost). The results were encouraging (and useful to my phone bill). Each of the original Minix distribution -AT floppies were 1.2Mbytes. The compressed versions were: -rw-rw-rw- 1 uucp 73082 Jan 31 05:41 minix.1.Z -rw-rw-rw- 1 uucp 28803 Jan 31 05:46 minix.2.Z -rw-rw-rw- 1 uucp 502798 Jan 31 07:04 minix.3.Z -rw-rw-rw- 1 uucp 566196 Jan 31 17:50 minix.4.Z -rw-rw-rw- 1 uucp 616796 Jan 31 16:20 minix.5.Z (1=boot, 2=/, 3=/usr, 4&5=other stuff) Rob Warnock Systems Architecture Consultant UUCP: {amdcad,fortune,sun}!redwood!rpw3 DDD: (415)572-2607 USPS: 627 26th Ave, San Mateo, CA 94403
rick@seismo.CSS.GOV (Rick Adams) (02/06/87)
> Since netnews runs on 11's, it can be ported to Minix. Note that all the > netnews software is public domain already (even though recent versions > have an invalid copyright notice by Rick Adams: the software was written > by many people and contributed by them to the public domain). The copyright notice IS valid. One of the things you can do with public domain software is stick your copyright notice on it. The point of my adding the copyright notice is to keep someone from doing the same thing and trying to sell it as theirs. Public domain means that anyone can do anything with it. That includes selling it, claiming they wrote it, etc. ---rick
beattie@netxcom.UUCP (02/06/87)
I don't know about others but I almost never run binaries that have come over the net. There are a couple of problems with binary only postings. First it is possible to pick up a Trojan Horse that way. Secondly environments vary I believe this will be doubly so with MINIX since everybody is going to be tweeking (breaking/desacrating/improving...) their own system in strange and wonderful ways that can not be forseen. I for one would discourage binary only postings. -- ----------------------------------------------------------------------- Brian Beattie | Phone: (703)749-2365 NetExpress Communications, Inc. | uucp: seismo!sundc!netxcom!beattie 1953 Gallows Road, Suite 300 | Vienna,VA 22180 |
thomps@gitpyr.UUCP (02/07/87)
> The problem I see with Shar is the absurdly-simple Trojan Horse > loophole it opens up. When you turn your machine over to somebody > else's command script, AT YOUR OWN COMMAND PROMPT, you're just asking > for trouble. A special "unshar" script, which only lets the shar run > certain commands and put files in certain directories, would be an > appropriate patch to this. I believe that there are unshar programs floating around. Perhaps we can find one and standardize on it. Using tar would be a serious mistake. The bandwidth of the net is not infinite and backbone sites must pay the bills for shuffling all the extra junk around. A move to use tar would likely be nixed by news administrators at backbone sites refusing to carry the large traffic I expect to be produced by this group. -- Ken Thompson Phone : (404) 894-7089 Georgia Tech Research Institute Georgia Insitute of Technology, Atlanta Georgia, 30332 ...!{akgua,allegra,amd,hplabs,ihnp4,seismo,ut-ngp}!gatech!gitpyr!thomps
rdh@sun.UUCP (02/08/87)
In article <962@osiris.UUCP> mjranum@osiris.UUCP (Marcus Ranum) writes: > To get to the point: I am an avid reader of comp.sources and >other software related netgroups. One of the things that has on a few >occasions had me frothing at the mouth is the incredible variety of >different formats in which binaries and sources are posted. What I >would like to suggest, since we are starting out afresh here, is that >we establish a MINIX source/binary posting standard. I agree with this, and also with John Gilmore about using shar as the standard format. There are many advantages to shar, most notably that anyone can simply READ it. Let's keep it as straightforward as possible. -bob.
rdh@sun.UUCP (02/08/87)
In article <552@atari.UUCP> apratt@atari.UUCP (Allan Pratt) writes: >The problem I see with Shar is the absurdly-simple Trojan Horse >loophole it opens up. When you turn your machine over to somebody >else's command script, AT YOUR OWN COMMAND PROMPT, you're just asking >for trouble. A special "unshar" script, which only lets the shar run >certain commands and put files in certain directories, would be an >appropriate patch to this. True, but since it's a text file and you can READ it, you can run grep on the shar to list all the command lines before running it. At (presumably) one command-line per source file in even the biggest shar, it wouldn't take long to find, edit out, and post any offending command lines back onto the net. -bob.
hve@cernvax.UUCP (02/09/87)
>In his article Ken writes: >.... >I believe that there are unshar programs floating around. Perhaps we >can find one and standardize on it. >.... Minix contains a "shar" program, which I used to "shar" all the source files before writing them with "doswrite", directory by directory, to floppy. I then used PC-DOS and a network connection to transfer them to our VAX UN*X system running BSD 4.2, and tried to "unshar" them with "sh". At this point I found out, that I had to port one program from Minix first, namely "gres", and install it on our UN*X system. Having done this, I had no problems to unshar Minix source. So, why look for a shar/unshar pair, if Minix provides it?
svirsky@ttidca.UUCP (02/10/87)
In article <12897@sun.uucp> rdh@sun.UUCP (Robert Hartman) writes: + In article <962@osiris.UUCP> mjranum@osiris.UUCP (Marcus Ranum) writes: ++ .... What I ++ would like to suggest, since we are starting out afresh here, is that ++ we establish a MINIX source/binary posting standard. + + I agree with this, and also with John Gilmore about using shar as the + standard format. There are many advantages to shar, most notably that + anyone can simply READ it. Let's keep it as straightforward as possible. I also vote for shar, but is there any way to optimize the shar'ed source, for instance, utilities to compress the whitespace in the source before shar'ing and then uncompressing after unshar'ing. When I say compress, by the way, I mean that the compressed source would still be readable as-is by the compiler. It just may not be readable to the programmer's eyes. Would the savings be worth the trouble? -- Bill Svirsky, Citicorp/TTI, 3100 Ocean Park Blvd., Santa Monica, CA 90405 Work phone: 213-450-9111 x2597 svirsky@ttidca | ...!{philabs,randvax,trwrb,vortex}!ttidca!svirsky
davidsen@steinmetz.UUCP (02/11/87)
If you're going to use compress, you might as well use atob and btoa
also, they produce less text than uuencode. I admit that a command
such as:
ls|cpio -oc|compress|btoa|mail joe -s 'text of source'
is not totally intuitive.
--
bill davidsen sixhub \
ihnp4!seismo!rochester!steinmetz -> crdos1!davidsen
chinet /
ARPA: davidsen%crdos1.uucp@ge-crd.ARPA (or davidsen@ge-crd.ARPA)rdh@sun.UUCP (02/15/87)
In article <402@ttidca.UUCP> svirsky@ttidcc.UUCP (William Svirsky) writes: > ... When >I say compress, by the way, I mean that the compressed source would >still be readable as-is by the compiler. It just may not be readable >to the programmer's eyes. Would the savings be worth the trouble? I don't think the savings would be worth the trouble. Much of the value of posting sources is the ability of the programmer to read them. Even if I decide not to use a posted program, I can still learn from it. There's no sense having to go through 5 or 6 format conversions, only to decide that I think the code's spagetti. Let's post sources that we can all read, if only for the educational value. -bob.