[comp.sys.apollo] Parallel pipes

martin@medix.UUCP (Brian K. Martin M.D.) (06/25/87)

> In article <8706151717.AA02467@yale-eli.arpa> ram-ashwin@YALE.ARPA (Ashwin Ram) writes:
> > >  Incidentally, I think someone once complained that the Aegis shell didn't even
> > >  have parallel pipes.  I almost changed that at SR8, only to discover that a
> > >  number of people were doing things like: "catf file | srf > file"
> > >  (translation: "cat file | sort > file") and making any changes would break
> > >  them quite badly.  Sigh.
> > 
> > My feeling is that disallowing "catf file | srf > file" is no worse than
> > disallowing "srf file > file" (which many naive users also tend to do), and
> > the advantages of parallel pipes outweigh this minor disadvantage anyway
> 
> Ah, but you don't have to answer the UCR from someone who didn't read the 
> release notes and just trashed a weeks worth of work. :-)  Seriously, as I've
> mentioned before, we tend to be very conservative when it comes to breaking
> things.  In this case I didn't feel that the advantages were worth the risk.

That's one of the many reasons why I don't use the Aegis shell.  The
"catf file | srf >file" problem doesn't exist under UNIX.  For example,
the UNIX sort command "sort -u -o foo foo" replaces the file foo with
a sorted version with all duplicated lines removed.  Perhaps part of the
problem is that Apollo's software engineers didn't give adequate thought
to how a sort (e.g., srf) command should work.  Since a sort command shouldn't
generate any output until the entire input has been processed, then it
seems reasonable to allow the user to replace the input file with the
output of the sort command.

When I first started working with Apollos, I decided to try to port
some Bourne shell scripts that I use for preliminary NLP in the
analysis of medical citations and databases for KB construction to a
strictly Aegis environment.  I gave up after just a few hours.  My big
motivation was a lack of disk space--a 72MB DN3000 with Aegis, bsd4.2
and system V and various compilers leaves very little real work space.
(I bought the DN3000 before the cartridge tape drives were available
and before Apollo announced bigger disk drives, else I would have
gotten the 348MB with a cartridge tape)

I found that I was unnecessarily creating temp files, and that I was
unable to locate the proper Aegis commands with which to replace the
UNIX commands.  For example, I extensively use the sort, uniq, join,
comm, awk, sed, lex and egrep commands. But I certainly haven't found
all these capabilities under Aegis.  So I gave up, loaded UNIX, and
rewrote the shell scripts in order use as little disk space as
possible.  I also started brow beating myself for not buying a Sun,
which would have given me twice the disk space, twice the main memory
(8MB vs 4MB), a cartridge tape instead of a floppy, bundled software
and all the UNIX tools I needed for the same price as my hamstrung
Apollo.  But I digress...

Regards,

    Brian K. Martin, M.D.
    Martin Information Systems, Ltd.
    3420-A Hinahina Street
    Honolulu, Hawai'i 96816

PHONE: (808) 735-5661
ARPA: uhccux!medix!martin@nosc.mil
UUCP: { ames!ihnp4,ucbvax,dcdwest,seismo }!sdcsvax!nosc!uhccux!medix!martin

nazgul@apollo.uucp (Kee Hinckley) (06/25/87)

In article <8706241948.AA11755@uhmanoa.ICS.HAWAII.EDU> martin@medix.UUCP (Brian K. Martin M.D.) writes:
> That's one of the many reasons why I don't use the Aegis shell.  The
> "catf file | srf >file" problem doesn't exist under UNIX.  For example,
> the UNIX sort command "sort -u -o foo foo" replaces the file foo with
> a sorted version with all duplicated lines removed.  Perhaps part of the
> problem is that Apollo's software engineers didn't give adequate thought
> to how a sort (e.g., srf) command should work.  Since a sort command shouldn't
> generate any output until the entire input has been processed, then it
> seems reasonable to allow the user to replace the input file with the
> output of the sort command.

The point is not the individual command in question, but rather the concept
of parallel pipes.  I could have just as well given the example of 
"catf file | fpat foo > file".  The "problem" isn't in the commands involved,
but in that users have taken advantage of the fact that pipes are not parallel.
In fact, consider the line:
    cmd | (read arg1 arg2; readln rest)
That kind of thing is used all the time in the Aegis shell, but wouldn't work
at all if every pipe resulted in a fork.

> When I first started working with Apollos, I decided to try to port
> some Bourne shell scripts that I use for preliminary NLP in the
> analysis of medical citations and databases for KB construction to a
> strictly Aegis environment.  I gave up after just a few hours.  My big
> motivation was a lack of disk space--a 72MB DN3000 with Aegis, bsd4.2
> and system V and various compilers leaves very little real work space.
> (I bought the DN3000 before the cartridge tape drives were available
> and before Apollo announced bigger disk drives, else I would have
> gotten the 348MB with a cartridge tape)
> 
> I found that I was unnecessarily creating temp files, and that I was
> unable to locate the proper Aegis commands with which to replace the
> UNIX commands.  For example, I extensively use the sort, uniq, join,
> comm, awk, sed, lex and egrep commands. But I certainly haven't found
> all these capabilities under Aegis.  So I gave up, loaded UNIX, and
> rewrote the shell scripts in order use as little disk space as

I'm totally confused.  A couple questions:

When you say that you decided to "port" the Bourne shell scripts do you
mean you tried to rewrite them into /com/sh scripts?  If that isn't the case
then what were the problems porting them?  If that is the case then how
did using /com/sh result in a lack of disk space problem?  (Whether you
use parallel pipes or temp files, the space issue isn't going to be much
different.)

In the next paragraph you say you gave up and loaded Unix, so I gather
that you didn't have it loaded before, in which case I don't understand
why you felt you didn't have enough disk space *before* you loaded Unix,
since in the first paragraph you state that it was the size of (both)
Unix environments that was causing the problem.

As I said, I'm confused.  Given the configuration I guess I would recommend
not having everything from both System V and BSD4.2 on the disk, particularly
if you have a network and can link elsewhere (most of us at Apollo run with
all of our Unix directories pointing to some common location).  If you only
have the one machine and need both System V and BSD4.2 then I agree that you 
are in a bind and about the only solution is to buy a larger disk.  On the
other hand, if you need both System V and BSD4.2 then a Sun probably isn't
going to do the trick either.

                                                Kee Hinckley
-- 
UUCP: {mit-erl,yale,uw-beaver}!apollo!nazgul  ARPA: apollo!nazgul@eddie.mit.edu
I'm not sure which upsets me more; that people are so unwilling to accept
responsibility for their own actions, or that they are so eager to regulate 
everyone else's.

nazgul@apollo.uucp (Kee Hinckley) (06/27/87)

In article <35af6e88.b0a1@apollo.uucp> nazgul@apollo.UUCP (Kee Hinckley) writes:
> then what were the problems porting them?  If that is the case then how
> did using /com/sh result in a lack of disk space problem?  (Whether you
> use parallel pipes or temp files, the space issue isn't going to be much
> different.)

Open mouth, insert foot.  Sorry about that, I was thinking of memory vs.
files (which, given memory mapped io isn't that different) and totally missing
a rather basic feature of parallelism; namely that you don't have to dump
out everything before the next guy starts reading it.

                                                    -kee

P.S.  Has anyone seen my brain lately?
-- 
UUCP: {mit-erl,yale,uw-beaver}!apollo!nazgul  ARPA: apollo!nazgul@eddie.mit.edu
I'm not sure which upsets me more; that people are so unwilling to accept
responsibility for their own actions, or that they are so eager to regulate 
everyone else's.