tcjones@watdragon.waterloo.edu (Crocodile Dundee) (11/09/87)
And now, a truly miscellaneous thing, a shell script to do multi-input
multi-output pipelining.
"An invaluable tool that you will probably never use."
Terry.
#!/bin/sh
# This is a shell archive. Remove anything before this line,
# then unpack it by saving it in a file and typing "sh file".
#
# Wrapped by watdragon!tcjones on Sat Nov 7 04:51:20 EST 1987
# Contents: pipe_doc pipe
echo x - pipe_doc
sed 's/^@//' > "pipe_doc" <<'@//E*O*F pipe_doc//'
Here is a thing for those times when the shell input/output facilities
just wont let you do what you want to in any simple manner. Basically
it's a multi-input multi-output pipe command.
Call it "pipe".
The idea is that the input is sequentially assembled from possibly several
sources and written (at possibly different times during the assembly) to
possibly several destinations.
The input sources can be the terminal or the output of any other command -
even the output of another "pipe".
The output can be to the terminal, or to any normal command - including a
normal pipeline, or to the input of another "pipe".
The syntax is easy. Command line arguments are one of three things,
1) the word "-in"
2) the word "-out"
3) a command
-in switches you to "pipe input mode", -out switches you to "pipe output mode".
A pipe always begins in input mode. An example might make this clear...
% pipe -in "echo hello" -out "|wc"
1 1 6
Which is to say, "Go into input mode, take input from the
command 'echo hello' (which of course produces the single word 'hello'),
switch to output mode, and send the output so far into the command '|wc'"
Notice that the commands above ("echo hello" and "|wc") must be passed to
pipe as a single argument. Thus it is necessary to enclose multi-word commands
in quotes. Also it is necessary to escape characters such as "|" and ">" from
the shell. "" quotes do both of these jobs nicely, while still allowing for the
expansion of environment variable and commands in backquotes.
"-in" and "-out" can be abbreviated to "-i" and "-o" respectively and so
this could have been written as
pipe -i "echo hello" -o "|wc"
or, (since we start off in input mode) as
pipe "echo hello" -o "|wc"
Now this could have been done easily with "echo hello | wc" using a normal
one-in, one-out pipeline. More sophisticated examples will be coming up.
To take input from standard input, or write to standard output, use a "-"
So,
% pipe - -o -
hello there
^D # have to close standard input with EOF
hello there
%
does the same thing as "cat" (alone on a line) would have done,
reading from the terminal and writing back to the terminal.
If you immediately go into output mode, pipe will assume you want to provide
initial input from standard input, so the above could be written as
"pipe -o -". If you don't give an output directive, pipe will assume you want
standard output, so the last example could have been written simply as "pipe".
That's about all there is to know, here are some examples
% pipe - "grep tcjones /etc/passwd" - -o "|mail tcjones" -
Here is my entry in /etc/passwd
^D
Hope it's what you want.
Terry.
^D
%
Here is my entry in /etc/passwd
tcjones:Wwo5wq.gu/wd.:134:113:Crocodile Dundee,MC6129x6687,8843852,88463
38,103 minota hagey residence:/u2/tcjones:/usr/public/itcsh
Hope it's what you want.
Terry.
%
Which takes a line from my terminal, concatenates the output of grep,
concatenates another line from my terminal, sends it all off in mail to me, and
puts the output on the screen as well. Decidably more useful than the earlier
examples.
Here's another,
% pipe - "cat last_lines" -o "|sort >sorted" ">unsorted" "|wc -l"
here is a sample line
here is another
and a third
^D
32
%
which takes lines from my terminal, concatenates the contents of the file
last_lines, sends all that into sort (which sorts the lot and leaves the output
in the file "sorted"), puts a copy in the file "unsorted" and also sends the
lot to "wc -l" so I can see how many lines there were.
(The last_lines file already contained 29 lines)
It is always possible to omit the "-" command from a "-o" or "-i" section,
provided that there is no other command in this section - which is to say that
the section must be left empty.
It is possible to switch back and forth between input and output modes,
% pipe -o "|wc" -i -o "|wc" -i -o "|wc"
this
^D
1 1 5
is
^D
2 2 8
the first
^D
3 4 18
%
which takes lines from the terminal until an EOF, sends them to "wc",
switches back to input mode (note there are no commands and so standard input
will be used) to read more lines from the terminal, sends the current input to
"wc" and so on and on... To have the whole thing sent to the screen at the end
I could have appended a "-" OR used another "-o" with no command.
Notice that the output of wc in each case reflects the cumulative way in which
the input is being assembled.
As a final example, you can intermix pipes with unix pipelines and i/o
redirection to create complicated things like
% pipe "echo this" | pipe - "pipe 'echo is'" | pipe - "echo dumb"
this
is
dumb
%
Which could be used to do more useful things. That's probably enough...
@//E*O*F pipe_doc//
chmod u=rw,g=,o= pipe_doc
echo x - pipe
sed 's/^@//' > "pipe" <<'@//E*O*F pipe//'
#!/bin/sh
#
# pipe -- multi-input, multi-output pipelining
#
# Terry Jones 19/10/87 (tcjones@watdragon)
#
#-------------------------------------------------------------------------------
# Department Of Computer Science, University Of Waterloo
# Waterloo Ontario Canada N2L 3G1
#
#{ihnp4,allegra,decvax,utzoo,utcsri,clyde}!watmath!watdragon!tcjones
#tcjones@dragon.waterloo.{cdn,edu} tcjones@WATER.bitnet
#tcjones%watdragon@waterloo.csnet [from oz, tcjones@dragon.waterloo.cdn@munnari]
#-------------------------------------------------------------------------------
#
myname=`basename $0`
if [ $# -eq 0 ]
then
while read line
do
echo $line
done
exit 1
fi
def_shell=/bin/sh
if [ -z "$SHELL" ]
then
echo No '$SHELL' variable set, using $def_shell
SHELL=$def_shell
fi
cum_in=/tmp/${USER}_pipe_$$
if [ -f $cum_in -a ! -w $cum_in ]
then
echo ${myname}: could not use temporary ${cum_in} - try again.
exit 1
fi
>$cum_in
tty=`tty`
#echo rm $cum_in >> rmt
IN=1
SOME_IN=0
SOME_OUT=0
ANY_OUT=0
while [ -n "$1" ]
do
case $1 in
-in|-i)
IN=1
SOME_IN=0
if [ "$SOME_OUT" = "0" ]
then
SOME_OUT=1
cat $cum_in
fi
shift;;
-out|-o)
IN=0
SOME_OUT=0
ANY_OUT=1
if [ "$SOME_IN" = "0" ]
then
SOME_IN=1
while read line
do
echo $line >> $cum_in
done
fi
shift;;
*)
if [ "$IN" = "1" ]
then
SOME_IN=1
if [ "$1" = "-" ]
then
while read line
do
echo $line >> $cum_in
done
else
echo $1 | $SHELL >> $cum_in
fi
else
SOME_OUT=1
if [ "$1" = "-" ]
then
cat $cum_in
else
eval "cat $cum_in" "$1"
fi
fi
shift;;
esac
done
if [ "$SOME_OUT" = "0" -o "$ANY_OUT" = "0" ]
then
cat $cum_in
fi
/bin/rm -f $cum_in
@//E*O*F pipe//
chmod u=rwx,g=,o= pipe
echo Inspecting for damage in transit...
temp=/tmp/shar$$; dtemp=/tmp/.shar$$
trap "rm -f $temp $dtemp; exit" 0 1 2 3 15
cat > $temp <<\!!!
141 848 5157 pipe_doc
113 228 2331 pipe
254 1076 7488 total
!!!
wc pipe_doc pipe | sed 's=[^ ]*/==' | diff -b $temp - >$dtemp
if [ -s $dtemp ]
then echo "Ouch [diff of wc output]:" ; cat $dtemp
else echo "No problems found."
eadeade