[comp.sys.ibm.pc] Redirection and piping

MHS108@PSUVM.BITNET (Mark Solsman) (12/12/89)

   Can any one out there explain how to use the redirection and the pipeline
features with ms-dos? I have tried these damn things since I started using
computers, yet have allways been unsuccessful. I can redirect the output to a
destination, and I can somewhat pipe the output to another program(maybe). My
problem is going the other way. I have a hard disk utility that makes you enter
the drive number from the keyboard. I would like to redirect the drive numbers
from another file, so that I could run that unattended. Any help would be
greatly appreciated!
   To sum it up, I am looking for an explaniation to use the redirect and pipe-
line functions in ms-dos.

   thanks in advance!


-------
Mark Solsman
US mail  -> 1012 Whippoorwill Drive   Clarks Summit, Pa   18411
BitNET   -> MHS108 @ PSUVM.BITNET
InterNET -> MHS108 @ PSUVM.PSU.EDU
RelayNet -> Mark Solsman  (direct mail to node OUTER)

ts@uwasa.fi (Timo Salmi LASK) (12/12/89)

In article <89345.110313MHS108@PSUVM.BITNET> MHS108@PSUVM.BITNET (Mark Solsman) writes:
>
>   Can any one out there explain how to use the redirection and the pipeline
>features with ms-dos? I have tried these damn things since I started using
>computers, yet have allways been unsuccessful. I can redirect the output to a
>destination, and I can somewhat pipe the output to another program(maybe). My
>problem is going the other way. I have a hard disk utility that makes you enter
>the drive number from the keyboard. I would like to redirect the drive numbers
>from another file, so that I could run that unattended. Any help would be
>greatly appreciated!

I am not going to try to give a piping lecture here, but if you have
the capability of getting files by anonymous ftp you might be
interested in getting /pc/ts/tsutlc16.arc and look at how the
DOUBLES.EXE program can be made to accept input from a YES files. 
But I am positive that someone will come with a good concise advice
here giving the outlines and principles rather than a mere reference
to an example. 

...................................................................
Prof. Timo Salmi                                (Site 128.214.12.3)
School of Business Studies, University of Vaasa, SF-65101, Finland
Internet: ts@chyde.uwasa.fi Funet: vakk::salmi Bitnet: salmi@finfun

fredex@cg-atla.UUCP (Fred Smith) (12/12/89)

In article <89345.110313MHS108@PSUVM.BITNET> MHS108@PSUVM.BITNET (Mark Solsman) writes:
>
>   Can any one out there explain how to use the redirection and the pipeline
>features with ms-dos? I have tried these damn things since I started using
>computers, yet have allways been unsuccessful. I can redirect the output to a
>destination, and I can somewhat pipe the output to another program(maybe). My
>problem is going the other way. I have a hard disk utility that makes you enter
>the drive number from the keyboard. I would like to redirect the drive numbers
>from another file, so that I could run that unattended. Any help would be
>greatly appreciated!
>-------
>Mark Solsman


Mark:

In the general case DOS i/o redirection is used (from the point of view
of the keyboard operator) in the same way as in Unix. However, what may be the
problem you have encountered here is that in DOS some programs do not use
stdin/stdout/stderr, but rather read from or write to the hardware directly,
or at least use BIOS services rather than those provided by DOS. It is DOS that
does the i/o redirection, so if your hard disk utility reads the keyboard
either by using BIOS services or going direct to the keyboard interrupt then
you are out of luck.

Many DOS programs are quite unfriendly in this manner, in that they assume that
theyown the whole machine and can do anything they please. This works fine only
in the case where they do not need to interact with other programs (obviously
the situation envisioned by the designers of that program).

As an aside, let me give you an illustration. I have written a unix-like more
program for DOS which reads the keyboard via BIOS services. The reason it does
this is that it is possible that more may be displaying data from a pipe or
otherwise redirected file, but still needs to be able to get commands fromthe
keyboard, even though stdin may be redirected.

Fred

pipkins@qmsseq.imagen.com (Jeff Pipkins) (12/13/89)

In article <89345.110313MHS108@PSUVM.BITNET> MHS108@PSUVM.BITNET (Mark Solsman) writes:
>   Can any one out there explain how to use the redirection and the pipeline
>features with ms-dos? [...]  I have a hard disk utility that makes you enter
>the drive number from the keyboard. I would like to redirect the drive numbers
>from another file, so that I could run that unattended [...].

A program that reads its input from stdin and writes it output to stdout is
known as a filter.  A pipe is an operating system method of linking several
of these filters together like a chain of elephants, so that the output of
one becomes the input of another.  Example:

C:\> echo A: | diskutil

The echo program is not, strictly speaking, a filter because it does not
take its input from stdin.  Instead, it takes its input from its command
line and merely writes that to stdout.  The pipe symbol '|' says that
the output (stdout) of the echo program is to be used as the input (stdin)
of the diskutil program.  NOTE that if the diskutil program reads its
input directly from the keyboard or with a BIOS call, then it will not
read the input from stdin and you will have to type the drive letter anyway.
That's why this doesn't work as you would expect:

C:\> echo Y | format c:     (kids, don't try this at home...)

Redirection is the act of specifying a file or device for stdin or stdout.
For example:

C:\> more <read.me

The more program is a filter.  The <read.me specifies that the input
for more is to come from the file read.me.  This is faster than saying:

C:\> type read.me | more

But has a similar effect.  The > symbol is used to redirect stdout.  Note
that stdin and stdout can be redirected to a device as well as a file:

C:\> echo ^L >lpt2

Will send a form feed character to the printer on LPT2.  So you might also
try something like this:

C:\> echo Y >yes.ans
C:\> diskutil <yes.ans

These techniques can be combined to save you from writing new programs.

C:\> dir c:\ | find '<DIR>' | sort >dirlist.c

This line will do a directory of drive C:, throw away all lines that don't
have '<DIR>' on them, sort the lines (by name, ext) and write it to a
file called dirlist.c.  In other words, dirlist.c will contain a sorted
list of the directories on drive C:

You can view this list later by typing C:\> more <dirlist.c

I hope the presence of pipes encourage you to write small filters that can
perform just one function.  A collection of these is quite powerful.  Instead
of having a program to convert MS-DOS and UNIX files back and forth, I use
pipes of small filters such as dtab (expands tabs to spaces), lf2crlf
(converts line feed characters to CR/LF sequence) and dz (removes control-Z
characters from end of file).  By doing this, these small filter programs
are reusable in many other situations.

Hope this helps.
-Jeff
pipkins@imagen.com

ear@wpi.wpi.edu (Eric A Rasmussen) (12/13/89)

In article <89345.110313MHS108@PSUVM.BITNET> MHS108@PSUVM.BITNET (Mark Solsman) writes:
>
>   Can any one out there explain how to use the redirection and the pipeline
>features with ms-dos?

It's relatively simple.

>  redirects standard output from a program, which would normally go to the
   screen, to whereever you redirect it to.

For example, typing "main > prn" at the dos prompt would cause the output
from the program main to go to the printer.  You can also redirect output to a
file.

<  redirects standard input to a program, generally from a file, instead of
   taking the input from the keyboard.

For example, typing "main < commands.txt" at the dos prompt would cause the
program main to act as if you had typed all the characters in the file
commands.txt from the keyboard.

|  takes the output from a command or program, instead of a file or device, and
   passes (pipes) it along to a second program for subsequent processing. 
   (Note that I'm not so familiar with this feature, but I know how to use it.) 

For example, typing "dir | more" at the dos prompt has about the same effect
as "dir/p", but in this case you are using the program MORE to display the
output from DIR one page at a time by intercepting its output and only letting
you see 24 lines of it at a time before making you press a key to continue. 
When you use the /p option with the DIR command, you are using a built in
feature of DIR.  (Note that for this to work, you must have the program 
MORE.whatever-the-extension-on-it-is somewhere in your directory path.)

Hope this helps.
 _                _                                         +-=-=-=-=-=-=-=-=-+
|_  ,_  .   _    |_}   _    _  ,_ _         _   _   _   ,_  | ear@wpi.wpi.edu |
|_  |   |  |_    | \  |_\  _>  | | |  |_|  _>  _>  |_'  | | | ear%wpi@wpi.edu |
--< A real engineer never reads the instructions first! >-- +-=-=-=-=-=-=-=-=-+

fisher@sc2a.unige.ch (Markus Fischer) (12/13/89)

In article <89345.110313MHS108@PSUVM.BITNET>, MHS108@PSUVM.BITNET
(Mark Solsman) writes:
>    Can any one out there explain how to use the redirection and the pipeline
> features with ms-dos? [...]
> I have a hard disk utility that makes you enter
> the drive number from the keyboard. I would like to redirect the drive numbers
> from another file, so that I could run that unattended. Any help would be
> greatly appreciated!

Well, basically, you have the output redirect ('>' and '>>'), the input redirect
('<'), and the pipe ('|').  The redirect is used to send the output of a program
to a file ('>' rewrites and '>>' appends), or to get the input from a file.
This works only for programs that use standart i/o.  Neither direct keyboard
read, nor direct video addressing can be redirected; this applies to the pipe,
also.  The pipe is used when you want the output of a program to be send as the
input to another.  Basically, you could say that
	...> foo | bar
expands to
	...> foo > tmp_file
	...> bar < tmp_file
	...> del tmp_file
where the tmp_file is some unique non-existing filename, provided by DOS.

As for you example, let's say you want to run Norton's SD (speed disk).  This
program requires you to enter a drive letter (as in you example), the default
being the default current drive, followed by a <CR>.  So you simply create a
file containing the drive letter and a <CR>, let's call it drive.txt, and type:
	...> SD < drive.txt
or, if you prefer not to have this file always in you disk, type:
	...> echo C | SD
which will create the needed temporary file for you.
NOTE: you will need to reboot afterwards, as SD requires you to type one last
key *after* the job is done.  As the program traps the keyboard (to avoid an
unexpected end, messing up the disk), the buffer will be flushed before asking
for that last character, or it will be read directly from the keyboard, which
you can no longer do...

It is of course possible to create more complex input files, for example for
an editor... (which uses standard input ! most do not...)  I have even written
a simple program, called `write', which works a little like the `echo' command,
but with more options. For example, 
	...> write C$_Y$_exit$_
produces a file containing:
	C<CR-LF>
	Y<CR-LF>
	exit<CR-LF>

But be warned: when unsing a pipe, the program will run with the file being it's
only input!  You can no longer answer to unexpected questions, nor can you type
that last character after the "All done, strike any key to return to DOS"...
So you better try everything out when you can harmlessly reboot any time!

Hope this helps

Markus Fischer                -|--|--|--|--|--|--I   Department of Anthropology
                        -|--|--|--|--|--|--|-(#)-I   University of Geneva
      -|--|--|--|--|--|--|--|--|-(#)-|-(#)(#)(_)-I   CH-1227  Carouge (GE)
   -&-(_)-|--|--|-(#)-&--|-(#)(#)(_)(#)-&-(_)(#)-I   Switzerland
            -|--|--|--|--|-(#)(_)-|-(_)(_)(_)(#)-I
black (#) to kill ! --|--|-(#)(_)(_)(_)(#)(#)(_)(_)  fisher@sc2a.unige.ch
=+==+==+==+==+==+==+==+==+==+==+==+==+==+==+=(#)=+   fisher@cgeuge52.bitnet

baird@cod.NOSC.MIL (John M. Baird) (12/14/89)

From article <89345.110313MHS108@PSUVM.BITNET>, by MHS108@PSUVM.BITNET (Mark Solsman):
> 
>    Can any one out there explain how to use the redirection and the pipeline
> features with ms-dos? 
What follows in not technical. It just gives short examples of how they
can be used, and warnings about when they can't. It is fairly long.

John Baird, Naval Ocean Systems Center, San Diego, CA USA
--------
From HELPSB, a DOS VMS-like help utility's files:
******** Information about |(piping) *******
To pipe (send, channel, direct) the output of one command into another command.

Output from a command that would normally appear on the screen can be used
as input to another command (that would normally get its input from the 
keyboard) by piping it. Piping is controlled by the use of the vertical bar 
(|) character. Some versions of DOS may not support the use of piping in BATCH 
files. 

If a text file, maybe some sort of document, had 10,000 lines in it, it could 
be displayed on the screen one full screen at a time, with the command

	TYPE BIGFILE.TXT | MORE

If a directory contained 20 entries, to get a listing of the contents of the 
directory in alphabetical order, the command

	DIR | SORT

could be used. If the directory had 120 entries, an alphabetical display one 
full screen at a time could be obtained by the command

	DIR | SORT | MORE
	ECHO Hello >> TST.TXT

will redirect the string "Hello" from the screen to the file TST.TXT. If 
TST.TXT did not previously exist, this will create it. 

	SORT < RANDOM.TXT >> SORTED.TXT

will redirect input from the keyboard to come from the file RANDOM.TXT and 
redirect output to the screen to be added to the end of the file SORTED.TXT.

	TYPE CON >> EXISTING.TXT 

will read everything entered from the keyboard (until a Ctrl-Z is entered) and 
add it to the end of file EXISTING.TXT, rather than displaying it on the 
screen. Another way to to the same thing is with 

	COPY EXISTING.TXT+CON

It may sometimes appear that >> didn't work. If a file has a Ctrl-Z character 
in it, >> will still add to the end of the file, AFTER the Ctrl-Z. Some 
editors, and the DOS TYPE and COPY /A commands, quit when they encounter the 
first Ctrl-Z.
    MORE < NEW.TXT

will redirect input which would normally come from the keyboard to come from 
the file NEW.TXT. MORE will still display its output on the screen. 

    SORT < RANDOM.TXT > SORTED.TXT

will redirect input from the keyboard to come from the file RANDOM.TXT
and redirect output to the screen to the file SORTED.TXT.

******** Information about < or > or >> (redirection) *******
To change the normal destination or source of input or output for a command
is done by redirection. Input that normally comes from the keyboard for a
command can be redirected to come from a file by the < symbol (INPUT gives
examples). Output from a command that normally goes to the screen can be
redirected to a file by the > symbol (OUTPUT gives examples) or >> symbols
(APPEND gives examples).

Device names are really useful with redirection when used by DOS commands 
like COPY, ECHO, ESC, SORT and TYPE. Separately purchased programs may or
may allow the use of redirection. If you encounter problems or erratic 
behavior in the use of redirection, it may be because the number of files 
that can be open at one time is set too low. This number is specified on 
the FILES line in the CONFIG.SYS file. 

******** Information about > (output) *******
	ECHO Hello > TST.TXT

will redirect the string "Hello" from the screen to the file TST.TXT. If 
TST.TXT already exists, its previous contents will be lost. Use >> to add
to an existing file.

	SORT < RANDOM.TXT > SORTED.TXT

will redirect input from the keyboard to come from the file RANDOM.TXT
and redirect output to the screen to the file SORTED.TXT.

	COPY OLD NEW > NUL

will copy the file OLD in the current directory to the file NEW, and the usual
COPY message "1 File(s) copied" will not appear on the screen. It will be
redirected to the NUL device. This is a way to reduce the number of messages 
that appear on the screen while a batch file is being executed.

	REM > EMPTYFIL

will create the file EMPTYFIL, a file that is empty (it has no data in it
and a DIR command will show its size is zero).

	ECHO remark 
	PAUSE > NUL
will redirect the usual 

	Strike any key when ready...

message to NUL, allowing a remark of your choosing to appear on the screen.

******** Information about < (input) *******
    MORE < NEW.TXT

will redirect input which would normally come from the keyboard to come from 
the file NEW.TXT. MORE will still display its output on the screen. 

    SORT < RANDOM.TXT > SORTED.TXT

will redirect input from the keyboard to come from the file RANDOM.TXT
and redirect output to the screen to the file SORTED.TXT.

******** Information about >> (append) *******
	ECHO Hello >> TST.TXT

will redirect the string "Hello" from the screen to the file TST.TXT. If 
TST.TXT did not previously exist, this will create it. 

	SORT < RANDOM.TXT >> SORTED.TXT

will redirect input from the keyboard to come from the file RANDOM.TXT and 
redirect output to the screen to be added to the end of the file SORTED.TXT.

	TYPE CON >> EXISTING.TXT 

will read everything entered from the keyboard (until a Ctrl-Z is entered) and 
add it to the end of file EXISTING.TXT, rather than displaying it on the 
screen. Another way to to the same thing is with 

	COPY EXISTING.TXT+CON

It may sometimes appear that >> didn't work. If a file has a Ctrl-Z character 
in it, >> will still add to the end of the file, AFTER the Ctrl-Z. Some 
editors, and the DOS TYPE and COPY /A commands, quit when they encounter the 
first Ctrl-Z. If a file has internal Ctrl-Z characaters in it, the EDLIN 
command can be used to remove them (see the EDLIN HINTS CONTROL_CHARACTERS 
topic.)

roy@comcon.UUCP (Roy M. Silvernail) (12/14/89)

In article <8161@cg-atla.UUCP}, fredex@cg-atla.UUCP (Fred Smith) writes:
} 
} Many DOS programs are quite unfriendly in this manner, in that they assume that
} theyown the whole machine and can do anything they please. This works fine only
} in the case where they do not need to interact with other programs (obviously
} the situation envisioned by the designers of that program).

Fred, I wonder if this hasn't crept in because DOS programmers moved up
from single-tasking micros. I definitely dislike programs that make the
assumption that they own the machine.

} 
} As an aside, let me give you an illustration. I have written a unix-like more
} program for DOS which reads the keyboard via BIOS services. The reason it does
} this is that it is possible that more may be displaying data from a pipe or
} otherwise redirected file, but still needs to be able to get commands fromthe
} keyboard, even though stdin may be redirected.
} 
An excellent idea! Don't suppose you have a MS-DOS version of cat laying
about... (;-)
} Fred


-- 
_R_o_y _M_. _S_i_l_v_e_r_n_a_i_l  | UUCP: uunet!comcon!roy  |  "No, I don't live in an igloo!"
[ah, but it's my account... of course I opine!]           -Sourdough's riposte
SnailMail: P.O. Box 210856, Anchorage, Alaska, 99521-0856, U.S.A., Earth, etc.

cs4g6ag@maccs.dcss.mcmaster.ca (Stephen M. Dunn) (12/16/89)

   Redirection and piping work the same, essentially, in DOS as they
do in UNIX, except they have some limitations.  DOS only allows you to
play with stdin and stdout, whereas UNIX lets you redirect any handle
you wish.  And, of course, when piping something, UNIX lets you run
both tasks simultaneously, while DOS dumps the output of the first
program into a disk file and then runs the second program with its
stdin coming from that file.  Other than that, they're pretty much
the same.

   The problem you're having probably stems from the fact that in
DOS, there are ways of reading input and writing output that have
nothing to do with stdin and stdout.  For example, it is faster to
either use BIOS calls or direct screen memory writes than go through
DOS when you want to print something on the screen, so many programs
do this.  As far as DOS is concerned, such programs do not produce
any output, since they don't do it through stdout, and therefore
they cannot have their output redirected.

   In an analogous fashion, keyboard input can be done with BIOS
calls rather than through DOS (although the speed of keyboard input
is not a factor here, since even the fastest typists among us don't
come anywhere close to the computer's I/O speed limit).  The reason
for doing this is that using the BIOS allows you to check for the
next character without having it removed from the queue, and check
to see if the queue's empty, and other such tricks that DOS's
stdin won't allow you to do.  Such programs do not read from stdin,
and therefore stdin redirection will have no effect upon them.

   The program that you can't get to read redirected input probably
does its keyboard input directly through the BIOS.  I'm afraid that
if this is the case, you're out of luck.

-- 
Stephen M. Dunn                               cs4g6ag@maccs.dcss.mcmaster.ca
          <std_disclaimer.h> = "\nI'm only an undergraduate!!!\n";
****************************************************************************
    If it's true that love is only a game//Well, then I can play pretend