[comp.os.minix] Pipes and other oddities...

Rbaker@afmpc-1.arpa (01/18/89)

1.  My instructor insists that under *nix and/or MINIX, a single pipe can
flow data in both directions between processes... ie., a pipe can be opened
for both read and write.  Please, someone back me up on this - I swear a 
pipe is a one-way flow - opened for read-only at one end and write-only
at the other.  Please flame one of us, gently.

2.  What does an executable file look like under MINIX as opposed to a 
MS-DOS .EXE file?  Is there somewhere I can get this info?  I have all
standard ref's, including Ast's book, just can't find it.

3.  Please, Dr. Tannenbaum, are you willing to share with your friends,
who was Sweety Pi?

Thanks!  -roger

rbaker@afmpc1.arpa

ast@cs.vu.nl (Andy Tanenbaum) (01/18/89)

In article <6815@louie.udel.EDU> Rbaker@afmpc-1.arpa writes:
>2.  What does an executable file look like under MINIX as opposed to a 
>MS-DOS .EXE file?  Is there somewhere I can get this info?  
The executable file format is identical to that of IBM's PC-IX.  It consists
of a header, followed by the text and data.  The header format is described
in the book starting on line 6046.

>who was Sweety Pi?
Sweetie \(pi was my rabbit, may she rest in peace in rabbit heaven.

Andy Tanenbaum (ast@cs.vu.nl)

knop@duteca (P. Knoppers) (01/20/89)

I think that this will be of more general interest. It applies to any version
of *nix that I know of.

In <6815@louie.udel.EDU> someone (I can't find the header of your article)
wrote:
> 1.  My instructor insists that under *nix and/or MINIX, a single pipe can
> flow data in both directions between processes... ie., a pipe can be opened
> for both read and write.  Please, someone back me up on this - I swear a 
> pipe is a one-way flow - opened for read-only at one end and write-only
> at the other.  Please flame one of us, gently.
You're both right.... and wrong
I'll explain.
The pipe system call creates a FIFO structure that can be accessed through
two file descriptors, one for write, one for read. The simplest way to set 
up a pipe between two processes is:
1: create pipe (with the pipe system call)
2: fork (to create the second process)
After this both processes can read and write the pipe. (Your instructor is
right.)

When the shell sets up a pipe between to processes it arranges things such
that one process can only write to the pipe and the other process can only
read from it. (You are right.) Things go like this:
1: create a pipe (with the system call)
2: fork the 1st process (the one that will later exec the process that will
   write to the pipe)
3: fork the 2nd process (the one that will later exec the process that will
   read from the pipe)
4: wait for both processes to complete after which the shell will prompt the
   user for a new command. (The shell doesn't notice that its children
   perform an exec system call, the wait system call returns only when one
   of the children exits or gets killed.) If a process, or a pipe of processes
   is started in the background (with &) the only difference is that the
   shell does not wait for the processes to complete, but instead it will
   prompt the user immediately for the next command.
Process 1 (which is a copy of the shell) now
1: closes file descriptor 1 (standard out)
2: dup's the write file descriptor of the pipe to file descriptor 1
3: closes it's read file descriptor of the pipe
4: exec's the command that is supposed to write in the pipe.
   (the exec-ed process will inherit the pipe as standard output)
Process 2 (which is also a copy of the shell) now
1: closes file descriptor 0 (standard in)
2: dup's the read file descriptor of the pipe to file descriptor 0
3: closes it's write file descriptor of the pipe
4: exec's the command that is supposes to read from the pipe.
   (the exec-ed process will inherit the pipe as standard input)

Comments:
- When the shell sets up a pipe between two process it ensures that the
  communication path will be one-way.
- When more than 2 processes must be created connected by pipes, the
  scheme gets slightly more complicated. I leave this as an exercise
  for the reader.
- The shell cannot create a circularly connected set of processes. There
  is simply no way to specify such things on the command line.
- It is possible to create a set of processes that is interconnected with
  pipes in any way you like. However, the shell will not do this for you.
  If at any time, any of a set of pipes that connects processes in a circular
  fashion, must contain more than (in most unix versions) 8 blocks of data, 
  deadlock will occur. Your *nix kernel cannot detect this situation. (Why ?
  Another exercise for the reader...)
- You will probably have to read this twice to understand it. The subject
  is not simple. You may have to (re-)read the documentation for the system
  calls pipe, dup, dup2, close, fork, exec and wait. 
-- 
Peter Knoppers - knop@duteca.UUCP

henry@utzoo.uucp (Henry Spencer) (01/21/89)

In article <6815@louie.udel.EDU> Rbaker@afmpc-1.arpa writes:
>1.  My instructor insists that under *nix and/or MINIX, a single pipe can
>flow data in both directions between processes... ie., a pipe can be opened
>for both read and write.  Please, someone back me up on this - I swear a 
>pipe is a one-way flow - opened for read-only at one end and write-only
>at the other.  Please flame one of us, gently.

[Turn on burner, low and gentle flame... :-)]

The usual Unix pipe has one read-only descriptor and one write-only
descriptor, but any number of processes can be using either and there
is no reason why one process can't use both.  Of course, if more than
one process is writing, you get their data more or less intermixed
unless they coordinate somehow, and if more than one process is reading,
which gets what data is random unless they coordinate somehow.

If he's thinking of a full-duplex connection, with an independent path
in each direction, he's wrong except for some research systems using
Dennis Ritchie's "streams" scheme for their pipes.
-- 
Allegedly heard aboard Mir: "A |     Henry Spencer at U of Toronto Zoology
toast to comrade Van Allen!!"  | uunet!attcan!utzoo!henry henry@zoo.toronto.edu

kent@tifsie.UUCP (Russell Kent) (01/24/89)

Just thought I'd chime in, and add some interesting new info too --

in article <449@duteca4.UUCP>, knop@duteca (P. Knoppers) says:
>> 1.  My instructor insists that under *nix and/or MINIX, a single pipe can
>> flow data in both directions between processes... ie., a pipe can be opened
>> for both read and write.  Please, someone back me up on this - I swear a 
>> pipe is a one-way flow - opened for read-only at one end and write-only
>> at the other.  Please flame one of us, gently.
> You're both right.... and wrong

Actually, I believe the student is only right.  It's the instructor who is
both right and wrong.  Of course, the student is guilty of omission:
what constitutes an "end" to a pipe?

A pipe is in fact a one-way flow.  It is open for reading at "one end"
and writing at "the other [end]."  The trick is that an "end"
can consist of more than one file descriptor.  So if two processes each
have a pair of file descriptors, one on the reading "end" and one on the
writing "end", then data may flow between the processes.  However, since
pipes are essentially character-oriented, nothing guarantees any sort of
sense to the delivery of byte read with respect to bytes written.  Example,
if process #1 writes "Hello", and process #2 writes "Bonjour", and then
both processes read from their respective read "end" descriptors, it would
be perfectly legal for them to read say "HeBnjlloo" and "our".
(Henry Spencer's reply may explain this better.)

> 
> Comments:
> - The shell cannot create a circularly connected set of processes. There
>   is simply no way to specify such things on the command line.
> Peter Knoppers - knop@duteca.UUCP

Actually Peter, if your "*nix" supports named pipes, then you can do some
interesting things like:

    mknod loopback p
    tweedle < loopback | diagnose > loopback

Theoretically, this should work.  Unfortunately, reality does not always
conform to theory.

(The following analysis is 20% fact and 80% hunch)
It seems that the shell hangs when opening "loopback" for reading (this
is on SunOS), waiting for a writing end of the named pipe to exist.  This
is probably to prevent the EOF from appearing on tweedle's stdin before
the writing has had a chance to come into existence, and only happens
when starting.  The unfortunate bi-product is that the shell is about to
create the writing end of the named pipe (diagnose's stdout).  Therefore,
instant self-deadlock.  The solution?  Don't cause the shell to open
"loopback" for reading:

    cat loopback | tweedle | diagnose > loopback

Interesting, no?

(For those of you I have baffled, I appologize.  There really is
no straightforward way to describe this without assuming the reader
has a fairly thorough understanding of how pipes work.  If you
really want to understand, start by reading the info on pipe(2)
in your local Unix machine.  If you still don't understand, then
find a copy of Maurice Bach's _Design_of_the_Unix_Operating_System_
(Prentice-Hall) and read the section on pipes and on name-to-inode
mapping.)

Author's Note:
  We (Texas Instruments) really use this construct.  We have a piece
of manufacturing equipment that comes with diagnostic software that is
flexible, but not too smart.  What we did was write a program which
reads the output of the diagnostics, analyzes it, and then "types" further
commands onto the diagnostic's stdin, much as a warm-bodied engineer
would do.
------------------------------------------------------------------
Russell Kent                  UUCP:         convex!smu!tifsie!kent
Texas Instruments                    sun!texsun!ti-csl!tifsie!kent 
PO Box 655012  M/S 3635           ut-sally!im4u!ti-csl!tifsie!kent
Dallas, TX 75265              Voice: (214) 995-3501   TI-MSG: RAK9
-- 
Russell Kent                  UUCP:         convex!smu!tifsie!kent
Texas Instruments                    sun!texsun!ti-csl!tifsie!kent 
PO Box 655012  M/S 3635           ut-sally!im4u!ti-csl!tifsie!kent
Dallas, TX 75265              Voice: (214) 995-3501   TI-MSG: RAK9