[comp.unix.questions] reliable reads from pipes

atul@NCoast.ORG (Atul Parulekar) (08/04/90)

The following program does not work all the time.  Most of the times it works
(gives the correct directory (/users/prog/test) followed by a carriage return)
but sometimes it does not print out the full directory name (prints (/) not
followed by a carriage return.)  I am not sure whether it is not reading from
the pipe completely or not printing out the complete message.  Any ideas and/or
suggestions for improvements?  I am trying to write a program which calls
another program passing it some parameters and getting back some output.

#include <stdio.h>

main ()
{
   int fifo[2],proc,n;
   char line[81];

   pipe (fifo);
   if ((proc=fork ()) = -1) {
      fprintf (stderr,"Cannot fork\n");
      exit (1);
   }
   if (proc == 0) {
      close (1);	/*close std o/p and dup write end of pipe to it */
      dup (fifo[1]);

      execlp ("pwd", "pwd", (char *) 0);

      fprintf (stderr, "Cannot exec pwd\n");
      exit (2);
   }

   n = read (fifo[0], line, 80);
   line[n] = '\0';
   printf ("Current directory is %s\n", line);
}

edward@ucbarpa.Berkeley.EDU (Edward Wang) (08/04/90)

In article <1990Aug3.233256.29659@NCoast.ORG> atul@NCoast.ORG (Atul Parulekar) writes:
>The following program does not work all the time.
>. . .
>   int fifo[2],proc,n;
>   char line[81];
>
>   pipe (fifo);
>   [fork an exec of pwd]
>   n = read (fifo[0], line, 80);
>   line[n] = 0;
>   . . .

This may be of general interest.

The problem is that pwd may be writing less than the who path at a time.
The solution is to keep reading until end of file (n = 0) or error (n < 0).
There's no danger of looping because some progress is always made.

In the best Unix style,

	char *p, *q;
	for (q = (p = line) + sizeof line - 1;
	     p < q && (n = read(fifo[0], p, q - p)) > 0;
	     p += n)
		;
	*p = 0;
	/* if desired, check for overflow (p == q) or read error (n < 0) */

Something like this is also a good idea when writing,
because signals may cause partial writes to return.
Standard IO is well known for not doing this correctly.

guy@auspex.auspex.com (Guy Harris) (08/05/90)

>The following program does not work all the time.  Most of the times it works
>(gives the correct directory (/users/prog/test) followed by a carriage return)
>but sometimes it does not print out the full directory name (prints (/) not
>followed by a carriage return.)  I am not sure whether it is not reading from
>the pipe completely or not printing out the complete message.  Any ideas and/or
>suggestions for improvements?  I am trying to write a program which calls
>another program passing it some parameters and getting back some output.

Given that, I assume the reason you're running "pwd" is to test the
logic you're using to run the subprocess, not because you actually want
to write your own code to get the current working directory.  If you
want to get the current working directory, check first whether your OS
supplies such a routine; it's likely to be called either "getwd()" or
"getcwd()".  "getcwd()" is the offical POSIX version, and as such will
eventually be more likely to be available than "getwd()".

(BTW, if you want to run "pwd", be warned that a bad setting of PATH
will, in your example, either not find "pwd" at all or find the wrong
one.  You might want to just have it "exec" "/bin/pwd" and, if that
fails, "/usr/bin/pwd" instead.)

If you want to run some program and read its standard output, check out
"popen()", which your UNIX system almost certainly has.  It does most of
the work your code is doing, and you thus don't have to write that code
yourself, or make it work.  "'libc' is your friend."

In addition, note that there is *no* guarantee that a single "read()"
will necessarily pick up all the data that the program will write to the
pipe; you need to keep reading until you get an EOF, or until you know
you've read all you need to read.  "popen()" gives you a standard I/O
stream, which can make it a bit more convenient to keep reading.

gwyn@smoke.BRL.MIL (Doug Gwyn) (08/05/90)

In article <1990Aug3.233256.29659@NCoast.ORG> atul@NCoast.ORG (Atul Parulekar) writes:
>but sometimes it does not print out the full directory name (prints (/) ...

"pwd" doesn't necessarily print its entire output in a single atomic write(),
so your read() on the pipe may return only a portion of what "pwd" will
eventually have written.  The simplest solution is to read until you see a
new-line, which will be the last thing output by "pwd".

andrew@alice.UUCP (Andrew Hume) (08/05/90)

	doug's suggestion to look for \n is doubly important
because your code has both ends of the pipe open in both processes
and thus you won't get end of file. the rule is that in the process
that does the writing should close the reading end of teh pipe
and vice versa.

rick@tetrauk.UUCP (Rick Jones) (08/06/90)

In article <1990Aug3.233256.29659@NCoast.ORG> atul@NCoast.ORG (Atul Parulekar) writes:
>The following program does not work all the time.  ...
>... I am trying to write a program which calls
>another program passing it some parameters and getting back some output.
>
>#include <stdio.h>
>
>main ()
>{
>   int fifo[2],proc,n;
>   char line[81];
>
>   [code for fork & child writing process]
>
>   n = read (fifo[0], line, 80);
>   line[n] = '\0';
>   printf ("Current directory is %s\n", line);
>}

In all implementations where I've used pipes, a read on the pipe returns as
many bytes as are in the pipe at the time of the read.  If the writing process
is not buffering its writes, then it may be scheduled out in the middle of a
line, and the read process will see the incomplete data.

You either need to loop on the read until you get EOF (i.e. when the writer has
closed the pipe), or better use fgets() which will only return on end-lines or
buffer full (so it does the looping for you).  If you have to pick up multiple
lines, then fgets() is definitely the way to do it.  Raw reads on pipes can be
as difficult as raw reads on serial lines.

Incidentally, you should always close the read end of the pipe in the writing
process, and the write end in the reading process immediately after the fork.
If you don't at least do the second of these, the read process will never see
EOF on the pipe when the writer exits, since it is keeping it open itself.

-- 
Rick Jones					You gotta stand for something
Tetra Ltd.  Maidenhead, Berks			Or you'll fall for anything
rick@tetrauk.uucp (...!ukc!tetrauk.uucp!rick)	     - John Cougar Mellencamp

gwyn@smoke.BRL.MIL (Doug Gwyn) (08/06/90)

In article <11145@alice.UUCP> andrew@alice.UUCP (Andrew Hume) writes:
>	doug's suggestion to look for \n is doubly important
>because your code has both ends of the pipe open in both processes
>and thus you won't get end of file. the rule is that in the process
>that does the writing should close the reading end of teh pipe
>and vice versa.

In fact, most of us would probably use popen() and let it take care of
the details of getting the pipes set up properly, etc.  (Yes, I know
that some popen() implementations have bugs, but I think they all work
well enough for applications like this one.)

andrew@alice.UUCP (Andrew Hume) (08/07/90)

	lest anyone start relying on reads returning whatever is in the pipe,
9th edition and later unices preserved the size of the writes which can
now also exceed the size of the pipe buffer (i think).

boyd@necisa.ho.necisa.oz (Boyd Roberts) (08/13/90)

In article <11155@alice.UUCP> andrew@alice.UUCP (Andrew Hume) writes:
>
>	lest anyone start relying on reads returning whatever is in the pipe,
>9th edition and later unices preserved the size of the writes which can
>now also exceed the size of the pipe buffer (i think).

Not to mention the 1 byte write nasty that will take out all your
stream message buffers.  The stream pipe fills when the write
side high water mark is hit; which is tunable.

Those M_DELIM's are neat...


Boyd Roberts			boyd@necisa.ho.necisa.oz.au

``When the going gets wierd, the weird turn pro...''

sar0@cbnewsl.att.com (stephen.a.rago) (08/14/90)

In article <1827@necisa.ho.necisa.oz>, boyd@necisa.ho.necisa.oz (Boyd Roberts) writes:
> In article <11155@alice.UUCP> andrew@alice.UUCP (Andrew Hume) writes:
> >
> >	lest anyone start relying on reads returning whatever is in the pipe,
> >9th edition and later unices preserved the size of the writes which can
> >now also exceed the size of the pipe buffer (i think).
> 
> Not to mention the 1 byte write nasty that will take out all your
> stream message buffers.  The stream pipe fills when the write
> side high water mark is hit; which is tunable.

SVR4 won't let someone "take all the stream message buffers" unless
they are running as root.  And it's not message buffers, its general
memory out of the kernel memory pool.

> 
> Those M_DELIM's are neat...
> 

M_DELIM's are old.  In V10 it's a flag in the message header (except for
the message line discipline).  SVR4 also has delimiters, if you want to
use them.

Steve Rago
sar@attunix.att.com