decot@hpisod2.HP.COM (Dave Decot) (03/13/90)
Hi. I am writing a C program, say prog1, that wants to work on System V.3 or later and BSD 4.2 or later. It wants to run an arbitrary shell command and collect all of that command's standard output into a buffer in memory, and all of its standard error output into another buffer. I need to know the exit status of the shell command to determine what to do next. I need to use prog1's original standard output and standard error for other purposes later. I don't know in advance how long either of the standard output or standard error of the command will be, but it is usually under 80 bytes. I don't want to use temporary files if at all possible. I used popen(cmd, "r") to run the command and fgets() to retrieve the output into a buffer. I shifted and masked the return value of popen() to get the exit status of the command, and then call pclose() to wait for the child process and get rid of the popen() stream. All of this is very straightforward. However, capturing the *standard error* output of the command was harder, since popen() makes no allowances for that, and I didn't want to use a temporary file. Here's what I tried. Prior to calling popen(), prog1 saves the original standard error file by duplicating it to another file descriptor, then creates a pipe, then uses dup2() to duplicate the write end of the pipe to file descriptor 2 (standard error). If you follow the above, prog1 is now all ready to read from the read end of the pipe whatever is written to its own (and any forked child's) standard error output. So, as described above, prog1 now calls popen(), reads the standard output from the popen() stream, and finally calls pclose() to wait for the child process and destroy the popen() stream. OK, now. The problem that arises is that the child process created by popen() cannot write any more than a pipe-ful bytes to the standard error pipe, because nobody reads any of it from the other end until after the child process has terminated and been waited for by the pclose() call. I thought of keeping the popen() stream open until after reading from the standard error pipe, but then I cannot determine when I have read all of the standard error output, since the child process may still be running. Has anyone any brilliant suggestions? Dave
lwall@jpl-devvax.JPL.NASA.GOV (Larry Wall) (03/14/90)
In article <14020086@hpisod2.HP.COM> decot@hpisod2.HP.COM (Dave Decot) writes: : Hi. I am writing a C program, say prog1, that wants to work on System V.3 : or later and BSD 4.2 or later. : : It wants to run an arbitrary shell command and collect all of that : command's standard output into a buffer in memory, and all of its standard : error output into another buffer. I need to know the exit status of the : shell command to determine what to do next. I need to use prog1's : original standard output and standard error for other purposes later. : I don't know in advance how long either of the standard output or : standard error of the command will be, but it is usually under 80 bytes. : I don't want to use temporary files if at all possible. : : I used popen(cmd, "r") to run the command and fgets() to retrieve the : output into a buffer. I shifted and masked the return value of popen() : to get the exit status of the command, and then call pclose() to wait : for the child process and get rid of the popen() stream. All of this is : very straightforward. : : However, capturing the *standard error* output of the command was harder, : since popen() makes no allowances for that, and I didn't want to use : a temporary file. : : Here's what I tried. Prior to calling popen(), prog1 saves the original : standard error file by duplicating it to another file descriptor, then : creates a pipe, then uses dup2() to duplicate the write end of the pipe : to file descriptor 2 (standard error). If you follow the above, prog1 : is now all ready to read from the read end of the pipe whatever is : written to its own (and any forked child's) standard error output. So, : as described above, prog1 now calls popen(), reads the standard output : from the popen() stream, and finally calls pclose() to wait for the : child process and destroy the popen() stream. : : OK, now. The problem that arises is that the child process created by : popen() cannot write any more than a pipe-ful bytes to the standard error : pipe, because nobody reads any of it from the other end until after the : child process has terminated and been waited for by the pclose() call. : : I thought of keeping the popen() stream open until after reading from : the standard error pipe, but then I cannot determine when I have read all : of the standard error output, since the child process may still be running. : : Has anyone any brilliant suggestions? You need a separate process to collect the stderr while your original process is collecting stdout. This can be a child process which runs the actual prog1. When prog1 dies, the child appends the collected error messages to stdout, plus the exit status. Your program then looks for the magic lines at the end of the ordinary stdout output. You can do it something like this: #include <stdio.h> main() { char stdoutbuf[8192]; /* or whatever */ char stderrbuf[8192]; int exitstatus; FILE *fp; char *bufptr,*bufend; fp = popen("cachestderr 'prog1'", "r"); /* prog1 is any shell command */ bufptr = stdoutbuf; bufend = bufptr + sizeof stdoutbuf; while (fgets(bufptr,bufend-bufptr, fp)) { if (strcmp(bufptr,"*** HERE ARE THE ERRORS ***\n") == 0) break; bufptr += strlen(bufptr); } *bufptr = '\0'; bufptr = stderrbuf; bufend = bufptr + sizeof stderrbuf; while (fgets(bufptr,bufend-bufptr, fp)) { if (strncmp(bufptr,"*** EXIT STATUS WAS",19) == 0) break; bufptr += strlen(bufptr); } *bufptr = '\0'; exitstatus = atoi(bufptr+20); pclose(fp); printf("The stdout was:\n"); fputs(stdoutbuf,stdout); printf("The stderr was:\n"); fputs(stderrbuf,stdout); printf("The exit status was %d, signal %d\n", exitstatus / 256, exitstatus % 256); } Of course, then you have to write cachestderr. The following is one way: #!/usr/bin/perl pipe(IN,OUT); # pipe() requires 3.0 patchlevel 9 open(STDERR,">&OUT"); # hook stderr to the write side of the pipe close OUT; # this filehandle not needed now fork || (close IN, exec @ARGV); # run command as a child close STDERR; # make sure only child has write side open. @cache = <IN>; # gets EOF when child dies wait; # harvest status print "*** HERE ARE THE ERRORS ***\n", @cache, "*** EXIT STATUS WAS $? ***\n"; Note that we do nothing with stdout except keep it open till prog1 is done, then scribble on it ourselves and exit. Here's a test prog1 for you, while we're at it: #!/usr/bin/perl print "Line 1 to stdout\n"; warn "Line 1 to stderr\n"; print "Line 2 to stdout\n"; warn "Line 2 to stderr\n"; exit 123; Larry Wall lwall@jpl-devvax.jpl.nasa.gov
maart@cs.vu.nl (Maarten Litmaath) (03/15/90)
In article <7406@jpl-devvax.JPL.NASA.GOV>, lwall@jpl-devvax.JPL.NASA.GOV (Larry Wall) writes: )... )When prog1 dies, the child appends the collected error messages to stdout, )plus the exit status. Your program then looks for the magic lines at the )end of the ordinary stdout output. [...] "Yech! This stuff tastes like poison." Variably-sized buffered IO etc. is left as an exercise. --------------------cut here-------------------- #include <stdio.h> #define OUT_SIZE (10 * 1024) #define ERR_SIZE 1024 char *Prog[] = { "/bin/cat", "/etc/passwd", "/non-existent", 0 }; main() { int pp_out[2], pp_err[2], pp[2], n_out, n_err, n, pid1, pid2, w; int status; /* allright, so it should be a union wait */ char out[OUT_SIZE], err[ERR_SIZE], *p; if (pipe(pp_out) < 0 || pipe(pp_err) < 0) panic("pipe"); switch (pid1 = fork()) { case -1: panic("fork"); case 0: dup2(pp_out[1], 1); close(pp_out[1]); close(pp_out[0]); dup2(pp_err[1], 2); close(pp_err[1]); close(pp_err[0]); execv(*Prog, Prog); panic(*Prog); } close(pp_out[1]); close(pp_err[1]); if (pipe(pp) < 0) panic("pipe"); switch (pid2 = fork()) { case -1: panic("fork"); case 0: for (p = err; p < err + sizeof err; p += n) if ((n = read(pp_err[0], p, (err + sizeof err) - p)) <= 0) break; if (p > err && write(pp[1], err, p - err) < 0) panic("write"); if (n < 0) panic("read"); exit(0); } close(pp_err[0]); close(pp[1]); for (p = out; p < out + sizeof out; p += n) if ((n = read(pp_out[0], p, (out + sizeof out) - p)) <= 0) break; if (n < 0) panic("read"); n_out = p - out; for (p = err; p < err + sizeof err; p += n) if ((n = read(pp[0], p, (err + sizeof err) - p)) <= 0) break; if (n < 0) panic("read"); n_err = p - err; close(pp_out[0]); close(pp[0]); for (n = 0; n < 2; ) { w = wait(&status); if (w == pid1 || w == pid2) { if (w == pid1) printf("status: %d\n\n", status >> 8); ++n; } } printf("out:\n%s\nerr:\n%s", out, err); } panic(message) char *message; { perror(message); exit(1); } --------------------cut here-------------------- -- 1) Will 4.5BSD have wait5()? |Maarten Litmaath @ VU Amsterdam: 2) Sleep(3) should be sleep(2) again.|maart@cs.vu.nl, uunet!mcsun!botter!maart