abair@oakhill.UUCP (Alan Bair) (02/16/89)
The group I work for would like to be able to pipe data to FTP instead of writting an intermediate file and then invoking FTP to transfer the file. As far as I know you cannot directly pipe to FTP or get it to read STDIN for the file to transfer. Therefore, I would like to know if there are any c callable functions that know the FTP protocol. Then the program that genreates the data file could open the connection and feed the data directly across the connection to the FTP server at the other end. If these types of rotuines do not already exist, how hard would it be to write a set of functions to do this. We only need a limited set of the capabilites of FTP, like only one direction, fixed record format, minimal error correction, etc. Please email, since I normally do not read this group. Thanks, Alan Bair SPS CAD Austin, Texas Motorola, Inc. UUCP cs.utexas.edu!oakhill!turbinia!abair
rpw3@amdcad.AMD.COM (Rob Warnock) (02/17/89)
In article <1856@turbinia.oakhill.UUCP> abair@oakhill.UUCP (Alan Bair) writes: +--------------- | The group I work for would like to be able to pipe data to FTP | instead of writting an intermediate file and then invoking FTP to | transfer the file. As far as I know you cannot directly pipe to | FTP or get it to read STDIN for the file to transfer. +--------------- Well, as a matter of fact, with Berkeley FTP you can, as least for plain ASCII files, though it's really a kludge. On a "put" command, the special filename "-" in the position of the input file says to use stdin, and on a "get" command, "-" in the position of the output file says to use stdout. Try this (setting $HOST, $USER, $PASS, and $DESTFILE as needed): $ cmd... | (echo $USER; echo $PASS; echo put - $DESTFILE) | ftp $HOST and the output of "cmd..." should end up in $DESTFILE on $HOST. Same thing works to retrieve output from a remote site: $ (echo $USER; echo $PASS; echo get $DESTFILE - ) | ftp $HOST | cmd... with the remote file showing up as the standard input of "cmd...". In fact, with some care, binary files usually work, too, but you may run into problems with error messages ending up in the stream sometimes, etc. As you say, what you really want is: +--------------- | Therefore, I would like to know if there are any c callable functions | that know the FTP protocol. +--------------- One might think about trying to take the 4.3bsd FTP client and molding it into a set of "library routines" for use inside random applications programs... but it's not really that cleanly structured inside, though some of the routines ("command()", "getreply()") are at about the right level. You could do it, but it would be a lot harder than the simple command-line client outlined below. +--------------- | If these types of rotuines do not already exist, how hard would it | be to write a set of functions to do this. We only need a limited | set of the capabilites of FTP, like only one direction, fixed record | format, minimal error correction, etc. +--------------- DISCLAIMER: The "netcp" mentioned below is an internal tool within AMD, and is not itself available (that I know of), though the general idea (an FTP client with an "rcp"-like command line) is easily duplicated. However, should you want to ask them about it, mail to "postmaster@amdcad.amd.com" directly (not to me). Several years ago, some folks here at AMD [where I am a consultant] wrote a program called "netcp" which is an FTP client, but which uses an "rcp"-like command line (you know, the "host:file" syntax). It also reads the "~/.netrc" file (if present), which makes shell-script and pipeline use extremely easy (if you are comfortable with the security implications of ".netrc"). Examples: $ netcp srchost:srcfile - # sort of an "rcat", but using FTP $ (cd gorp; tar cvf - . ) | compress | netcp -b - desthost:gorp.tar.Z $ netcp -b srchost:~ftp/pub/foo.tar.Z - | uncompress | tar xvf - Since it is merely an FTP client, it works with any standard FTP server. Thus, there are a bunch of options for doing things you would do interactively if you were running the usual FTP client, such as "-b" above for "binary" (a.k.a. "MODE IMAGE"). They started with the source to the standard Berkeley 4.3bsd FTP client, which is now freely available as one of the "liberated" files on the 4.3-Tahoe release, and you could do the same. Just doing the basics above is pretty straightforward: Whenever the original code would read an interactive command from the user, fake up the right command based on the arguments given on the command line. And watch out that you don't start proliferating options... ;-} Note that -- at least in a Unix environment -- even though "netcp" is a program rather than a set of library routines, it is still a useful tool for faking a "remote file open". Just use the Unix "popen()" function. For example, to open a file remotely and count form-feeds (minus any error-checking): FILE *remote_input; char buf[BUFSIZ], *host = "somehost"; *filename = "somefile"; int c, n = 0; sprintf(buf, "netcp -b %s:%s -", host, filename); remote_input = popen(buf, "r"); while ((c = getc(remote_input)) != EOF) { if (c == '\f') ++n; } fclose(remote_input); /* remember to gather zombie */ printf("File %s on host %s has %d form-feeds\n", filename, host, n); You can even do this with the standard FTP client (modulo the caveats above about getting error messages mixed in the stream), as many Unix implmentations allow pipelines in "popen()" arguments: remote_input = popen("echo get somefile - | ftp somehost", "r"); or, if you don't use "~/.netrc": sprintf(buf, "(echo %s; echo %s; echo get %s - ) | ftp %s", username, password, filename host); remote_input = popen(buf, "r"); while ((c = getc(remote_input)) != EOF) { ...and so on... If even that fails, you can write your own version of popen() which hands two pipes (or stdio FILEs might be better) back to you for both stdin and stdout (and stderr, for that matter), and then pump the commands "interactively" to your system's normal FTP client: FILE *fdi, *fdo, *fde; sprintf(buf, "ftp %s", host); my3popen(buf, "r", &fdi, &fdo, &fde); fprintf(fdo, "%s\n%s\nget %s -\n", username, password, filename); fclose(fdo); /* push it out to child */ while ((c = getc(fdi)) != EOF) { if (c == '\f') ++n; } fclose(fdi); ...and so on... Just a few of the many ways to hack out a solution... Rob Warnock Systems Architecture Consultant UUCP: {amdcad,fortune,sun}!redwood!rpw3 ATTmail: !rpw3 DDD: (415)572-2607 USPS: 627 26th Ave, San Mateo, CA 94403