austin@bucsf.bu.edu (Austin H. Ziegler, III) (02/03/90)
What is it? How can I get it? How is it used? I recently got an account with FTP access and read in the SIMTEL35 docs that it outputs these files in AutoFTP format.... help? austin -- +--------------------------+-----------------------+-----------------------+ | Austin Herbert Ziegler | austin@ bucsf.bu.edu | "The Hammer has | | 700 Commonealth Box 2094 | bucsb.bu.edu | fallen." | | Boston, MA 02215 | buengf.bu.edu | -- Dan Forrester | | (617) 375-8272 | engc8vc@buacca.bu.edu | _Lucifer's Hammer_ | +--------------------------+-----------------------+-----------------------+
w8sdz@WSMR-SIMTEL20.ARMY.MIL (Keith Petersen) (02/03/90)
#! /bin/sh # This is a shell archive, meaning: # 1. Remove everything above the #! /bin/sh line. # 2. Save the resulting text in a file. # 3. Execute the file with /bin/sh (not csh) to create: # README # autoftp30.sh # checkout.c # ftpget.c # nextfile.c # sample.dat # This archive created: Tue Jul 18 16:29:20 1989 export PATH; PATH=/bin:/usr/bin:$PATH echo shar: "extracting 'README'" '(13337 characters)' if test -f 'README' then echo shar: "will not over-write existing file 'README'" else sed 's/^ X//' << \SHAR_EOF > 'README' X X AutoFtp Version 3.0 X (Previously GET21) X X FreeWare X X Mingqi Deng X July 6, 1989 X X X Contents X 1) .......................................... Description X 2) .............................. Contents of Version 3.0 X 3) ............................................... Set Up X 4) ............................................ Execution X 5) .............................................. History X X X AutoFtp is a UNIX Bourne shell program that automates the ftp processes Xto get a number of files from a site that allows ftp file transfer. It Xwill repeatedly try for a good connection until all the files requested Xare transferred. A user need not baby-sit the ftp process, thus is Xrelieved from the frustration of numerous unsuccessful ftp attempts. X X The program is based on GET20 ftp script by Ferd Boundick, and was Xoriginally written for my own use after a long suffering from ftp into XSIMTEL20. It has been fully tested. This release has the following new Xfeatures: X X 1) A bug that allows a request for a non-existing file to go on X forever has been fixed. X 2) Users now can run a process of name [*]sleep[*] while running the X autoftp without having any interferences in between. This is X because the new version of autoftp uses process ID in its actions X instead of their names as in the old version. X 3) Changing local directory during ftp and getting directory list are X supported now. X 4) This versio will not affect the .netrc file. It simply avoids X using it. X 5) A randomized retry timing sequence has been implemented. In the X previous version, there is a chance that several copies of the X auto-ftp can get into synchrony. Consequently, none of them can get X logged into SIMTEL20. This release minimizes this possibility and X also smoothes the load at SIMTEL20 thus all users will have a better X chance to login to SIMTEL20 on the average. X 6) More and better error handlings are implemented. In particular, X it will complain if the ALARM is too small for a large file. X 7) The awkward run.sh file in release 2.1 has been replaced by a X simpler input file. X 8) The number of working files is reduced to 5 for the entire run X from 5 for each file requested. Moreover, there is just one X file for the entire ftp history instead one for each file X requested. X X This release is a complete rewriting of the previous version, as Xa result of my discussion with Keith Peterson at SIMTEL20. The major Xpart of the shell script has been replaced by three C programs. This Xprovides a more natural and better program structure, thus more freedom Xin analyzing each ftp session. X X This is a freeware, meaning that it can be copied and used freely. XBut the programs must NOT be sold for profit. X X Problems, improvements, comments and suggestions can be sent to the Xauthor at: X XE-mail: Postal Mail: X X deng@shire.cs.psu.edu 333 Whitmore Lab X deng@psuvaxs.bitnet Computer Science Department X deng@psuvaxs.uucp The Pennsylvania State University X University Park, PA 16802 X======================================================================== X XAll of the files contained in version 3.0 are: X X README this file X autoftp30.sh the main program X nextfile.c used by autoftp30.sh to prepare for an ftp attempt X ftpget.c used by autoftp30.sh for one ftp attempt X checkout.c used by autoftp30.sh to analyze an ftp's result X sample.dat a sample input data file for autoftp30.sh, X consisting of a name list of the requested files X to be transferred from WSMR-SIMTEL20.ARMY.MIL. X X======================================================================== X XSet Up: X X 1. Do a "man ftp" to see if the constant TENEX defined on line 49 in X nextfile.c needs to be adjusted. Refer to Note #2 in the head X comment block in nextfile.c for more instructions. X X 2. Compile the three C programs by typing: X X cc -o checkout checkout.c X cc -o ftpget ftpget.c X cc -o nextfile nextfile.c X X This will produce three executable files named "checkout", "ftpget" X and "nextfile". X X *NOTE*: Each of the programs has a "Note" section in their head X comment block that provides information about the pre- X defined constants used in the program. If anything goes X suspiciously, adjust them properly. X X 3. Place the three compiled C programs in a directory of your choice. X However, if the directory is not named "bin" and in your home X directory, you will have to adjust the shell variable FtpLibDir X in autoftp30.sh (cf. autoftp30.sh). The default is $HOME/bin. For X example, if your home directory is "/usr/usr/deng", then the X default directory is "/usr/usr/deng/bin". X X 4. Adjust the following three shell variables in autoftp30.sh: X X RemoteHost, ALARM, FtpLibDir X X Instructions and examples are given in autoftp30.sh. X X *NOTE*: You may have to change LocalHost in autoftp30.sh (on the X lines next to RemoteHost) from "guest" to your machine's X address, if login to SIMTEL20 is always denied. Refer X to autoftp30.sh. X X 5. If you have any executable files of your own of the following names: X X cat chmod cp echo expr grep mv rm sed sh sleep test X X then rename them using different names. These are UNIX commands' X names, thus should not be taken over by a user's own executables. X X======================================================================== X XExecution: X X 1. Prepare an input file that contains a name list of files you want X to get from a remote host (cf. sample.dat). The file consists of X two types of lines: device-directory line (DDL) and file-line (FL). X X a) Device-directory line (DDL): X Format: X -d|c|l device_directory_name [dir_file] X where X d,c,l are flags indicating a DDL line: X "-d" :- the name on that line is a device_directory_name X that defines the directory where a subsequently X requested file resides, at the remote host. X This flag requires one name after the option. X "-c" :- a request to change the local directory so that X requested files will be put into that directory. X The name on that line is an existing local X directory. This flag requires zero or one name X after the option. If no name is followed, then X a change to the home directory is performed. X "-l" :- a request to obtain a directory listing for the X remote directory named on that line. This flag X requires two names after the option: the name X of the remote directory, and the name of the X file to which the listing is to be stored. X Example: X -d PD1:<MSDOS.DSKUTL> X -l PD:<ANONYMOUS> root.dirlst X -c download X Function: X A "-d" DDL line defines the directory from which all files X on the subsequent file-lines, up to a new "-d" DDL line, X are to be fetched. X A "-l" DDL line gets a directory listing to aid the download. X A "-c" DDL line changes the local directory PERMANENTLY upto X the next "-c" DDL line to help organize the downloaded X file. X X *NOTE*: PS:<ANONYMOUS> directory contains general information X and many directory lists at SIMTEL20. New users are X particularly encouraged to look into this directory. X X b) File lines (FL): X Format: X [-a|b|8|t] RemoteFile [LocalFile] X where: X a,b,8,t are flags: X "-a" :- a file is to be transferred in ASCII mode X "-b" :- a file is to be transferred in binary mode X "-t" or "-8" X :- a file is to be transferred in tenex mode X (32->8 bits conversion, which is the mode X you should use if you are ftp'ing into X SIMTEL20 from a DEC VAX or a SUN) X The option can be omitted (indicated by the bracket X "[","]".) If it is omitted, the default is "-t" X (identical to "-8"). X RemoteFile is the requested file's name at the remote host X LocalFile (optional) is the file name under which the X RemoteFile is to be received at the local machine. X If it is omitted, the default is RemoteFile. X Examples: X -a 00-INDEX.TXT DSKUTL.IDX X -8 BACKEZ.ARC X DISCACHE.ARC CACHE.ARC X TIMEPARK.ARC X -a VDSK.ASM X Functions: X An FL line specifies the name of a requested file (whose X directory is specified by a preceding DDL line), and the X transfer mode for the file. X X *NOTE* 1) An option on an FL line must NOT be omitted if the X RemoteFile starts with a "-". And only the first letter X of an option is effective. X 2) Options and names are separated by blanks or TABs. X No other delimiters can be used. X 3) There can be more than one blank or TAB between the X options and names. X 4) There can be blanks or TABs before an option. But X nothing should be inserted between "-" and a flag. X 5) A DDL or FL line can appear anywhere in an input file X though some sequence may not be meaningful. X 6) The input file may contain any number of blank line X anywhere. They are simply ignored. X 7) The file name on an FL line, and directory names on a X "-d" or "-l" DDL line can be either in lower or upper X case. This makes no difference as far as autoftp30.sh is X concerned. However, all options must be in lower case. X X 2. Run the program: X There are two ways: X i) run as a background job (strongly recommended) by typing: X sh autoftp30.sh in_file & X ii) run as a foreground job (not recommended) by typing: X sh autoftp30.sh in_file X where in_file is the name of the input file prepared as above. X X If the program is run as a background job (indicated by "&" at the X end of the line, do a "man sh" for more information), then the X terminal on which this command is issued is freed right away. Thus X the user can immediately issue whatever other commands he may want. X Otherwise, the terminal is taken by the autoftp process, and the X user has to wait until its completion (upto several hours) before X he can issue any other command. X X While the program is running background, its status can be checked X by issuing "ps -xg" (do a "man ps" for more information). It can X be terminated, if the user wishes, by "kill -9 pid" where "pid" is X the process id, shown by "ps", of the very process to be terminated X (do a "man kill" for more information.) X X 3. The results are a number of files: the files received, and 6 working X files created by the execution. Among the 6 files, 5 of them only X exist during the execution. X X a) XXXXXmsg :- (XXXXX is a process ID number, as shown by a "ps" X command in UNIX.) X This is the only working file that is usually left after the X execution and the only working file the user should be concerned X with. It contains the history of the ftp attempts. Check this file X to see if each requested file is successfully transferred. A file X transfer fails only when it is too large or has a wrong name, X which will be reported in this file. (If your local system goes X down, or the process is killed by a "kill" command, the transfer X will of course be interrupted and no message will be reported in X these cases.) X X b) XXXXXstdout, XXXXXstderr, XXXXXftp.script, XXXXXtmp :- X (XXXXX is again a process ID number, as shown by a "ps" X command in UNIX.) X These files are temporary working files. XXXXXtmp only exists X momentarily. If the autoftp30.sh terminates gracefully, they will X all be erased upon its completion. Otherwise, the user can always X delete them manually (this happens if the autoftp30.sh is killed X by a kill command of UNIX, the local machine went down in the X middle of the execution, or due to a situation unanticipated by X the AutoFtp.) X X======================================================================== X XHistory: X X Version 2.1 (GET21), by Mingqi Deng, Jan. 12, 1989 X -------------------------------------------------- X X 1. Unlike Get20, Get21 will not give up when connection is refused X or disrupted. Therefore the users will be relieved from the X frustration of running it over and over again for a good X connection. X 2. To get out from a silent connection, there is a time out for X each connection. The default is one hour. For large files or X slow communication lines, the user is advised to increase the X value. X 3. A separate shell program sample is included for transferring X multiple files from SIMTEL20. X X----------------------------------------------------------------------- X XVersion 2.0 (GET20) by : Ferd Boundick X--------------------------------------- X X date written: some time in 1983 X modification: 22 March 84 X X 1. ftp modes are referred to as "ascii, binary, and tenex" X 2. The option -8 is identical to -t for tenex (36->8) transfers. X 3. Remote host name now is stored in the variable "RemoteHost". X 4. Local host name now is stored in the variable "LocalHost" for X remote login usage. SHAR_EOF if test 13337 -ne "`wc -c < 'README'`" then echo shar: "error transmitting 'README'" '(should have been 13337 characters)' fi fi echo shar: "extracting 'autoftp30.sh'" '(5769 characters)' if test -f 'autoftp30.sh' then echo shar: "will not over-write existing file 'autoftp30.sh'" else sed 's/^ X//' << \SHAR_EOF > 'autoftp30.sh' X#! /bin/sh X# X# AutoFtp Version 3.0, by Mingqi Deng, July 6, 1989 X# ------------------------------------------------- X# X# Refer to file README for instructions to run the program X X# Set-up: X# X# 1. Set the following 3 parameters in this program on lines 23-62: X# X# ALARM, RemoteHost, FtpLibDir X# X# Instructions and examples are given as comments in the following. X# X# 2. Make sure that you do NOT have any executable files of the same X# name as the following UNIX commands' names: X# X# cat chmod cp echo expr grep mv rm sh sleep sed test X# X# Otherwise, rename your files to different names. X X# An ftp attempt will be aborted after ALARM many seconds. It must at X# least be equal to 300. The default # is 3600, eg., ALARM=3600 (do X# not leave blanks between '=' and '3600'!) X XALARM=3600 X X# Define host names: X# Use one of the following addresses for simtel20 archive site for the X# parameter RemoteHost below: X# "26.2.0.74" X# "WSMR-SIMTEL20.ARMY.MIL" X# "SIMTEL20.ARPA" X# The first one is highly recommended if it works, since it is the X# most time-saving address to use. The second is the official name. The X# third is an alias, thus least recommended. However, you may find only X# one of them works for you, depending on how your UNIX installation X# handles the addresses. One symptom is a quickly failed ftp (less than X# 5 seconds), which can be noticed by the ps command of UNIX. X# X# Also note that it has been noticed that some system does not recognize X# the address in upper case. Therefore, if none of the above three X# works for you, change the names to lower case and try again. X# X# LocalHost is set to "guest". If you find it not working, replace X# it with the host name of your machine on which this program is run. X# eg., RemoteHost="26.2.0.74" X# LocalHost="shire.cs.psu.edu" X XRemoteHost="26.2.0.74" XLocalHost="guest" X X# Set the path name for the directory where three compiled C programs X# ftpget.c, nextfile.c and checkout.c are. This program assumes as X# default that the three compiled programs are in $HOME/bin (a directory X# "bin" in your home directory). If they are to be put in a different X# directory, say, lib in your home directory, you should set FtpLibDir X# as: X# FtpLibDir="$HOME/lib" X# Otherwise, just leave 'FtpLibDir=""' on next line as it is. XFtpLibDir="" X X# X#-------AUTOFTP30.SH Set-up Stops Here: No further set-up needed------ X# X Xtest 1$FtpLibDir = 1 && FtpLibDir=$HOME/bin X_ftpget="$FtpLibDir/ftpget" X_checkout="$FtpLibDir/checkout" X_nextfile="$FtpLibDir/nextfile" X Xtest -s "$_ftpget" || X { echo "***Cannot find file '$_ftpget'!" X echo " Please compile 'ftpget.c', name the compiled program as" X echo " 'ftpget' and place it in your '$FtpLibDir' directory." ; X exit 2 ; } Xtest -s "$_checkout" || X { echo "***Cannot find file '$_checkout'!" X echo " Please compile 'checkout.c', name the compiled program as" X echo " 'checkout' and place it in your '$FtpLibDir' directory." ; X exit 2 ; } Xtest -s "$_nextfile" || X { echo "***Cannot find file '$_nextfile'!" X echo " Please compile 'nextfile.c', name the compiled program as" X echo " 'nextfile' and place it in your '$FtpLibDir' directory." ; X exit 2 ; } X Xtest $# != 1 && X { echo "***Usage: sh autoftp30.sh in_file" ; exit 2; } X Xexec 1>$$stdout 2>$$stderr X X# Try FTP into simtel20 SLEEP many seconds until being connected X# SLEEP will be increased and decreased randomly for subsequent tries. XSLEEP=60 X X# save messages Xcat $$stdout $$stderr > $$msg X X#initialize the loop XWS=`expr \( $SLEEP \* 47 + 31 \) \% 89 + 50` X Xctr=1 Xcp $1 $$input Xretry=no X X# Repeat if there is more request in the input file or if the last X# attempt failed and a retry can be performed. Xwhile { test -s $$input || test $retry = yes ; } Xdo X X > $$stderr X > $$stdout X X# If not a re-try for a requested file, then fetch next requested X# file name. If the fetch fails, then quit autoftp30.sh. X test $retry = no && X { echo "%%%%%%% Process File Request #$ctr %%%%%%%" >> $$msg X ctr=`expr $ctr + 1` X $_nextfile $$input $RemoteHost anonymous $LocalHost $$ftp.script >> $$msg || X { status=92; break;} X } X X# Attempt to get the file. If not able to make an attempt, then quit. X# Note that exit status=99 means there is no alarm call. X chmod og-rx $$ftp.script X $_ftpget $ALARM $$ftp.script || X { test $? = 99 && # alarm called, do not quit yet, $_checkout X # will respond X { status=93; break; } ; } X X# Check the result of the transfer. If a retry is not to be made, then X# set "retry" to no, otherwise set it to yes. X $_checkout $$stdout $$stderr $$ftp.script $$tmp >> $$msg X status=$? X { test $status = 99 && { status=94; break ; } ; } || X { test $status = 0 && retry=no ; } || retry=yes X X# if not the last request, i.e., $$input is not empty or retry=yes, X# then wait for a while before next attempt. Otherwise done!!! X test -s $$input || test $retry = yes || { status=95; break ; } X X# save the messages for only the last attempt on a request X test $retry = no && cat $$tmp >> $$msg X rm $$tmp X X sleep $WS X WS=`expr \( $WS \* 13 \/ 10 \) \% 900 + 60` X Xdone X Xtest $status -ge 92 && X { test $status -le 93 && cat $$stderr >> $$msg || X { cat $$tmp >> $$msg ; rm $$tmp ; } # status=94,95 X } X Xcase $status in X 92) echo "*******Fatal error: when preparing an ftp attempt!*******" >> $$msg;; X 93) echo "*******Fatal error: during an ftp attempt!*******" >> $$msg;; X 94) echo "*******Fatal error: when checking last ftp result!*******" >> $$msg;; X 95) echo "###################Execution Completed###################" >> $$msg;; X *) ;; Xesac X X#remove ftp command and working files Xrm $$ftp.script $$stderr $$stdout $$input X Xexit 0 SHAR_EOF if test 5769 -ne "`wc -c < 'autoftp30.sh'`" then echo shar: "error transmitting 'autoftp30.sh'" '(should have been 5769 characters)' fi chmod +x 'autoftp30.sh' fi echo shar: "extracting 'checkout.c'" '(10184 characters)' if test -f 'checkout.c' then echo shar: "will not over-write existing file 'checkout.c'" else sed 's/^ X//' << \SHAR_EOF > 'checkout.c' X/* X * checkout.c by Mingqi Deng, July 6, 1989 X * X * checks the result of an ftp attempt made by autoftp30.sh (cf. X * ftpget.c) to get a file from a remote host. X * X * Compile: X * cc -o checkout checkout.c X * Execute: X * checkout f_stdout f_stderr ftp_script f_tmp X * where: X * f_stdout :- the standard output file used by ftpget.c X * f_stderr :- the standard error-message file used by ftpget.c X * ftp_script :- the ftp command script file created by nextfile(.c) X * f_tmp :- a temporary file for deleting null chars in X * f_stderr and f_stdout, also used as stderr by X * checkout(.c) for most of its error messages. X * X * Exit status: X * 99 : fatal error, quit this session of autoftp30.sh X * 0 : finish an attempt, no further attempt should be made X * on this requested file (either file successfully X * received, ALARM in autoftp30.sh too small, or a wrong X * file name) X * 2 : a failed attempt that should be retried X * X * Example: X * checkout 1234stdout 1234stderr 1234ftp.script 1234tmp X * X * Note : X * Two constants max_line_length and max_line defined on the X * "#define" lines below specify the maximum length of lines in X * f_stderr and f_stdout, and the maximum number of lines in them X * altogether.In case they are not large enough (your will get a X * message saying so), increase them. X */ X X#include <stdio.h> X#include <sys/types.h> X#include <sys/stat.h> X#include <sys/timeb.h> X X#define max_line_length 160 X#define max_line 400 X Xchar hold[max_line][max_line_length+1]; Xint ctr, /* number of lines in f_stderr and f_stdout */ X letter; /* number of letters in f_stderr and f_stdout */ XFILE *f_tmp; X Xmain(argc, argv) Xint argc; Xchar *argv[]; X{ X int flag,ftp_get_file; X char cmd[80],s[80],s1[80],s2[80],*rfile,*lfile,*sprintf(),*malloc(); X FILE *f_err, *f_out, *f_ftp; X struct stat *buf; X time_t tloc,timeofday; X X f_tmp = fopen(argv[4],"w"); X if (argc != 5) { X fprintf(stderr,"***Usage: checkout stdout stderr ftp_script f_tmp\n"); X exit(99); X } X f_out = fopen(argv[1],"r"); X f_err = fopen(argv[2],"r"); X f_ftp = fopen(argv[3],"r"); X if (f_err == NULL || f_out == NULL || f_ftp == NULL) X { fprintf(f_tmp,"***File(s) '%s', '%s' or '%s' do not exist\n", X argv[1],argv[2],argv[3]); X exit(99);} X X ctr=0; letter=0; /* read in the stdout file of the ftp session*/ X while ((ctr<max_line) && (getline(hold[ctr],f_out,f_tmp)!=0)) ctr++; X if (ctr == max_line) { X fprintf(f_tmp,"%s\n%s\n%s\n", X "***Increase the constant 'max_line' on the '#define' line", X " in file checkout.c, recompile it, then try again!"); X exit(99); X } X X /* read in the stderr file of the ftp session*/ X while ((ctr<max_line) && (getline(hold[ctr],f_err,f_tmp)!=0)) ctr++; X if (ctr == max_line) { X fprintf(f_tmp,"%s\n%s\n%s\n", X "***Increase the constant 'max_line' on the '#define' line", X " in file checkout.c, recompile it, then try again!"); X exit(99); X } X if (letter==0) /* empty file: a silent connection, try again */ X { printf("A silent connection, will try again.\n"); exit(2);} X X flag = detect("unknown host"); X flag = (flag)? 1: detect("Unknown host"); X if (flag) { X /* wrong remote host name, quit attempt for this run */ X fprintf(f_tmp,"***host unknown: check RemoteHost in autoftp30.sh!\n"); X exit(99); X } X X /* get the requested file names from the ftp script file X if a 'get' is performed. */ X ftp_get_file=0; X while( fgets(s,max_line_length,f_ftp) != NULL) X if (strncmp(s,"get",3) == 0) { X extract3(s,s1,s1,s2); X if (s1[0]!='\0') { rfile = s1; } X else { X fprintf(f_tmp,"***Invalid ftp_script file '%s'\n",argv[3]); X exit(99); X } X if (s2[0]!='\0') lfile=s2; X else lfile=s1; X ftp_get_file=1; X } X X flag = detect("Invalid use of terminal designator"); X flag = (flag)? 1:detect("File not accessible"); X flag = (flag)? 1:detect("File not accessable"); X flag = (flag)? 1:detect("file not found"); X flag = (flag)? 1:detect("File not found"); X flag = (flag)? 1:detect("not found"); X flag = (flag)? 1:detect("No such file"); X flag = (flag)? 1:detect("No such directory"); X X if (flag) { X /* wrong remote file name, quit attempt for this file */ X if (ftp_get_file) X fprintf(f_tmp, X "***Remote file/direcortory %s not found!\n",rfile); X else X fprintf(f_tmp, "***Remote direcortory not found!\n"); X exit(0); /* abort the current request */ X } X X flag = detect("refused"); X flag = (flag)? 1: detect("unreachable"); X flag = (flag)? 1: detect("failed"); X flag = (flag)? 1: detect("Connection timed out"); X flag = (flag)? 1: detect("timed out"); X flag = (flag)? 1: detect("Not connected"); X flag = (flag)? 1: detect("not connected"); X flag = (flag)? 1: detect("not accessable"); X flag = (flag)? 1: detect("not accessible"); X flag = (flag)? 1: detect("not available"); X flag = (flag)? 1: detect("time out"); X flag = (flag)? 1: detect("times out"); X X if (flag) { /* a bad connection, try again */ X printf("A bad connection, will try again.\n"); X /* remove the incomplete file if it exists */ X sprintf(cmd,"test -f '%s' && rm '%s'",lfile,lfile); X system(cmd); X exit(2); /* try again */ X } X X flag = detect("Transfer complete"); X flag = (flag)? 1:detect("received"); X if (flag != 0) { X if (ftp_get_file) { X fprintf(f_tmp,"=======File '%s'\n",rfile); X fprintf(f_tmp, X " successfully received as '%s'.=======\n",lfile); X } else X fprintf(f_tmp,"=======Transfer successfully completed.\n"); X exit(0); /* done for this RemoteFile */ X } else { X flag = detect("Alarm call"); X if (ftp_get_file==0) { X printf(" A silented connection. Will try again.\n"); X exit(2); X } X X /* then check if too large a requested file */ X buf=(struct stat *)malloc(sizeof(struct stat)); X stat(lfile,buf); /* get GMT time in seconds (from 12/31/1969 X 19:00:00) for the file being received X since last modification. If the file X does not exist, 0 is returned. */ X timeofday = time(&tloc); /* get current time */ X /* if the file was modified in last 5 minutes*/ X if (timeofday - buf->st_mtime < 300) { X if (flag == 0) fprintf(f_tmp, X "***I am confused. But most likely the chance is:\n"); X fprintf(f_tmp, "%s\n%s%s%s%s%s\n%s%s%s\n", X "***'alarm' in autoftp30.sh appears too small for remote file", X " '",rfile,"' (to be received as '",lfile,"').", X " File '",lfile,"' removed, increase the 'alarm' and try again."); X /* remove the incomplete file (it must exist X since the difference is < 300) */ X sprintf(cmd,"rm '%s'",lfile); X system(cmd); X exit(0); /* abort this request, do not try any more. */ X } else { X if (flag == 0) X printf("***I am confused. But most likely the chance is:\n"); X printf(" A silented connection. Will try again.\n"); X /* remove the file if it exists. */ X sprintf(cmd,"test -f '%s' && rm '%s'",lfile,lfile); X system(cmd); X exit(2); /* do not abort this request, try again. */ X } X } X} X X/* detect(s) X * returns 1 if string s is in the file f_err and f_out that have been X * read into array hold. X */ Xdetect(s) Xchar *s; X{ X int i,flag; X X for (i=0; i<ctr && ((flag=substring(hold[i],s))==0); i++); X return(flag); X} X X/* substring: X * returns 1 if s2 is a substring of s1, 0 otherwise. X */ Xsubstring(s1,s2) Xchar *s1,*s2; X{ X int i,l1,l2,found; X X l1 = strlen(s1); X l2 = strlen(s2); X X found = 1; X for (i=0; i<=l1-l2 && found!=0; i++) X found = strncmp(s1+i,s2,l2); X X return(found==0); X} X X/* getline(s,fin,fout) X* reads an input line from file fin, and skips null chars, count X* the number of letters in the input file (result kept in global var X* letter). All non-null characters are written to fout to eliminate X* numerous annoying null character (inserted by ftp?) that causes very X* slow scrolling when using MORE of UNIX to display the file fin. X* return 0 if eof. X*/ Xgetline(s,fin,fout) Xchar s[]; XFILE *fin,*fout; X{ X char c; X int i,cc; /* cc used to avoid machine dependent comparison X of getc()'s return value to EOF */ X X i = 0; X while( ((cc=getc(fin)) != EOF) && i <= max_line_length) { X c=cc; X if (c=='\n') X { s[i]='\0'; putc(c,fout); return(1); } X else X if (c != '\0') { X putc(c,fout); X if (c > 64 && c < 91 || c > 96 && c < 123 ) X { letter++; s[i++]=c; } X else X if (c==' ') s[i++]=c; X } X } X X if (i>max_line_length) X { fprintf(f_tmp,"%s\n%s\n", X "***'max_line_length' in checkout.c appears too small.", X " Please increase it, recompile and try again!"); X exit(99); X } X X s[i] = '\0'; X return(0); X} X X/* extract3(s,s0,s1,s2) X* extracts 3 substring from s to s0, s1 and s2, where X* blanks, tabs are delimiters, new line character is ignored. S is X* terminated by a null character as usual, so will s0, s1 and s3. X*/ Xextract3(s,s0,s1,s2) Xchar s[],s0[],s1[],s2[]; X{ X int i,head,tail,ptr,l; X X l = strlen(s); tail=0; X for (i=0;i<3;i++) { X ptr=0; X head=tail; X while(s[head]==' ' || /* skip blanks, tabs and new line */ X s[head]==9 || X s[head]=='\n') head++; X tail=head; X X while(tail<l && X s[tail]!=' '&& /* stop if a blank, tab or NL is seen */ X s[tail]!=9 && X s[tail]!='\n') X switch(i) { X case 0: s0[ptr++]=s[tail++]; break; X case 1: s1[ptr++]=s[tail++]; break; X case 2: s2[ptr++]=s[tail++]; break; X } X switch(i) { X case 0: s0[ptr]='\0'; break; X case 1: s1[ptr]='\0'; break; X case 2: s2[ptr]='\0'; break; X } X } X} X SHAR_EOF if test 10184 -ne "`wc -c < 'checkout.c'`" then echo shar: "error transmitting 'checkout.c'" '(should have been 10184 characters)' fi fi echo shar: "extracting 'ftpget.c'" '(2525 characters)' if test -f 'ftpget.c' then echo shar: "will not over-write existing file 'ftpget.c'" else sed 's/^ X//' << \SHAR_EOF > 'ftpget.c' X/* X * ftpget.c by Mingqi Deng, July 6, 1989 X * X * makes one attempt to get a file from a remote host. X * X * Compile: X * cc -o ftpget ftpget.c X * Execute: X * ftpget alarm remotehost ftp_script X * where X * alarm :- the number in seconds that an ftp attempt can last X * ftp_script :- ftp command script created by nextfile.c X * X * Example: X * ftpget 3600 wsmr-simtel20.army.mil 123ftp_script X * Note : X * 1. The maximum number of characters contained on ftpget's command X * line is defined in the constant ftpget_cmd_line_length. The X * default is 160. X * 2. The host name is assumed to have no more than 80 characters. X * Adjust the constant hostname_length if necessary. X */ X X#include <stdio.h> X#include <signal.h> X Xint id,pgid,mask; X X#define ftpget_cmd_line_length 160 X#define hostname_length 80 X Xmain(argc, argv) Xint argc; Xchar *argv[]; X{ X unsigned t_alarm; X char cmd[ftpget_cmd_line_length+1]; X void handler(); X FILE *f; X X if (argc != 3) { X fprintf(stderr,"***Type 'ftpget alarm ftp.script' to run\n"); X exit(99); X } X /* check if file ftp_script exists. */ X f = fopen(argv[2],"r"); X if (f == NULL) { X fprintf(stderr,"***Ftp script file '%s' does not exist.\n",argv[2]); X exit(99); X } X fclose(f); X X sscanf(argv[1],"%u",&t_alarm); X if (t_alarm < 300) { X fprintf(stderr,"***'alarm'(= %d < 300) in autoftp30.sh too small!\n",t_alarm); X exit(99); X } X X mask=sigsetmask(0); /* do not block any signals */ X id=getpid(); pgid=getpgrp(id); /* get process group id */ X setpgrp(id,pgid+1); /* modify process group id so that only this X and all of its subprocesses (with this id) X can be terminated by a signal */ X signal(SIGALRM,handler); /* catch the alarm signal and use procedure X "handler" to process it */ X /* create a command "ftp remotehost < ftp_script" to X be run in current shell (instead of a subshell)*/ X sprintf(cmd,"exec ftp -n < '%s'",argv[2]); X X alarm(t_alarm); /* timing the system call next: send a SIGALRM X signal to the current process after t_alarm X many seconds */ X system(cmd); /* execute "ftp remotehost < ftp_script" */ X alarm(0); /* stop the alarm if "cmd" ends before the alarm X call*/ X} X X/* SIGALRM signal handling */ Xvoid handler() X{ X fprintf(stderr,"Alarm call!\n"); X killpg(pgid+1,SIGHUP); /* kill the current process and its X subprocess (ftp ....) */ X exit(5); X} SHAR_EOF if test 2525 -ne "`wc -c < 'ftpget.c'`" then echo shar: "error transmitting 'ftpget.c'" '(should have been 2525 characters)' fi fi echo shar: "extracting 'nextfile.c'" '(12936 characters)' if test -f 'nextfile.c' then echo shar: "will not over-write existing file 'nextfile.c'" else sed 's/^ X//' << \SHAR_EOF > 'nextfile.c' X/* X * nextfile.c by Mingqi Deng, July 6, 1989 X * X * get next requested file from an input file and prepare an ftp X * script file for ftpget.c (cf. ftpget.c) The requested file X * will be deleted from the input file. X * X * Compile: X * cc -o nextfile nextfile.c X * Execute: X * nextfile in_file RemoteHost login password ftp_script X * where X * in_file :- input file to autoftp30.sh (cf. README) that consists X * of two types of lines: X * a) Device-directory line (DDL): X * Format: X * -d|c|l device_directory_name [dir_file] X * where X * "-d" :- the name on that line is a device_directory_name X * that defines the directory where a subsequently X * requested file resides, at the remote host. X * This flag requires one name after the option. X * "-c" :- a request to change the local directory so that X * requested files will be put into that directory. X * The name on that line is an existing local X * directory. This flag requires one name after X * the option, too. X * "-l" :- a request to obtain a directory listing for the X * remote directory named on that line. This flag X * requires two names after the option: the name X * of the remote directory, and the name of the X * file to which the listing is to be stored. X * Example: X * -d PD1:<MSDOS.DSKUTL> X * -l PD:<ANONYMOUS> root.dirlst X * -c download X * Function: X * A "-d" DDL line defines the directory from which all X * files on the subsequent file-lines, up to a new "-d" X * DDL line, are to be fetched. X * A "-l" DDL line gets a directory listing. X * A "-c" DDL line changes the local directory PERMANANTLY X * upto next "-c" DDL line. X * X * b) File lines (FL): X * Format: X * [-a|b|8|t] RemoteFile [LocalFile] X * where: X * a,b,8,t are flags of options: X * "-a" :- a file is to be transferred in ASCII mode X * "-b" :- a file is to be transferred in binary mode X * "-t" or "-8" X * :- a file is to be transferred in tenex mode X * (32->8 bits conversion, which is the mode X * you should use if you are ftp'ing into X * SIMTEL20 from a DEC VAX or a SUN) X * If the option is omitted, the default is "-t". X * RemoteFile is the requested file's name at the remote host X * LocalFile (optional) is the file name under which the X * RemoteFile is to be received at the local machine. X * If it is omitted, the default is RemoteFile. X * Examples: X * -a 00-INDEX.TXT DSKUTL.IDX X * -8 BACKEZ.ARC X * DISCACHE.ARC CACHE.ARC X * TIMEPARK.ARC X * -a VDSK.ASM X * Functions: X * An FL line specifies the name of a requested file X * (whose directory is specified by a preceding DDL line), X * and the transfer mode for the file. X * *NOTE* X * 1) An option on an FL line must NOT be omitted if the X * RemoteFile starts with a "-". And only the first letter X * of an option is effective. X * 2) Options and names are separated by blanks or TABs. X * No other delimiters can be used. X * 3) There can be more than one blank or TAB between the X * options and names. X * 4) There can be blanks or TABs before an option. But X * nothing should be inserted between "-" and a flag. X * 5) A DDL or FL line can appear anywhere in an input file X * though some sequence may not be meaningful. X * 6) The input file may contain any number of blank line X * anywhere. They are simply ignored. X * 7) The file name on an FL line, and directory names on a X * "-d" or "-l" DDL line can be either in lower or upper X * case. This makes no difference as far as autoftp30.sh is X * concerned. However, all options must be in lower case. X * X * RemoteHost:- the host name of the anonymous ftp session X * login :- the login id for the anonymous ftp session X * password :- the password for the anonymous ftp session X * ftp_script:- the output file that consists of ftp commands X * X * Example: X * nextfile 123input WSMR-SIMTEL20.ARMY.MIL anonymous guest 123ftp.script X * Note : X * 1. The maximum number of characters contained on each line in X * file in_file (cf. sample.dat) is no more than max_line_length. X * The default is 80. Adjust it if necessary. X * 2. The constant TENEX in the second "#define" may need adjusting. X * Instead of being '"tenex"', it may have to be '"type L 8"', X * '"type local"' or '"type local byte"'. Do a "man ftp" to see X * how your system wants it. X */ X X#include <stdio.h> X#include <string.h> X X#define max_line_length 80 X#define TENEX "tenex" X Xmain(argc, argv) Xint argc; Xchar *argv[]; X{ X int no_more, /* 1 if end of file */ X dir_path, /* 1 if a DDL line of -d option seen */ X chdir_req, /* 1 if an lcd request on current DDL*/ X not_done, /* contol while loop */ X to_chdir, /* 1 if a pending lcd request */ X chdir_flg, /* 1 if an lcd request seen */ X action; /* 1 if usful request (dir, get) seen */ X char cmd[80],type[10],*dummy=NULL, X op_flg, option(), X s1[max_line_length+1], X s2[max_line_length+1], X chdir_name[max_line_length+1],/* local dir name to chage to */ X dir1[max_line_length+1], /* remote dir name to get dir list */ X dir2[max_line_length+1], /* file to get remote dir list */ X dpath[max_line_length+1]; /* the directory path in DDL line*/ X FILE *fin,*fftp; X X if (argc != 6) { X fprintf(stderr, X "***Usage: nextfile input RemoteHost login password ftp.script\n"); X exit(3); X } X if ( (fin = fopen(argv[1],"r")) == NULL) { X fprintf(stderr,"***The input file does not exist!\n"); exit(3); X } X X /* Create an ftp command file which will be executed once X ftp starts up. This is true no matter whether a login X failed or not. A quit command is always issued. */ X fftp = fopen(argv[5],"w"); X fprintf(fftp,"open %s\n",argv[2]); X fprintf(fftp,"user %s %s\n",argv[3],argv[4]); X fprintf(fftp,"type ascii\n"); X fprintf(fftp,"verbose\n"); X X action=0; dir_path=0; to_chdir=0; chdir_flg=0; chdir_flg=0; X not_done=1; X while (not_done) { X /* process a sequence of DDL lines with -c option */ X no_more=lcd(fin,chdir_name,&chdir_req,&op_flg,s1,s2); X if (chdir_req) { to_chdir = 1; /* delay issuing lcd command, since X only the last one is effective */ X chdir_flg = 1; /* neet to remember the lcd for X subsequent file transfer */ X } X X if (no_more) { X printf("=======Last request .......\n"); X if (action==0) X printf( "***Warning: No files requested!\n"); X fprintf(fftp,"bye\n"); X fclose(fftp); X update_fin(fin,argv[1],dummy,0,dummy); X exit(0); X } X /* if not eof, is it a DDL or FL line? */ X if (op_flg == 'l') { X if (s1[0] == '\0' || s2[0] == '\0') { X fprintf(stderr, X "***Invalid '-l' DDL line in the input file!\n"); X exit(3); X } X if (to_chdir) X { fprintf(fftp,"lcd %s\n",chdir_name); to_chdir=0; } X fprintf(fftp,"cd %s\n",s1); /* note that ascii mode was set */ X fprintf(fftp,"dir *.* %s\n",s2); X action=1; X } else X if (op_flg == 'd') { X if (s1[0]=='\0') { X fprintf(stderr, X "***Invalid device-directory-line(DDL) in the input file!\n"); X exit(3); X } X dir_path=1; X strcpy(dpath,s1); X } else { /* not a -c, -l, or -d line, must be an FL line */ X if (dir_path==0) { /* no -d DDL line seen yet */ X fprintf(stderr, X "***Device-directory line missing in the input file!\n"); X exit(3); X } X not_done = 0; X } X } /* of while */ X if (to_chdir) /* delayed lcd: only the last lcd is effective*/ X fprintf(fftp,"lcd %s\n",chdir_name); X X /******* process an FL line *****/ X if (s1[0]=='\0') { X fprintf(stderr,"***The requested file name is blank!\n"); X exit(3); X } X if (s2[0]=='\0') strcpy(s2,s1); X X /* set the file transfer mode */ X if (op_flg == 'b') strcpy(type,"binary"); X if (op_flg == 't') strcpy(type,TENEX); X if (op_flg == '8') strcpy(type,TENEX); X X printf("=======Start requesting file %s .........\n",s1); X /* type ascii has been set at the beginning */ X if (op_flg != 'a') fprintf(fftp,"type %s\n", type); X fprintf(fftp,"get %s%s %s\nbye\n",dpath,s1,s2); X fclose(fftp); X X update_fin(fin,argv[1],dpath,chdir_flg,chdir_name); X} X X/* gets next non-blank line from file fin, return 1 if EOF */ Xnextline(s,f) Xchar *s; XFILE *f; X{ X char *p; X X while( (p=fgets(s,max_line_length,f)) != NULL) X if (nonblank(s)) break; X return(p==NULL); X} X X/* nonblank(s) X* returns true (non-zero) if s contains non-blank chars. X*/ Xnonblank(s) Xchar *s; X{ X int i,flag; X X if (s==NULL) return(0); X i = 0; flag = 0; X for (i=0; i<strlen(s) && flag==0; i++) X flag = (s[i]==' ' || s[i]==9 || s[i]=='\n' )? 0:1; /* 9 is tab*/ X return(flag); X} X X/* extract(s,num,s0,s1,s2) X* extracts num (1-3) many substring from s to s0, s1 and s2, where X* blanks, tabs are delimiters, new line character is ignored. S is X* terminated by a null character as usual, so will s0, s1 and s3. X* Depending on s, any of s0, s1 or s2 can be an empty string. X*/ Xextract(s,num,s0,s1,s2) Xchar s[],s0[],s1[],s2[]; Xint num; X{ X int i,head,tail,ptr,l; X X l = strlen(s); tail=0; X for (i=0;i<num;i++) { X ptr=0; X head=tail; X while(s[head]==' ' || /* skip blanks, tabs and new line X ('\0' is after '\n') */ X s[head]==9 || X s[head]=='\n') head++; X tail=head; X X while(tail<l && /* stop if end of the string s */ X s[tail]!=' '&& /* stop if a blank, tab or NL is seen */ X s[tail]!=9 && X s[tail]!='\n') X switch(i) { X case 0: s0[ptr++]=s[tail++]; break; X case 1: s1[ptr++]=s[tail++]; break; X case 2: s2[ptr++]=s[tail++]; break; X } X switch(i) { X case 0: s0[ptr]='\0'; break; X case 1: s1[ptr]='\0'; break; X case 2: s2[ptr]='\0'; break; X } X } X} X X/* X * extract options of a non-blank input line, detects invalid options, X * and will "insert" '-t' option to a default FL line. When returning, X * s2 and s3 are set to the 2nd and 3rd string on the line. X */ Xchar option(s,n,s2,s3) Xchar *s,*s2,*s3; Xint n; X{ X char s1[max_line_length+1]; X X extract(s,n,s1,s2,s3); X if (s1[0] != '-') { /* default FL line option (-t) */ X strcpy(s3,s2); /* re-set s2 and s3 */ X strcpy(s2,s1); X return('t'); X } X if (s1[1] == '8') return('8'); X if (s1[1] == 'a') return('a'); X if (s1[1] == 'b') return('b'); X if (s1[1] == 'c') return('c'); X if (s1[1] == 'd') return('d'); X if (s1[1] == 'l') return('l'); X if (s1[1] == 't') return('t'); X /* unrecognizable option */ X fprintf(stderr, "***Invalid option in the input file!\n"); X exit(3); X} X X/* X * process a sequence of DDL lines with -c flags. note that only X * last -c line retains its effect. X */ Xlcd(f,chdir_name,chdir_req,op_flg,s1,s2) Xint *chdir_req; Xchar *op_flg,*chdir_name,*s1,*s2; XFILE *f; X{ X int chdir_line,flg; X char c,s0[max_line_length+1]; X X *chdir_req=0; X X chdir_line = 1; X while (chdir_line) { X X flg=nextline(s0,f); X if (flg) return(1); /* eof encountered, s0 is NULL */ X X c=option(s0,3,s1,s2); X if (c=='c') { *chdir_req=1; strcpy(chdir_name,s1); } X else chdir_line = 0; X } X *op_flg = c; X return(0); X} X X/* X * update the input file to delete the processed requests X */ Xupdate_fin(fin,nfin,dpath,chdir_flg,chdir_name) Xint chdir_flg; /* 1 if an lcd opertion was performed. */ Xchar *dpath, /* directory path in the last DDL (-d) line */ X *nfin, /* name of the input file */ X *chdir_name; /* name of the local directory changed to */ XFILE *fin; /* file pointer for the input file */ X{ X char name[10],s[max_line_length+1]; X int pid; X FILE *ftmp; X X pid=getpid(); X sprintf(name,"%d%tmp",pid); X ftmp = fopen(name,"w"); X /* save the DDL line if there is more FL in the input*/ X if ( nextline(s,fin) == 0 ) { /* if more non-blank lines */ X if (chdir_flg) X fprintf(ftmp,"-c %s\n",chdir_name); X fprintf(ftmp, "-d %s\n",dpath); X fputs(s,ftmp); X while ( fgets(s,max_line_length,fin) != NULL) fputs(s,ftmp); X } X fclose(fin); X fclose(ftmp); X X /* replace in_file by the temporary file (maybe empty) */ X sprintf(s,"rm '%s'",nfin); X system(s); X /* rename the tmp_file to in_file */ X sprintf(s,"mv '%s' '%s'",name,nfin); X system(s); X} SHAR_EOF if test 12936 -ne "`wc -c < 'nextfile.c'`" then echo shar: "error transmitting 'nextfile.c'" '(should have been 12936 characters)' fi fi echo shar: "extracting 'sample.dat'" '(262 characters)' if test -f 'sample.dat' then echo shar: "will not over-write existing file 'sample.dat'" else sed 's/^ X//' << \SHAR_EOF > 'sample.dat' X-l PS:<ANONYMOUS> root.dirlst X-d PD1:<MSDOS.DSKUTL> X-a 00-INDEX.TXT DSKUTL.IDX X-8 DISCACHE.ARC CACHE.ARC X TIMEPARK.ARC X-a VDSK.ASM X-d PD1:<MSDOS.SYSUTL> X-a 00-INDEX.TXT SYSUTL.IDX X BENCH410.ARC X-a BOOTCODE.ASM X CNFGTXT.ARC X DOS33HLP.ARC X DOS33PAT.ARC SHAR_EOF if test 262 -ne "`wc -c < 'sample.dat'`" then echo shar: "error transmitting 'sample.dat'" '(should have been 262 characters)' fi fi exit 0 # End of shell archive
mikek@col.hp.com (Mike Karin) (02/06/90)
Ok, I tried using the autoftp that is contained in this basenote and I saw a behavior that I thought was a little strange. After each file, autoftp logs off! This means that it has to try to log on again for each file. After trying for a long time to get into SIMTEL, doesn't it make sense to transfer *ALL* of the desired files instead of just one? I don't understand the rationale here or maybe something is broken. Mike Karin Hewlett-Packard Co Colorado Springs Division mikek@hp-col.col.hp.com
austin@bucsf.bu.edu (Austin H. Ziegler, III) (02/08/90)
Speaking of AutoFTP, being the one who originally requested it as of late, I compiled the C programs and then used SIMTEL35 to create a script file, uploaded it to the UNIX machine, removed the "^M"'s made in MS-DOS, and then 'sh autoftp.sh qmodem.scr' to get the files I didn't have from the PD2:<MSDOS2.QMODEM> directory. It simply would not execute the ftp (and I had a ~/bin directory) and the message files were filled with garbage. I have the original shar file, and will try again if someone needs to see the garbage, but if you can tell me what might be wrong, I'd appreciate it. just my two bytes, austin
w8sdz@smoke.BRL.MIL (Keith Petersen) (02/09/90)
In article <3870001@col.hp.com> mikek@col.hp.com (Mike Karin) writes: >After each file, autoftp logs off! This means that it has to try to log >on again for each file. After trying for a long time to get into >SIMTEL, doesn't it make sense to transfer *ALL* of the desired files >instead of just one? I don't understand the rationale here or maybe >something is broken. That logic was put into the program to prevent users from hogging the system. If someone changes that it will become necessary for SIMTEL20 to add a timer to the FTP server to drop jobs that have been logged in for long periods. I don't think anyone wants to see that because it could mean losing your connection in the middle of a long file transfer. The idea here is to give everyone a chance to get in. There are a limited number of net ports available for FTP jobs. Keith -- Keith Petersen Maintainer of SIMTEL20's CP/M, MSDOS, & MISC archives [IP address 26.2.0.74] Internet: w8sdz@WSMR-SIMTEL20.Army.Mil, w8sdz@brl.arpa BITNET: w8sdz@NDSUVM1 Uucp: {ames,decwrl,harvard,rutgers,ucbvax,uunet}!wsmr-simtel20.army.mil!w8sdz