tchrist@convex.com (Tom Christiansen) (10/16/90)
I've looked at several alternatives to rdist (coda, track) and
I still like rdist best overall. (Having to convert lots of Distfile,
some of ~1000 lines, has a little to do with it. :-)
Here's a list of bugs and wishes I've come up with for rdist. I'm posting
it here and taking it off to LISA for brainstorming. Additions or
solutions welcome. I'll check back in a week when I get back from LISA.
--tom
* You have to have the master in everyone's /.rhosts files. The
master must initiate the connection, not vice versa. I want
a subscription service version of rdist VERY BADLY. This is
my number one wish.
* It doesn't do the follow symlinks -- it clears them. Sometimes
I like this, sometimes I don't. The problem is that when space
fills, the real executable is often replaced with a symlink
on the host with problem, which rdist happily destoys.
* There's no include file syntax, which makes it difficult for
common definitions to be shared amongst various distfiles.
* You can only do add to, never subtract from, lists. I want
to define host lists that have removals on them. Like
PRODUCTION = ( ${ENGR} - ${LAB} )
Or
${FILES} -> ( ${HOSTS} - {PRECIOUS} )
If subtraction is used, it would be best if what got removed
were rememberd be remembered so that if you later say
TECH = ( ${PRODUCTION} ${SUPPORT} )
That and ${LAB} machines wouldn't be added back in even if they
were in ${SUPPORT}, since they'd been included in ${PRODUCTION}.
I realize this is complex, but it's also very useful.
* If the remote machine hangs (which seems to happen a lot),
or the special command never returns, rdist can't recover from this.
* If a host is down and 30 targets go the that host, then
it'll try 30 times, wasting a lot of time and generating
a lot of error messages.
* It sends a zillion notifies to a user rather than bundling
them all together. That's because no state is kept between
execution of targets; the same could also be said of the previous
problem.
* It forgets to set the access time.
* If you are in the middle of executing a special clause on a remote
host and hit ^C, the rdist returns but the command keeps running. The
signal should propagate as in rsh.
* If you are rdisting some hard-linked pair of files /foo/bar/x and
/foo/bar/y to a host on which /foo/bar is a symlink to a legitimate
directory, then the install will fail saying: "rdist: host:/foo/bar/y:
No such file or directory (no parent)".
* If you execute rdist and specify a rule to perform that
does not exist in the rdistfile, then rdist does not print
anything nor exit non-zero. You cannot tell if there was nothing
to rdist or if I made a mistake. Rdist should print a diagnostic
that says "rdist: don't know how to rdist <target>".
* When installing new directories, rdist does not (always) preserve the
original's mode. I haven't tracked this one down conclusively yet.
* The install clause doesn't take -q like the command line. It
would be nice if it did.
* When using the -v option of rdist to verify a distfile and your
install rule specifies a different directory to place the files in,
the "need to update" message indicates the wrong directory.
* This distfile doesn't work:
( /v8/foo ) -> ( hosts )
install /usr/local/bin;
This does:
( /v8/foo /v8/foo ) -> ( hosts )
install /usr/local/bin;
It seems to care unduly much about having more than one element
in its file list.
* You can't make rdist work across several machines in parallel.
It needs some kind of "go parallel" option that made rdist
fork itself some number of times and have all files going
to some host be handled by each child. This would probably
end up being faster as well, since due to lack of state,
it isn't keeping connections around to a host while it's
off playing with a different one for a while.
* It's hard to have a master with a set a several trees that go
to common subdirs depending on the machine. For example,
/usr/local/bin.v8 and /usr/local/bin.v9 should go to
/usr/local/bin depending on which o/s I'm talking. I've put
something together, but it's pretty lame. Perhaps there's a
better way, but I don't know it.jms@romana.Tymnet.COM (Joe Smith) (10/17/90)
To add to the list of things that rdist does not do for me:
*) No bidirectional younger copy. If remote file is newer, I want it
copied to the master. (See enclosed script for a fix.)
*) Complains about remote directory having the wrong permissions instead
of fixing. (See enclosed script for a fix.)
*) If sending selected files from wildcarded directories, (such as
"/home/delta/*/{.login,.cshrc}"), it does not create missing directories.
(See enclosed script for a fix.)
*) When output is redirected to a file, rdist does not flush the buffers
after each file, and stdio buffers up 8K bytes of output. This means
that "tail -f logfile" cannot be used to monitor the progress in realtime.
(I don't have the sources to rdist to fix this.)
*) Changes to the owner, group, or mode of a file are not propagated to
the remote system unless the date (or checksum) of the file changes.
(I don't have the sources to rdist to fix this.)
----------%<-------- cut here for "fix-warnings" --------->%----------
#!/bin/sh
# Problems in the rdist procedure are located and fixed.
# In particular, wrong protection on directories and "remote file is newer".
cmds=/tmp/fix-warnings$$.1
warn=/tmp/fix-warnings$$.2
localhost=`hostname`
# The log from rdist is piped into this script as stdin.
# Typical use is:
# (echo; echo -n "==$$== --- starting $0 $*"; date) >>$logfile
# rdist -y -c some files to remotehost >>$logfile
# sed "1,/=$$=/D" $logfile | /etc/rdist/fix-warnings >>$logfile
# Where the use of =$$= is to select only messages from this run.
egrep "Warning|host|directory" | sed 's/:/ /g' >$warn # keep only good stuff
sendcommands() {
if [ -s $cmds ]; then # check if size of file is nonzero
echo; echo " Commands to be executed on $remotehost:"
cat $cmds # list the chmod commands here
fi
if [ "$newfiles" != "" ]; then # add "rdist" to list of commands
echo "remotehost=$remotehost" >>$cmds
echo 'echo -n "$remotehost - "' >>$cmds # rdist says "updating host"
echo "rdist -y -c $newfiles $localhost" >>$cmds
echo 'echo -n "$remotehost - ";date' >>$cmds
fi
if [ -s $cmds ]; then
echo
rsh $remotehost /bin/sh <$cmds # do chmod or mkdir as needed
echo
: >$cmds # zero out the file
fi
if [ "$dirfiles" != "" ]; then # parent directory was missing
rdist -y -c $dirfiles $remotehost # try sending file again
fi
}
: >$cmds
remotehost="$1" newfiles="" dirfiles="" fixed=""
# Get rid of extra stuff (such as a timestamp I put) in front of rdist's
# "updating host" message so that the name of the remote host can be found.
sed -e 's/^.*updating host/updating host/' $warn | \
(while read x1 x2 x3 x4 x5 x6 x7 x8 x9 x10
do
#"updating host boomer"
if [ "$x1 $x2" = "updating host" ]; then
if sendcommands; then hosts="$hosts $remotehost"; fi # previous host
remotehost=$x3 # remember name of remote host
newfiles="" # reset list of newer files
dirfiles="" # and list of files missing parent dirs
fi
#"/home/gemini/some/dir: Warning: remote mode 775 != local mode 2775"
if [ "$x3 $x4" = "remote mode" ]; then
sgid=`expr $x9 - $x5`
if [ "$sgid" = "2000" ]; then x9="g+s"; fi
if [ "$sgid" = "-2000" ]; then x9="g-s"; fi
echo "chmod $x9 $x1" >>$cmds # rdist should do this itself!
fixed=yes
fi
#"Warning: /mnt/xd4h/2/02042/c00.log: remote copy is newer"
if [ "$x5 $x6" = "is newer" ]; then
newfiles="$newfiles $x2"
fixed=yes
fi
#"rdist: boomer:/tym/7/07000/rdista08190: No such file or directory"
if [ "$x4 $x5 $x8" = "No such directory" ]; then
missingdir=`dirname $x3`
dup=no
for dir in $dirfiles
do
if [ $missingdir = $dir ]; then dup=yes; fi
done
if [ $dup = no ]; then
# Assume that just one directory is missing. This will create it.
rdist -c $missingdir $remotehost # create dir, right owner.group, mode
# But if more than one directory is missing, "mkdir -p" creates them all.
echo "mkdir -p $missingdir" >>$cmds
dirfiles="$dirfiles $missingdir" # double check for dir being updated
fi
fi
done
sendcommands # do last (or only) host
#if [ "$fixed" = "" ]; then echo "$0: no warnings"; fi
)
rm $cmds $warn
#[End of /etc/rdist/fix-warnings]#
--
Joe Smith (408)922-6220 | SMTP: jms@tardis.tymnet.com or jms@gemini.tymnet.com
BT Tymnet Tech Services | UUCP: ...!{ames,pyramid}!oliveb!tymix!tardis!jms
PO Box 49019, MS-C41 | BIX: smithjoe | 12 PDP-10s still running! "POPJ P,"
San Jose, CA 95161-9019 | humorous dislaimer: "My Amiga 3000 speaks for me."rob@violet.berkeley.edu (Rob Robertson) (10/24/90)
add to the list: *) i wanna be able to group files better so that i can selectively rdist out packages, not just till the from a a label to the end of the distfile. *) macros (which i think tom mentioned). *) wild card expansion on the remote host statements like: execpt /etc/hosts.* get expanded on the LOCAL SERVER not on the remote side. *) expansion of tilda's occurs locally, thus if your sending from /home/test/rob to /home/xxx/rob by (~rob/ ) -> xxx it fails. (i have fixed this). *) this is tough: hard links on the server file system that end up being separate file systems on the remote end. there should be a way to specify what to do. *) i also tried to install a file called "install" once it bombed due to a problem in the yacc code. (in the queue to be fixed). rob -- william robertson rob@violet.berkeley.edu "crack has ruined the drug culture" - H S Thompson
seindal@skinfaxe.diku.dk (Rene' Seindal) (10/24/90)
rob@violet.berkeley.edu (Rob Robertson) writes: > *) i also tried to install a file called "install" once it bombed due > to a problem in the yacc code. (in the queue to be fixed). Write ./install in you Distfiles. Rene' Seindal (seindal@diku.dk)