[net.unix-wizards] Incremental dumps using cpio

weber@nadc.arpa (01/17/86)

We recently installed Sys-V, release 2 and would like to use the cpio
command to do daily incremental dumps.  AT&T has hinted that cpio
should be used but doesn't want to give out any more info without a
service contract in place.

We like the dump utility on Berkely 4.1 and would like something
comparable.  Is this possible?   Has anyone already done this?
Any info would be greatly appreciated.

Please mail any responses directly to me.  Many thanks.

					    Jane M. Weber
					    weber@NADC.ARPA

bzs%bostonu.csnet@csnet-relay.arpa (Barry Shein) (01/18/86)

>From: weber@nadc.ARPA
>We recently installed Sys-V, release 2 and would like to use the cpio
>command to do daily incremental dumps.  AT&T has hinted that cpio
>should be used but doesn't want to give out any more info without a
>service contract in place.

Sure, we do that as a regular shell procedure on our 3B5, although
the shell script is a little peculiar to our installation, here's
the gist of it:

	for j in 5 6 7 8	# whatever, for /usr5, /usr6...
	do
		/bin/find /usr$j -type f $duration -print >> $logfile
		/bin/cpio -oB < $logfile > /dev/rmt/0m
	done

where we set logfile to a file name that indicates the date (eg.
Jan.17.86) and the duration is going to be something like "-mtime -7"
which means back up all files modified in the last 7 days. The purpose
of the logfile is that it will end up to be one of the first files
backed up and contains the names of all the files on the tape if we
need that for reference later (could extract it and print it out as a
first step for recovering something, much faster than cpio -itB, you
could also move them to an operator's area so a quick grep will tell
you which tapeset should have a file in question.) Note that if the
system is running things will change while the find is running and
while cpio is running, we figure all that means is that you will miss
backing up some files that were just this minute created or changed
which is equivalent to having started the backups an hour ago, we'll
get it next time. For fulls (very similar code) we go single-user to
make sure we get everything. It's not 4.2's dump by any means but that
wasn't a choice. Restores can be done with the cpio pattern matching
(cpio -ivBmd '/usr5/*' < /dev/rmt/0m [restore everything on /usr5,
eg]). One more thing, when you run out of tape cpio will say something
like 'output write error; enter device name to continue', mount a
fresh tape with a write ring and type the device name again
(/dev/rmt/0m or whatever) and it will just continue, you could back up
your whole file system to one tapeset though I wouldn't recommend it
(a bad spot in one tape could ruin the set.)

Hope it helps. Needless to say do a few dry runs with whatever you choose.

	-Barry Shein, Boston University

P.S. The above procedure ignores empty directories, FIFOs and device
files, you might want to run a 'find' to at least create a file which
lists those in a file so, worst comes to worst, you could re-create
by hand or by editing the output into a shell script, I would use:

	/bin/find / -type p -a -exec echo mknod '{}' p ';' >> FIFO.jan17.86

or somesuch (the result should be a shell script which would recreate FIFOs.)

sam@delftcc.UUCP (Sam Kendall) (01/22/86)

In article <1729@brl-tgr.ARPA>, bzs@bostonu.csnet (Barry Shein) writes:
> Sure, we do [incremental dumps using cpio] as a regular shell procedure
> on our 3B5 ....
> [shell procedure essentially "find filesystem -mtime 7 -print | cpio"]
> P.S. The above procedure ignores empty directories, FIFOs and device
> files ....

It also ignores:
(1) files that have been linked, chmod'd, or backdated (using tar or
    cpio);
(2) contents of directories that have been mv'd; and
(3) files that have been unlinked (i.e., restoring a full dump, then an
    incremental dump, will leave in their old place files which were
    rm'd or mv'd in between the dumps).

Using "find -ctime" instead of "find -mtime" will solve (1).  This makes
the backups okay for the purpose of restoring accidentally deleted
files, but (2) and (3) make it messy for restoring a lost filesystem --
things won't be quite the same after the restore.  Take it from me, this
"near restoration" is annoying as hell.

Both find and cpio are underpowered for this application.  find becomes
awkward here when you have one filesystem mounted under another mounted
filesystem (e.g., /usr and /usr/spool), and its granularity of 1 day
with -ctime is annoying.  -newer has a finer granularity, but doesn't
deal with change times.  As for cpio, it should have an option to cope
with (3) by recording the contents of directories.

Backup tools, even more than interactive debuggers, have traditionally
been neglected under UNIX.  (Backup *devices* have also been neglected
by (super)micro manufacturers, of course, but now that cartridge tape
drives are getting cheaper, that is changing.)  I have gotten mailings
for the "UBACKUP" tools from Unitech Software, but I don't know much
about them.

----
Sam Kendall			     allegra \
Delft Consulting Corp.		seismo!cmcl2  ! delftcc!sam
+1 212 243-8700			       ihnp4 /
ARPA: delftcc!sam@nyu.ARPA

kimcm@olamb.UUCP (Kim Chr. Madsen) (01/30/86)

[come on ye little faithfull goumet]

How about this little strategy:

	#!/bin/sh
	backupdate=/usr/local/bakupdate
	if [ -f $backupdate ] ; then
		find $1 -newer $backupdate -print | cpio -ov >/dev/whatever
		touch $backupdate
		echo "Incremental backup finished."
	else
		find $1 -print | cpio -ov > /dev/whatever
		touch $backupdate	# create empty file.
		echo "Full backup finished"
	fi

This program above is NOT guaranteed to work, but is just an idea of
the approach. In the case above you can use ncpio if you're using one
of at&t's 3B machines.
					Kim Chr. Madsen
					kimcm@olamb.UUCP