[net.bugs.usg] A good incremental backup for system V

dwd@ttrda.UUCP (Dave Dykstra ) (04/21/86)

I now see that a good incremental backup for system V is possible.

> As has been known at least since the V7 dump/restor system was written
> (1978?), you should dump based on the most recent of the mod time and
> the change time.  The change time changes when the inode changes, and
> since a rename involves (temporarily) changing the link count, the change
> time will reflect it.

With the above suggestion of Henry Spencer, I put in a quick addition to
find(1) to look for files that are newer based on change time instead of
modification time, calling the option '-cnewer'.  Now all I need is
the change to cpio that was formerly suggested to have an option to
remove files which no longer exist.  All it needs to do (I think) is
to save the complete contents of directories which have been changed
and unlink the files which aren't in the directories upon restore.

> I'm planning to post afio, my homegrown cpio replacement, to mod.sources
> Real Soon Now*. Among its excessive features are automatic recognition of
> ASCII, binary and byte-swapped binary cpio archives. There are many other
> neat things; the present manual page follows.

How about adding the option to afio, Mark Brukhartz?  

			- Dave Dykstra @ AT&T Teletype
			  ihnp4!ttrdc!ttmcsa!dwd

mdb@laidbak.UUCP (Mark Brukhartz) (05/11/86)

In article <151@ttrda.UUCP>, dwd@ttrda.UUCP (Dave Dykstra ) writes:
> I now see that a good incremental backup for system V is possible.
> 
> With the above suggestion of Henry Spencer, I put in a quick addition to
> find(1) to look for files that are newer based on change time instead of
> Now all I need is the change to cpio that was formerly suggested to have
> an option to remove files which no longer exist.  All it needs to do (I
> think) is to save the complete contents of directories which have been
> changed and unlink the files which aren't in the directories upon restore.

Directories are recursive. A change to the root directory would turn the
next incremental backup into a full one. When "filehog" removes the vnews
core file from his home directory (:-), all of his files will be backed
up again. Ugh!

> How about adding the option [to remove files] to afio, Mark Brukhartz?  

I've thought about this for the past several months. It's easy to remove
files when reading the archive.  How does one tell afio (when writing the
backup) which files are to be removed? Note that order is significant; it
is neccessary to remove a directory (and, of course, its contents) before
creating a file of the same name.

	o Name a file containing the list. This requires that the list
	  be generated before afio is invoked, preventing pipelineing of
	  incremental backups.

	o Assume that nonexistent files are intended to be removed. This
	  sounds dangerous... It's too easy for a mortal user to "backup"
	  and "restore" a mode 777 directory containing inaccessible mode
	  400 files, inadvertently removing the files. Yes, it's possible
	  to distinguish "no permission" from "no such file" errors, but
	  the possibility for bugs and other problems is disturbing.

	o Add a "key" as the first character of each input pathname, with
	  ">" meaning "copy" and "<" meaning "remove". This would have to
	  be Yet-Another-Option* to preserve what little cpio invocation
	  compatibility remains. Consider, too, of all of the other keys
	  which could be added by future featurefesters (...say that five
	  times quickly!).

Lacking other ideas or suggestions, I'll probably go with the "key" scheme.
None of the present alternatives sound very inviting, though.

						Mark Brukhartz
						Lachman Associates, Inc.
						..!ihnp4!laidbak!mdb

* Yet-Another-Option ought to be trademark of the
  Regents of the University of California at Berkeley

davidsen@steinmetz.UUCP (Davidsen) (05/22/86)

I am currently personally backing up three systems totaling about 200MB of
hard disk. This doesn't sound impressive, but I currently have to do it to
floppy disk! Honest, I think about backup *a lot*! I use backup/restore
(or dump/restore as it's called on some) whenever posible. I have had to
restore one system from cold twice due to hardware changes. Until cpio
will take care of deleting deleted files, I won't use it for regular
system backup. I run out of disk every time I try.

I have no ultimate answer, but I will offer one trick which has cut down
my hassles. Although I don't use cpio for system backup, I sometimes want
to backup a small directory for transfer to another system or for storage
before deleting the contents. The output of cpio may be piped through
compress (I have v4.0 from mod.sources) which reduces the volume of output
data to fit on one disk (usually). A friend uses this to backup a 75MB
system on a 40MB streaming tape. It eats CPU, but I don't have to sit and
change media, and he doesn't pay a third shift operator.

Second trick... if you have a large database which gets "changed" every
day, you may eliminate the backup by keeping one backup on disk and saving
an "incremental" of just that file. The program I wrote looks like a one
night stand between diff and cmp, but it works. You can backup the
database once a week (or whatever) and just take an incremental daily.
            ================ warning ================
This doesn't work on all databases, use your regular backup before you try
it. Some databases modify most of the file just to change a few records
(you can tell by response), while others are sensitive to the date changed
or something. I am ashamed of the code I use to do this so don't even ask.
If I ever clean it up I'll post it, after I have a lawyer read the
disclamer.

-- 
	-bill davidsen

  ihnp4!seismo!rochester!steinmetz!--\
                                       \
                    unirot ------------->---> crdos1!davidsen
                                       /
         sixhub ---------------------/        (davidsen@ge-crd.ARPA)

"Stupidity, like virtue, is its own reward"