aris@tabbs.UUCP (Aris Stathakis) (03/29/91)
The title says it all. What *IS* the best way to do a backup? At the
moment all I do is make sure I keep all important data on a seperate
filesystem, then periodically backup up that filesystem using tar.
If I ever had a crash, i'd have to re-install the Operating System
(Xenix 386 2.3.3) then restore my data from tape.
I'd like to know the best way to do a backup so that I can recover from
a FULL crash i.e. having to re-install on a different machine from
a tape backup. I'm sure there are lots of ways to do it, like using
the standard backup proggram SCO give you, but find it too inflexible.
There are several commercial programs that do this kind of thing for
you (ctar, Lone tar), but they cost money :-)
I'm basically looking for the best way(s) to do a secure backup for
a quick recovery from a crash. Any other interesting information
on backup methodology would also be appreciated. If it helps, I'm
backing up to a 150 meg tape drive.
Post here or mail me direct, i'll post a summary if there is any
interest.
Thanks
Aris
--
Aris Stathakis | Bang: ..!uunet!ddsw1!olsa99!tabbs!aris or aris@tabbs.UUCP
- First time, it's a KLUDGE - The second, a trick. -
- Later, it's a well-established technique! -
-- Mike Broido, Intermetricsdavidsen@sixhub.UUCP (Wm E. Davidsen Jr) (04/02/91)
In article <aris.670179095@tabbs> aris@tabbs.UUCP (Aris Stathakis) writes: | I'd like to know the best way to do a backup so that I can recover from | a FULL crash i.e. having to re-install on a different machine from | a tape backup. I'm sure there are lots of ways to do it, like using | the standard backup proggram SCO give you, but find it too inflexible. Since you say Xenix and I've been running a bunch of these for years, I would first say don't use tar, it skips empty directories, and they may be needed to make things work. A new version might have cured that, but there are other reasons. First, of course, you use mkdev fd to create a bootable floppy with your current kernel and devices. When you make a new kernel for any reason, make a new disk (don't rewrite the old one). For each partition take a level zero dump using the error correcting tape device. I suggest making two and verifying them if you're paranoid. I am, and I store one copy off site. Then take regular level one dumps to pick up the changes. Dump saves the data a lot faster than tar, and restores slower. Since these are disaster backups you won't restore much, and if you do you will want reliability rather than speed. When/if the level one dumps get large, switch to level two. At that point I take another level zero, and save one of my original level zero dumps. This gives you a good full restore capability, and since it's Xenix you have the X option to allow slow but reliable restore of selected files. As an alternative for the incrementals, create a file via touch after the level zero, and then use find and cpio to save the modified files. The only advantage is that if you have to restore individual file often this is a more convenient and faster way to do it. You can also use the -mtime and/or -ctime options on find to select files. This allows taking the level zero dumps of each individual partition, then taking one incremental for all filesystems (if it will fit on a single tape) to save time. If you use cpio use the -depth option, as it will insure correct time modified on directories when restoring. Just in case you need it. Summary: - always level zero dump - then level one dump or cpio - dump saves faster, restores slower, needs one tape/filesys - cpio is convenient, save all files on one tape Example: from floppy boot, single user mode: dump 0ufk /dev/erct0 55000 /dev/hd0root dump 0ufk /dev/erct0 55000 /dev/ru incremental, single user mode highly preferred! (five days after level zero) find / \( -mtime -5 -o -ctime -5 \) -depth -print | cpio -oBc > /dev/erct0 Hope this isn't a lot more detail than you wanted! -- bill davidsen - davidsen@sixhub.uucp (uunet!crdgw1!sixhub!davidsen) sysop *IX BBS and Public Access UNIX moderator of comp.binaries.ibm.pc and 80386 mailing list "Stupidity, like virtue, is its own reward" -me
fred@compu.com (Fred Rump) (04/03/91)
davidsen@sixhub.UUCP (Wm E. Davidsen Jr) writes: >In article <aris.670179095@tabbs> aris@tabbs.UUCP (Aris Stathakis) writes: >| I'd like to know the best way to do a backup so that I can recover from >| a FULL crash i.e. having to re-install on a different machine from >| a tape backup. I'm sure there are lots of ways to do it, like using >| the standard backup proggram SCO give you, but find it too inflexible. [long description deleted] >Hope this isn't a lot more detail than you wanted! On the other hand you could simply pick up an inexpensive program like CTAR that does all that for you nightly. When you first install the program it makes rootable/bootable diskettes for you and you could be off and running by either slipping a new disk into a crashed system or taking everything to a different similar system. We have this at all of our customer sites and even keep one set of disks with us just in case the customer loses his. fr -- Fred Rump | Home of Brother John Software CompuData, Inc. | SCO Advanced Product Center 10501 Drummond Rd. | Bang: {uunet dsinc}!cdin-1!fred (800-223-DATA) Philadelphia, Pa. 19154| Internet: fred@COMPU.COM (215-824-3000)
curt@cynic.wimsey.bc.ca (Curt J. Sampson) (04/03/91)
In article <3599@sixhub.UUCP> davidsen@sixhub.UUCP (bill davidsen) writes: > In article <aris.670179095@tabbs> aris@tabbs.UUCP (Aris Stathakis) writes: > > | I'd like to know the best way to do a backup so that I can recover from > | a FULL crash i.e. having to re-install on a different machine from > | a tape backup. I'm sure there are lots of ways to do it, like using > | the standard backup proggram SCO give you, but find it too inflexible. > > [ Bill Davidson's hints deleted. ] I now feel qualified to speak on this, having just installed my tape drive and completed a marathon 12 hour backup/reformat/restore on my system (SCO Xenix 2.3.2). I have considered using backup/restore, but because you need one tape per partition (and I hear that restore has problems restoring to a partition that is a different size from the one backup backed up from) I discarded any notions I had of using those. As well, you also can't tell backup to avoid certain subtrees which aren't worth backing up (/usr/spool/news, for example) unless you give them a separate partition. I think that cpio is the best of the standard utilities for backup for the following reasons: - you can store all of your files on one tape (if they fit) - your backups are completely independent of your partitioning scheme - you can easily choose which files you want to back up - you can easily restore single files (which is good when you have users who are a little trigger-happy with the delete command and come whining to you afterwards :-)) My standard backup scheme is cd / ; find . -depth -print | grep -v '^\./usr/spool/news | \ cpio -oBcv >/dev/rct0 This will back up everything except the /usr/spool/news tree. (There's not much point, IMHO, in backing up a file that's going to be gone in four days anyway. And if you back up news, your incrementals get out of hand very quickly. :-)) You can insert more grep -v commands to avoid other stuff as well, if you like. You could also back up just particular sections if you like (e.g., "cd / ; find ./usr/local -depth -print | cpio ..."). This is my only backup scheme at the moment (no incrementals). I can afford to do a full backup quite often (every night or two), because it's only 40 MB and it takes less than twenty minutes. (Also, I only have two tapes right now. :-)) One other thing to note is that when I do a find I cd to root and use 'find .' rather than just using 'find /'. This lets me easily restore stuff when I've booted from a floppy and mounted the hard drive root partition on /mnt by just cd'ing to /mnt and doing a cpio -i. For my first backup with this drive I backed up everything, including news, because I needed to repartition my disk. This was something of an acid test of my backup system, actually, since I was essentially simulating the dire situation proposed by Mr. Stathakis above. You suddenly begin to realize what a large and unfriendly place the universe is when you realize that your entire system is sitting only on a couple of DC600A tapes and your hard drive is empty. :-) At any rate, I booted from my floppy, ran fdisk and divvy, created the stub directories, mounted all my partitions, and restored from the tape (using cpio with the d and m options, of course). I came back after half an hour and all the files had been restored to their proper places on their proper partitions with no intervention from me. I rebooted and everything worked just fine. (Ok. I'll admit it. That's a baldfaced lie. I broke my system quite badly, and it took me many hours to get it running again. But now that I've done it once, I could do it again in less than three hours with no problem whatsoever.) So there you have it. I think that find, grep and cpio are your best friends when it comes to backing up. Backup/dump and restore are nonsense, IMHO, especially if you're like me and you have five partitions, all of which are fairly small (25-40 MB each). For those interested, I mailed an entertaining (or perhaps not-so-entertaining) summary of those eventful twelve hours to the sco-list mailing list. Hopefully it will appear there in the next couple of days. If not, I can mail a copy to anybody who asks. cjs -- | "It is actually a feature of UUCP that the map of curt@cynic.uucp | all systems in the network is not known anywhere." curt@cynic.wimsey.bc.ca | --Berkeley Mail Reference Manual (Kurt Schoens)
bob@consult.UUCP (Bob Willey) (04/03/91)
In article <3599@sixhub.UUCP> davidsen@sixhub.UUCP (bill davidsen) writes: >In article <aris.670179095@tabbs> aris@tabbs.UUCP (Aris Stathakis) writes: >| I'd like to know the best way to do a backup so that I can recover from >| a FULL crash i.e. having to re-install on a different machine from >| a tape backup. I'm sure there are lots of ways to do it, like using >| the standard backup proggram SCO give you, but find it too inflexible. > > Since you say Xenix and I've been running a bunch of these for years, >I would first say don't use tar, it skips empty directories, and they >may be needed to make things work. A new version might have cured that, > > First, of course, you use mkdev fd to create a bootable floppy with > For each partition take a level zero dump using the error correcting > If you use cpio use the -depth option, as it will insure correct time >Example: > from floppy boot, single user mode: > dump 0ufk /dev/erct0 55000 /dev/hd0root > dump 0ufk /dev/erct0 55000 /dev/ru > incremental, single user mode highly preferred! > (five days after level zero) > find / \( -mtime -5 -o -ctime -5 \) -depth -print | > cpio -oBc > /dev/erct0 There is another alternative, use a commerical program like CTar. It is more in-depth program than tar, and backs up ALL the files and directories of a system (including device drivers, char files, zero dirs, and sorts of other interesting items).. CTAR also comes with a utility program to create a rootable/bootable diskette with all the necessary utilities and even a menu front-end to make disaster recovery much easier. (especially at 3am, when your eyes don't work so well). You can create all of the above yourself, or use CTAR to do all the work for you, and not have to worry about (did I get ALL the programs I wil need in a disaster on my recovery diskette). CTAR also includes scripts to automatically create an entry in cron for automated backups at whatever time is appropriate, will auto- failure scripts to do whatever you want if the backup fails. In other words, you can fish thru the manuals and create everything yourself if you have the inclination, or a commercial program to do all of the work for you. We have been using Ctar for several years now on all of our client's machines and have been very impressed. We are not affiliated with Microlite (CTAR) other than a satisfied user. -- >.. CCS Enterprises, Inc. .. Bob Willey, CDP ..< >.. P.O. Drawer 1690 .. uunet!consult!bob ..< >.. Easton, Maryland 21601 .. (301) 820-4670 ..< >.......................BBS: (301) 476-5098.....................<
macleod@cmllab.rgb.sub.org (Connor MacLeod) (04/05/91)
In article <aris.670179095@tabbs> aris@tabbs.UUCP (Aris Stathakis) wrote: | The title says it all. What *IS* the best way to do a backup? At the | moment all I do is make sure I keep all important data on a seperate | filesystem, then periodically backup up that filesystem using tar. | If I ever had a crash, i'd have to re-install the Operating System | (Xenix 386 2.3.3) then restore my data from tape. Well... I don't know if the way I use is the best but it's ok for me. I'm using cpio to backup my system. The advantage in using cpio instead of tar is that cpio is able to backup special devices, too. The syntax for using cpio is: cd /path_to_backup; find ./ -print | cpio -oB >/dev/rct0 I'm using 150/250 MB tapes according to the usage of my filesystems. Make sure to use "find ./" to get a backup with relative paths. The cpio has no option -A like tar to suppress the leading / when restoring. The command for the restore is: cd /path_to_restore_to; cpio -iBvdum </dev/rct0 Have a look at the various options of cpio. The ones mentioned above are ok with me. You might change the blocksize or something else. The tape you get when doing the backup with cpio is your complete system. So you only have to install the basic OS - it's N1 B1 and B2 for Xenix - and then restore your tape. You can decide to create a bootable filesystemfloppy containing the tools you need to set up a new harddisk so you could use this "blaster" to build a bootable filesys on your hd in case you've had a "final" crash. I never had a blaster disk for Xenix 'cause it's only N1 B1 and B2 to install the basic OS but I have one for SCO Unix. To install the basic OS of SCO Unix it'll take half an hour and a lotta disks! The blaster set contains only of two: a bootfloppy and a filesystemfloppy with the tools to bring up a bootable filesys (or more) on a new hd. Rgds -- Uwe Obst # {connor|macleod}@cmllab.rgb.sub.org (aka Connor MacLeod) # "Trust me, I know what I'm doing!" -- Sledge Hammer
srodawa@vela.acs.oakland.edu (Ron Srodawa) (04/06/91)
In article <1991Apr3.121959.627@cynic.wimsey.bc.ca> curt@cynic.wimsey.bc.ca (Curt J. Sampson) writes: >I think that cpio is the best of the standard utilities for backup for >the following reasons: > > - you can store all of your files on one tape (if they fit) > - your backups are completely independent of your partitioning > scheme > - you can easily choose which files you want to back up > - you can easily restore single files (which is good when you have > users who are a little trigger-happy with the delete command and > come whining to you afterwards :-)) I wholeheartedly agree! I just went through the grief of restructuring my system and ended up rebuilding it from scratch because I learned from the tough teacher--experience--some of these same points. cpio is indeed the best way to go. The 2.3.3 version even has the K option to stop at the end of each tape. Be sure to add cpio (and lots of other things) to the bootable floppy you of course will remember to build. It isn't there by default. Build two floppies while you're at it..they are indispensible. Be sure to do the suggested find . rather than find / trick. I didn't and couldn't process the cpio tapes I had which is one reason I had to start all over from scratch. Cpio is extremely flexible. If you use the -c option, systems other than Xenix can process the files. This is especially useful when your Xenix box has big brothers down the hall. I am backing up to a tapedrive on a machine down the hall with something like.. cpio -oBcK <size> | rcmd <bigbrother> "dd obs=10 > /dev/rmt0" This even allows reel changes during the backup. Because my "real" backups are taken over the network, I plan to, "any day now", take one set of root backups on floppies just so I can restore without the grief or rebuilding from the distribution diskettes. Ron. -- | Ronald J. Srodawa | Internet: srodawa@vela.oakland.edu | | School of Engineering and CS | UUCP: srodawa@vela.UUCP | | Oakland University | Voice: (313) 370-2247 | | Rochester, Michigan 48309-4401 | |
aty@ucselx.sdsu.edu (young a t) (04/07/91)
In article <5664@vela.acs.oakland.edu> srodawa@vela.acs.oakland.edu (Ron Srodawa) writes: >In article <1991Apr3.121959.627@cynic.wimsey.bc.ca> curt@cynic.wimsey.bc.ca (Curt J. Sampson) writes: >>I think that cpio is the best of the standard utilities for backup for >>the following reasons: >> (much stuff deleted) > >Cpio is extremely flexible. If you use the -c option, systems other >than Xenix can process the files. This is especially useful when >your Xenix box has big brothers down the hall. I am backing up >to a tapedrive on a machine down the hall with something like.. Got to be careful with this. We recently had the disk controller crash on our 3B2 system, which had been backed up with cpio as you describe. The disks we lost had all the users' home directories. So we said, OK, we'll just restore those guys on the SUN workstation from the backup tapes, right? Wrong. The SUN version of cpio can't read multi-volume backups; we only saved the guys who were on the first tape. There are probably other Berkeley-derived systems whose cpio can only read one tape. BEWARE! -- A.T.Young (aty@mintaka.sdsu.edu)
ronald@robobar.co.uk (Ronald S H Khoo) (04/07/91)
macleod@cmllab.rgb.sub.org (Connor MacLeod) writes: > The advantage in using cpio instead > of tar is that cpio is able to backup special devices, too. Well, one of the advantages, anyway. Yes, cpio is a good choice. > Have a look at the various options of cpio. The ones mentioned above > are ok with me. You might change the blocksize or something else. If you're running SCO Xenix 2.3.x, you'll get more cpio options if you install SLS xnx 155b a.k.a. UFM. Available for anon UUCP from sosco, anon FTP from uunet.uu.net, or from whoever sells you support, I guess. This new cpio is taken from Sys V R3.2 (in fact I think it's actually a COFF executable) and has nice options like how big your tape is, and lets you use huge buffers to make your tape stream nicely. If you're running a cartridge tape drive, there's also an improved device driver in that update which makes multivolume backups a lot easier. -- Ronald Khoo <ronald@robobar.co.uk> +44 81 991 1142 (O) +44 71 229 7741 (H)
louis@cs.athabascau.ca (Louis Schmittroth) (04/07/91)
In article <1991Apr7.044627.5298@ucselx.sdsu.edu>, aty@ucselx.sdsu.edu (young a t) writes: > In article <5664@vela.acs.oakland.edu> srodawa@vela.acs.oakland.edu (Ron Srodawa) writes: > >In article <1991Apr3.121959.627@cynic.wimsey.bc.ca> curt@cynic.wimsey.bc.ca (Curt J. Sampson) writes: > >>I think that cpio is the best of the standard utilities for backup for > >>the following reasons: > >> > (much stuff deleted) I run Xenix 2.3.2 hare at home as a personal computer. For what it's worth, here is my experience with backups. First I installed a Colorado Jumbo tape system, and had both hardware and software problems. So, I went to Archive, and have been very happy. About a year or so ago I bought CTAR, a commercial tar system what also backs up special device files. This has worked flawlessly. About 6 months or so ago, I added a second disk to my controller. Instead of re-installing I was able to use the file system recovery diskettes from the CTAR system to restore. I am a great fan of CTAR. Microlite Corp. 1021 Sutherland Street Pittsburgh, PA 15204. 412-771-4901.
sean@utoday.com (Sean Fulton) (04/08/91)
In article <1991Apr7.044627.5298@ucselx.sdsu.edu> aty@ucselx.sdsu.edu (young a t) writes: > >Got to be careful with this. We recently had the disk controller crash >on our 3B2 system, which had been backed up with cpio as you describe. >The disks we lost had all the users' home directories. So we said, >OK, we'll just restore those guys on the SUN workstation from the >backup tapes, right? Wrong. The SUN version of cpio can't read >multi-volume backups; we only saved the guys who were on the first >tape. There are probably other Berkeley-derived systems whose cpio >can only read one tape. BEWARE! > We're running SCO Unix 3.2.2 with a 40 M Irwin tape drive. I consistently run into cpio problems restoring from more than one tape. Note that this does *not* occur when using sysadmsh to do the backup, but when I try to do it freehand, I run into problems. The method to date has been: find / -print | cpio -ovcB >>/dev/rctmini When restoring, I use: cpio -ivcdB </dev/rctmini but always hit a tape read error at the end of the first tape and can't pick up on my second or third. Is it possible I'm just running out of tape too soon (I have the default at 37 M on an allegedly 40 M tape), or is it something else? -- Sean Fulton sean@utoday.com UNIX Today! (516) 562-5430 /* The opinions expressed above are not those of my employer */
adeboer@gjetor.geac.COM (Anthony DeBoer) (04/09/91)
In article <1991Apr7.044627.5298@ucselx.sdsu.edu> aty@ucselx.sdsu.edu (young a t) writes: >Got to be careful with this. We recently had the disk controller crash >on our 3B2 system, which had been backed up with cpio as you describe. >The disks we lost had all the users' home directories. So we said, >OK, we'll just restore those guys on the SUN workstation from the >backup tapes, right? Wrong. The SUN version of cpio can't read >multi-volume backups; we only saved the guys who were on the first >tape. There are probably other Berkeley-derived systems whose cpio >can only read one tape. BEWARE! From a lot of experiences in our shop restoring backups on machines other than the one that made them, or of needing one file from the fifth tape of a set, my conclusion is that multi-tape backups are evil. What I've started using is a small program that acts something like the C-News batcher, generating backup lists each containing no more than a certain number of megs worth of data, and then backing up each list to a single tape. That way I can restore all or part of any individual tape independently, I'm not completely dead in the water if tape two dies, and I don't have to worry about differing end-of-tape continuation implementations. Any other views on this area? -- Anthony DeBoer NAUI#Z8800 | adeboer@gjetor.geac.com | Programmer (n): One who Geac J&E Systems Ltd. | uunet!geac!gjetor!adeboer | makes the lies the Toronto, Ontario, Canada | #include <disclaimer.h> | salesman told come true.
n025fc@tamuts.tamu.edu (Kevin Weller) (04/10/91)
I've been making backups with find and cpio for over a year now, and I have a (relatively) friendly shellscript front-end for it all which I'd post if I could solve a thorny problem with the compression scheme. The problem is that, since I pipe the cpio output through compress and my own volume splitter (restore is just the reverse), I can't start a restore in the middle of a multi-volume backup. This means that a bad volume in the middle stops me from restoring the rest of the way. I've tried and tried to figure out compress's scheme for packing the data so I could uncompress starting in the middle, but no luck. I find the idea of backing up 180 megs of data without compression ludicrous, so is there another high-performance compression utility out there that will do the job (let me start uncompressing from the middle of a file, which is essentially what a multi-volume backup is)? Any pointers would be appreciated, and if and when I make my backup scheme adequately reliable, I'll post the package of files to get it going. Thanks -- Kev PS: I use diskettes to make backups since I can't afford a decent tape drive, but the principle is the same for tape backups. Even if I COULD afford a tape drive, I don't think I have another hardware interrupt slot available for the controller (can't use the floppy controller either). -- ------------------------------------------------------------------------------ Kevin L. Weller /-------+--------------------\ internet: n025fc@tamuts.tamu.edu | aTm | GIG 'EM, AGGIES! | CIS: 73327,1447 (but I rarely log on) \-------+--------------------/ ------------------------------------------------------------------------------ %SYS-E-BADOPSYS, Fatal system error, DEC VMS halting / "And now for something -SYS-I-GETUNIX, Replace with UNIX immediately! / completely different." ----------------------------------------------------------------- Monty Python
wul@sco.COM (Wu Liu) (04/11/91)
/--sean@utoday.com (Sean Fulton) said... | In article <1991Apr7.044627.5298@ucselx.sdsu.edu> aty@ucselx.sdsu.edu (young a t) writes: | > | >Got to be careful with this. We recently had the disk controller crash | >on our 3B2 system, which had been backed up with cpio as you describe. | >The disks we lost had all the users' home directories. So we said, | >OK, we'll just restore those guys on the SUN workstation from the | >backup tapes, right? Wrong. The SUN version of cpio can't read | >multi-volume backups; we only saved the guys who were on the first | >tape. There are probably other Berkeley-derived systems whose cpio | >can only read one tape. BEWARE! | | We're running SCO Unix 3.2.2 with a 40 M Irwin tape drive. I | consistently run into cpio problems restoring from more than one tape. | Note that this does *not* occur when using sysadmsh to do the backup, | but when I try to do it freehand, I run into problems. The method to | date has been: | | find / -print | cpio -ovcB >>/dev/rctmini | | When restoring, I use: | | cpio -ivcdB </dev/rctmini | | but always hit a tape read error at the end of the first tape | and can't pick up on my second or third. | | Is it possible I'm just running out of tape too soon (I have | the default at 37 M on an allegedly 40 M tape), or is it something | else? | /* The opinions expressed above are not those of my employer */ \-- Try specifying the size of the tape with the -K option. For a 40mb tape, use either 40000 or something slightly less, depending on how confident you are that the tape is of the correct length. Also, you might consider backing up by filesystems, with the -mount option to find. For example: # cd / # find . -mount -print | cpio -ocdmuvB -O/dev/rctmini -K40000 will back up your entire root filesystem onto QIC-40 tape(s) without straying into any user filesystems you have set up. The -K option to cpio is documented in the cpio(C) on-line man page.