fnf@mcdsun.UUCP (Fred Fish) (01/01/70)
In article <285@unisol.UUCP> haral@unisol.UUCP (Haral Tsitsivas) writes: >Another issue though is, how many system administrators can keep the system >down long enough to do block by block checking of dump tapes? If you have a >production level system (or a large development system) with several eagles >(and GBytes) you may only have enough time to do complete backups and not >block by block checking of the tape and disk files (users never want to lose >the machine for long periods of time, or be slowed down while a backup is >done when they are logged in). A very valid point, unfortunately. I myself have been bitten badly a couple of times when I simply did not want to take the time to verify the backup. I once lost several days worth of work to flakey hardware that gave no indication of any error on write, and as luck would have it, nuked only file data blocks on my backup and not file header blocks. The typical method of "verifying" tar tapes by doing a table of contents would not have caught this either. The alternative to keeping the filesystems quiescent long enough to verify the backup is to turn the users loose and do the verify anyway, and then simply manually scan the list of differences for those that look like they might indicate a problem (/unix is different for example). Whether or not this is a practical solution obviously depends on your circumstances. There are couple of other uses for a tape versus filesystem comparison facility that I have found to be quite handy: (1) Write an archive of only the files you consider to be critical to system operation (I.E. "system" files as opposed to "user" files). Keep this around, keep it current, and when you experience any "flakiness" with the system, diff your current system against this standard. This is great for catching all sorts of nasty things like hardware induced filesystem errors, or malicious hacking at your system files. (2) Use it to verify that two trees in a given system are identical, including permissions, ownership, file contents, dates, etc. Very quick and easy to start two copies, one to create an archive in the "source tree" and pipe it to another to diff the archive in the "destination tree". > If a tape-speed (or near tape-speed) method >for verifying the sanity of backups exists and works reliably (under most >systems, although not as reliably as the block comparison method) then most >people will use that... This is one of the reasons why I also have an "inspection" mode, that simply verifies that all the blocks are readable, they are in the correct order, each block has the proper checksum, etc. Not quite as comforting as a full tape versus filesystem verify, but as you noted, much faster and less impact on the system with downtime. This also means you can pull any tape out of storage, years later, and still verify that it is a complete and viable backup (I understand some installations rewrite old tapes periodically to counteract some sort of "bitrot" that all magnetic media is subject to). -Fred -- = Drug tests; just say *NO*! = Fred Fish Motorola Computer Division, 3013 S 52nd St, Tempe, Az 85282 USA = seismo!noao!mcdsun!fnf (602) 438-3614
seb022@tijc02.UUCP (Scott Bemis ) (01/01/70)
9/8/87 Concerning Haral Tsitsivas of UniSolutions Associates comments on verifying UNIX tape backups, > Another issue though is, how many system administrators can keep the system > down long enough to do block by block checking of dump tapes? If you have a > production level system (or a large development system) with several eagles > (and GBytes) you may only have enough time to do complete backups and not ..... Why can I afford to verify backup tapes on a VAX using the VMS operating system, but may not have the time nor utilites to verify backup tapes on a VAX using the UNIX operating system? The VMS operating system - DEC's proprietary operating system for it's VAXes - has a verify option. It is known as BACKUP/VERIFY referred to in DEC's VAX VMS Backup Utility Reference Manual, page 38. Both the VMS VAX(es) 11/785s and the UNIX VAX 8600 at my place of employment are production systems with gigabytes of data stored on DEC's RA81 and RA60 disk drives. There are second and third operators to perform tape backups. Scott Bemis Texas Instruments, Johnson City, TN !mcnc!rti!tijc02!root Phone: (615) 461-2959
seb022@tijc02.UUCP (Scott Bemis ) (08/14/87)
Has anyone used Ubackup software from Unitech Software, Inc., Autoback from Werner & Monk, Inc., or Reel Backup from COSI, Inc. I am considering purchasing a UNIX tape backup utility. What are your comments, compliments, complaints with any of these tools? Particular questions Can this tool (from whatever company) a. handle multiple tape reels to backup a single file system if necessary? (I have large file systems) b. keep an online catalog of files backuped to each particular backup tape (tell me which tape a file may be on)? c. perform both a full and incremental backup on a MOUNTED file system which may be modified as it is being backed up? d. be able to restore individual files from either an incremental or full backups, Not require one to copy the entire tape to a spare area on a disk drive (if a such an area exists) like volcopy? e. be installed easily and easy to use, and well documented? OR has someone (a genius) found a way to meet these requirements using standard AT&T UNIX V utilities? Below explains why I am interested in purchasing a tape backup utility. First, I am using a port of AT&T UNIX V Release 2.0 Version 2 on a VAX 8600. I would like to be able to verify my incremental and full tape backups. I currently use find piped to cpio to make /bin/find . -mtime -"$days" -print | cpio -oaB >/dev/rmt/0h incremental backups (backup only files modified within the last day, week, etc) and volcopy to make full tape backups. /etc/volcopy -a -bpi6250 -feet2400 $fsname /dev/rdsk/$2 - /dev/rmt/0h - > /dev/console 2>&1 With these methods, I can not easily compare the contents with of the tape with what is on the disk. I want to be sure that the backup tape really holds what it should. Users and managers can get very if files can not be restored from backups. Yes, this has happened before. With full backups using with volcopy, I need a spare area on a disk drive or in some cases, a spare disk drive to restore files from full backups. In addition, I do not know what files are on the full backups. Great Fun to restore several full backup tapes to find just one critical file a user deleted!!! With volcopy, the file system has to be unmounted to make sure it is not being modified during the backup procedure, making the computer unusable for many computer users. Scott Bemis Texas Instruments, Johnson City, TN rti!tijc02!root Phone: (615) 461-2959
mkhaw@teknowledge-vaxc.ARPA (Michael Khaw) (08/15/87)
in article <153@tijc02.UUCP>, seb022@tijc02.UUCP (Scott Bemis ) says: > Xref: teknowledge-vaxc comp.unix.questions:2433 comp.unix.wizards:2599 > Has anyone used Ubackup software from Unitech Software, Inc., > Autoback from Werner & Monk, Inc., or Reel Backup from COSI, Inc. Please post a summary of the responses you get. I asked the question some time back (I'd only heard of Ubackup and Reel Backup until now). I got one response back, and it was a favorable review of Ubackup. I think Ubackup does most of what you ask for, but it uses tar format -- which I infer to mean that it has tar's pathname limit of 100 characters, and that's not acceptable in my environment (some of our users love deep trees AND long nodenames). Mike Khaw -- internet: mkhaw@teknowledge-vaxc.arpa usenet: {hplabs|sun|ucbvax|decwrl|sri-unix}!mkhaw%teknowledge-vaxc.arpa USnail: Teknowledge Inc, 1850 Embarcadero Rd, POB 10119, Palo Alto, CA 94303
mike@pyrdc.UUCP (Mike Whitman) (08/16/87)
In article <15702@teknowledge-vaxc.ARPA>, mkhaw@teknowledge-vaxc.ARPA (Michael Khaw) writes: > I got one response back, and it was a favorable review of Ubackup. I think > Ubackup does most of what you ask for, but it uses tar format -- which I > infer to mean that it has tar's pathname limit of 100 characters, and that's > not acceptable in my environment (some of our users love deep trees AND > long nodenames). I am using Ubackup right now to do dumps on my machine and it uses either/or tar/cpio you get to pick which you wish to use. Rumors are of Unitech comming out with an ANSI tape version as well giving you 3 choices to use. All in all a good package. -- Pyramid Technology Corp. Mike Whitman 8320 Old Courthouse Rd. (703)848-2050 Suite 440 ...uunet!pyrdc!mike Vienna Virginia 22180 "language is a virus from outer space"
davids@iscuva.ISCS.COM (David Schmidt) (08/17/87)
In article <15702@teknowledge-vaxc.ARPA> mkhaw@teknowledge-vaxc.ARPA (Michael Khaw) writes: >in article <153@tijc02.UUCP>, seb022@tijc02.UUCP (Scott Bemis ) says: >> Has anyone used Ubackup software from Unitech Software, Inc., >> Autoback from Werner & Monk, Inc., or Reel Backup from COSI, Inc. > >Ubackup does most of what you ask for, but it uses tar format -- which I >infer to mean that it has tar's pathname limit of 100 characters, and that's >not acceptable in my environment (some of our users love deep trees AND >long nodenames). We use Ubackup here where we work and we are very pleased with it. The biggest problem we have had with it is the documentation (which has been re-written for the next release I've been told). As to the tape format, Ubackup will use *either* tar or cpio. You can change the format with a simple change to a config file. You can also specify the blocking factor to increase tar performance if your drive can use large tape blocks. We use a blocking factor of 126 and it cut our backups to about half the time (TK50's on MicroVAXen...SLOW!). -- David Schmidt UUCP: davids@iscuva.ISCS.COM ISC Systems Corporation (seismo!uunet!iscuva!davids) East 22425 Appleway Phone: +1 509 927-5479 Liberty Lake, WA 99019
zemon@felix.UUCP (Art Zemon) (08/18/87)
I haven't used Ubackup because I was put off by their $6000 price tag. Maybe it's cheaper for machines smaller than a VAX 8700. -- -- Art Zemon FileNet Corporation Costa Mesa, California ...!hplabs!felix!zemon
root@ttidca.TTI.COM (The Root of all Evil) (08/21/87)
After extensive shopping ....
I got SysAdmin from UniSolutions Associates...great stuff.
SysAdmin lets you backup in either dump or cpio format (for logical file
backups), dd or volcopy (for physical backups) and dump, cpio or tar
(for file archives). The tapes are verified after each backup and an
on-line log file is created listing the files on each tape. Logfiles are
kept on-line (their duration depends on the system administrator -- the
expiration happens through the "delopr" script from crontab) and are
searched before file restores to locate the most recent tape that
the files can be found on. The package also provides operator, security,
system admin, archive, system config menus and _lots_ of other stuff ,alone
well worth the price, which is a third of the others. Install and documentation
is clear and complete. Adding local tools/menus and such easy.
SysAdmin can also be taught to perform non-UNIX filesystem backups
(such as Unify database backups) through the filesystem switch
interface (etc/fstab.opr).
Vendor Info: UniSolutions Associates (213) 641-6739
6520 Green Valley Circle, Suite 13-203
Culver City, Ca 90230
- Stephen
---------------------------- Stephen Mullin ---------------------------------
Citicorp / TTI
3100 Ocean Park Blvd. (213) 450-9111, ext. 2820
Santa Monica, CA 90405 {csun,philabs,randvax,trwrb}!ttidca!smfnf@mcdsun.UUCP (Fred Fish) (08/24/87)
In article <1143@ttidca.TTI.COM> sm@ttidcc.UUCP (The Root of all Evil) writes: >SysAdmin lets you backup in either dump or cpio format (for logical file >backups), dd or volcopy (for physical backups) and dump, cpio or tar >(for file archives). The tapes are verified after each backup and an I have seen this claim of "verifying the backup" now for a couple of "tape management" style backup systems that use cpio or tar for the actual backup. Have these vendors modified tar/cpio to do actual data verification or does their idea of verification mean being able to get a listing of the table of contents without getting any errors, or possibly extracting all the files into some spare disk partition and doing some sort of file comparison? -Fred -- = Drug tests; just say *NO*! = Fred Fish Motorola Computer Division, 3013 S 52nd St, Tempe, Az 85282 USA = seismo!noao!mcdsun!fnf (602) 438-3614
guardian@laidbak.UUCP (Harry Skelton) (08/25/87)
In article <358@mcdsun.UUCP> fnf@mcdsun.UUCP (Fred Fish) writes: > >Have these vendors modified tar/cpio to do actual data >verification or does their idea of verification mean being able to >get a listing of the table of contents without getting any errors, or >possibly extracting all the files into some spare disk partition and >doing some sort of file comparison? > >-Fred I have worked with tape drives and software under SCO Xenix and a couple of other drives under other Xenix's and found one thing that most end up doing: read under rewind or write-read durring output. A general read under rewind does ok but does not totally insure data but most tape problems that I have found have been hard errors anyway. The write-read functions do insure data but will cost you in backup time . Like 'tar' under any system, most of the Xenix based 'tar' utilities are streamers (from manufacturer) and are just as stupid as the original tar. ( script does not recieve error from tar although tar said it had problems ) I am certain that this may have been fixed by now but was brain dammaged when I worked on it (couple of months ago). 'dd' I can see being able to verify the tape with a streaming readback but with utilities like 'tar' or 'cpio', I too can't think of a 'good' way of verification other than a funky readback or write-read function. Quick question: Anyone know why 'dd' would not wait for a device? I have a tape drive attached to a large unix box (and found this true on other systems) and I use dd to read in from the tape. When the tape drive takes a moment to advance the tape, dd seems to continue without checking on the status of the device. Is this normal? Is this a problem with the device? Is this a problem with 'dd'? reply 'RE: dd and devices' Many thanks... Harry Skelton
haral@unisol.UUCP (09/02/87)
In article <153@tijc02.UUCP>, seb022@tijc02.UUCP (Scott Bemis) writes: >> I am considering purchasing a UNIX tape backup utility. What are your >> comments, compliments, complaints with any of these tools? Hi, I am just catching up with news so I'll try to answer your questions regarding our software... Essentially, Steve Mullin (sm@ttidca) was correct in describing the software, so I will try to answer or clarify only some of the points. >> Can this tool (from whatever company) >> a. handle multiple tape reels to backup a single file system if >> necessary? (I have large file systems) Yes. SysAdmin provides a front-end or a back-end to cpio (if needed -- when your cpio cannot handle multiple tapes) to span multiple tapes and to optionally label tapes. >> b. keep an online catalog of files backuped to each particular backup >> tape (tell me which tape a file may be on)? Yes. You decide how long you keep the on-line logs or you can recreate them using SysAdmin when you are looking for something that has already rolled off your on-line log files. >> c. perform both a full and incremental backup on a MOUNTED file system >> which may be modified as it is being backed up? Yes, but your mileage may vary... you can get away with it for a number of times, but if you heavily modify files as they are being backed up no program will guarantee you that your files will be consistent. There is no way under UNIX to suspend a file's changes (or to stop changes to a file) while you are backing up (a point already made in a previous message)... The only UNIX vendor that has even come close to providing a real tape checking utility is MASSCOMP which provides an option (c) to their dump or restor program (can't remember which one right now) that compares the data on the dump tape with the filesystem and reports the differences (i-nodes, blocks, etc). >> d. be able to restore individual files from either an incremental >> or full backups, Not require one to copy the entire tape to a >> spare area on a disk drive (if a such an area exists) like volcopy? Yes. >> e. be installed easily and easy to use, and well documented? Yes... Read the July issue of UNIX WORLD for a review of SysAdmin. >> OR has someone (a genius) found a way to meet these requirements >> using standard AT&T UNIX V utilities? SysAdmin uses volcopy for full physical dumps and cpio for full or incremental logical dumps. SysAdmin provides the front-end under which you can backup at various levels just like under BSD systems but still using cpio under System V. Of course, on BSD systems, SysAdmin uses dump and restore for logical dumps and dd for physical dumps. >> Scott Bemis --Haral Tsitsivas UniSolutions Associates (213) 641-6739 ...!seismo!scgvaxd!ashtate!unisol!haral
fnf@mcdsun.UUCP (Fred Fish) (09/03/87)
So far I have resisted jumping into this backup discussion to avoid flames about commercialism but I'd just like to note the availability of another option that does most of what everyone has been talking about. For availability, see disclaimer below. In article <284@unisol.UUCP> haral@unisol.UUCP (Haral Tsitsivas) writes: >message)... The only UNIX vendor that has even come close to providing >a real tape checking utility is MASSCOMP which provides an option (c) to >their dump or restor program (can't remember which one right now) that >compares the data on the dump tape with the filesystem and reports the >differences (i-nodes, blocks, etc). Not quite, Motorola has been shipping a product called bru (Backup and Restore Utility) for several years as a standard part of their Unix system, that also does archived-file versus filesystem-file comparisions as an optional pass to do backup verification. It also does checksums on each and every block so there is yet another mode (inspection) which goes through and checks the archive for self-consistency and proper checksums. >SysAdmin uses volcopy for full physical dumps and cpio for full or >incremental logical dumps. SysAdmin provides the front-end under which you >can backup at various levels just like under BSD systems but still using >cpio under System V. Of course, on BSD systems, SysAdmin uses dump and >restore for logical dumps and dd for physical dumps. It was precisely the problems with volcopy/cpio/tar/dump/restore etc that caused me to retch and write bru in the first place. All archives are fully compatible across all supported machines ("unix boxes", Amiga) and all supported OS's (BSD, SysV, Xenix, AmigaDOS). It differs from packages like UBackup and SysAdmin in that it does very little system administration type things and concentrates on providing the most robust and portable "low level backup mechanism" possible. You might say they are complimentary packages. Disclaimer: I'm rather biased as I am the author of bru. It has been licensed by Motorola (along with Intel and "others") for unlimited binary distribution as a part of their Unix product. Don't call either Motorola or I about it, as all marketing is done by an marketing agent. His phone number is (602) 820-0042. -Fred -- = Drug tests; just say *NO*! = Fred Fish Motorola Computer Division, 3013 S 52nd St, Tempe, Az 85282 USA = seismo!noao!mcdsun!fnf (602) 438-3614
fnf@mcdsun.UUCP (Fred Fish) (09/04/87)
In article <367@mcdsun.UUCP> fnf@mcdsun.UUCP (Fred Fish) writes: >"low level backup mechanism" possible. You might say they are complimentary ^^^^^^^^^^^^^ Though I have received compliments that should have been "complementary" :-) -Fred -- = Drug tests; just say *NO*! = Fred Fish Motorola Computer Division, 3013 S 52nd St, Tempe, Az 85282 USA = seismo!noao!mcdsun!fnf (602) 438-3614
haral@unisol.UUCP (Haral Tsitsivas) (09/05/87)
In article <367@mcdsun.UUCP> fnf@mcdsun.UUCP (Fred Fish) writes: >In article <284@unisol.UUCP> haral@unisol.UUCP (Haral Tsitsivas) writes: >>message)... The only UNIX vendor that has even come close to providing >>a real tape checking utility is MASSCOMP... >Not quite, Motorola has been shipping a product called bru (Backup and >Restore Utility) for several years as a standard part of their Unix >system... I do remember BRU now from earlier days when it was posted to the net in binary form for the 4.1C release of BSD, but, since I don't work with any Motorola releases, I couldn't comment. Another issue though is, how many system administrators can keep the system down long enough to do block by block checking of dump tapes? If you have a production level system (or a large development system) with several eagles (and GBytes) you may only have enough time to do complete backups and not block by block checking of the tape and disk files (users never want to lose the machine for long periods of time, or be slowed down while a backup is done when they are logged in). If a tape-speed (or near tape-speed) method for verifying the sanity of backups exists and works reliably (under most systems, although not as reliably as the block comparison method) then most people will use that... --Haral Tsitsivas UniSolutions Associates (213) 641-6739 ...!seismo!scgvaxd!ashtate!unisol!haral
hedrick@topaz.rutgers.edu (Charles Hedrick) (09/09/87)
Our backup scripts all begin by cat'ing a random binary file to the tape, cat'ing it back into a temp file, and diff'ing the results. This should catch any consistent tape drive failures. So far we haven't seen any problems that this didn't catch. One can certainly imagine intermittent failures that would get by it, but there's a lot to be said for a test that is cheap and catches most of the problems.
gew@dnlunx.UUCP (Weijers G.A.H.) (09/11/87)
In article <160@tijc02.UUCP>, seb022@tijc02.UUCP (Scott Bemis ) writes: > Why can I afford to verify backup tapes on a VAX using the VMS operating > system, but may not have the time nor utilites to verify backup tapes on a VAX > using the UNIX operating system? > > The VMS operating system - DEC's proprietary operating system for it's VAXes - > has a verify option. It is known as BACKUP/VERIFY ......... BACKUP/VERIFY does *not* check whether the data on tape is consistent with the data on disk. On the other hand, BACKUP stores its data in a redundant way, which is a lot safer than using tar/cpio etc. -- | Ge' Weijers | Disclaimer: the views expressed | | PTT Dr. Neher Laboratories | are usually entirely my own, | | Leidschendam, the Netherlands | not my employers'. |
mangler@cit-vax.Caltech.Edu (System Mangler) (09/21/87)
In article <160@tijc02.UUCP>, seb022@tijc02.UUCP (Scott Bemis ) writes: > Why can I afford to verify backup tapes on a VAX using the VMS operating > system, but may not have the time nor utilites to verify backup tapes on a VAX > using the UNIX operating system? Verifying backup tapes can be done to varying degrees, depending upon the resources you're willing to commit: running a tape exerciser when the tape is bought, to reject defectives. Does not slow down backups at all. reading the tape end-to-end looking for I/O errors. Can run at full speed of tape drive. read tape, checking checksums. Double-buffering can make this run near full speed. read tape, checking that files have the number of blocks claimed. Less amenable to double-buffering. Still does not catch directory inconsistencies, nor does it verify contents. Verify that every file on the tape has a parent directory. Requires building large data structure while reading tape. (E.g. "restore t"). do backup a second time, and compare block-for-block. Gets out of sync at first difference - not useful for active filesystems such as the root. Parallelism possible, so compare can probably be made to run only slightly slower than the real backup. do backup a second time, and "diff" against the tape, file by file. Usable on active filesystems. Inherently sequential. Restore backup onto spare partition. Slow... also does not tell you whether the result is equivalent to the filesystem you started with. Which of these are worth the cost?
daveb@geac.UUCP (Brown) (09/22/87)
In article <160@tijc02.UUCP>, seb022@tijc02.UUCP (Scott Bemis) writes: > Why can I afford to verify backup tapes on a VAX using the VMS operating > system, but may not have the time nor utilities to verify backup tapes on a VAX > using the UNIX operating system? In article <4040@cit-vax.Caltech.Edu> mangler@cit-vax.Caltech.Edu (System Mangler) writes: >Verifying backup tapes can be done to varying degrees, depending upon >the resources you're willing to commit: >[a long list of possible meanings of "verify"] >Which of these are worth the cost? Methinks mangler missed a very popular meaning of verify e: run the backup again, but this time have the program only compare the contents of the tape to the files/directory-surrogates it would be writing normally. This is the usual meaning of "verify" on a commercial OS. The price is 1) one complete backup's worth of time, 2) many warnings of the form "tape written correctly but doesn't match disk" if you're doing a backup on a live system, 3) you have to get a backup program which *does* verify. --dave -- David Collier-Brown. {mnetor|yetti|utgpu}!geac!daveb Geac Computers International Inc., | Computer Science loses its 350 Steelcase Road,Markham, Ontario, | memory (if not its mind) CANADA, L3R 1B3 (416) 475-0525 x3279 | every 6 months.
tihor@acf3.NYU.EDU (Stephen W. Tihor) (10/29/87)
gew@dnlunx.UUCP (Weijers G.A.H.) writes that: BACKUP/VERIFY does *not* check whether the data on tape is consistent with the data on disk. which is strange since it acts like it does. I know its not buffering the data in memory when it rereads the tape to compare with and it seems to access disk just as much. I know BACKUP/COMPARE does since it gives error message by disk bblock of file for differences. Come to think of it so does BACKUP/verify ... now if DEC would just release VMS BACKUP for Ultrix (Ala their FOrtran, C and LISP products.)