david@ci-dandelion.UUCP (04/28/87)
I'm starting to look for any software that provides a superset of the Berkeley-style dump(8) and restore(8) utilities. Of course, it would be great if we came across such a package at no cost, but we don't really expect to find one with everything we want without having to pay for it. If we find a product we like, we will seek to acquire a license to distribute it with our Ultrix-based product which runs on a DEC Microvax II. The users of our product are mechanical engineers who should not have to learn about the many details of system management. Here are some capabilities that would be nice: - Granularity smaller than a filesystem - At least ten incremental levels - Large written tape blocks for reasonable speed to a TK-50 tape - Easy, reliable recovery from read or write errors - Support for large archive files and multiple tape volumes - Maintenance of a database of the names of the files archived - A command shell usable by non-computer-oriented people, possibly using full-screen forms and curses routines - Source code availability Please let me know if you have experience with any software package to replace or augment dump and restore, or if your company offers one, or if you are interested in information about one. If the interest warrants it, I'll post a summary of all responses I get. Thanks, -David Watson Cognition, Inc. (617) 667-4800 Billerica, MA
mkhaw@teknowledge-vaxc.ARPA (Michael Khaw) (04/28/87)
In article <1308@ci-dandelion.UUCP> david@ci-dandelion.UUCP (David Watson) writes: > >I'm starting to look for any software that provides a superset of the >Berkeley-style dump(8) and restore(8) utilities. Of course, it would be [ more detailed specs omitted ] >Please let me know if you have experience with any software package >to replace or augment dump and restore, or if your company offers one, or >if you are interested in information about one. If the interest warrants it, >I'll post a summary of all responses I get. Here's my vote that you post a summary. I've only seen discussion of vanilla tar and dump/restore (for 4bsd) and cpio, volcopy, and labelit (Sys V) on net news. And there's multivol and cousins from mod.sources. Of commercial offerings, I've only heard of Ubackup (Unitech, Inc) and Reel Utilities (COSI), but have no idea how good/bad either is. Both advertise in "Unix Review" and such. (OK, inews, here's extra lines to keep you happy) Mike Khaw -- internet: mkhaw@teknowledge-vaxc.arpa usenet: {hplabs|sun|ucbvax|decwrl|sri-unix}!mkhaw%teknowledge-vaxc.arpa USnail: Teknowledge Inc, 1850 Embarcadero Rd, POB 10119, Palo Alto, CA 94303
stever@videovax.Tek.COM (Steven E. Rice, P.E.) (04/28/87)
In article <1308@ci-dandelion.UUCP>, David Watson (david@ci-dandelion.UUCP) writes: > I'm starting to look for any software that provides a superset of the > Berkeley-style dump(8) and restore(8) utilities. . . . I would like to find something that is a *real* superset! One feature that would be invaluable is the ability to back up the filesystem while the machine is loaded with users and running. The present backup is supposed to cause unspecified evil to occur if all users aren't off the machine. . . Particularly, I would like to find an incremental backup utility that would lurk in the background and periodically, at user-specified times (every hour, twice a day, whatever), back up files that have been changed since the last run. Since our VAX is small (an 11/750) and runs unattended, it would need the ability to keep writing to the same tape until the tape is filled, and then holler for help (preferably by mail, or some such). It would also need to be reasonable about its use of the tape drive -- there must be some way to get it to give up the drive so other things could be done, without at the same time requiring the operator to mount a tape each time it wanted to run. All of this was running on the Sigma 7 at Montana State University, Bozeman, Montana, almost 20 years ago! Surely, someone has done the same kind of backup utility for the VAX!!! Steve Rice ----------------------------------------------------------------------------- new: stever@videovax.tv.Tek.com old: {decvax | hplabs | ihnp4 | uw-beaver | cae780}!tektronix!videovax!stever
jules@zen.UUCP (Julian Perry) (05/01/87)
In article <1308@ci-dandelion.UUCP> david@ci-dandelion.UUCP (David Watson) writes: >Here are some capabilities that would be nice: > > - Granularity smaller than a filesystem > - At least ten incremental levels > - Large written tape blocks for reasonable speed to a TK-50 tape > - Easy, reliable recovery from read or write errors > - Support for large archive files and multiple tape volumes > - Maintenance of a database of the names of the files archived > - A command shell usable by non-computer-oriented people, > possibly using full-screen forms and curses routines > - Source code availability I have just started to rewrite our current backup system to be the "all new super-duper do-it-all definitive backup (honestly)". Features will include: Table driven including: a) List of available backup devices, for each device: i) The system the device is connected to ii) The path name of the device iii) The capacity of the device iv) The command used to write data to the device (optional) v) The command to rewind and unload the device (optional) b) List of which systems to backup on a network c) What to backup and in what order Each tape (or disc, or whatever) will be in cpio format (but this could be changed) and multiple copies can be made simultaneously. Every tape is a complete cpio archive (i.e. you don't need to read the first tape to get at what's on the second etc) and the first file is an index of what is on the tape. [This we already have in our current system] Features will not include anything user-friendly, sorry! Having said that, what could you want to be user-friendly? Only 'qualified' administrators will need such elaborate backup schemes, and they wouldn't be seen dead using 'user-friendly' software anyway! :-) Any SENSIBLE suggestions and ideas on the subject would be appreciated. By the way, this is NOT all pie-in-the-sky stuff, much of it already works and I am committed to finishing it soon. Aside from that, I need the damn thing yesterday! Jules [the fully-backed up] -- IN-REAL-LIFE: Julian Perry E-MAIL: jules@zen.co.uk || ...!mcvax!ukc!zen.co.uk!jules PHONE: +44 532 489048 ext 217 ADDRESS: Zengrange Limited, Greenfield Road, Leeds, England, LS9 8DB
wcs@ho95e.ATT.COM (Bill.Stewart) (05/02/87)
In article <4360@videovax.Tek.COM> stever@videovax.Tek.COM (Steven E. Rice, P.E.) writes: >In article <1308@ci-dandelion.UUCP>, David Watson (david@ci-dandelion.UUCP) >> I'm starting to look for any software that provides a superset of the >> Berkeley-style dump(8) and restore(8) utilities. . . . >that would be invaluable is the ability to back up the filesystem while >the machine is loaded with users and running. ..... >Particularly, I would like to find an incremental backup utility that >would lurk in the background and periodically, at user-specified times >(every hour, twice a day, whatever), back up files that have been >changed since the last run. Incremental backups have their risks - they don't notice if files have been *removed*, and can get confused if links change. But they're easy to use with live systems, and are normally adequate when used with weekly or biweekly full backups. On System V, you don't have dump/restor - a common approach is to use volcopy (a souped-up dd which can handle multiple tapes for a single disk slice) for image copies, and do incrementals with find/cpio. ("ff" is a much faster find which reads the ilist instead of recursively searching directories, but the effect is the same.) Here's a crude backup program: mv /etc/backuptimes/$i /etc/backuptimes/old$i date >/etc/backuptimes/$i find $i -newer /etc/backuptimes/old$i -print | \ cpio -ocBv >/dev/rmt/0hn 2>>/etc/backuptimes/log$i ### ^^tape-drive, no-rewind If you don't like cpio, use xargs tar $TAROPTIONS. We have spare disk space, so we dedicate a disk slice to incrementals, and copy the slice to (the same) tape during the day. -- # Bill Stewart, AT&T Bell Labs 2G-202, Holmdel NJ 1-201-949-0705 ihnp4!ho95c!wcs
ed@mtxinu.UUCP (Ed Gould) (05/04/87)
>Incremental backups have their risks - they don't notice if files have >been *removed*, and can get confused if links change. But they're easy >to use with live systems, and are normally adequate when used with >weekly or biweekly full backups. Real incremental dumps - like those made with dump - *do* notice that files have been removed or that links have changed. This has been true since the Sixth Edition (although there were bugs in the V6 dump/restor that were fairly serious). The serious bugs were fixed in V7, and facilities for easier extraction of single files from a dump were added in 4.2BSD restore (other facilities were there all along, but they were clumsy to use). >On System V, you don't have dump/restor - a common approach is to use >volcopy (a souped-up dd which can handle multiple tapes for a single >disk slice) for image copies, and do incrementals with find/cpio. Why USG decided to drop dump/restor from System III (or maybe it was still distributed with SysIII but discouraged), I'll never understand. Extracting one file from a volcopy tape is near on impossible unless there's a spare disk partition available; many sites don't have enough free disk to allow that. Especially with large disks and large partitions, it's hardly economical to keep 150Mbytes (a typical largest partition size) free all the time. (It actually is possible to extract one file from a volcopy of a filesystem, but in general it can take as many as four passes over all of the tapes: One to read the inode and the single indirect blocks, a second to read double indirect blocks, a third to read triple indirect blocks and a fourth to read the actual data. Of course, for a small file, fewer passes may suffice.) -- Ed Gould mt Xinu, 2560 Ninth St., Berkeley, CA 94710 USA {ucbvax,decvax}!mtxinu!ed +1 415 644 0146 "A man of quality is not threatened by a woman of equality."
jgp@moscom.UUCP (05/07/87)
There are at least two companies (Exabyte and Honeywell) offering tape
systems with > 2000 Mb storage per tape (using normal video tape). How
will systems like these affect current backup practices?
Since entire filesystems will fit on a single tape, doing
full dumps instead of incrementals becomes more attractive.
While dumps and tars will still need time to run (to traverse
the filesystem) a dd image could be nice and quick. Awkward
to restore individual files from but nice protection against
head crashes ("System pausing for 10 minutes for disk check-
pointing, please stand by.").
It is a lot less work to do the full dump, mkfs, full restore
procedure to optimize disk performance with a single tape,
especially if you want to have more than one copy of your
full dump before you do the mkfs. Of course more optimized
disks allows for faster full dumps too.
Anybody else have any comments? I'd be especially interested in
hearing from anybody who has actually used one of these systems.
The ads and glossies make them sound great but thats what ads and
glossies are for.
--
Jim Prescott rochester!moscom!jgpmangler@cit-vax.Caltech.Edu (System Mangler) (05/11/87)
In article <4360@videovax.Tek.COM>, stever@videovax.Tek.COM (Steven E. Rice, P.E.) writes: > One feature > that would be invaluable is the ability to back up the filesystem while > the machine is loaded with users and running. This is not generally workable. Consider a filesystem with three directories, A, B, and C. After the hypothetical backup program finishes looking through A, and is busy looking through B, some user moves a file from C to A. It will not be backed up. You can also work it the other direction, and get two copies of the file. I understand that some Burroughs operating system solves this by putting its filesystems into "snapshot mode" for the duration of the backup, where all files/directories to be backed up are treated as copy-on-write. Requires adequate free space... Don Speck speck@vlsi.caltech.edu {seismo,rutgers,ames}!cit-vax!speck
mangler@cit-vax.Caltech.Edu (System Mangler) (05/11/87)
In article <353@mtxinu.UUCP>, ed@mtxinu.UUCP (Ed Gould) writes: > Why USG decided to drop dump/restor from System III (or maybe it was > still distributed with SysIII but discouraged), I'll never understand. It was probably too much work to make restor work on a dual-blocksize filesystem such as that found in Sys V. Restor believes that disk and tape data both come in units of BSIZE. Getting it to convert 512-byte tape records to 1024-byte disk blocks (or vice versa - take your pick) is a *pain*. Dump's speed is limited by the small blocksize of System V filesystems. Volcopy can read in large chunks. On the other hand, dump doesn't waste time copying free blocks like volcopy does. With slow tape drives (or UN52 controllers), dump wins; with fast tape drives, volcopy wins. Don Speck speck@vlsi.caltech.edu {seismo,rutgers,ames}!cit-vax!speck
stever@videovax.UUCP (05/12/87)
In article <2653@cit-vax.Caltech.Edu>, Don Speck (alias mangler@cit-vax.Caltech.Edu) writes: > In article <4360@videovax.Tek.COM>, stever@videovax.Tek.COM (Steven E. Rice, P.E.) writes: >> One feature >> that would be invaluable is the ability to back up the filesystem while >> the machine is loaded with users and running. > This is not generally workable. Consider a filesystem with three > directories, A, B, and C. After the hypothetical backup program > finishes looking through A, and is busy looking through B, some > user moves a file from C to A. It will not be backed up. You > can also work it the other direction, and get two copies of the > file. . . . But in fact, it is generally workable! In the worst case, you get no backup, which is just exactly what happens right now [ 8^( ]. In all other cases, there is at least one backup [ 8^) ]. And unless the file that was moved and missed being backed up is moved every hour on the hour (or twice a day, or however frequent the backup is), it will get caught the next time around. The frustration I have with all this "modern," "friendly," "powerful" software is that the backup system I have described was running on the Scientific Data Systems (SDS) Sigma 7 (later Xerox Sigma 7, then Honeywell Sigma 7, but that's another story. . .) that was installed at Montana State University, Bozeman, Montana, in 1967!!! In all the years I spent pounding on the Sigma 7 (1969 to 1975), the most I ever lost was 2 hours of work (a crash occurred just as an incremental backup tape was being mounted). On a VAX which is brought down for an hour each evening for incremental backups, a crash can easily cost an entire day's work. Perhaps by 1990 we'll be able to catch up to where we were in 1970??? Steve Rice ----------------------------------------------------------------------------- new: stever@videovax.tv.Tek.com old: {decvax | hplabs | ihnp4 | uw-beaver | cae780}!tektronix!videovax!stever