jensen%cs.uiowa.edu@relay.cs.net (Nancy Jensen) (11/20/86)
Somewhere, sometime in the past I think I saw a reference to problems encountered when trying to restore from Level 0 dumps taken on an active filesystem. Are there indeed such problems? (In particular on a bsd system). Thanks, Nancy Jensen jensen@cs.uiowa.edu
Rudy.Nedved@h.cs.cmu.edu (11/20/86)
Nancy, When I worked on dump for 4.1BSD (I have not worked on the 4.2 or 4.3 versions but believe the design is still same), the act of running several passes over the inode list for a filesystem was prone to race conditions and the act of chasing down disk addresses without going thru the operating system was prone to problems. The classic failure mode was when a very large file was truncated at just the right time and another large file was being created. Dump would be processing the disk addresses from its copy of the inode. If the released disk addresses are reused fast enough...you will get the wrong data recorded on tape. If the file was large enough and you were chasing down indirect pointers and the indirect pointer block had been replaced by random data... you would get a large number of block read (bread) failures. The only solution that CMU came up with for running backups was to run level 0 dumps on an inactive file system so we *knew* the data was valid and to modifiy dump so when we ran higher level dumps on active filesystems...a bread failure during the processing of an indirect block would cause the rest of the file to be ignored. We seemed to have gotten away with it but we are relying more and more on incrementals which have a large potential of recording the wrong data for a file. The only logic that works is that old stable files are backed up and that tends to be good enough for us. -Rudy
billd@celerity.UUCP (Bill Davidson) (10/25/88)
I am trying to write tapes from Microport System V/AT (80286 yech!) and
I would like to use a format which can be read by BSD 4.3's dump/restore
programs (so that I can read my backups from a real computer). I have
the source for dump/restore on Berkeley but it's very much bound to the
Berkeley file system and it appears that it would be rediculously difficult
to port. The BSD man pages and the header files describe what the dump
headers look like but not really how a typical tape archive is written and
arranged. I would appreciate any info from anyone who could give me a better
description of the arrangement of the data on the tape. If anyone knows
of a public domain version of this progam that runs on System V/AT I
would also like to know of it.
--Bill Davidson
P.S. Don't tell me to use tar or cpio. That's what I do now and it is
severely inadequate.
-------------------------------------------------------------------------------
<usual disclaimers>
UUCP: .....!{ucsd|sdcsvax}!celerity!billdwnp@dcs.UUCP (Wolf N. Paul) (10/26/88)
In article <178@celerity.UUCP> billd@celerity.UUCP (Bill Davidson) writes: >I am trying to write tapes from Microport System V/AT (80286 yech!) and >I would like to use a format which can be read by BSD 4.3's dump/restore >programs (so that I can read my backups from a real computer). I have >the source for dump/restore on Berkeley but it's very much bound to the >Berkeley file system and it appears that it would be rediculously difficult >to port. I'd be careful with that, anyway. I have run into problems with the dump format being different on different machines, all running BSD. Why not use TAR? or cpio or its public-domain brother afio? Both of these options would be much more portable than dump. -- Wolf N. Paul * 3387 Sam Rayburn Run * Carrollton TX 75007 * (214) 306-9101 UUCP: killer!dcs!wnp ESL: 62832882 DOMAIN: dcs!wnp@killer.dallas.tx.us TLX: 910-380-0585 EES PLANO UD
davidsen@steinmetz.ge.com (William E. Davidsen Jr) (10/27/88)
In article <178@celerity.UUCP> billd@celerity.UUCP (Bill Davidson) writes: | I am trying to write tapes from Microport System V/AT (80286 yech!) and | I would like to use a format which can be read by BSD 4.3's dump/restore | programs (so that I can read my backups from a real computer). I have | the source for dump/restore on Berkeley but it's very much bound to the | Berkeley file system and it appears that it would be rediculously difficult | to port. Since you're summary was "cpio is not a real backup program" I have to ask "why not?" I go between SysIII, SysV, micropost, xenix, Ultrix and SunOS using cpio, and it seems real enough for general use. If there's a problem you have, warn us about it. Hopefully one of the things which will come out of the xenix SysV merge is dump again. The V.4 info seems to say you can dump the fast filesystem stuff but not the SysV f/s's. Since xenix does this, it can't be *that* hard! (the REAL bill davidsen) -- bill davidsen (wedu@ge-crd.arpa) {uunet | philabs}!steinmetz!crdos1!davidsen "Stupidity, like virtue, is its own reward" -me
jpd@usl-pc.usl.edu (DugalJP) (10/28/88)
In article <12433@steinmetz.ge.com> davidsen@crdos1.UUCP (bill davidsen) writes: > Since you're summary was "cpio is not a real backup program" I have to >ask "why not?" I go between SysIII, SysV, micropost, xenix, Ultrix and >SunOS using cpio, and it seems real enough for general use. If there's a >problem you have, warn us about it. Having seen the dreaded "Out of phase, get help" message one too many times, I no longer use cpio for system backups. Dump/restore lets me skip over tape errors, resync, and access the remainder of a tape [this is the most often error I encounter]. If I get a write error on an output tape, Dump will let me rewrite that reel without having to start from the beginning reel (cpio would have died horribly!). Cpio has its uses, but (in my opinion) producing reliable system backups is NOT one of them. -- James Dugal, N5KNX USENET: ...!{dalsqnt,killer}!usl!jpd Associate Director Internet: jpd@usl.edu Computing Center US Mail: PO Box 42770 Lafayette, LA 70504 University of Southwestern LA. Tel. 318-231-6417 U.S.A. -- -- James Dugal, N5KNX USENET: ...!{dalsqnt,killer}!usl!jpd Associate Director Internet: jpd@usl.edu Computing Center US Mail: PO Box 42770 Lafayette, LA 70504 University of Southwestern LA. Tel. 318-231-6417 U.S.A.
guy@auspex.UUCP (Guy Harris) (10/28/88)
> Hopefully one of the things which will come out of the xenix SysV >merge is dump again. The V.4 info seems to say you can dump the fast >filesystem stuff but not the SysV f/s's. Since xenix does this, it can't >be *that* hard! The history here is a bit interesting. Basically, "dump"/"restor" (no "e", notice) came from V7. AT&T marked it as "deprecated" in S3, and dropped it in S5. Other vendors, such as, presumably, Microsoft, saw no reason to drop it.... 4BSD didn't drop it either; in fact, they beefed "dump" up quite a bit. "dump" was then rewhacked quite a bit to make the 4.2BSD version (and then rewhacked some more to make the 4.3BSD version). Probably, if you want "dump" for the System V file system, you should start with the 4.1BSD version, and whack it as necessary to support file systems with a block size other than 1K (and maybe throw in goodies from the 4.2BSD and 4.3BSD ones as well, such as "remote magtape" support and the multiple buffering stuff). "restor(e)" is a different question. You should definitely start with the 4.1BSD one (since, as I remember, it understands the "s_tfree" and "s_tinode" fields, while the V7 one and even the S3 one didn't). You definitely want to add the "restore by name" feature that "restore" has - it's a colossal win. I don't know whether it should restore only to a directory, rather than a raw file system, as "restor" does. I suspect that, if S5R4 doesn't have "dump" or "restor(e)" for the S5 file system, it's only because they don't like it and think "volcopy"/"finc"/"frec"/"frick"/"frack"/... are better, or because they don't have the effort to invest in bringing it back.
billd@celerity.UUCP (Bill Davidson) (10/28/88)
A couple of postings and several mail messages to me all say about the same thing about my desire to use dump/restore. They all essentially say, "What's wrong with tar and cpio?" If you've been following a number of discussions concerning floppy backups recently in comp.unix.microport you'd know that cpio occasionally loses something. I've have it die after reading about 25Meg of a tape that had 35Meg on it originally. This is completely unacceptable. The reason that tar is unacceptable it that I can not back up my entire hard disk on one tape (130Meg hard disk and ~40Meg tape) and I have to figure out by hand how many tapes and which directories to tar to each tape. It doesn't work at all for incremental backups. I have to make a complete list of files for an incremental and that list is usually too long for tar. Cpio takes file names from stdin so I have to use it for incrementals and because of the problem stated above, that worries me. Also, neither program is particularly convient to restore a file with. dump/restore is a much friendlier program. One person did mention that the format does change somewhat from machine to machine. That could cause some problems but I would like to at least try. Oh, and for the xenix-person that mentioned it, System V/AT does not come with any version of dump/restore (although it has a header file describing the format which is almost completely different from it's BSD counterpart). -- Bill Davidson .....!ucsd!celerity!billd
mcneill@eplrx7.UUCP (mcneill) (10/28/88)
In article <12433@steinmetz.ge.com>, davidsen@steinmetz.ge.com (William E. Davidsen Jr) writes: > In article <178@celerity.UUCP> billd@celerity.UUCP (Bill Davidson) writes: > | I am trying to write tapes from Microport System V/AT (80286 yech!) and > | I would like to use a format which can be read by BSD 4.3's dump/restore > | programs (so that I can read my backups from a real computer). I have > | the source for dump/restore on Berkeley but it's very much bound to the > | Berkeley file system and it appears that it would be rediculously difficult > | to port. > > Since you're summary was "cpio is not a real backup program" I have to > ask "why not?" I go between SysIII, SysV, micropost, xenix, Ultrix and > SunOS using cpio, and it seems real enough for general use. If there's a > problem you have, warn us about it. > Many versions of cpio (from original AT&T code) have a bug which makes multiple volume cpio's unreliable. Many times I have tried to restore multi-volume cpio archives without success. The error occurs when you switch tapes. Cpio sometimes does not correctly write the info needed to switch to the next archive tape. -- Keith D. McNeill | E.I. du Pont de Nemours & Co. uunet!eplrx7!mcneill | Experimental Station (302) 695-7395 | P.O. Box 80357 | Wilmington, Delaware 19880-0357
karl@sugar.uu.net (Karl Lehenbauer) (10/29/88)
cpio is supposed to have a salvage mode in Bell Tech Sys V/386 3.1 and presumably in others. It would be relatively easy to read the tape no-rewind-on-close, then after the error, read the tape until you saw something that looked like a cpio header, then feed the rest back to cpio. I started writing something like this, then decided to hack the PD cpio to do it instead, but then I saw that 3.1 had it anyway, so I blew it off. Note that salvaging cpio archives is a potential avenue for Trojan Horses as users could put fake tricked-up cpio entries in big files and hope that they would be seen as valid entries after an error. Note also that cpio archives within cpio archives can screw up salvage mode -- it can see the embedded files, recover them, then see the trailer record of the embedded file and exit, blowing off the rest of the archive. -- -- "We've been following your progress with considerable interest, not to say -- contempt." -- Zaphod Beeblebrox IV -- uunet!sugar!karl, Unix BBS (713) 438-5018
erik@mpx2.UUCP (Erik Murrey) (10/29/88)
All of this tar/cpio/dump discussion raises some questions for
me:
Sys-V tar doesn't copy directory permissions (or even directory
entries!) to the tape. This means that empty directories don't
get backed up and permissions get guessed during a restore. This
stinks. (I know BSD's tar does this correctly)
I like cpio becuase it dumps everything to tape. Even directories
and special files like named pipes, etc. I does not, however, allow
you to split the backup across several tapes (or disks). This
is a big loss for people without tape drives (me).
Dump/restore is very quick, and it backs up everything, but I have
a few questions: It seems to dump the entire filesystem, including
the superblock/inode dumps. It also seems to me that it dumps
disk blocks in order of the disk itself, rather than the order of the
file. This means that restoring a filesystem will *not* reduce
the fragmentation. (Which is often why I backup/restore in the
first place.) Does it also require the exact same filesystem
to by restored on? If this is so, then I can't use dump/retore to
expand a full filesystem.
The docs I have for dump/restore don't explain details like
this to me. Please fill me in!
---
Erik Murrey
MPX Data Systems, Inc.
erik@mpx1.UUCP
...!{bpa,spl1,cbmvax,vu-vlsi}!mpx1!erikhsu@santra.HUT.FI (Heikki Suonsivu) (10/30/88)
In article <12433@steinmetz.ge.com> davidsen@crdos1.UUCP (bill davidsen) writes: > Since you're summary was "cpio is not a real backup program" I have to >ask "why not?" I go between SysIII, SysV, micropost, xenix, Ultrix and >SunOS using cpio, and it seems real enough for general use. If there's a My CT miniframe refuses to read cpio output from 386 microport or afio. I got around the problem by compiling afio on my computers and it works nicely, I just patched it to ask next floppy after it really has flushed the buffers, original did ask for new floppy when still writing on to it. I use it for all my backups. Not as fast as fastbacks etc on dos, but maybe I haven't checked out all the options. Does microport 386 use different DMA channel for floppies? If yes, then it could be faster?
friedl@vsi.COM (Stephen J. Friedl) (10/31/88)
In article <2921@sugar.uu.net>, karl@sugar.uu.net (Karl Lehenbauer) writes: > cpio is supposed to have a salvage mode in Bell Tech Sys V/386 3.1 and > presumably in others. The 3B2 has this in Sys V Rel 3.1 as well. It is *really* handy... -- Steve Friedl V-Systems, Inc. +1 714 545 6442 3B2-kind-of-guy friedl@vsi.com {backbones}!vsi.com!friedl attmail!vsi!friedl ----Nancy Reagan on 120MB SCSI cartridge tape: "Just say *now*"----
rick@seismo.CSS.GOV (Rick Adams) (10/31/88)
cpio is not a backup program. Neither is tar (nor tp for that matter). cpio and tar are ARCHIVE programs. A BACKUP program should be able to restore the disk to the state it was at the time of the backup. It should also offer incremental backups. The reason cpio can't handle it is that it has no way to not restore files that have been removed since the initial backup. This is real important as anyone who has tried to restore a corrupted disk from a cpio "backup" and has found that the "backup" is too big to fit on the disk it came from can tell you. volcopy barely qualifies as a backup program, but it is much too clumsy to use in general. (especially if your removable media is smaller that the filesystem you want to back up) --rick
ggs@ulysses.homer.nj.att.com (Griff Smith) (10/31/88)
In article <326@auspex.UUCP>, guy@auspex.UUCP (Guy Harris) writes: > ... Probably, if you want "dump" for the System V file system, you should > start with the 4.1BSD version, and whack it as necessary to support file > systems with a block size other than 1K (and maybe throw in goodies from > the 4.2BSD and 4.3BSD ones as well, such as "remote magtape" support and > the multiple buffering stuff). I (and others, I suspect) have done part of this. I started with the 4.1dump that comes with 4.[23]BSD and then made it work with System Vr3. An associate added network support to my stuff. Performance is unimpressive; similar to tar and cpio. The 1k blocks really slow it down. I haven't had the courage (or time) to do all the things required to deal with other block sizes. > "restor(e)" is a different question. You should definitely start with > the 4.1BSD one (since, as I remember, it understands the "s_tfree" and > "s_tinode" fields, while the V7 one and even the S3 one didn't). If you are starting with 4.3BSD source, the 4.3BSD restore is a good starting point. The `4.2BSD compatibility mode' works just fine. 4.3 restore was an easier port than 4.1 dump. > I suspect that, if S5R4 doesn't have "dump" or "restor(e)" for the S5 > file system, it's only because they don't like it and think > "volcopy"/"finc"/"frec"/"frick"/"frack"/... are better, or because they > don't have the effort to invest in bringing it back. My guess is that most of us don't know much about it any more. Tribal memory is short. Many of the old guard who knew about dump/restor have `died'. Some others have heard legends of a place where backups are fast and convenient, but they are usually dismissed as the ravings of the old generation. Another point: backup devices on the 3B2 processors I have used have been incredibly slow, which lowers the advantage of fast dump programs. Few of our System V users realize how fast backup can be when you use the 4.3BSD dump with a 125 ips tape drive and a fast file system. I'll take part of that back. Volcopy onto a 200 ips tape drive on the IBM mainframe at the Comp Center goes like a bat out of hell, but you don't maintain the speed when you do incrementals with cpio. -- Griff Smith AT&T (Bell Laboratories), Murray Hill Phone: 1-201-582-7736 UUCP: {most AT&T sites}!ulysses!ggs Internet: ggs@ulysses.att.com
gwyn@smoke.BRL.MIL (Doug Gwyn ) (10/31/88)
In article <262@mpx2.UUCP> erik@mpx2.UUCP (Erik Murrey) writes: >Sys-V tar doesn't copy directory permissions (or even directory >entries!) to the tape. This means that empty directories don't >get backed up and permissions get guessed during a restore. This >stinks. (I know BSD's tar does this correctly) UNIX System V's "tar" was provided simply for compatibility reasons. It is essentially the 7th Edition version and has numerous problems. Like 7th Edition, directories are not in the archive; whenever a directory is needed but not present during extraction, it is created using the current default umask. This feature is necessary whether or not directories are recorded in the archive. One of Berkeley's infamous "better ideas" was to stash directory entrites in the archive before their contents. Unfortunately, if a directory did not have write permission, when restoring a batch of files from the archive creation of the directory will preclude successful extraction of the subsequent files within it. >I like cpio becuase it dumps everything to tape. Even directories >and special files like named pipes, etc. I does not, however, allow >you to split the backup across several tapes (or disks). This >is a big loss for people without tape drives (me). UNIX has never had a good, standard way to deal with multi-volume files.
robert@pvab.UUCP (Robert Claeson) (10/31/88)
In article <12433@steinmetz.ge.com>, davidsen@steinmetz.ge.com (William E. Davidsen Jr) writes: > I go between SysIII, SysV, micropost, xenix, Ultrix and > SunOS using cpio, and it seems real enough for general use. If there's a > problem you have, warn us about it. How do I make backups of special files with cpio?-- Robert Claeson, ERBE DATA AB, P.O. Box 77, S-175 22 Jarfalla, Sweden Tel: +46 758-202 50 Fax: +46 758-197 20 EUnet: rclaeson@erbe.se ARPAnet: rclaeson%erbe.se@uunet.uu.net
debra@alice.UUCP (Paul De Bra) (11/01/88)
In article <44433@beno.seismo.CSS.GOV> rick@seismo.CSS.GOV (Rick Adams) writes: >... >A BACKUP program should be able to restore the disk to the state it >was at the time of the backup. It should also offer incremental backups. Right, and since there is no way to reset the create-time on a Unix system (except by setting the date and resetting it but that's awful and can never be used in a multiuser environment) there are NO backup programs that can restore the disk to the state it was at the time of the backup since this is simply not possible in Unix. The only way to restore the disk is if you make a complete image, using volcopy or dskcpy or whatever it is called (depending on the Unix version) but those programs cannot offer incremental backups. Paul. -- ------------------------------------------------------ |debra@research.att.com | uunet!research!debra | ------------------------------------------------------
guy@auspex.UUCP (Guy Harris) (11/01/88)
>Dump/restore is very quick, and it backs up everything, but I have >a few questions: It seems to dump the entire filesystem, including >the superblock/inode dumps. It also seems to me that it dumps >disk blocks in order of the disk itself, rather than the order of the >file. This means that restoring a filesystem will *not* reduce >the fragmentation. False. It does not dump disk blocks in the order of the disk, and it definitely doesn't *restore* disk blocks in the order of the disk itself. Dumping/restoring a file system will reduce fragmentation, especially on systems such as V7, S3, or 4.1BSD that have the V7 file system and also have dump/restor (not "restore"). Unfortunately, S5 has the V7 file system, but generally doesn't have dump/restor.... >Does it also require the exact same filesystem to by restored on? >If this is so, then I can't use dump/retore to expand a full filesystem. No, it doesn't; one of the common uses of dump/restor(e) is to expand full file systems.
guy@auspex.UUCP (Guy Harris) (11/01/88)
>This is real important as anyone who has tried to restore a corrupted >disk from a cpio "backup" and has found that the "backup" is too big to >fit on the disk it came from can tell you. Furthermore (although this isn't as likely to be a problem), neither "cpio" nor "tar" know about files with "holes" - they just read the file and, if they hit a hole, the file system code dutifully gives them a block full of zeroes, which as far as they're concerned is a disk block full of zeroes. This means that if you use "cpio" or "tar" to dump and restore a file system containing files with holes, you may again fild that the backup won't fit, because those holes will get assigned blocks when you restore the dump....
jds@mimsy.UUCP (James da Silva) (11/01/88)
In article <262@mpx2.UUCP> erik@mpx2.UUCP (Erik Murrey) writes: > > ... It also seems to me that it dumps >disk blocks in order of the disk itself, rather than the order of the >file. This means that restoring a filesystem will *not* reduce >the fragmentation. (Which is often why I backup/restore in the >first place.) ... > The BSD dump format goes roughly like this: Dump Header File Removal List (an inode bitmap) File Dump List ( " ) The directories The files The files are dumped in inode order (file with inode 10 is dumped before inode 11 and so forth), and the files are contiguous rather than fragmented. - Jaime ---------------------------------------------------------------------- usenet: uunet!mimsy!jds James da Silva internet: jds@mimsy.umd.edu
mangler@cit-vax.Caltech.Edu (Don Speck) (11/01/88)
In article <8373@alice.UUCP>, debra@alice.UUCP (Paul De Bra) writes: > since there is no way to reset the create-time on a Unix system > (except by setting the date and resetting it but that's awful and can never > be used in a multiuser environment) there are NO backup programs that can > restore the disk to the state it was at the time of the backup restor (counterpart of V7/SysIII/4.1bsd dump) would restore ctimes, inode numbering, holes, blocks past EOF, link counts, and even unreferenced files to the state at the time of the (incremental) backup. It could restore 100% full disks. It did not preserve the disorder of the free list. It was ugly and horrible (it wrote the raw device) but it CAN be done. AT&T and Berkeley both dropped it due to the difficulty of porting to filesystems whose blocksize may differ from the (fixed) tape blocksize. Berkeley sacrificed the ctimes, and wrote restore, which is much cleaner. AT&T gave up on incremental backups, and wrote volcopy (a glorified 'dd'). When we got a System V machine, nobody had the patience to mount 7 tapes per volcopy every day. After the first head crash, I ported restor. I regret this; it should have been restore. I can stand loss of ctimes. What do Amdahl UTS and Cray Unicos use? Don Speck speck@vlsi.caltech.edu {amdahl,ames!elroy}!cit-vax!speck
ggs@ulysses.homer.nj.att.com (Griff Smith) (11/01/88)
In article <262@mpx2.UUCP>, erik@mpx2.UUCP (Erik Murrey) writes: > Dump/restore is very quick, and it backs up everything, but I have > a few questions: It seems to dump the entire filesystem, including > the superblock/inode dumps. It dumps most of the inode information so it can restore file attributes. It also dumps a block map so `holes' can be restored in the right places. > It also seems to me that it dumps > disk blocks in order of the disk itself, rather than the order of the > file. This means that restoring a filesystem will *not* reduce > the fragmentation. Which version of `restore' are you using? Blocks are dumped in file order. The 4.[23]BSD version of restore puts files into a directory tree using standard `write' and `seek' operations. Fragmentation IS reduced, but the original fragmentation seems to be insignificant on the disks I have seen. > (Which is often why I backup/restore in the first place.) Are you using something other than the BSD fast file system? The last time I saw studies of the fragmentation of file systems that use allocation bit-tables instead of free lists, the fragmentation reached a steady state in about two weeks. This is much shorter than the pay-back time for a full dump/restore. > Does it also require the exact same filesystem to by restored on? No. Unless you are talking about 4.1BSD. I never used `restor'. > If this is so, then I can't use dump/restore to expand a full filesystem. No problem. You can also use `restore' to recover selected trees in a filesystem. > The docs I have for dump/restore don't explain details like > this to me. Please fill me in! Do you have `dump(8)' and `restore(8)' in the (Berkeley) UNIX System Manager's Manual? > --- > Erik Murrey > MPX Data Systems, Inc. > erik@mpx1.UUCP > ...!{bpa,spl1,cbmvax,vu-vlsi}!mpx1!erik -- Griff Smith AT&T (Bell Laboratories), Murray Hill Phone: 1-201-582-7736 UUCP: {most AT&T sites}!ulysses!ggs Internet: ggs@ulysses.att.com
paula@bcsaic.UUCP (Paul Allen) (11/01/88)
Several people have complained that tar cannot write multi-volume
backups. While the use of tar for backups may be debatable, it is
fairly simple to write a multi-volume tar archive. The basic idea is
to pipe the tar output to a script that has a loop containing a prompt for
the next {tape,disk} to be mounted and a dd command to copy just the
right number of blocks from stdin to the device. I use something
similar to this in my unattended-dump-to-disk scheme, and it works quite
nicely.
Several other complaints about tar have been voiced during this
discussion. I would suggest that people get a copy of John Gilmore's
public domain tar program. It's in the archives. It supports strange
things like named pipes and devices. It has limited support for reading
a damaged archive. It can take a list of files to copy from stdin.
If dump/restore are not available, this program is a reasonable
alternative.
Paul Allen
--
------------------------------------------------------------------------
Paul L. Allen | paula@boeing.com
Boeing Advanced Technology Center | ...!uw-beaver!ssc-vax!bcsaic!pauladhesi@bsu-cs.UUCP (Rahul Dhesi) (11/01/88)
Well, does anybody know WHY it is that AT&T does not supply "restore",
or an equivalent backup program, with System V?
--
Rahul Dhesi UUCP: <backbones>!{iuvax,pur-ee}!bsu-cs!dhesierik@mpx2.UUCP (Erik Murrey) (11/01/88)
In article <10789@ulysses.homer.nj.att.com>, ggs@ulysses.homer.nj.att.com (Griff Smith) writes: > Which version of `restore' are you using? Blocks are dumped in file > order. The 4.[23]BSD version of restore puts files into a directory > tree using standard `write' and `seek' operations. Fragmentation IS > reduced, but the original fragmentation seems to be insignificant on > the disks I have seen. I should have said that I am running on a SCO Xenix 2.3.1 system. It now (as of this release) uses the SysV "fsphoto" stuff which seems to be equivelent (on the surface.. i.e. I, II, III, IV, etc appear when dumping) to the dump/restore that I had on the older versions of Xenix. The only docs I have are for fsphoto and fssave, etc. Somewhere buiried in the docs I found that you couldn't restore onto a filesystem that has a different size. Any idea of what dump/restore version fsphoto/fssave came from? Is it completely different? ... Erik-- Erik Murrey MPX Data Systems, Inc. erik@mpx2.UUCP ...!{bpa,spl1,cbmvax,vu-vlsi}!mpx1!erik
guy@auspex.UUCP (Guy Harris) (11/02/88)
>>A BACKUP program should be able to restore the disk to the state it >>was at the time of the backup. It should also offer incremental backups. > >Right, and since there is no way to reset the create-time on a Unix >system That's "inode change time", not "create time"; neither the V7/S5 file system, nor the 4.2BSD file system, stores the create time. >(except by setting the date and resetting it but that's awful and can never >be used in a multiuser environment) there are NO backup programs that can >restore the disk to the state it was at the time of the backup since this >is simply not possible in Unix. You completely missed the point. I can live with the inode change time not being restored. I'd rather not live with a full restore done from a full and an incremental backup restoring files that had been deleted by the time the incremental was done - I want those files gone. I don't *have* to live with that if I use "dump" and "restore"; I do if I use "cpio" or "tar" for incremental dumps. (Oh, and by the way, although the 4.[23]BSD "restore" restores to a mounted file system, the V7/S3/4.1BSD "restor" restored to a raw disk, so it could set the change time to whatever it wanted.)
beattie@visenix.UUCP (Brian Beattie) (11/02/88)
In article <8373@alice.UUCP> debra@alice.UUCP () writes: >In article <44433@beno.seismo.CSS.GOV> rick@seismo.CSS.GOV (Rick Adams) writes: >>... >>A BACKUP program should be able to restore the disk to the state it >>was at the time of the backup. It should also offer incremental backups. > >Right, and since there is no way to reset the create-time on a Unix system >(except by setting the date and resetting it but that's awful and can never or writting the inode by writting the raw device as dump/restor does. >be used in a multiuser environment) there are NO backup programs that can you should never dump an active filesystem. >restore the disk to the state it was at the time of the backup since this >is simply not possible in Unix. >The only way to restore the disk is if you make a complete image, using >volcopy or dskcpy or whatever it is called (depending on the Unix version) or dump/restor if ATT has not castrated you system. >but those programs cannot offer incremental backups. dump/restor will do incremental backups just fine. It is true that an _incremental_ _restor_ will not set the create time back but that too could be worked if necessary. > >Paul. > >-- >------------------------------------------------------ >|debra@research.att.com | uunet!research!debra | >------------------------------------------------------ l i n e c o u n t f o d d e r -- _ANYONE_ | Brian Beattie (703)471-7552 can sell software| 11525 Hickory Cluster, Reston, VA. 22090 that has already | beattie@visenix.UU.NET been written | ...uunet!visenix!beattie
jerry@olivey.olivetti.com (Jerry Aguirre) (11/02/88)
In article <8373@alice.UUCP> debra@alice.UUCP () writes: >In article <44433@beno.seismo.CSS.GOV> rick@seismo.CSS.GOV (Rick Adams) writes: >>A BACKUP program should be able to restore the disk to the state it >>was at the time of the backup. It should also offer incremental backups. >Right, and since there is no way to reset the create-time on a Unix system >(except by setting the date and resetting it but that's awful and can never >be used in a multiuser environment) there are NO backup programs that can >restore the disk to the state it was at the time of the backup since this >is simply not possible in Unix. Actually, as others have pointed out the "restor" (pre 4.2bsd) version did restore the ctime and other information because it wrote to an unmounted raw disk. In the case of "restore" it is not enough to add some way of setting the ctime. The inodes are being allocated at the whim of the system. It uses the file "restoresymboltable" to keep track of the old to new inode mapping between restores of different levels. Because of this, new incremental dumps of the file system would not be compatable with previous dumps. You HAVE to do a new full dump. Restore is also damn slow.
guy@auspex.UUCP (Guy Harris) (11/02/88)
>How do I make backups of special files with cpio?--
You say "find ... | cpio -o ...". "cpio" is smart enough not to try to
open and read special files; it writes special entries that tell the
"cpio" that reads the tape to "mknod" the appropriate file.
If you're using SunOS 3.2 or later, it also understands symbolic links
(I expect the S5R4 "cpio" to handle them compatibly).rbj@nav.icst.nbs.gov (Root Boy Jim) (11/02/88)
From: Rahul Dhesi <dhesi@bsu-cs.uucp>
Keywords: cpio is not a real backup program
Well, does anybody know WHY it is that AT&T does not supply "restore",
or an equivalent backup program, with System V?
Because Berkeley does.
Rahul Dhesi UUCP: <backbones>!{iuvax,pur-ee}!bsu-cs!dhesi
(Root Boy) Jim Cottrell (301) 975-5688
<rbj@nav.icst.nbs.gov> or <rbj@icst-cmr.arpa>
Careful with that VAX Eugene!rbj@nav.icst.nbs.gov (Root Boy Jim) (11/02/88)
From: Guy Harris <guy@auspex.uucp> Furthermore (although this isn't as likely to be a problem), neither "cpio" nor "tar" know about files with "holes" - they just read the file and, if they hit a hole, the file system code dutifully gives them a block full of zeroes, which as far as they're concerned is a disk block full of zeroes. Which doesn't bother me too much that the zeros are written on tape; after all, people have more tape than disk. And to change it would require storing the disk block addresses, most likely increasing the dump size for most normal dumps, which contain no real or potential holes. This means that if you use "cpio" or "tar" to dump and restore a file system containing files with holes, you may again fild that the backup won't fit, because those holes will get assigned blocks when you restore the dump.... But disk is another matter again. Isn't it about time the kernel created holes via writing as well as seeking? I can see possible performance problems with all those `looking for zero' scanning loops; perhaps a flag to open(2), O_HOLY could announce the user's intention to create such files. Alternatively, restore could do it for all files, but this would require knowing the block and fragment size of the filesystem. I suspect that none of this has been done because holy files are relatively rare, and the savings wouldn't really be that great. (Root Boy) Jim Cottrell (301) 975-5688 <rbj@nav.icst.nbs.gov> or <rbj@icst-cmr.arpa> Careful with that VAX Eugene!
andrew@alice.UUCP (Andrew Hume) (11/03/88)
debra states "there are NO backup programs that can restore the disk to the state it was at the time of the backup since this is simply not possible in Unix." this is of course only true if the backup programs restore as a user-level program. as an obvious example, dump doesn't cause the access times to be updated. but then again, with the modern trend of more complicated file systems continuing, no one can write these disk-level programs any more anyhow.
andrew@alice.UUCP (Andrew Hume) (11/03/88)
actually, Ninth edition has a -z flag to cp that creates holes in the destination if they may have been in the source. (of course, user-level programs can't tell.) it works well.
guy@auspex.UUCP (Guy Harris) (11/03/88)
> Well, does anybody know WHY it is that AT&T does not supply "restore", > or an equivalent backup program, with System V? > >Because Berkeley does. Can you substantiate this, or is it just a random flame?
jc@minya.UUCP (John Chambers) (11/09/88)
In article <8470@cit-vax.Caltech.Edu>, mangler@cit-vax.Caltech.Edu (Don Speck) writes: > In article <8373@alice.UUCP>, debra@alice.UUCP (Paul De Bra) writes: > > since there is no way to reset the create-time on a Unix system Of course not, since Unix systems don't keep the creation time of a file. There is no field in an inode that contains this information. > restor (counterpart of V7/SysIII/4.1bsd dump) would restore ctimes, Yet another person who thinks 'ctime' means "creation time". It doesn't. According to my Sys/V manual, the st_ctime field in an inode is the "Time of last file status change". In other words, it's the last time the inode was changed (e.g., to change permissions). It isn't the creation time. -- John Chambers <{adelie,ima,maynard,mit-eddie}!minya!{jc,root}> (617/484-6393) [Any errors in the above are due to failures in the logic of the keyboard, not in the fingers that did the typing.]
Makey@logicon.arpa (Jeff Makey) (11/10/88)
(Root Boy) Jim Cottrell <rbj@nav.icst.nbs.gov> writes: >Isn't it about time the kernel created holes via writing as well as >seeking? I can see possible performance problems with all those >`looking for zero' scanning loops; Few files on most UNIX systems need the sparse-file feature. However, I once was messing with some custom database files that were very large and very sparse. In order to copy one of these database files and not fill up the file system I had to write my own version of "cp" which scanned each input block and didn't write it out if it contained all zeroes. The one exception was the very last block, which had to be written out to disk regardless of contents to properly set the file size. Copying large sparse files this way is *much* faster than actually writing every block of zeroes. Even for normal files the performance difference between this and the standard "cp" was so slight that I just put my modified "cp" in my personal "bin" directory and used it all the time. :: Jeff Makey Makey@LOGICON.ARPA