madd@adt.UUCP (jim frost) (04/17/89)
>In article <8904131500.AA20088@adt.uucp> adt!madd@bu-it.bu.edu (jim frost) writes: >>It is possible (even pretty easy) to build a very >>good, very reliable backup program which works through the filesystem. > >Restoring access times properly is needed (without an additional system call >and losing the ctime information). When doing a lot of i/o you just don't loose very bad doing the system call, especially since you're guaranteed that the inode is in memory. I don't consider the loss of ctime to be such a tragedy, after all your restore did technically change the status of the file. >Dealing with files with holes is also >mildly tricky (you do want restores of a dumped fs to fit back :-). Here I have to agree with you, but it's pretty easy to build a program which not only handles the holes, but creates new holes where they didn't exist before so your restore is smaller than the original but still perfectly intact. >But most >of all, I don't think incurring the overhead of going through the file system >is justifiable once you have large amounts of disk space. If you don't go through the filesystem, you have to build most of the filesystem code into the dump program. Most likely you're not going to go to the effort to streamline accesses as nicely as the filesystem does (just how much effort do you want to put into your dump, anyway?), so you're likely to loose anyway. If you're really looking for performance, you can take advantage of parallelism to collect the disk information while waiting for the tape portion of the program to complete; almost all of the time spent by dump is waiting for the tape. This technique has been used on (at least) non-UNIX systems and resulted in very high throughput even with FS/OS overhead, although the throughput was lower than if the same technique were used on the raw filesystem. jim frost madd@bu-it.bu.edu
vjs@rhyolite.SGI.COM (Vernon Schryver) (04/18/89)
In article <8904171554.AA19754@adt.uucp>, madd@adt.UUCP (jim frost) writes: > ... [about how to handle holes] ... > jim frost > madd@bu-it.bu.edu There are no holes in EFS. Some consider that a feature, since you get more consistency (think of cp and dd in addition to tar/cpio/bru). Most of us consider that an oversight which no one has had time to fix. Holes in an Extent FS are messier than in a fixed or semi-fixed size block FS like FFS or BFS. What do you do when someone writes 1 byte in the middle of a hole? How big an extent do you allocate? If you allocate the largest possible, you are almost always wrong, and waste lots of blocks. If you don't, and the user eventually writes nearby, then you have unnecessarily fragemented the file. This absense of holes in EFS may affect the utility (but not the possibility) of omniscient backup tools. Me?--I use tar & cpio. Vernon Schryver Silicon Graphics vjs@sgi.com
madd@adt.UUCP (jim frost) (04/18/89)
I didn't realize that EFS worked that way. Interesting. How do you know how large an extent to allocate anyway? Or do you attempt to allocate extents within differing cylinders (groups) so you can possibly continue expanding the extent until either the cylinder (group) is full or you run out of cylinders (groups)? And what exactly defines an extent (ie it's not likely to be <start block>,<run> for obvious reasons). I'd really like some reading material on EFS if you have it. Inquiring minds want to know. >Me?--I use tar & cpio. > >Vernon Schryver >Silicon Graphics >vjs@sgi.com Tar isn't too useful when you're backing up more than a few hundred megabytes (actually even a hundred). It's slow and very unreliable. Given Murphy's Law and the nature of tape drives, the one backup you really need will be corrupted. A good backup/restore program would be able to get a lot of information off the tape anyway; tar would barf and you'd end up bit-fiddling to get the file. Blech. jim frost madd@bu-it.bu.edu
rpaul@dasys1.UUCP (Rod Paul) (04/22/89)
>>Me?--I use tar & cpio. >> >>Vernon Schryver >>Silicon Graphics >>vjs@sgi.com > >Tar isn't too useful when you're backing up more than a few hundred >megabytes (actually even a hundred). It's slow and very unreliable. >Given Murphy's Law and the nature of tape drives, the one backup you >really need will be corrupted. A good backup/restore program would be >able to get a lot of information off the tape anyway; tar would barf >and you'd end up bit-fiddling to get the file. Blech. > How about "bru"? Anybody else out ther using it? I've had no problems at all, and it's great over the net. It also uses the fastest damn copression inplementation (option) I've seen yet. To specify media size use the -s option, when "bru" figures it needs another tape it stops and asks for one. You can also write a label in the header field. I ended up writing a shell as a general front end at our facility (as "bru" has so many flags), I used some other shell from the SGI release as a template. I don't remember what it was now, just check out every damn program that does backups. One thing I wonder though, how portable is a file backed up with "bru"? -- Rodian Paul | Big Electric Cat Public UNIX | Just say YES to UII ! ..!cmcl2!hombre!dasys1!rpaul |
jmb@patton.SGI.COM (Jim Barton) (04/25/89)
In article <9405@dasys1.UUCP>, rpaul@dasys1.UUCP (Rod Paul) writes: ... > I ended up writing a shell as a general front end at our facility (as > "bru" has so many flags), I used some other shell from the SGI release > as a template. I don't remember what it was now, just check out every > damn program that does backups. The base level shell for modifying is called 'beer' (pun on 'bru' obvious), which sets up a simple daily, weekly and quarterly backup schedule for an operator to follow. I believe it's installed in /etc and is standard with the release. > One thing I wonder though, how portable is a file backed up with "bru"? I took pains to keep bru compatible with other EST implementations of bru, so tapes written with it should be readable on anybody else's bru. > -- > Rodian Paul | > Big Electric Cat Public UNIX | Just say YES to UII ! > ..!cmcl2!hombre!dasys1!rpaul | -- Jim Barton Silicon Graphics Computer Systems "UNIX: Live Free Or Die!" jmb@sgi.sgi.com, sgi!jmb@decwrl.dec.com, ...{decwrl,sun}!sgi!jmb "I used to be disgusted, now I'm just amused." - Elvis Costello, 'Red Shoes' --
henry@utzoo.uucp (Henry Spencer) (04/26/89)
In article <8904181440.AA11202@adt.uucp> madd@adt.UUCP (jim frost) writes: >Tar isn't too useful when you're backing up more than a few hundred >megabytes (actually even a hundred). It's slow and very unreliable. >Given Murphy's Law and the nature of tape drives, the one backup you >really need will be corrupted. A good backup/restore program would be >able to get a lot of information off the tape anyway; tar would barf >and you'd end up bit-fiddling to get the file. Blech. Don't confuse the program with the format. It is not hard to write a program that scavenges files from a tar tape. In fact, guess how we just did a restore off backups on our 4D/60T. Surprise surprise, SGI tar does *not* know how to restore off a multi-tape backup, although it's perfectly happy to create them... We'd have been Up S**t Creek if we didn't have the scavenging programs; SGI basically just said "uh, yeah, that's a problem" when we asked. -- Mars in 1980s: USSR, 2 tries, | Henry Spencer at U of Toronto Zoology 2 failures; USA, 0 tries. | uunet!attcan!utzoo!henry henry@zoo.toronto.edu
mberger@RELAY.NSWC.NAVY.MIL (05/03/89)
.