michael@yonder.UUCP (Michael E. Haws) (05/26/91)
I am trying to use afio to back up my 386 based SCO Xenix 2.3.2 system. The command I use is find /usr /u -print | afio -oZ -L/back/log | dd of=/dev/rct0 ibs=10b obs=1000b Part way thru the backup, messages appear in the log file something to the effect of "File table overflow" repeated many times. On the console the message "Inode table overflow" appears many times. When the job completes the "df" command shows 1000's of free inodes. I suspect it may have something to do with the compressing of the data but does any one know for sure. Thanks, Michael -- Michael E. Haws "Keep the blue side up" w - (303) 986-2370 boulder!yonder!michael h - (303) 232-0628 michael%yonder@csn.org
chip@chinacat.unicom.com (Chip Rosenthal) (05/27/91)
In article <446@yonder.UUCP> michael@yonder.UUCP (Michael E. Haws) writes: >find /usr /u -print | afio -oZ -L/back/log | dd of=/dev/rct0 ibs=10b obs=1000b > >Part way thru the backup, messages appear in the log file something to the >effect of "File table overflow" repeated many times. On the console the >message "Inode table overflow" appears many times. When the job >completes the "df" command shows 1000's of free inodes. Those messages have nothing to do with what's on your disk. It means that the fixed size kernel tables which hold active file and inode information have run out of space. Try this: # pstat | grep '^[^ ]' 139 active inodes 39 processes 126 open files # egrep 'NINODE|NFILE' /usr/sys/conf/master inodes NINODE 250 files NFILE 250 (Note that you probably need root privs to access the `master' file, but not to run `pstat'.) My first guess would be that you are on the hairy edge, and the backup is overflowing these tables. If I remember correctly, XENIX is distributed for 100 active inodes/files apiece. As you can see, I upped mine to 250 - and needed to do so badly. The machine isn't doing much at the moment and it still needs over 100 apiece. If you find that this is the problem, run `/usr/sys/conf/configure' and select option 3 (Files, Inodes, and Filesystems). Adjust the inode/file values, and then run `link_xenix' and `hdinstall' (both in `/usr/sys/conf') to create and install a new kernel. It is quite common to have to increase NINODE and NFILE. NPROC and NCLIST are the two others which frequently need to be adjusted. BTW...I don't recognize your `afio' options. Are you using a hacked version? (Or am I using an outdated version???) Do you realize you are missing out on the *best* part of afio? The `-f' option makes afio scream. If you are able, junk the dd and use the afio pseudo-double-buffering. Also, I'm concerned that the `Z' option might be compressing the backup. If this is the case, methinks you are making a *big* mistake. For all intents and purposes, if one bit is corrupt in an LZW compressed file, the following contents are unrecoverable. Using compress on backup volumes is penny-wise and megatons-foolish. -- Chip Rosenthal <chip@chinacat.Unicom.COM> | Don't play so Unicom Systems Development 512-482-8260 | loud, Mr. Collins.
michael@yonder.UUCP (Michael E. Haws) (05/28/91)
In article <1991May26.203103.19212@chinacat.unicom.com>, chip@chinacat.unicom.com (Chip Rosenthal) writes: > In article <446@yonder.UUCP> michael@yonder.UUCP (Michael E. Haws) writes: > >find /usr /u -print | afio -oZ -L/back/log | dd of=/dev/rct0 ibs=10b obs=1000b > I'm concerned that the `Z' option > might be compressing the backup. It is. > If this is the case, methinks you > are making a *big* mistake. For all intents and purposes, if one bit > is corrupt in an LZW compressed file, the following contents are > unrecoverable. Using compress on backup volumes is penny-wise and > megatons-foolish. > > -- > Chip Rosenthal <chip@chinacat.Unicom.COM> | Don't play so > Unicom Systems Development 512-482-8260 | loud, Mr. Collins. It is my impression that you are suggesting that I never keep compressed data on my hard disk, since this data would be subject to the same potential problem when trying to recover it from a tape archive. Or have I missed something? As for the contents following a corrupt bit being unrecoverable, I intentionally corrupted a random bit in a compressed file and was able to successfully uncompress the file and although it did have the random character corrupted, the remainder of the file was intact. -- Michael E. Haws "Keep the blue side up" w - (303) 986-2370 boulder!yonder!michael h - (303) 232-0628 michael%yonder@csn.org
bill@camco.Celestial.COM (Bill Campbell) (05/28/91)
In <446@yonder.UUCP> michael@yonder.UUCP (Michael E. Haws) writes:
:I am trying to use afio to back up my 386 based SCO Xenix 2.3.2 system.
:The command I use is
:find /usr /u -print | afio -oZ -L/back/log | dd of=/dev/rct0 ibs=10b obs=1000b
:Part way thru the backup, messages appear in the log file something to the
:effect of "File table overflow" repeated many times. On the console the
:message "Inode table overflow" appears many times. When the job
:completes the "df" command shows 1000's of free inodes. I suspect it may
:have something to do with the compressing of the data but does any one know
:for sure.
I've used afio extensively here for backups and use:
find . -print | afio -ov -c 100 -f -s <tapesize_in_blocks> /dev/rct0
This handles multiple volumes properly, is fast, and has never
failed on any of my systems. What more can I ask?
Bill
--
INTERNET: bill@Celestial.COM Bill Campbell; Celestial Software
UUCP: ...!thebes!camco!bill 6641 East Mercer Way
uunet!camco!bill Mercer Island, WA 98040; (206) 947-5591
ken@dali.cc.gatech.edu (Ken Seefried iii) (05/28/91)
In article <447@yonder.UUCP> michael@yonder.UUCP (Michael E. Haws) writes: > >It is my impression that you are suggesting that I never keep compressed >data on my hard disk, since this data would be subject to the same >potential problem when trying to recover it from a tape archive. Or have >I missed something? > Single and multi-bit media errors are several orders of magnatude more likely on tapes than on hard disks. You will have to forgive me for not being able to quote chapter and verse, but as I remember (and my memory is quite foggy, forgive me), the figures are something like 1 bit in 10^7 for tape media and 1 bit in 10^9 for hard disks. Perhaps someone with better (and more recent) data will care to speak up. Obviously there is some risk for both media, but the risk is vastly smaller for hard disks. -- ken seefried iii "I'll have what the gentleman ken@dali.cc.gatech.edu on the floor is having..."
michael@yonder.UUCP (Michael E. Haws) (05/28/91)
In article <29927@hydra.gatech.EDU>, ken@dali.cc.gatech.edu (Ken Seefried iii) writes: > In article <447@yonder.UUCP> michael@yonder.UUCP (Michael E. Haws) writes: > > > >It is my impression that you are suggesting that I never keep compressed > >data on my hard disk, since this data would be subject to the same > >potential problem when trying to recover it from a tape archive. Or have > >I missed something? > > > > Single and multi-bit media errors are several orders of > magnatude more likely on tapes than on hard disks. ... > the figures are something like 1 bit in 10^7 > for tape media and 1 bit in 10^9 for hard disks. ... > Obviously there is some risk for both media, but the risk > is vastly smaller for hard disks. Agreed, but the point I was trying to make was that when the compressed file from the hard disk is now backed up to tape the chances of recovering it are no better(or worse) than the file compressed during the backup process. Since I was told that it was not safe to compress files during backup, I assumed that already compressed files would be just as susceptible to non-recoverability(is that a word?) from a tape archive. -- Michael E. Haws "Keep the blue side up" w - (303) 986-2370 boulder!yonder!michael h - (303) 232-0628 michael%yonder@csn.org
adeboer@gjetor.geac.COM (Anthony DeBoer) (05/29/91)
In article <450@yonder.UUCP> michael@yonder.UUCP (Michael E. Haws) writes: >In article <29927@hydra.gatech.EDU>, ken@dali.cc.gatech.edu (Ken Seefried iii) writes: >> In article <447@yonder.UUCP> michael@yonder.UUCP (Michael E. Haws) writes: >> > >> >It is my impression that you are suggesting that I never keep compressed >> >data on my hard disk, since this data would be subject to the same >> >potential problem when trying to recover it from a tape archive. Or have >> >I missed something? > >Agreed, but the point I was trying to make was that when the compressed file >from the hard disk is now backed up to tape the chances of recovering >it are no better(or worse) than the file compressed during the >backup process. > >Since I was told that it was not safe to compress files during backup, I >assumed that already compressed files would be just as susceptible to >non-recoverability(is that a word?) from a tape archive. The difference is in what you mean by "compress during backup". The easy way to do it is "find | cpio | compress | dd", to compress the whole backup in one pass. An error in the middle of the tape could screw up the decompression for the whole remainder of the tape. Compressing each file individually, though, would make the tape look at if you went through your hard disk beforehand and compressed everything. (That's probably the "easiest" way of trying to implement this; doing it properly is nontrivial.) An error might screw up the remainder of the file in which it lies, but provided you can read the data off the rest of the tape cpio could be re-syncronized on the next header and be able to recover the rest of the files on the tape. There's nothing inherent about compressed files that makes them less restorable from tape than any other file, except for the factor of not being able to decompress past a glitch. Other files, like executables, may also be useless if they're corrupt. On the other hand, of course, a compressed file IS a much smaller target for a glitch to try to hit. :-) -- Anthony DeBoer NAUI#Z8800 | adeboer@gjetor.geac.com | Programmer (n): One who Geac Canada Ltd. | uunet!geac!gjetor!adeboer | makes the lies the Toronto, Ontario | #include <disclaimer.h> | salesman told come true.
michael@yonder.UUCP (Michael E. Haws) (05/29/91)
In article <1991May28.172728.26275@gjetor.geac.COM>, adeboer@gjetor.geac.COM (Anthony DeBoer) writes: > The difference is in what you mean by "compress during backup". The easy way > to do it is "find | cpio | compress | dd", to compress the whole backup in one > pass. An error in the middle of the tape could screw up the decompression for > the whole remainder of the tape. The version of afio that I am using came from the Ohio State archives and the compression extension was added by Jeff Buhrt. It does not compress the the cpio archive as a whole but compresses each file individually before it puts it in the archive. -- Michael E. Haws "Keep the blue side up" w - (303) 986-2370 boulder!yonder!michael h - (303) 232-0628 michael%yonder@csn.org
les@chinet.chi.il.us (Leslie Mikesell) (05/29/91)
In article <447@yonder.UUCP> michael@yonder.UUCP (Michael E. Haws) writes: >> >find /usr /u -print | afio -oZ -L/back/log | dd of=/dev/rct0 ibs=10b obs=1000b [compressing with -oZ...] >> If this is the case, methinks you >> are making a *big* mistake. For all intents and purposes, if one bit >> is corrupt in an LZW compressed file, the following contents are >> unrecoverable. Using compress on backup volumes is penny-wise and >> megatons-foolish. >It is my impression that you are suggesting that I never keep compressed >data on my hard disk, since this data would be subject to the same >potential problem when trying to recover it from a tape archive. Or have >I missed something? The problem with compressing backups is that normally the entire stream is compressed as a whole on its way to the archive device. In that case, a media error early in the tape will make it essentially impossible to recover any data past that point (but some tape drivers won't continue past a media error anyway, so in that case it wouldn't matter). However, this is a new option in a modified version of afio that was posted to comp.sources.3b1 that compresses on a per-file basis by actually making a temporary compressed copy on disk first. This means that the hunt-for-the-next-cpio-header method of error recovery will still work. Other additions involve improvements for floppy access (make an in-core copy to allow verify/rewrite per disk and ability to format a new disk before continuing if there are problems). I'd dump the pipe to dd and let afio handle the device directly, though. If you use the -f option it will go a little faster. Also, I'd still only use the -oZ if it's necessary to make a volume fit on a tape. The compressed files won't automatically be uncompressed when you restore and there is no way to tell which files were compressed on the disk before the backup and which were compressed on the fly so there would be a lot of work involved in getting a filesystem back to its original state if you have to restore from one of those tapes. Les Mikesell les@chinet.chi.il.us