heiser@sud509.ed.ray.com (02/01/90)
This message is addressed to anyone who uses an Exabyte on a Sun 3/280 (SunOS4.0.3). What parameters have you found to work best (for correctly determining tape length, blocking factor for quickest operation, etc) for backing up to the Exabyte using 'dump'? Thanks in advance! Bill Heiser | Work: heiser@tdw201.ed.ray.com | {decuac,necntc,uunet}!rayssd!tdw201!heiser | Home: Bill.Heiser@f240.n322.z1.fidonet.org (Fidonet 1:322/240) | The Think_Tank BBS (508) 655-3848 | Other: 75106.2332@compuserve.com
merlyn@iwarp.intel.com (Randal Schwartz) (02/07/90)
In article <4618@brazos.Rice.edu>, heiser@sud509 writes: | This message is addressed to anyone who uses an Exabyte on a Sun 3/280 | (SunOS4.0.3). What parameters have you found to work best (for correctly | determining tape length, blocking factor for quickest operation, etc) for | backing up to the Exabyte using 'dump'? | | Thanks in advance! Here it is, stolen directly out of a handful of Perl scripts that I use to back up 16GB of disk in 146 filesystems across 30 machines to any one of 13 Exabyte drives... dump ${level}dsbf 43000 12000 64 - $device | dd of=/dev/nrst1 obs=64b where $level is like "0u", and $device is the device to be dumped, like "/dev/xd2c". The "dd" command is possibly on a different system via rsh, which is why I have the pipe and dd. I don't know where I got the numbers... I think the local rep gave them to me. Also, I have found 64 to be a reasonable compromise between speed and running out of buffers on one of our heavily used machines. Just another sysadm, /=Randal L. Schwartz, Stonehenge Consulting Services (503)777-0095 ==========\ | on contract to Intel's iWarp project, Beaverton, Oregon, USA, Sol III | | merlyn@iwarp.intel.com ...!any-MX-mailer-like-uunet!iwarp.intel.com!merlyn | \=Cute Quote: "Welcome to Portland, Oregon, home of the California Raisins!"=/
laukee@nsfnet-relay.ac.uk (David Lau-Kee) (02/12/90)
>From: merlyn@iwarp.intel.com (Randal Schwartz) > >In article <4618@brazos.Rice.edu>, heiser@sud509 writes: >| This message is addressed to anyone who uses an Exabyte on a Sun 3/280 >| (SunOS4.0.3). What parameters have you found to work best (for correctly >| determining tape length, blocking factor for quickest operation, etc) for >| backing up to the Exabyte using 'dump'? >| >| Thanks in advance! > >Here it is, stolen directly out of a handful of Perl scripts that I use to >back up 16GB of disk in 146 filesystems across 30 machines to any one of >13 Exabyte drives... > >dump ${level}dsbf 43000 12000 64 - $device | dd of=/dev/nrst1 obs=64b > >where $level is like "0u", and $device is the device to be dumped, like >"/dev/xd2c". The "dd" command is possibly on a different system via rsh, >which is why I have the pipe and dd. > >I don't know where I got the numbers... I think the local rep gave them to >me. Also, I have found 64 to be a reasonable compromise between speed and >running out of buffers on one of our heavily used machines. > Drawing from the Sun enhanced SCSI manual: The Exabyte organises its data as tracks of 8192 bytes. For a 90 minute tape (cartridge type 2048, e.g., Canon P5-90), the tape is 340 feet long. For a 346 foot tape (4152") there are 261660 tracks of 8192 bytes, (i.e., 261660 / 4152 tpi = 63 tpi = (63 * 8192 = 516096)*8 = 4128768 bpi). To allow for defects you should assume a density of 4100000. *If* you are using dump to write the tape (not dd) then you should correct the tape size by increasing by 840/blocksize to account for the fact that dump assumes a larger interblock gap than the exabyte leaves. Since the exabyte organises data in tracks of 8192 bytes (and any unwritten data in a track is wasted space) maximum utilisation occurs when the blocking factor is a multiple of 8192 bytes. Now under 4.0 (and I guess 3.X) the maximum size of a data transfer over the SCSI bus is 63k bytes (viz 126 blocking factor you'll often see with tar). For maximum throughput and minimum waste Sun recommend a blocking factor which results in the nearest multiple of 8192 to 63k bytes, i.e., 56k bytes, therefore: use tar b 112 use dump b 56 use dd bs 56k So for a straight dump onto a SCSI'd exabyte you'd want: dump 0ucbsdf 56 5190 4100000 <exabyte raw device> <file system> I don't know about the 64b = 32k blocking factor suggested previously for dumping to a pipe, but it is certainly reasonable for getting to the exabyte (since it is a full mutiple of 8192), however it does look like 32k is well below the 63k SCSI bus transfer maximum. There are probably good reasons for wanting to pipe into an rsh to dd onto the exabyte, but for dumping across a network I usually go for specifying a remote tape unit to dump. Something like this should work as a general exabyte dump script to be rsh'd on various machines on your network: #!/bin/sh PATH=:/bin:/usr/bin:/etc IFS= default=9 ebhost= <default exabyte host> level=${1-$default} machine=`/bin/hostname` tapehost=${2-ebhost} if [ "$tapehost" = "$machine" ]; then prefix="" else prefix="${tapehost}:" fi echo "Making a level $level dump for $machine onto exabyte on $tapehost" for i in h g a; do echo "dumping sd0$i" /etc/dump ${level}ucfbsd ${prefix}/dev/nrst2 56 5190 4100000 /dev/rsd0$i sleep 120 done echo "Level $level dumps for $machine finished." Note the 120 second sleeps between filesystem dumps. This is because the exabyte pauses for several seconds after a filemark has been written. The Sun driver has a pause built in to prevent operation during this period, but in my experience this has proved insufficient. I don't know whether the beast needs to "cool down" before starting a second dump file, but since I do them overnight I'm not worried as long as it works. Also note that multi tape dumps will not work since I am not recalculating tape remaining between files (but with 2.04 Gig and only 7 machines I'm not too bothered!).
merlyn@iwarp.intel.com (Randal Schwartz) (02/14/90)
In article <4943@brazos.Rice.edu>, canon!laukee@nsfnet-relay (David Lau-Kee) writes: | I don't know about the 64b = 32k blocking factor suggested previously for | dumping to a pipe, but it is certainly reasonable for getting to the | exabyte (since it is a full mutiple of 8192), however it does look like | 32k is well below the 63k SCSI bus transfer maximum. Yes, I've done dumps at 63k, but it panic'ed one of our Sun3's for running out of buffer space (something else was going on at the time that probably contributed), and I cranked the number back to play it safe. I haven't had any problems since then, *or* enough "spare time" to try raising it back to 63k to see if anything ever breaks again. | There are probably good reasons for wanting to pipe into an rsh to dd onto | the exabyte, but for dumping across a network I usually go for specifying a | remote tape unit to dump. I've got the script designed to work with both our Suns and our Microvaxes (blech!) and the microvax /dev/rmt *seems* to have problems talking to the scsi. Once again... if I had the time (does *any* SA ever have time?) I'd try to work out exactly what broke and fix it. | Note the 120 second sleeps between filesystem dumps. This is because the | exabyte pauses for several seconds after a filemark has been written. The | Sun driver has a pause built in to prevent operation during this period, | but in my experience this has proved insufficient. Yeah, I found that out "the hard way" too. Started first with no delay, then 60 seconds, then 120 seconds, which is my current value. Been using that for a few months. | Also note | that multi tape dumps will not work since I am not recalculating tape | remaining between files (but with 2.04 Gig and only 7 machines I'm not too | bothered!). Yeah, I have a whole bunch of support now for taking the 146 "worthwhile" filesystems and mapping them into tape collections that are all under 2G. I also have scripts to stat the current sizes to find out if the level 0 tape collections can be further collected. (All in Perl, of course... :-) Just another Perl hacker, exabyte user, and sysadmin, /=Randal L. Schwartz, Stonehenge Consulting Services (503)777-0095 ==========\ | on contract to Intel's iWarp project, Beaverton, Oregon, USA, Sol III | | merlyn@iwarp.intel.com ...!any-MX-mailer-like-uunet!iwarp.intel.com!merlyn | \=Cute Quote: "Welcome to Portland, Oregon, home of the California Raisins!"=/
laplant@engr.wisc.edu (02/14/90)
>>From: merlyn@iwarp.intel.com (Randal Schwartz) > > So for a straight dump onto a SCSI'd exabyte you'd want: > > dump 0ucbsdf 56 5190 4100000 <exabyte raw device> <file system> I've just found out my dumps to 8mm are pretty much useless. Dump doesn't give any errors, says it dumped 50MB (or whatever) to 0.01 tape(s). then, when you try to restore, you get tons of resync restore, skipped xx blocks errors (man on restore says these are read errors) It ends up restoring about half of what was dumped. I tried the numbers above with the same result. Other notes: done on a sun 4/110, SunOS 4.0, tar to 8mm works fine. any ideas? Dave LaPlant laplant@engr.wisc.edu University of Wisconsin Computer Aided Engineering Center