nomad@verdix.com (Lee Damon) (03/08/89)
We need to back up some _very_ large disks and it takes way to long
using /etc/backup (like 12+ hours - we only have cart' tape
drives...). I am trying remshing to a dd on a BSD system, but that is
slow and won't enable us to use multi-volume backups.
What I am looking for is something like RDUMP, or any kind of _fast_
backup. Basicly something that would preserve multi-volumes and would
not take more than a few hours to backup 500MB.
Any hints, clues, etc. would be greatly appreciated. Please e-mail to
me and I'll sumarize to the net if there is demand.
ADVthanksANCE!
nomad@verdix.com
============= Lee Damon work: verdix!nomad or nomad@verdix.com \
play: {agora,tessi,verdix}!castle!nomad or nomad@castle.fidonet.org \
fidonet: The Castle BBS, 1:105/302, 503-641-3161 / \
"God" created man in its image, and man being ever humble returned the favor. al@cs.strath.ac.uk (Alan Lorimer) (03/10/89)
In article <263@verdix.verdix.com> nomad@verdix.com (Lee Damon) writes: >We need to back up some _very_ large disks and it takes way to long >using /etc/backup (like 12+ hours - we only have cart' tape >drives...). I am trying remshing to a dd on a BSD system, but that is >slow and won't enable us to use multi-volume backups. > >What I am looking for is something like RDUMP, or any kind of _fast_ >backup. Basicly something that would preserve multi-volumes and would >not take more than a few hours to backup 500MB. > Hear Hear......... Hope someone at HP is listening. Those of us with large installations can see no reason to have to buy a 9144 tape drive for every system with a disc, especially when there is a perfectly adequate 6250bpi drive attached to something else. Come on HP, let's see the 9000/300 series integrate that little bit better with existing systems but allowing remote access to backup equipment - you've given us dump, so why not rdump? Alan. -- ____________________________________________________________________________ Alan G. Lorimer, Strathclyde University, 26 Richmond Street, Glasgow G1 1XH. UUCP: ...!uunet!mcvax!ukc!strath-cs!al DARPA: al%cs.strath.ac.uk@ucl-cs JANET: al@uk.ac.strath.cs
rclark@speclab.bgp-usgs.gov (Roger N. Clark) (03/10/89)
> What I am looking for is something like RDUMP, or any kind of _fast_ > backup. Basicly something that would preserve multi-volumes and would > not take more than a few hours to backup 500MB. What you need is a DAT tape drive. A single cartridge tape holds 1 to 2 gigabytes (depending on format) and throughput can be ~200 kbytes/sec. A single cartridge is about $10. Any remote shell over the ethernet will be limited by the net throughput (which seems typically about 50 kbytes/sec., but some machines are slow and can only do ~10 kbytes/sec.). So when is HP comming out with a drive????? I want one for my 825! Roger N. Clark
vic@zen.co.uk (Victor Gavin) (03/13/89)
In article <1345@stracs.cs.strath.ac.uk> al@cs.strath.ac.uk writes: >In article <263@verdix.verdix.com> nomad@verdix.com (Lee Damon) writes: >>What I am looking for is something like RDUMP, or any kind of _fast_ >>backup. Basicly something that would preserve multi-volumes and would >>not take more than a few hours to backup 500MB. >> >Come on HP, let's >see the 9000/300 series integrate that little bit better with existing >systems but allowing remote access to backup equipment - you've given >us dump, so why not rdump? > >Alan. We've got our 300's connected on a network with a machine with a .5in tape drive. We use NFS to mount remote file-systems and then backup them up with (cd / ; find RemoteNode -type f -print | cpio -oacxBC > /dev/rmt/0h) [Yes, we don't back up the directories because cpio complains a lot; this is a limitation we live with for now -- we're primarily interested in preserving user data files.] At the moment we're looking at these new Exabyte tape drives, but the company that is distributing them in this country haven't got any for HP computers (even though there adverts say they have!). They think that they'll have something Real Soon Now (but don't hold your breath :-). vic -- Victor Gavin Zengrange Limited vic@zen.co.uk Greenfield Road ..!mcvax!ukc!zen.co.uk!vic Leeds England +44 532 489048 LS9 8DB
dunlap@apl-em.UUCP (John Dunlap) (03/13/89)
> We need to back up some _very_ large disks and it takes way to long > using /etc/backup (like 12+ hours - we only have cart' tape > drives...). I have seen ads for 1.2 GByte tape drives from Bering and IEM that connect between the computer and the disk drive. At 3 a.m. (or whatever) they will disconnect the computer and copy the disk drive(s) to the tape at about 14 MBytes/minute. These drives cost about $9000 as I recall and use DATs (digital audio tapes). John Dunlap ...!hpubvwa!apl-em!dunlap dunlap@uw-apl.apl.washington.edu Applied Physics Lab University of Washington 1013 NE 40th St Seattle WA 98105
jim@cs.strath.ac.uk (Jim Reid) (03/15/89)
In article <1345@stracs.cs.strath.ac.uk> al@cs.strath.ac.uk writes: >................................................ Come on HP, let's >see the 9000/300 series integrate that little bit better with existing >systems but allowing remote access to backup equipment - you've given >us dump, so why not rdump? Cynical people like myself might think that one possible reason is a sales ploy by HP to force people to buy HP's expensive 6250bpi tape drive. [Yes, I know it does data compression on the fly, but that doesn't justify a price that's 3-5 times a Fujitsu 2444 tape drive.] HP wouldn't sell us a controller card for the tape drive on its own either. (The controller had to come with a tape drive too.) This meant we couldn't plug a 9000/300 say into an existing 6250bpi drive through a tape switch. Sheesh. Jim -- ARPA: jim%cs.strath.ac.uk@ucl-cs.arpa, jim@cs.strath.ac.uk UUCP: jim@strath-cs.uucp, ...!uunet!mcvax!ukc!strath-cs!jim JANET: jim@uk.ac.strath.cs "JANET domain ordering is swapped around so's there'd be some use for rev(1)!"
cheeks@xochitl.uucp (Mark Costlow) (03/15/89)
We've got several 300-series machines on an ethernet, intermixed with Suns.
We have 2 8-banger (model number escapes me) tape drives for backing them up.
We are currently using dump(1M) to back up the HPs. I do this with shell script
which run on the host machines which have the tape drives connected to them.
Essentially they just do: "remsh RemoteMachine dump 4uf - /". This dumps
the device to standard output, and the output of the remsh (which is the
dump itself) gets piped to tcio and out to the tape drive. It would be
trivial to make this work from a Sun, and with Exabytes on the way, we may
just do that.
Caveats: restore(1M) DOES NOT KNOW ABOUT SYMBOLIC LINKS! This is, frankly,
a Real Pisser (tm). It also cannot restore the net/special files
which are used for rfa, but with nfs around that doesn't bother
us.
Restores have to be done in a rather strange manner ... the command
looks something like: "tcio -i /dev/rct8 | restore -if -", which
does an interactive restore from standard input. Suprisingly enough,
this actually works, but you obviously can't use BSD restore's
"which tape do you want?" feature so you have to read through ALL
the tapes of the dump to do it. Full restoration of a partition
works just as you'd expect it to and rather quickly too.
I was a bit misleading when I said you could do this on Suns easily.
You can do it, but due to the nature of dumping this thing through
the network as standard output, you would need a device large enough
to hold any given dump in order to do it. HP's 8-banger tape-drive
works for this, but you would run into problems with smaller things
since dump sees standard out as a huge infinite file, as opposed to
several discrete files (tapes). Again, with Exabytes, this would
not pose a problem.
FLAME ON
I hope someone at HP is reading this ... PLEASE fix dump to handle
all HP-UX file types. Like I said, we don't use RFA so the net/specials
don't bother me, but I would imagine that it does bother somebody. And
symbolic links are a must in a network environment.
While I was able to solve our problem with dump to standard out, it's
still an ugly hack. Not everyone's got big disk-partition-sized archive
devices. Rdump is REALLY what is needed here.
Fix find(1)!!!!! It doesn't support the -prune option like BSD find,
so if you try to use the find | cpio scripts which HP seems to expect
us to use, find digs down through ALL MOUNTED NFS FILE SYSTEMS! This
makes dumps of that ilk take SEVERAL HOURS! Again, if you use RFA, find
doesn't like the net/special files either so your find commands end up
looking like: find /net/machine/usr/../. <find_options>. This too is
an ugly hack.
FLAME OFF
I apologize if these things have been fixed in the 6.5 release which I
haven't received yet.
Mark Costlow
System Bozo @ Large
Mail to: ... sun!texsun!xochitl!cheeksstevewy@hpgrla.HP.COM (Steve Wiley) (03/16/89)
Here's some facts about HP's RDAT product.
RDAT:
Capacity: 1.3 GBytes (HP/SONY DDS Format "The highest capacity format")
Transfer Rate: 183 KBytes/Sec (HP/SONY DDS Format "The fastest format")
Error Rate: <10E-15 Errors/Bit
MR Date: Sometime this fall.
Interfaces: Single Ended SCSI
HPIB
What is it? The DDS format was designed to make RDAT as fast and as high
capacity as possible. Competing formats are both slower and
lower capacity. The product will look much like any sequential
tape drive to the system with a few exceptions. One of these
is the creation of a new type of tape mark, the Save Set Mark.
This mark can be used to group files, similar to how a file mark
groups records. With this mark, each client on the LAN could
be backed up within its own save sets.
One of the most interesting facts about DAT is that a record
boundary ("gap"), filemark, and save set mark only eat up 4
bytes of tape space, hense not degredation in capacity or
transfer rate. Also, a DDS RDAT cartridge requires no pre-
formating.
Finally, it is possible to partition a DDS cartridge into 2
partitions. This first partition (the smaller one) could
be used as a directory of the second partition. This first
partition does require the use of a format statement.
Concerns: If the best the LAN can do is 50 KBytes/Sec the RDAT will
reposition quite a bit. This is not the idea case but should
not cause a great deal of trouble.decot@hpisod2.HP.COM (Dave Decot) (03/16/89)
Thanks for taking the time to let us see your comments.
I will let someone more familiar with dump/restore(1M) respond to the problems
you experienced with those utilities.
> Fix find(1)!!!!! It doesn't support the -prune option like BSD find,
HP-UX find(1) has been enhanced in Release 6.5 to include the -prune primary,
along with several other primaries and performance improvements.
Dave Decot
HPnomad@verdix.com (Lee Damon) (03/16/89)
Here are the solutions that were suggested. I thank all of you,
plus the people who posted. Yes, RDUMP _is_ necessary, unfortunatly...
nomad
-------------
From: Steve Fullerton <uunet!CS.ORST.EDU!statware!scf@verdix.com>
You might take a look at the afio program which should be on most of the
usenet archives. It supports very large blocks as well as some
network support (afio must be installed on both systems). I'm not sure
of the speed as it also utilizes remsh. You could also try to write
a simple program on the HP that opens a socket and sends across large
buffers and on the remote BSD system, utilize the inetd daemon to
monitor the socket, and then schedule the necessary program to open
the tape drive and write. I did this a while back and it went something
like:
find / -depth -print | cpio -oBc | netsend -h hostname
The netsend program opened the socket and sent across buffers of 5120
bytes (standard cpio). On the other system, I had a program called
netrecv that was scheduled by inetd that opened the tape device, read
the buffers, and wrote them to tape. Simple handshaking was implemented
to support multiple volume tapes.
I haven't used this since I started with afio, but I still build enough
of a local backup to restore a system that can then restore files
across the net; otherwise, that backup on the other system doesn't
do much good.
------------
From: Dave Taylor <uunet!hplabsz.hpl.hp.com!taylor@verdix.com>
Organization: guest of Hewlett-Packard Laboratories
Something that I like is the backup scheme being implemented
by Epoch Systems for their Epoch-1 NFS-based network file
server; the client machines 'subscribe' to a backup service
on the Epoch-1, and then simply make their file systems
available for remote NFS mounting. The server regularly
initiates an NFS connection with the client and grabs a copy
of whatever has changed on that file system, copying it
onto the local file system (either winchester or optical, it
supports both).
An easy way to apply this in a typical situation would be to
devote a machine + disk or two to "backup services" and then
to have the machine regularly mount the 500 Meg (Eagle?)
disk remotely and spin across it, copying any changed files
onto its *own* hard disk. Once that is done, the connection
is dropped and the backup server would then begin the slow
process of writing the data onto a slower media.
Actually, thinking about it, this algorithm could be considerably
speeded up by having the machine with the 500 Meg disk create
what we could call a 'prefetch data file' that contains the
fully-qualified names of all files that have been changed or
otherwise require backing up. The remote connection would then
be much much faster because instead of opening every directory
on the remote file system, it would request just the files
that it needs to have. In addition, the prefetch file could
also contain an indication of file sizes, aiding the remote
machine in ascertaining on-the-fly whether there is enough
local storage space available.
Another approach would be to have a shadow or other duplicate
of the 500 Meg disk and swap 'em back and forth as a backup
strategy... this is not a great solution unless your site is
under great stress since it's very performance expensive to
do true disk shadowing, and the off-line alternatives end up
with just as much of a performance hit as doing the slow
backup in the first place! The places this makes sense are
typically high demand data processing centers, like banks,
airline reservation systems, and so on.
An easier solution would be to purchase a faster backup
media or system. There are a number of different solutions
from Hewlett-Packard in this domain that you might consider,
including the relatively new 9145 dual-density cartridge tape
drive, which gives you twice the capacity on the same tape
AND is supposed to write to the tape significantly faster.
HP is also rumored to have a digital audio tape backup
system queued for release some time in early 1989, if they
haven't already released it to the public. That will give
you, through the HP/Sony DAT format, massively greater
storage facilities (like you could make a COMPLETE copy of
your disk on one tape, I believe!!) and considerably faster
I/O throughput speed.
[HP-IB is slow...]
It's because of this bus problem that writing directly to
the network could prove to be a win; I don't know what the
exact performance numbers are, but I wouldn't be suprised
to find that you could dump data out onto a low-traffic
network faster than you could to a slow HP-IB device.
------------
From: uunet!stepstone.com!aad@verdix.com (Anthony A. Datri)
My solution has been to mount the things I want backed up via nfs and
tar it off onto tape. This works for me becuase the total of our hpux disks
is less thana 2400 foot tape. If your total is bigger than your biggest
remote tape drive, then you might use the publically-available pdtar, which will
run everything through compress on its way to the tape.
Or, buy an exabyte, like I'm trying to.
--------------------
============= Lee Damon work: verdix!nomad or nomad@verdix.com \
play: {agora,tessi,verdix}!castle!nomad or nomad@castle.fidonet.org \
fidonet: The Castle BBS, 1:105/302, 503-641-3161 / \
"God" created man in its image, and man being ever humble returned the favor. jim@b-mrda.Boeing (Jim Sadler) (03/16/89)
/ b-mrda:comp.sys.hp / dunlap@apl-em.UUCP (John Dunlap) / 9:21 pm Mar 12, 1989 / >I have seen ads for 1.2 GByte tape drives from Bering and IEM that >connect between the computer and the disk drive. At 3 a.m. (or whatever) >John Dunlap >...!hpubvwa!apl-em!dunlap ---------- John where did you see the adds ? Jim sadler
dunlap@apl-em.UUCP (John Dunlap) (03/28/89)
Bering, Inc. has model Echo-4920 which uses the Exabyte recording technique. It puts 2.0 GByte on a DAT tape. Emulates CS/80 tape drives. Has a high speed (10 MByte/min) off line backup capability. Their phone is (800) 237-4641 or (408) 379-6900. IEM, Inc. has model ET8200-HP which has nearly identical specs. Their phone is (800) 321-4671 or (303) 223-6071. All this from their flyers -- I don't own one nor am I pushing them. John Dunlap dunlap@uw-apl.apl.washington.edu (206) 543-1300 Applied Physics Lab University of Washington 1013 NE 40th St Seattle, WA 98105