[net.unix-wizards] Flames on system backups / The software tool approach

dan@SRI-TSC (04/04/83)

When I worked at Air Force Data Services Center, we came up against the same
problem of trying to use archival programs to store files on multiple reels of
tape.  We took the general case, however, and came up with a "tool" for
writing/reading multi-reel tapes.  I'm suprised noone else has thought of it,
since it falls right in with the UNIX concept of filters and having many small
programs that do a single task each, allowing you to combine them to solve
your problems.  Basically, what we did was create a filter you could pipe
into, which did all the dirty work of prompting for reels, watching for end of
tape, writing the appropriate block size, etc.  There were flags for block
size, BPI, etc, as well as a -r flag to go the otherway and read from tape
(and pipe the output into tar or whatever).

It seems that the concept of UNIX software tools has died.  Everybody
is writing gargantuan programs that have a million flags and "do everything".
Or even worse, taking a simple, elegant program designed to do a single task
well, and hacking in a million bells and whistles so that the program does
a hundred tasks, but none of them well.  Unix is beginning to feel a lot like
TSO...

	-Dan (I grew up on V6 -- a "real man's UNIX") Chernikoff
	dan@sri-tsc

smh (04/17/83)

Chernikoff's exposition on using a filter to handle multi-reel
packing on tape backups seems pretty neat.  Of course, high-volume
pipes on small machines (PDP-11's) without huge buffer cache's
might slow down backups somewhat, but Chernikoff neglected to
mention what might be another BIG win:
	Tape backups on UNIX suffer because IO is synchronous.  Some
number of blocks of data is gathered into a buffer (typically 20 blocks)
and then the backup process waits while the tape drive does its thing.
Having a pipe between the disk-reading process and the tape-writing
process could provide desirable IO overlaps, and could significantly
increase throughput.  The tape blocksize might have to be tuned to
agree with pipe size.
	Has Dan or anyone had any experience with this?
				Steve Haflich
				MIT Experimental Music Studio
				...genrad!mit-eddie!smh