[comp.unix.wizards] tar -r on a diskfile/tar arg too long

ssi@ziggy.EDU (Ssi) (06/08/90)

Does anyone know any draw backs/problems with using the -r option of
tar(1) on disk files, as in:

cat /dev/null > TARFILE
tar -cvf TARFILE long_list       <<-- table of contents as first file
for i in `cat long_list`
do
tar -rvf TARFILE $i             <<-- append each file to TARFILE
done
tar -cvf /dev/rmt0 TARFILE      <<-- put the whole thing to tape 
rm TARFILE


note:

tar -cvf /dev/rmt0 `cat long_list` 

Would give tar argument too long error.



                     Greg Ripp (813)628-6100 x5123
                     greg@system1.usfvax2.edu
                     ...!uunet!ateng!usfvax2!system1!greg

tr@samadams.princeton.edu (Tom Reingold) (06/08/90)

In article <1339@ziggy.EDU> ssi@ziggy.EDU (Ssi) writes:
$ 
$ 
$ Does anyone know any draw backs/problems with using the -r option of
$ tar(1) on disk files, as in:
$ 
$ cat /dev/null > TARFILE
$ tar -cvf TARFILE long_list       <<-- table of contents as first file
$ for i in `cat long_list`
$ do
$ tar -rvf TARFILE $i             <<-- append each file to TARFILE
$ done
$ tar -cvf /dev/rmt0 TARFILE      <<-- put the whole thing to tape 
$ rm TARFILE
$ 
$ 
$ note:
$ 
$ tar -cvf /dev/rmt0 `cat long_list` 
$ 
$ Would give tar argument too long error.

One disadvantage is that it's going to be very slow.  Each file you add
will take longer than the previous one.

This is why "cpio" takes its input names from its standard input.  It's
an unusual way of doing things, but very appropriate for a file
archiver.

Also, not all files can be appended to, such as exabyte tapes.  It
simply won't work.
--
                                        Tom Reingold
                                        tr@samadams.princeton.edu
                                        rutgers!princeton!samadams!tr
                                        201-560-6082

bill@twg.UUCP (Bill Irwin) (06/09/90)

In article <1339@ziggy.EDU< ssi@ziggy.EDU (Ssi) writes:
<Does anyone know any draw backs/problems with using the -r option of
<tar(1) on disk files, as in:
<
<cat /dev/null > TARFILE
<tar -cvf TARFILE long_list       <<-- table of contents as first file
<for i in `cat long_list`
<do
<tar -rvf TARFILE $i             <<-- append each file to TARFILE
<done
<tar -cvf /dev/rmt0 TARFILE      <<-- put the whole thing to tape
<rm TARFILE

I have always used "tar cvf TARFILE `cat long_list`" to do this.

There is also the -F option to specify the file:

tar cvfF TARFILE long_list

Using one of these methods should eliminate your error message.
-- 
Bill Irwin - TWG The Westrheim Group - Vancouver, BC, Canada
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
uunet!van-bc!twg!bill     (604) 431-9600 (voice) |     UNIX Systems
Bill.Irwin@twg.UUCP       (604) 431-4629 (fax)   |     Integration

ssi@ziggy.EDU (Ssi) (06/09/90)

This message is empty.

karish@mindcrf.UUCP (06/09/90)

In article <1340@ziggy.EDU> ssi@ziggy.cmd.usf.edu.UUCP (Ssi) writes:
>In article <1339@ziggy.EDU> ssi@ziggy.EDU (Ssi) writes:
>$ Does anyone know any draw backs/problems with using the -r option of
>$ tar(1) on disk files, as in:
>$ 
>$ cat /dev/null > TARFILE
>$ tar -cvf TARFILE long_list       <<-- table of contents as first file
>$ for i in `cat long_list`
>$ do
>$ tar -rvf TARFILE $i             <<-- append each file to TARFILE
>$ done
>$ tar -cvf /dev/rmt0 TARFILE      <<-- put the whole thing to tape 
>$ rm TARFILE

Unless nested tar files are somehow advantageous, I'd use 'dd' to
put TARFILE to tape.  This lets you choose the biggest block size
your tape drive can handle, which often results in a substantial
increase in tape capacity.  The tradeoff is in portability; drives
vary in the block size they can handle.

>$ note:
>$ tar -cvf /dev/rmt0 `cat long_list` 
>$ Would give tar argument too long error.

If 'tar -r' turns out to be the best answer, use 'xargs' instead of
the shell loop.  It'll minimize the number of seeks and process
creations needed.

>Maybe I should mention that this is part of a bigger picture, to
>backup a network that requires more than one tape, and has to
>run on a `live' system.

In this context, the proposed scheme makes less sense than it did
before.  Where do you find disk space for the tremendous working file?
You can't back up the whole system unless half the server's disk
is kept free.

As previously noted, cpio works fine with an arbitrarily long
input list (on stdin, not on the command line).  If your system
doesn't have it, or you don't care for the user interface or
the error messages, try one of the public domain or freely-
distributable tar programs like pax, pdtar, or GNUtar.  I think
pax and GNUtar can read long input lists.  With the source,
it's not hard to make any of 'em do it.

The real trick is to handle multi-volume archives properly.  Some tape
drives can sense end of tape and report it in a timely fashion, and
others can't; ditto for the drivers that should listen for those
reports.  The usual remedy has been to tell the program how long your
tape is and what average data density you expect, and let the program
guess how much will fit on each volume.  You should adjust these
parameters (yes, lie to tar!) to take into account the block size
you're using and the characteristics of your tape drive: different
drives put different-length gaps between blocks.
-- 

	Chuck Karish		karish@mindcraft.com
	Mindcraft, Inc.		(415) 323-9000