[comp.unix.wizards] tar frustration

lew@gsg.UUCP (Paul Lew) (08/11/88)

From article <2858@ttrdc.UUCP>, by levy@ttrdc.UUCP (Daniel R. Levy):
> 
> All this points up a "feature" of tar which I find frustrating:  if I want
> tar to tape-archive a large number of files randomly scattered all over the
> file system (such as for an incremental backup) I'm SOL because tar wants
> to be told either a directory to completely search or file names to archive,
> via the argument list.  "cpio" circumvents this problem, since I can feed it
> a list of files, but what if I don't WANT to use cpio?

Check with public domain tar posted to comp.sources.unix volumn 12.  There is
a flag 'T' which will take filenames from a file.  I used it to save sources
files like:

	find $src -print | sed	\
		-e '/\.o$/d'	\
		-e '/\.a$/d'	\
		-e '/~$/d'	\
		-e '/\/core$/d'	\
		-e '/\/a\.out$/d' | tar -c -T -

It works out great.  If $src is an absolute pathname, this tar will remove
the leading /s when writing to tape.
-- 
Paul Lew			{oliveb,harvard,decvax}!gsg!lew	(UUCP)
General Systems Group, 5 Manor Parkway, Salem, NH 03079	(603) 893-1000

levy@ttrdc.UUCP (Daniel R. Levy) (08/12/88)

> >tar cf /dev/whatever *

> I would suggest using "." rather than "*" to avoid the expansion of the commmand
> line to ridiculously long lengths.  While the meaning is certainly different, I
> have yet to think of any problems with this method when the intent is to tar up
> the contents of my current directory.

All this points up a "feature" of tar which I find frustrating:  if I want
tar to tape-archive a large number of files randomly scattered all over the
file system (such as for an incremental backup) I'm SOL because tar wants
to be told either a directory to completely search or file names to archive,
via the argument list.  "cpio" circumvents this problem, since I can feed it
a list of files, but what if I don't WANT to use cpio?  (Say, in a situation
which would trigger a known cpio bug, like inode numbers greater than 65535 or
uid's less than 0 [a la SUN] when doing cpio -c.)  Using the "r" option of tar
with repeated invocations of tar would work all right, but would be blastedly
slow because it would rewind the tape over and over and over.  If I used the
no-rewind tape device, I'd get a whole bunch of little tar archives, one for
each invocation.
-- 
|------------Dan Levy------------|  THE OPINIONS EXPRESSED HEREIN ARE MINE ONLY
| Bell Labs Area 61 (R.I.P., TTY)|  AND ARE NOT TO BE IMPUTED TO AT&T.
|        Skokie, Illinois        | 
|-----Path:  att!ttbcad!levy-----|

ron@topaz.rutgers.edu (Ron Natalie) (08/13/88)

If anybody is interested, I've got kicking around a public domain version of
TAR that takes the CPIO user interface, that is, the list of file names is
provided on the standard input.  While I was writing this, I noticed that
while every version of the TAR manual page that I've come accross describes
the format calls for zero-filling the fields, every implementation I've
seen actually space fills them (as if they had done it with Printf("%12d")).

Fortunately, every one is liberal in reading the archives (scanf doesn't care).

-Ron

pope@vatican (John Pope) (08/13/88)

In article <2858@ttrdc.UUCP>, levy@ttrdc (Daniel R. Levy) writes:
>
>All this points up a "feature" of tar which I find frustrating:  if I want
>tar to tape-archive a large number of files randomly scattered all over the
>file system (such as for an incremental backup) I'm SOL because tar wants
>to be told either a directory to completely search or file names to archive,
>via the argument list.  "cpio" circumvents this problem, since I can feed it
>a list of files, but what if I don't WANT to use cpio?

To feed tar a list of files, I just keep the directories I want in a file called
"save_list" and do:

	tar cf /dev/rst8 `cat save_list` 

As a side note, SunOS has a handy "X" option to tar, which specifies a filename
containing files to exclude from the backup:

	tar cfX /dev/rst8 exclude_list `cat save_list` 

This lets me back up everything in /usr/foo, but exclude the subdirectory
/usr/foo/bar, for example.
-- 
-- 
John Pope
	Sun Microsystems, Inc. 
		pope@sun.COM

james@bigtex.uucp (James Van Artsdalen) (08/13/88)

In article <2858@ttrdc.UUCP>, levy@ttrdc.UUCP (Daniel R. Levy) wrote:

> All this points up a "feature" of tar which I find frustrating:  if I want
> tar to tape-archive a large number of files randomly scattered all over the
> file system (such as for an incremental backup) I'm SOL because tar wants
> to be told either a directory to completely search or file names to archive,
> via the argument list.

Am I the only one to use John Gilmore's tar that was posted a while
back?  It solves all of the problems I've so far (including this one -
his can take files from stdin).  It does need some work to get it
working under SysV, but definitely worthwhile.
-- 
James R. Van Artsdalen    ...!uunet!utastro!bigtex!james     "Live Free or Die"
Home: 512-346-2444 Work: 328-0282; 110 Wild Basin Rd. Ste #230, Austin TX 78746

root@conexch.UUCP (Larry Dighera) (08/13/88)

In article <2858@ttrdc.UUCP< levy@ttrdc.UUCP (Daniel R. Levy) writes:
<< <tar cf /dev/whatever *
<
<< I would suggest using "." rather than "*" to avoid the expansion of the commmand
<< line to ridiculously long lengths.  While the meaning is certainly different, I

<All this points up a "feature" of tar which I find frustrating:  if I want
<tar to tape-archive a large number of files randomly scattered all over the
<file system (such as for an incremental backup) I'm SOL because tar wants
<to be told either a directory to completely search or file names to archive,
<via the argument list.  "cpio" circumvents this problem, since I can feed it
<a list of files, but what if I don't WANT to use cpio?  (Say, in a situation
<which would trigger a known cpio bug, like inode numbers greater than 65535 or
<uid's less than 0 [a la SUN] when doing cpio -c.)  Using the "r" option of tar
<with repeated invocations of tar would work all right, but would be blastedly
<slow because it would rewind the tape over and over and over.  If I used the
<no-rewind tape device, I'd get a whole bunch of little tar archives, one for
<each invocation.

This is so simple that it makes me feel like I don't understand the problem.
If you want tar to take the names of the files it is to put into the archive
from a file which contains the names of the files, just do this:

	tar cvf /dev/whatever `cat file_of_names`

You can generate file_of_names with find just like is normally done with
cpio.  Ain't UNIX grand?

Larry Dighera



-- 
USPS: The Consultants' Exchange, PO Box 12100, Santa Ana, CA  92712
TELE: (714) 842-6348: BBS (N81); (714) 842-5851: Xenix guest account (E71)
UUCP: conexch Any ACU 2400 17148425851 "" "" ogin:-""-ogin:-""-ogin: nuucp
UUCP: ...!uunet!turnkey!conexch!root || ...!trwrb!ucla-an!conexch!root

mouse@mcgill-vision.UUCP (der Mouse) (08/20/88)

In article <2858@ttrdc.UUCP>, levy@ttrdc.UUCP (Daniel R. Levy) writes:
> All this points up a "feature" of tar which I find frustrating:
> [...can't take random filenames except in argument list...].  "cpio"
> circumvents this problem, since I can feed it a list of files, but
> what if I don't WANT to use cpio?

You use a tar that can take filenames from stdin.  Mine can.  I think
Gilmore's can.  (Mine also treats absolute pathnames on the tape
specially, and has other frills.)

					der Mouse

			old: mcgill-vision!mouse
			new: mouse@larry.mcrcim.mcgill.edu

mark@iccdev.UUCP (Mark Wutka) (08/27/88)

In article <1249@mcgill-vision.UUCP>, mouse@mcgill-vision.UUCP (der Mouse) writes:
> In article <2858@ttrdc.UUCP>, levy@ttrdc.UUCP (Daniel R. Levy) writes:
> > All this points up a "feature" of tar which I find frustrating:
> > [...can't take random filenames except in argument list...].  "cpio"
> > circumvents this problem, since I can feed it a list of files, but
> > what if I don't WANT to use cpio?
> 
> You use a tar that can take filenames from stdin.  Mine can.  I think
> Gilmore's can.  (Mine also treats absolute pathnames on the tape
> specially, and has other frills.)

If you don't have a tar that will take files from stdin, you can try what
I have used here:

	tar <whatever> `cat listoffiles`

Hopefully this will work right on your shell. You can list the files
one per line if you like. I'm not sure if all the shells do this, but
the one I use - ksh - will concatenate the files into one long line
separated by spaces. Be careful, though, it may bomb if you give it
some enormous number of files. I haven't had a blow-up with several
hundred files, though.

If you want to make tar take from stdin, just try:

	tar <whatever> `cat`

This will works with other commands, of course. The only one that comes
to my mind right now is "ar". If you want to create a library and can't
really use wildcards you can do it this way.


-- 
...!gatech!ncrats!iccdev!mark

This is what happens when I roll my head on the keyboard:
kijmuhnyjuygikmluhygbtnjkm,l.jhnubgyvfnjmuki,lmnjhbgv

MorsinAc@econ.vu.nl (Triple A) (09/27/88)

My summary says it all...

This is my roll:
sdewxz 0-omk-56l;b vnjbgmh