rbp@investor.UUCP (Bob Peirce) (09/06/85)
> In article <221@drivax.UUCP> alan@drivax.UUCP (Alan Fargusson) writes: > >You might try tar(1), however you will have to fix it. If you have source > >that is. As in the example in the manual type: > > > > cd fromdir; tar cf - . | ( cd todir; tar xf - ) > > > >The problem with this is that when tar tries to make a directory in todir > >it execs mkdir(1), then waits for ALL of its children to complete. But in > >the bourne shell the first tar is a child of the second tar, so it can hang > >forever. All you have to do is fix tar(1) so it only waits for the process > >that it forked. Wait a minute! I have a Bourne shell script called cphier which contains the following and runs fine. if [ $# -ne 2 ] then echo usage: cphier frompath topath fi exec tar cf - $1 | (cd $2; tar xf - ) What am I doing wrong? -- Bob Peirce, Pittsburgh, PA uucp: ...!{allegra, bellcore, cadre, idis} !pitt!darth!investor!rbp 412-471-5320 NOTE: Mail must be < 30K bytes/message
alan@drivax.UUCP (Alan Fargusson) (09/16/85)
> > Wait a minute! I have a Bourne shell script called cphier which contains the > following and runs fine. > > if [ $# -ne 2 ] > then > echo usage: cphier frompath topath > fi > exec tar cf - $1 | (cd $2; tar xf - ) > > What am I doing wrong? It doesn't always fail. The problem is that sh(1) makes the first tar a child of the second tar. If the second tar has to create a directory it does a fork/exec of mkdir, then waits for all of its children to die, including the first tar. Now if the first tar is nearly finished then it will write its output to the pipe and exit, which allows the second tar to continue. However, if the first tar has to write more than what will fit in the pipe then the whole mess hangs up, because the first tar is waiting for the second tar to read from the pipe, and the second tar is waiting for the first tar to exit. -- Alan Fargusson. { ihnp4, amdahl, mot }!drivax!alan
rbp@investor.UUCP (Bob Peirce) (09/19/85)
>> following runs fine. >> >> if [ $# -ne 2 ] >> then >> echo usage: cphier frompath topath >> fi >> exec tar cf - $1 | (cd $2; tar xf - ) > > directory it does a fork/exec of mkdir, then waits for all of its > children to die, including the first tar. Now if the first tar is > nearly finished then it will write its output to the pipe and exit, > which allows the second tar to continue. However, if the first tar > has to write more than what will fit in the pipe then the whole mess > hangs up, because the first tar is waiting for the second tar to > read from the pipe, and the second tar is waiting for the first tar > to exit. How big is the pipe? I read somewhere pipes use the root file system, which on our computer has about 1100 free blocks. Yet, I have used cphier to transfer, for example, the entire netnews source, including rn, from one drive to another. All I created was the highest level directory. Cphier made all the rest. Will it sometimes work and sometimes fail on a transfer of this size? -- Bob Peirce, Pittsburgh, PA uucp: ...!{allegra, bellcore, cadre, idis} !pitt!darth!investor!rbp 412-471-5320 NOTE: Mail must be < 30K bytes/message
david@ukma.UUCP (David Herron, NPR Lover) (09/21/85)
In article <249@investor.UUCP> rbp@investor.UUCP (Bob Peirce) writes: >>> following runs fine. >>> >>> if [ $# -ne 2 ] >>> then >>> echo usage: cphier frompath topath >>> fi >>> exec tar cf - $1 | (cd $2; tar xf - ) >> >> directory it does a fork/exec of mkdir, then waits for all of its >> children to die, including the first tar. Now if the first tar is >> nearly finished then it will write its output to the pipe and exit, >> which allows the second tar to continue. However, if the first tar >> has to write more than what will fit in the pipe then the whole mess >> hangs up, because the first tar is waiting for the second tar to >> read from the pipe, and the second tar is waiting for the first tar >> to exit. huh? This can't work as you're suggesting. If so, then the standard tar pipeline (as documented in tar(1)) wouldn't work at all. I've often (read, any time I copy directory heirarchies) used that pipeline, and IT'S JUST WORKED. This is for both simple directory trees (one or two levels) all the way to very complicated ones (like, the System V troff/nroff tree). Tar HAS to be able to exec mkdir at any time and work correctly. It can't be waiting for all it's children to exit like you say..... By The Way ... I always use the complete form as in: (cd $1; tar cf - .) | (cd $2; tar xf -) but there was this wonderfull shell script posted to net.sources around last may or so that did all sorts of error checking and creating of target directories and all. So that's what I use instead of doing the above directly. (Look in your sources archive for "cpdir(1)"). -- --- David Herron --- ARPA-> ukma!david@ANL-MCS.ARPA --- UUCP-> {ucbvax,unmvax,boulder,oddjob}!anlams!ukma!david --- {ihnp4,decvax,ucbvax}!cbosgd!ukma!david Hackin's in me blood. My mother was known as Miss Hacker before she married!
alan@drivax.UUCP (Alan Fargusson) (09/23/85)
> How big is the pipe? I read somewhere pipes use the root file system, > which on our computer has about 1100 free blocks. Yet, I have used > cphier to transfer, for example, the entire netnews source, including > rn, from one drive to another. All I created was the highest level > directory. Cphier made all the rest. Will it sometimes work and sometimes > fail on a transfer of this size? Pipes use an inode on the root file system. They are limited to 5120 bytes on System V, and 4096 on some other systems, by code in the kernel. -- Alan Fargusson. { ihnp4, amdahl, mot }!drivax!alan
guy@sun.uucp (Guy Harris) (09/24/85)
> Pipes use an inode on the root file system. Not in systems since V7; in V7, a "pipe device" was added, so that you can put pipes on other file systems if you want. Other systems, like 4.2BSD, use mechanisms other than pseudo-files to buffer pipe data. > They are limited to 5120 bytes on System V, and 4096 on some other systems, > by code in the kernel. To clarify - this is not a limit on the total amount of data which can be poured down a pipe (which seems to be what the previous poster was worried about, since they were using something like two piped-together "tar"s to copy a huge file system). This is a limit on how much data can be buffered in a pipe before the writer is blocked. Chances are slim (if not nil) that you won't be able to copy a big file system using piped-together "tar"s because of the pipe filling up the file system (unless, of course, you're moving data to the file system on which the pipes exist). Guy Harris
gwyn@brl-tgr.ARPA (Doug Gwyn <gwyn>) (09/25/85)
> Tar HAS to be able to exec mkdir at any time and work correctly. It > can't be waiting for all it's children to exit like you say..... But it did; that was a silly bug that many of us have long since fixed.
pls@soessx.UUCP (P.L.Sullivan) (09/28/85)
In article <239@drivax.UUCP> alan@drivax.UUCP (Alan Fargusson) writes: >> How big is the pipe? I read somewhere pipes use the root file system, > >Pipes use an inode on the root file system. They are limited to 5120 bytes >on System V, and 4096 on some other systems, by code in the kernel. On System V, pipes are 10 physical blocks long, with the 11th thru 13th block pointers reserved for pipe activity descriptions. This makes pipes 10240 bytes max on systems with 1k byte physical blocks (e.g. 3b2 and 3b20), 20480 bytes max on those with 2k byte physical blocks (e.g. 3b5 and 3b15). Guy Harris posted a reply to the above article, but didn't mention this - is this peculiar to AT&T hardware?? ================================================================== Pat Sullivan - {akgua|akguc|ihnp4}!soessx!pls - voice 404-257-7382
seth@megad.UUCP (Seth H Zirin) (10/02/85)
> > How big is the pipe? I read somewhere pipes use the root file system, > > which on our computer has about 1100 free blocks. Yet, I have used > > cphier to transfer, for example, the entire netnews source, including > > rn, from one drive to another. All I created was the highest level > > directory. Cphier made all the rest. Will it sometimes work and sometimes > > fail on a transfer of this size? > > Pipes use an inode on the root file system. They are limited to 5120 bytes > on System V, and 4096 on some other systems, by code in the kernel. Not entirely true, pipes do use an inode, but its from the "pipedev" not the "rootdev" in SIII and SV. The "pipedev" is however usually the same as the "rootdev", although it doesn't have to be. The data in pipes is stored in blocks of the buffer cache. When the buffer cache fills and a buffer is needed for perhaps another process, the data from a pipe could find itself physically on the disk (especially with named pipes). I have seen instances (SIII) where the system crashed or was rebooted when there was data in opened named pipes, and the data remained accessable after reboot. If both ends of the named pipe are closed before reboot, the data is flushed. -- ------------------------------------------------------------------------------- Name: Seth H Zirin UUCP: {decvax, ihnp4}!philabs!sbcs!megad!seth Keeper of the News for megad
vince@leadsv.UUCP (Vince Westin) (10/03/85)
In article <2226@ukma.UUCP>, david@ukma.UUCP (David Herron, NPR Lover) writes: > > (cd $1; tar cf - .) | (cd $2; tar xf -) > Why not ( cd $2; tar -cf - -C $1 . | tar xf - ) ? -- Vince Westin { Live and learn ... Die and forget it all } {decwrl,dual,ihnp4}!{amdcad!cae780,sun!sunncal}!leadsv!vince
guy@sun.uucp (Guy Harris) (10/14/85)
> > (cd $1; tar cf - .) | (cd $2; tar xf -) > > Why not ( cd $2; tar -cf - -C $1 . | tar xf - ) ? Because the -C flag to "tar" was added by Berkeley in 4.2BSD; it's not present in all versions of "tar". Guy Harris
acheng@uiucdcs.CS.UIUC.EDU (10/16/85)
>/* Written 3:30 am Oct 14, 1985 by guy@sun.uucp in uiucdcs:net.unix-wizar */ >> > (cd $1; tar cf - .) | (cd $2; tar xf -) >> >> Why not ( cd $2; tar -cf - -C $1 . | tar xf - ) ? > >Because the -C flag to "tar" was added by Berkeley in 4.2BSD; it's not >present in all versions of "tar". > > Guy Harris Our 4.2 version of tar.c doesnot have a -C option nor does the man page says so. What is it suppose to do? In the search of the 'C' option, i found an 'F' option. Again, no comment, no man page coverage.
davy@pur-ee.UUCP (Dave Curry) (10/17/85)
In article <13700116@uiucdcs> acheng@uiucdcs.CS.UIUC.EDU writes: > >Our 4.2 version of tar.c doesnot have a -C option nor does the man page >says so. What is it suppose to do? > >In the search of the 'C' option, i found an 'F' option. Again, no comment, >no man page coverage. The 'F' option is neat.... if you double it, 'FF', then tar will not put any files whose names end in ".o" onto the tape. For example: tar cvFF /usr/src would put all the sources on, but leave the objects off. I don't know what a single 'F' does... --Dave Curry
mikel@codas.UUCP (Mikel Manitius) (10/19/85)
> (cd $1; tar cf - .) | (cd $2; tar xf -)
or (cd $1; find . -print | cpio -o .) | (cd $2; cpio -idu)
for those without tar.
What I really wish our System V people would do is put in the -r flag
(for recursive) on cp(1), like Berkeley did a long time ago.
--
Mikel Manitius - ...{ihnp4|akguc|attmail|indra!koura}!codas!mikelgwyn@brl-tgr.ARPA (Doug Gwyn <gwyn>) (10/21/85)
> or (cd $1; find . -print | cpio -o .) | (cd $2; cpio -idu)
Better still,
cd $1; find . -print -depth | cpio -pdu $2
If links are preferable, add the "l" flag to the cpio options.guy@sun.uucp (Guy Harris) (10/22/85)
> Our 4.2 version of tar.c doesnot have a -C option nor does the man page > says so. Wanna bet? Here's the relevant page, snatched from "/usr/man/man/tar.1" right off the 4.2 tape: If a file name is preceded by .BR \-C , then .I tar will perform a .IR chdir (2) to that file name. This allows multiple directories not related by a close common parent to be archived using short relative path names. For example, to archive files from /usr/include and from /etc, one might use .ti +0.5i tar c -C /usr include -C / etc It's in the "tar" source as well. You just didn't check the whole manual page or the whole source file. > In the search of the 'C' option, i found an 'F' option. Again, no comment, > no man page coverage. It is rather a hack - it prevents directory files with the name SCCS and plain files which: have a name of one or two characters in length have a name ending in ".o" have the name "core" or "errs" or, in some circumstances, have the name "a.out" from being put onto a tape. Presumably, this is so that only source files - not SCCS files, object files, executable images (there is a comment saying "SHOULD CHECK IF IT IS EXECUTABLE", although since they have a "stat" of the file I have no idea why they *don't* so check), or droppings like core dumps and "errs" files - I believe it is a custom to redirect the output of compiles, or the error output anyway, to "errs". It is probably not documented 1) because it *is* such a hack and 2) because there is no guarantee that it will be in all subsequent 4.xBSD releases. Guy Harris
guy@sun.uucp (Guy Harris) (10/22/85)
> The 'F' option is neat.... if you double it, 'FF', then tar will > not put any files whose names end in ".o" onto the tape. This is more correct than my explanation. "F" leaves out SCCS directories and "core" and "errs" files; "FF" also leaves ".o" and "a.out" files (files named "a.out", not arbitrary executable images). One of "cpio"s two redeeming features (the other is that it understands file which aren't plain files or symlinks) is that it reads a list of files to write from its standard input, so you can construct arbitrary filters which will specify which files do and don't go. (It still doesn't make up for "cpio"s deficiencies - somebody proposed, and may have implemented, an option to "tar" to have it read its list of files to write from its standard input.) Guy Harris
guy@sun.uucp (Guy Harris) (10/22/85)
> > (cd $1; tar cf - .) | (cd $2; tar xf -) > > or (cd $1; find . -print | cpio -o .) | (cd $2; cpio -idu) > for those without tar. Or (cd "$1"; find . -depth -print | cpio -pdl "$2"), according to the S5 "cpio" manual page. > What I really wish our System V people would do is put in the -r flag > (for recursive) on cp(1), like Berkeley did a long time ago. It's not too hard to do (I did it a while ago, but don't have the sources handy). Both "cp -r" and "tar"/"cpio" have their uses. "cp -r" is more convenient, but back-to-back "tar"s and "cpio" will preserve more information. (Beware of symbolic links if you do a "cp -r", for instance; "cp" doesn't know from symlinks.) Guy Harris
lasse@daab.UUCP (Lars Hammarstrand) (10/26/85)
In article <313@codas.UUCP> mikel@codas.UUCP (Mikel Manitius) writes: >> (cd $1; tar cf - .) | (cd $2; tar xf -) > >or (cd $1; find . -print | cpio -o .) | (cd $2; cpio -idu) >for those without tar. > >What I really wish our System V people would do is put in the -r flag >(for recursive) on cp(1), like Berkeley did a long time ago. >-- *** NO!!!!!, THAT'S NOT GOOD ENOUGH. **** Why doesn't anybody write a GOOD AND PORTABLE cptree(1) with some flags like this.... (It shall manage to copy special files too) cptree [flags...] fromdir todir [file list or pattern...] (copy only files matching [file list or pattern...]) (*?[]) -f force (overwrite any file with the same name). -t time. -v vebose. -n not matching [file list or pattern...] (*?[]) -i take pattern from [inputfile] or standard-input. -c create dir if it's not there. .. forgotten anything? ..and you say, why don`t you? I have tried but I don't have the time to make it complete so I let someone else to do it instead! Lars Hammarstrand. Datorisering AB, Stockholm, SWEDEN.
acheng@uiucdcs.CS.UIUC.EDU (10/29/85)
>> Our 4.2 version of tar.c doesnot have a -C option nor does the man page >> says so. > >Wanna bet? Here's the relevant page, snatched from "/usr/man/man/tar.1" >right off the 4.2 tape: >... > Due to temporary senility, I didnot read the whole man page before hitting the keyboard. Yes, the -C option is described black and white in the man page. =albert cheng
geoff@desint.UUCP (Geoff Kuenning) (10/31/85)
In article <169@daab.UUCP> lasse@daab.UUCP (Lars Hammarstrand) writes: >Why doesn't anybody write a GOOD AND PORTABLE cptree(1) with some flags >like this.... (It shall manage to copy special files too) [list of features follows] AT&T did, a while ago. It's called cpio. Most of your suggested features are supported directly; the rest are easy with 'find' and 'comm' pipes. -- Geoff Kuenning {hplabs,ihnp4}!trwrb!desint!geoff
lasse@daab.UUCP (Lars Hammarstrand) (11/06/85)
> >In article <169@daab.UUCP> lasse@daab.UUCP (Lars Hammarstrand) writes: > > > >Why doesn't anybody write a GOOD AND PORTABLE cptree(1) with some flags > >like this.... (It shall manage to copy special files too) > > > >[list of features follows] > >In article <136@desint.UUCP> geoff@desint.UUCP (Geoff Kuenning) writes: >AT&T did, a while ago. It's called cpio. Most of your suggested features >are supported directly; the rest are easy with 'find' and 'comm' pipes. >-- > > Geoff Kuenning > {hplabs,ihnp4}!trwrb!desint!geoff Are you sure that you can find CPIO on every system round the world? When I said "GOOD AND PORTABLE", I meant that everybody should have the oppotunity to use a true utlilty like cptree(1)... Lars Hammarstrand.
jsdy@hadron.UUCP (Joseph S. D. Yao) (11/09/85)
In article <169@daab.UUCP> lasse@daab.UUCP (Lars Hammarstrand) writes: >Why doesn't anybody write a GOOD AND PORTABLE cptree(1) with some flags >like this... An early USENIX distribution had a command called move (ln move copy) that did almost exactly what you ask. It originated at the Rand Corporation. It was either recursive or not, even linked or not, per your request. I don't know if it has been updated for FIFO's or symlinks or whatever is current special on your system. -- Joe Yao hadron!jsdy@seismo.{CSS.GOV,ARPA,UUCP}