pcg@aber-cs.UUCP (Piercarlo Grandi) (02/02/90)
In article <15142@bfmny0.UU.NET> tneff@bfmny0.UU.NET (Tom Neff) writes: In article <1990Jan31.034746.8408@virtech.uucp> cpcahil@virtech.UUCP (Conor P. Cahill) writes: >BTW - when I use this kind of backup I use the following: > > find ... | cpio ... | compress | dd of=/dev/rmt0 obs=2048k > >This will run much faster and most of the time will use less space on >the tape since it will stream 2MB segments. You should be using 'ddd' (which has been posted to comp.sources.misc) or, even faster, my own 'team' (which has been posted to alt.sources; a slightly improved version will be submitted to comp.sources.misc, so don't ask me for a copy now). These use two (or more) processes with local buffering to overlap input with output, and smooth out variations in the speed with which data is read from the disc. I normally run my own 'team' with four processes each with a 30K buffer, and it always streams continuously. Actually on my box, using "bs=2048k" (both input and output) rather than just "obs=2048k" (output) runs even faster. It seems as though with the one block size and no conversion, 'dd' reads and writes simultaneously with good efficiency. Using 'obs' is a bad idea; 'dd' will allocate *two* buffers, one 'ibs' (default 512?) bytes long, and the other 'obs' bytes long, and *copy* between the two before writing. If you use 'bs', 'dd' just allocate one buffer, and this is vastly faster. -- Piercarlo "Peter" Grandi | ARPA: pcg%cs.aber.ac.uk@nsfnet-relay.ac.uk Dept of CS, UCW Aberystwyth | UUCP: ...!mcvax!ukc!aber-cs!pcg Penglais, Aberystwyth SY23 3BZ, UK | INET: pcg@cs.aber.ac.uk
mike@antel.uucp (Michael Borza) (02/07/90)
[sorry if I've got the quoting wrong here-- m.b.] In article <1611@aber-cs.UUCP> pcg@cs.aber.ac.uk (Piercarlo Grandi) writes: >In article <15142@bfmny0.UU.NET> tneff@bfmny0.UU.NET (Tom Neff) writes: > In article <1990Jan31.034746.8408@virtech.uucp> cpcahil@virtech.UUCP (Conor P. Cahill) writes: >>>BTW - when I use this kind of backup I use the following: >>> >>> find ... | cpio ... | compress | dd of=/dev/rmt0 obs=2048k >>> >>>This will run much faster and most of the time will use less space on >>>the tape since it will stream 2MB segments. > >comp.sources.misc) or, even faster, my own 'team' (which has >been posted to alt.sources; a slightly improved version will be >submitted to comp.sources.misc, so don't ask me for a copy .... looking forward to seeing it .... >>>Actually on my box, using "bs=2048k" (both input and output) rather >>>than just "obs=2048k" (output) runs even faster. It seems as though >>>with the one block size and no conversion, 'dd' reads and writes >>>simultaneously with good efficiency. > >Using 'obs' is a bad idea; 'dd' will allocate *two* buffers, >one 'ibs' (default 512?) bytes long, and the other 'obs' bytes >long, and *copy* between the two before writing. If you use >'bs', 'dd' just allocate one buffer, and this is vastly faster. Actually, this seems to be very machine dependent. On my 20 MHz machine (10 MB of memory, no caching, ESDI disk system, Everex tape controller, Archive tape drive), `dd bs=1024k' gives very poor performance, as does `dd' with the default buffer sizes. `dd ibs=1024k obs=1024k' gives much better performance, keeping the tape streaming about 80% of the time. The all time best performance, though, is `dd obs=1024k', using the default input buffer size for tape writes. On my machine, this keeps the tape streaming >90% of the time the drive motor's running. (Reading from the tape, ibs is substituted for obs, naturally.) It appears that the situation is not so clear-cut as it might first appear. I'd say that some experimentation with particular hardware is in order to find what works best on a given system. >Piercarlo "Peter" Grandi | ARPA: pcg%cs.aber.ac.uk@nsfnet-relay.ac.uk mike borza. -- Michael Borza Antel Optronics Inc. (416)335-5507 3325B Mainway, Burlington, Ont., Canada L7M 1A6 work: mike@antel.UUCP or uunet!utai!utgpu!maccs!antel!mike home: mike@boopsy.UUCP or uunet!utai!utgpu!maccs!boopsy!mike
michael@fts1.UUCP (Michael Richardson) (02/10/90)
In article <1611@aber-cs.UUCP> pcg@cs.aber.ac.uk (Piercarlo Grandi) writes: >You should be using 'ddd' (which has been posted to >comp.sources.misc) or, even faster, my own 'team' (which has >been posted to alt.sources; a slightly improved version will be Now, I haven't tried `team', so I can't comment on that, but I did run some preliminary tests with ddd vs SVR3.2 (ICS 386/ix) dd, and found that there was a negligible difference between the two. I have not gotten around to doing disk to disk tests, I started with {dd/ddd} if=/dev/rmt0 of=junk and of=/dev/null with block sizes varying from 200k down to 1k. The tape (an AT style Archive tape controller in an AMI386 Mark II motherboard) had about 10meg of stuff on it. (Had to up the ulimit to run the tests :-) ) ddd seemed to occasionally change the file mode to 000, and then die on the next write. I never did track this down. Is it possible that the tape controller was the critical factor, and ddd does many times better on disk to disk? Or writes to the drive? -- :!mcr!: Michael C. Richardson HOME: mcr@julie.UUCP SCHOOL: mcr@doe.carleton.ca WORK: michael@fts1.UUCP I never liked staying in one place too long, but this is getting silly...