larry@focsys.UUCP (Larry Williamson) (12/10/88)
Streaming tape I/O with 386/ix seems to be rather slow. The drive
is not streaming very well. It spends most of it's time stopping
and starting. I'm using an Everex Excel 60 with a long controller
card.
The command I've used is
find . -print | cpio -ovc >/dev/rmt0
Microport 286 SV/AT has a nice little utility called strm that
buffers a large amount of data for the tape drive so that the drive
is able to spend more time streaming and less time stop-starting.
Is there any such utility for 386/ix? Is there some other secret way
to get better efficiency out of the driver?
In a foolish act of desperation, I tried the strm program, but
although it ran and did not degrade the tape operation, it did not
help either.
Thanks,
Larry
--
Larry Williamson -- Focus Systems -- Waterloo, Ontario
watmath!focsys!larry (519) 746-4918james@bigtex.cactus.org (James Van Artsdalen) (12/11/88)
In <317@focsys.UUCP>, larry@focsys.UUCP (Larry Williamson) wrote: > Streaming tape I/O with 386/ix seems to be rather slow. The drive > is not streaming very well. It spends most of it's time stopping > and starting. Try using GNU tar. I use a rather old version, but the current version should be at least as good. I do backups with a blocking factor of 6000, or three megabytes per write to the tape. tar has portability advantages over cpio anyway. -- James R. Van Artsdalen james@bigtex.cactus.org "Live Free or Die" Dell Computer Corp, 9505 Arboretum Blvd Austin TX 78759 512-338-8789
debra@alice.UUCP (Paul De Bra) (12/11/88)
In article <317@focsys.UUCP> larry@focsys.UUCP (Larry Williamson) writes: ]Streaming tape I/O with 386/ix seems to be rather slow. The drive ]is not streaming very well. It spends most of it's time stopping ]and starting. I'm using an Everex Excel 60 with a long controller ]card. ] ]The command I've used is ] find . -print | cpio -ovc >/dev/rmt0 ] A solution which also works for other applications is to use "cat": find . -print | cpio -ovc | cat | cat > /dev/rmt0 Hope this helps. Paul. -- ------------------------------------------------------ |debra@research.att.com | uunet!research!debra | ------------------------------------------------------
wnp@dcs.UUCP (Wolf N. Paul) (12/11/88)
In article <317@focsys.UUCP> larry@focsys.UUCP (Larry Williamson) writes: >Streaming tape I/O with 386/ix seems to be rather slow. The drive >is not streaming very well. It spends most of it's time stopping >and starting. I'm using an Everex Excel 60 with a long controller >card. > >The command I've used is > find . -print | cpio -ovc >/dev/rmt0 > It seems to me that you should be able to achieve the effect of the "strm" utility on Uport 286 (and other similar utilities, i.e. "stream" as supplied with the Bell Tech driver, etc.) by using "dd" with a large block size. You may need to experiment to find out how large a block size your system will allow, but try something like this: find . -print | cpio -ocv | dd of=/dev/rmt0 bs=64k Then try increasing the "bs" value until you either (a) get no more performance improvement by increasing, or (b) run out of memory. (A buffer size of 64k should work even on a 286 system). -- Wolf N. Paul * 3387 Sam Rayburn Run * Carrollton TX 75007 * (214) 306-9101 UUCP: killer!dcs!wnp ESL: 62832882 DOMAIN: dcs!wnp@killer.dallas.tx.us TLX: 910-380-0585 EES PLANO UD
pim@ctisbv.UUCP (Pim Zandbergen) (12/11/88)
In article <317@focsys.UUCP> larry@focsys.UUCP (Larry Williamson) writes: >Streaming tape I/O with 386/ix seems to be rather slow. > >The command I've used is > find . -print | cpio -ovc >/dev/rmt0 > A better command would be: find . -print | cpio -ovcBO /dev/rmt0 This tells cpio to use 5120 byte blocks. You can also use dd(1) to have arbitrary blocksizes, like in: find . -print | cpio -ovc | dd of=/dev/rmt0 obs=20k Try some different blocksizes for optimum results. -- --------------------+------------------------------------+--------------------- Pim Zandbergen | CTI Software BV | Phone: +31 70 542302 pim@ctisbv.UUCP | Laan Copes van Cattenburch 70 | Fax: +31 70 512837 ..!mcvax!ctisbv!pim | 2585 GD The Hague, The Netherlands | Telex: 32133 CTI NL
plocher@uport.UUCP (John Plocher) (12/12/88)
In article <317@focsys.UUCP> larry@focsys.UUCP (Larry Williamson) writes: >Streaming tape I/O with 386/ix seems to be rather slow. > > find . -print | cpio -ovc >/dev/rmt0 > >Microport 286 SV/AT has a nice little utility called strm that Try (on 386 unix only) find . -print | cpio -ovcB | dd of=/dev/rmt0 bs=500k or find . -print | cpio -ovcBC500000 > /dev/rmt0 The latter command is undocumented and may not work <std_declaimer.h> syntax is -B Use large block size -Cbuffersize Use an output buffer of buffersize -John Plocher
wietse@wzv.UUCP (Wietse Z. Venema) (12/13/88)
In article <317@focsys.UUCP> larry@focsys.UUCP (Larry Williamson) writes: >Streaming tape I/O with 386/ix seems to be rather slow. The drive >is not streaming very well. It spends most of it's time stopping >and starting. I'm using an Everex Excel 60 with a long controller >card. You may consider using the ddd command that was posted earlier this year in one of the source groups by Tapani Lindgren <nispa@cs.hut.fi>. The following is an excerpt from the man page: Ddd works almost the same way as dd(1), but it has a much better throughput, especially when used with slow i/o- devices, such as tape drives. The improvement is achieved mainly by dividing the copying process into two processes, one of which reads while the other one writes and vice versa. Also all code conversion capabilities are omitted. There is no additional overhead copying data between various conversion buffers. Ddd was inspired by the vast difference in speed between BSD4.2 and BSD4.3 dumps - in BSD4.3 dump(8) writes to raw magnetic tape with several processes, thus keeping the tape continuously in motion. I wanted to get the same improvement to remote dumps, so this filter was needed. Adding it to any pipeline of commands usually increases the throughput (that is, if you have enough MIPS). -- work: wswietse@eutrc3.uucp | Eindhoven University of Technology work: wswietse@heitue5.bitnet | Mathematics and Computing Science home: wietse@wzv.uucp | 5600 MB Eindhoven, The Netherlands
news@littlei.UUCP (12/13/88)
#In article <317@focsys.UUCP> larry@focsys.UUCP (Larry Williamson) writes: # >Streaming tape I/O with 386/ix seems to be rather slow. The drive # >is not streaming very well. It spends most of it's time stopping # >and starting. I'm using an Everex Excel 60 with a long controller # >card. # > # >The command I've used is # > find . -print | cpio -ovc >/dev/rmt0 # > There are two problems here: 1) The data transfers are done on small buffers (5K bytes, I believe), and 2) cpio blocks in the write when it should be reading the disk to create more output. In article <251@dcs.UUCP> wnp@dcs.UUCP (Wolf N. Paul) writes: #It seems to me that you should be able to achieve the effect of the #"strm" utility on Uport 286 (and other similar utilities, i.e. "stream" #as supplied with the Bell Tech driver, etc.) by using "dd" with a large #block size. You may need to experiment to find out how large a block #size your system will allow, but try something like this: # # find . -print | cpio -ocv | dd of=/dev/rmt0 bs=64k This decouples the reading done by CPIO from the writing done by DD, and sometimes CPIO will be able to compute output fast enough to keep the tape streaming. This does not work well on my machine, though. Here's the command that I use: find . -print | cpio -oBcv | dd of=/dev/rmt0 ibs=5k obs=500k With this command, DD will always wait until 500K have accumulated before writing the tape (except at EOF, of course). Then the tape is guaranteed to stream at least that 500K. Since my tape is still generally faster than my disk, this command tends to read the disk more or less continuously, writing the tape in long bursts, each of which keeps the tape streaming for a minute or so. The net result begins to feel like: find . -print | cpio -oBcv >/dev/null except that a backup tape actually exists at the end.
karl@sugar.uu.net (Karl Lehenbauer) (12/13/88)
> In article <317@focsys.UUCP> larry@focsys.UUCP (Larry Williamson) writes: > >Streaming tape I/O with 386/ix seems to be rather slow. In article <276@uport.UUCP>, plocher@uport.UUCP (John Plocher) writes: (lots deleted) > -Cbuffersize Use an output buffer of buffersize I have found this to be the key to streamer performance, much moreso than -B. It works on Bell Tech, too. I use: find ... | cpio -ocvC 1024000 >/dev/tape ...to allocate a meg for tape buffers. Even with a lot less it still helps tremendously. What'll happen is the system will fill the buffer 'til it's full, then cpio'll write it out in one long stream. Note that you probably want to do this with little or no other activity as the state of files that are being written to at the time they're backed up makes it possible that the backups won't be any good for those files. -- -- "We've been following your progress with considerable interest, not to say -- contempt." -- Zaphod Beeblebrox IV -- uunet!sugar!karl, Unix BBS (713) 438-5018
david@varian.UUCP (David Brown) (12/13/88)
In article <317@focsys.UUCP>, larry@focsys.UUCP (Larry Williamson) writes: > Streaming tape I/O with 386/ix seems to be rather slow. The drive > is not streaming very well. It spends most of it's time stopping > and starting. I'm using an Everex Excel 60 with a long controller > card. > > The command I've used is > find . -print | cpio -ovc >/dev/rmt0 > I use an AT&T 6386, but I've heard that the UNIX sold by AT&T for the 6386 passed through Interactive. I've gotten much enhanced performance by using the -B flag to cpio: find . -print | cpio -ovcB >/dev/rmt/c0s0 The difference (on an external Mountain 60mb (DC600A) drive) was something like 45min vs. 4 hours for a full tape. It still sounds like it's stopping and starting a lot, though. -- David Brown 415-649-4000 Orion Network Systems 1995 University Ave. Suite 350 Berkeley CA 94704 {pacbell,lll-crg,zehntel,ista,rtech,csi,kinetics}!varian!david
clewis@ecicrl.UUCP (12/15/88)
In article <317@focsys.UUCP> larry@focsys.UUCP (Larry Williamson) writes: >Streaming tape I/O with 386/ix seems to be rather slow. The drive >is not streaming very well. It spends most of it's time stopping >and starting. I'm using an Everex Excel 60 with a long controller >card. > >The command I've used is > find . -print | cpio -ovc >/dev/rmt0 Try this instead find . -print | cpio -ovcC1048576 >/dev/rmt0 Mentioned in the HCR release notes. Can be retrofitted into the sysadm backup and restore scripts if you want (the HCR note contains details of which files and where to fix). Another alternative which works in most 386 UNIXs (not Xenix) is piping the output of your program thru dd with a bs of 1mb. Eg: tar cvf - <files> | dd bs=1024k > /dev/rmt0 (Xenix dd can't handle that much bs ;-) Similar tricks are also useful for speeding up cpio's, tar's or similar programs to floppies. Bell Tech UNIX (actually a somewhat older version of 386/ix) had utilities such as "star" and "stream". All they were were pipes of some utility thru a program with a large (default 3Mb!) buffer. -- Chris Lewis, Markham, Ontario, Canada {uunet!attcan,utgpu,yunexus,utzoo}!lsuc!ecicrl!clewis Ferret Mailing list: ...!lsuc!gate!eci386!ferret-request (or lsuc!gate!eci386!clewis or lsuc!clewis)