kai@uicsrd.csrd.uiuc.edu (08/29/88)
If you are writing data from within a C program, try:
fp = popen ("/usr/ucb/rsh vaxhost /bin/dd of=/dev/rmt8", "w");
Or, if you just want to write tar files, get the public domain tar program
from the comp.sources.unix archives. This program has the capability to
write to remote tape drives.
Patrick Wolfe (pwolfe@kai.com, kailand!pwolfe)mouse@mcgill-vision.UUCP (der Mouse) (09/02/88)
In article <43200031@uicsrd.csrd.uiuc.edu>, kai@uicsrd.csrd.uiuc.edu writes: > If you are writing data from within a C program, try: > fp = popen ("/usr/ucb/rsh vaxhost /bin/dd of=/dev/rmt8", "w"); You probably don't want to do this. It's likely to produce unreadable tapes. Why? Because a tape is not just a bytestream; it's got a record structure to it as well. When writing a local tape, each write() call produces one record. When writing through the network like this, this is not guaranteed, and in practice will probably not be so. Experiment first. For example, put a tar tape on the drive and do tapehost% dd if=/dev/rmt12 of=/tmp/foo tapehost% rcp /tmp/foo otherhost:/tmp/foo then change tapes (put a scratch tape on) and do otherhost% rsh tapehost dd of=/dev/rmt12 < /tmp/foo Then run tar on the resulting tape and notice the complaints. der Mouse old: mcgill-vision!mouse new: mouse@larry.mcrcim.mcgill.edu
kai@uicsrd.csrd.uiuc.edu (09/06/88)
> When writing a local tape, each write() call produces one record. Not true when a blocking factor > 1 is used. Each write just adds a little to a buffer, and when the buffer is full, THEN one physical record is actually produced. > When writing through the network like this, this is not guaranteed, > and in practice will probably not be so. The best way I've found to write through the network is to use fixed size blocks. Make sure the program that is reading from the network knows the correct blocksize and doesn't assume it will all show up after one single read. Multiple reads may (or may not) be neccessary Fortunately "dd" can do this: dd bs=20b of=/dev/rmt8 The "tar" program has an option (-B) to force input/output blocking to 20 "so that tar can work across a communications channel where the blocking may not be maintained." > Experiment first. For example, put a tar tape on the drive and do > tapehost% dd if=/dev/rmt12 of=/tmp/foo > tapehost% rcp /tmp/foo otherhost:/tmp/foo >then change tapes (put a scratch tape on) and do > otherhost% rsh tapehost dd of=/dev/rmt12 < /tmp/foo >Then run tar on the resulting tape and notice the complaints. dd's default input and output blocksizes are 512 bytes. Tar's default blocksize for writing is 10240 bytes. Try: tapehost% dd bs=20b if=/dev/rmt12 of=/tmp/foo tapehost% rcp /tmp/foo otherhost:/tmp/foo otherhost% rsh tapehost dd bs=20b of=/dev/rmt12 < /tmp/foo and dd will make sure it reads 10240 bytes (tar's default blocksize) each time. Patrick Wolfe (pwolfe@kai.com, kailand!pwolfe)
chris@mimsy.UUCP (Chris Torek) (09/07/88)
>>When writing a local tape, each write() call produces one record. This is correct. In article <43200035@uicsrd.csrd.uiuc.edu> kai@uicsrd.csrd.uiuc.edu writes: >Not true when a blocking factor > 1 is used. This is nonsense; the write() syscall does not have a blocking factor. (dd and tar have blocking factors, but rsh does not maintain them.) >The best way I've found to write through the network is to use fixed size >blocks. Make sure the program that is reading from the network knows the >correct blocksize and doesn't assume it will all show up after one single >read. Multiple reads may (or may not) be neccessary > >Fortunately "dd" can do this: > dd bs=20b of=/dev/rmt8 `dd' can indeed do the trick, but not with the options you gave. With just a `bs' option and no conversions, dd defaults to copying the input record size to its output, so that if it reads 32, then 1024, then 12 bytes, it writes 32, then 1024, then 12 bytes. >The "tar" program has an option (-B) to force input/output blocking to 20 "so >that tar can work across a communications channel where the blocking may not >be maintained." This only matters for input. To make tapes that those without the B flag can read, use % tar cf - trees | rsh tapehost "dd obs=20b of=/dev/tape_device" To read a tape, the command % rsh tapehost "dd if=/dev/tape_device bs=32k" | tar xf - suffices on 4.3BSD and recent SunOSes, since `f -' implies `B'; under 4.2BSD, you must write `... | tar xBf -', and without the B flag you would have had to use % rsh tapehost "dd if=/dev/tape_device bs=32k" | dd obs=XXb | tar xf - where `XX' is any value in 1..40. `dd if=/dev/tape_device bs=32k' could be replaced with any other program that copies its input to its output without change, as long as that program reads large enough blocks (as determined by the actual tape block size). The local re-blocking conversions (`dd obs=20b' on either host) can be made somewhat more efficient by using a larger ibs as well: % tar cf - trees | rsh tapehost "dd ibs=20b obs=20b of=/dev/tape_device" (20b = 10k = more than 4.2 and 4.3 BSD send across the net at any one time by default). Note that, as I mentioned above, dd bs=N and dd ibs=N obs=N have different meanings. -- In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 454 7163) Domain: chris@mimsy.umd.edu Path: uunet!mimsy!chris
roy@phri.UUCP (Roy Smith) (09/07/88)
kai@uicsrd.csrd.uiuc.edu writes: > > When writing a local tape, each write() call produces one record. > > Not true when a blocking factor > 1 is used. Each write just adds a little > to a buffer, and when the buffer is full, THEN one physical record is > actually produced. Say what? On any Unix system I've ever seen, each write(2) system call produces exactly one physical tape record. If you wanted to, you could do single character writes and get 1 character records (assuming this didn't put your tape drive/controller into spasms). The kernel has no concept of blocking factor. Programs that use blocking factors (dump, tar, dd, etc) do the buffering you describe in user code and present full tape records to the kernel in a single write(2). From the 4.3 mtio(4) man page (other systems will differ in wording but should say essentially the same thing): "A standard tape consists of a series of 1024 byte records terminated by an end-of-file." I'm not sure what this is doing in a section 4 man page. Certainly the kernel has no concept of what a standard tape looks like, nor does it prefer to write 1024 byte records over any other size. "Each read or write call reads or writes the next record on the tape. In the write case the record has the same length as the buffer given." The heart of the matter. You get records on the tape exactly as long as the byte count you pass to write(2). -- Roy Smith, System Administrator Public Health Research Institute {allegra,philabs,cmcl2,rutgers}!phri!roy -or- phri!roy@uunet.uu.net "The connector is the network"
roy@phri.UUCP (Roy Smith) (09/07/88)
In article <13412@mimsy.UUCP> chris@mimsy.UUCP (Chris Torek) writes: > Note that, as I mentioned above, > dd bs=N > and > dd ibs=N obs=N > have different meanings. This may be true, but you wouldn't guess it from the 4.3BSD dd man page: ibs=n input block size n bytes (default 512) obs=n output block size (default 512) bs=n set both input and output block size, superseding ibs and obs; also, if no conver- sion is specified, it is particularly effi- cient since no copy need be done If I didn't know better, and I was reading the man page for the first time, I would sure assume that "bs=N" and "ibs=N obs=N" would have exactly the same effect, modulo efficiency considerations. -- Roy Smith, System Administrator Public Health Research Institute {allegra,philabs,cmcl2,rutgers}!phri!roy -or- phri!roy@uunet.uu.net "The connector is the network"
kai@uicsrd.csrd.uiuc.edu (09/08/88)
>/* Written 7:58 pm Sep 6, 1988 by chris@mimsy.UUCP in uicsrd.csrd.uiuc.edu:comp.unix.wizards */ >`dd' can indeed do the trick, but not with the options you gave. With >just a `bs' option and no conversions, dd defaults to copying the input >record size to its output, so that if it reads 32, then 1024, then 12 >bytes, it writes 32, then 1024, then 12 bytes. > dd bs=N >and > dd ibs=N obs=N >have different meanings. With NO options, dd just copies input to output. If you specify "bs=20b", or "ibs=20b obs=20b", dd reads and writes 10240 byte blocks from input to output. Sorry, but these two parameter settings are the same, except that using "bs" to specify both input and output buffer sizes is more efficient, since copying between buffers need not be done, unless you are doing ASCII to EBCDIC conversions or some other type of data conversion. Patrick Wolfe (pwolfe@kai.com, kailand!pwolfe)
m5@lynx.UUCP (Mike McNally) (09/09/88)
In article <3463@phri.UUCP> roy@phri.UUCP (Roy Smith) writes: >kai@uicsrd.csrd.uiuc.edu writes: >> > When writing a local tape, each write() call produces one record. >> >> Not true when a blocking factor > 1 is used. Each write just adds a little >> to a buffer, and when the buffer is full, THEN one physical record is >> actually produced. Clearly there's some confusion between what tar does internally and what the OS does. > > Say what? On any Unix system I've ever seen, each write(2) system >call produces exactly one physical tape record. Most cheap SCSI tape drives have a fixed blocksize, often 512 bytes. No matter how much the OS wants to write 10240-byte blocks, the tape will have 10 separate 512 byte blocks on it. -- Mike McNally of Lynx Real-Time Systems uucp: lynx!m5 (maybe pyramid!voder!lynx!m5 if lynx is unknown)
kai@uicsrd.csrd.uiuc.edu (09/09/88)
>/* Written 7:07 am Sep 7, 1988 by roy@phri.UUCP in uicsrd.csrd.uiuc.edu:comp.unix.wizards */ > The kernel has no concept of blocking factor. You're right, I'm wrong. Every once in a while part of my brain reverts to my old IBM MVS days, where device selection and file blocking is controlled external to the actual program in the JCL. Patrick Wolfe (pwolfe@kai.com, kailand!pwolfe)
jeremy@chook.ua.oz (Jeremy Webber) (09/09/88)
In article <3463@phri.UUCP> roy@phri.UUCP (Roy Smith) writes: > "A standard tape consists of a series of 1024 byte records > terminated by an end-of-file." > > I'm not sure what this is doing in a section 4 man page. Certainly > the kernel has no concept of what a standard tape looks like, nor does it > prefer to write 1024 byte records over any other size. This probably refers to the "cooked" tape devices, in which the tape driver handles the blocking independently of read(2) and write(2). Since most people prefer to specify their own tape blocking I have never seen anything which uses cooked tape devices. They are of no use when using utilities such as tar and dump. -jeremy webber (jeremy@chook.ua.oz.au)
idall@augean.OZ (Ian Dall) (09/09/88)
In article <3463@phri.UUCP> roy@phri.UUCP (Roy Smith) writes: > The heart of the matter. You get records on the tape exactly as >long as the byte count you pass to write(2). My QIC-24 streaming tape drive (and I suspect most others) always writes 512 byte blocks. Writes get cached in the drive itself. If you don't write fast enough the drive will stop streaming and backtrack but the record structure will be unaffected. -- Ian Dall "In any argument there will be people on your side who you wish were on the other side." idall@augean.oz
jfh@rpp386.Dallas.TX.US (The Beach Bum) (09/14/88)
In article <JEREMY.88Sep9164651@chook.ua.oz> jeremy@chook.ua.oz (Jeremy Webber) writes: >In article <3463@phri.UUCP> roy@phri.UUCP (Roy Smith) writes: >> "A standard tape consists of a series of 1024 byte records >> terminated by an end-of-file." > Since most people >prefer to specify their own tape blocking I have never seen anything which uses >cooked tape devices. They are of no use when using utilities such as tar and >dump. but they are useful as file systems. you can create a file system on a tape, mount it read-only, and reference files on it just like any other disk-based file system. you might grow old waiting for your files, but it should work. -- John F. Haugh II (jfh@rpp386.Dallas.TX.US) HASA, "S" Division "If the code and the comments disagree, then both are probably wrong." -- Norm Schryer
guy@gorodish.Sun.COM (Guy Harris) (09/15/88)
> but they are useful as file systems.
They can be used as file systems. This does not necessarily mean they are
*useful* as file systems; are there cases where you would really want to use
one as a file system, and where using one as a file system is better than the
alternatives?
For example, you might want to do so to run a "stand-alone" UNIX to run disk
formatters, or to install your system; however, in 4.0 we have a "memory-only"
UNIX for the former, and have always used the Berkeley "mini-root on the swap
area" hack for the latter, and I've yet to hear of a reason why running UNIX
off a tape would be better. Note that, while on disks a physical "read" of
4096 bytes will generally just read e.g. 8 512-byte sectors, on tapes a
physical "read" of 4096 bytes will tend to want to read a 4096-byte block; this
may make the Berkeley file system unhappy when run off a tape.