dean@coplex.uucp (Dean Brooks) (05/02/91)
We just recently acquired an Exabyte 2.3 gig tape drive for our Motorola SYS V.3.2 (V/88) 8864 system, and I have noticed something unusual. If I do the following to place a file on to the tape: $ cp /tmp/BIGFILE /dev/exabyte And then immediately do the following to extract it again: $ cp /dev/exabyte /tmp/newfile The two files will have different lengths, varying anywhere from 1,000 bytes difference to 38,400 bytes difference. *However*, if I use "cpio" to backup the file and then restore it, it works perfectly. I have tried this test over and over with different files, and it *never* works with the "cp" or "cat" command, but *always* works with cpio. Is there a problem with "cat" or "cp" not doing an fflush or something ignorant like that? By the way, the /dev/exabyte file is a character based device, not a block based device. Any clues? -- dean@coplex.uucp (Dean Brooks) Copper Electronics, Inc. Louisville, Kentucky
scott@.convergent.com (Scott Lurndal) (05/03/91)
In article <1991May2.044713.12583@coplex.uucp>, dean@coplex.uucp (Dean Brooks) writes: |> If I do the following to place a file on to the tape: |> |> $ cp /tmp/BIGFILE /dev/exabyte |> |> And then immediately do the following to extract it again: |> |> $ cp /dev/exabyte /tmp/newfile |> |> The two files will have different lengths, varying anywhere from |> 1,000 bytes difference to 38,400 bytes difference. *However*, if I |> use "cpio" to backup the file and then restore it, it works perfectly. |> |> I have tried this test over and over with different files, and it *never* |> works with the "cp" or "cat" command, but *always* works with cpio. |> |> Any clues? It doesn't work because there is a fixed block size on the tape drive and no inode to keep the eof information. When you restore from the tape, you are going to get all the data up to the next tape block boundary. cpio and tar know this and can deal with it, cp and cat don't and can't.
allbery@NCoast.ORG (Brandon S. Allbery KB8JRR/AA) (05/03/91)
As quoted from <1991May2.044713.12583@coplex.uucp> by dean@coplex.uucp (Dean Brooks): +--------------- | We just recently acquired an Exabyte 2.3 gig tape drive for our | Motorola SYS V.3.2 (V/88) 8864 system, and I have noticed something | unusual. | | $ cp /tmp/BIGFILE /dev/exabyte | $ cp /dev/exabyte /tmp/newfile | | The two files will have different lengths, varying anywhere from | 1,000 bytes difference to 38,400 bytes difference. *However*, if I | use "cpio" to backup the file and then restore it, it works perfectly. +--------------- Character special mass storage devices, like tapes and disks, generally require the program to do block-mode access itself. The difference in sizes probably depends on how large the first write to the tape is. Instead of using cp or cat, try dd: dd if=/tmp/BIGFILE of=/dev/exabyte bs=20b The restored file will almost certainly be larger, because it will be padded to a whole block size (10K in the example above). (Why do tapes and disks require this in character mode when block special devices do it for you? Efficiency: the block special device has to go through the Unix buffer pool, the character special device can write directly from your buffer or even (if the device is smart enough) memory map your buffer onto the device's own buffer.) ++Brandon -- Me: Brandon S. Allbery Ham: KB8JRR/AA 10m,6m,2m,220,440,1.2 Internet: allbery@NCoast.ORG (restricted HF at present) Delphi: ALLBERY AMPR: kb8jrr.AmPR.ORG [44.70.4.88] uunet!usenet.ins.cwru.edu!ncoast!allbery KB8JRR @ WA8BXN.OH
martin@adpplz.UUCP (Martin Golding) (05/03/91)
In <1991May2.044713.12583@coplex.uucp> dean@coplex.uucp (Dean Brooks) writes: > If I do the following to place a file on to the [exabyte] tape: >$ cp /tmp/BIGFILE /dev/exabyte > And then immediately do the following to extract it again: >$ cp /dev/exabyte /tmp/newfile > The two files will have different lengths, varying anywhere from >1,000 bytes difference to 38,400 bytes difference. The exabyte is (internally) fixed very large block. Filemarks are only found between blocks. When you cp the file onto the exabyte, somebody somewhere is giving you padding for the last block. When you read it back, you get the padding ABSOLUTELY FREE!. You would find the same effect on a cartridge tape, but not quite as much extra data (512). cpio, because it is designed to pack multiple data sets into a single physical file, has byte counts for the files, so when you cpio one or more files out and back, cpio cheerfully ignores the padding. Use cpio (which is also ABSOLUTELY FREE) or ar or tar or maybe invent your own exciting storage/retrieval mechanism. Martin Golding | sync, sync, sync, sank ... sunk: Dod #0236 | He who steals my code steals trash. A poor old decrepit Pick programmer. Sympathize at: {mcspdx,pdxgate}!adpplz!martin or martin@adpplz.uucp
glenn@bryant.NCD.COM (Glenn Shapland) (05/04/91)
In article <1991May2.044713.12583@coplex.uucp>, dean@coplex.uucp (Dean Brooks) writes: |> |> We just recently acquired an Exabyte 2.3 gig tape drive for our |> Motorola SYS V.3.2 (V/88) 8864 system, and I have noticed something |> unusual. |> |> If I do the following to place a file on to the tape: |> |> $ cp /tmp/BIGFILE /dev/exabyte |> |> And then immediately do the following to extract it again: |> |> $ cp /dev/exabyte /tmp/newfile |> |> The two files will have different lengths, varying anywhere from |> 1,000 bytes difference to 38,400 bytes difference. *However*, if I |> use "cpio" to backup the file and then restore it, it works perfectly. |> |> I have tried this test over and over with different files, and it *never* |> works with the "cp" or "cat" command, but *always* works with cpio. |> |> Is there a problem with "cat" or "cp" not doing an fflush or something |> ignorant like that? By the way, the /dev/exabyte file is a character |> based device, not a block based device. |> |> Any clues? |> |> -- |> dean@coplex.uucp (Dean Brooks) |> Copper Electronics, Inc. |> Louisville, Kentucky cp and cat are typically not a preferred way to store data on a tape. The reason is most cartridge tape formats only support fixed blocks (512 for QIC). This means that a file written with a utility that does not keep track of the size of files, will never be read back with the same size as the original file (unless filesize mod tapeblocksize == 0). Use tar or cpio. You can test for this problem by taking a file that you read back from the tape drive, write it to tape drive, read it back and compare these two files. These two files should be the same since they should be an even number of tape blocks long. glenn@ncd.com
jbrennan@urbana.mcd.mot.com (Jim Brennan) (05/06/91)
In article <1991May2.044713.12583@coplex.uucp> dean@coplex.uucp (Dean Brooks) writes: -> -> We just recently acquired an Exabyte 2.3 gig tape drive for our -> Motorola SYS V.3.2 (V/88) 8864 system, and I have noticed something -> unusual. -> -> The two files will have different lengths, varying anywhere from -> 1,000 bytes difference to 38,400 bytes difference. *However*, if I -> use "cpio" to backup the file and then restore it, it works perfectly. -> -> I have tried this test over and over with different files, and it *never* -> works with the "cp" or "cat" command, but *always* works with cpio. -> -> Is there a problem with "cat" or "cp" not doing an fflush or something -> ignorant like that? By the way, the /dev/exabyte file is a character -> based device, not a block based device. -> -> Any clues? You must access character special tape devices on R32V2 in multiples of the logical block size for the device. Most tape devices have a logical block size of 512 bytes (streaming tapes). The exabyte uses a logical block size of 1024 bytes (this is the most efficient record size for the exabyte). This is why cp and cat do not work. You can use dd to do what you want, and specify a block size of 1024 bytes: dd if=/tmp/myfile of=/dev/exabyte bs=1024 Note: see the MVME328 manual page for more information. -> -- -> dean@coplex.uucp (Dean Brooks) -> Copper Electronics, Inc. -> Louisville, Kentucky -------- jbrennan@urbana.mcd.mot.com || uiucuxc!udc!jbrennan Motorola Computer Group, Core Technologies Urbana Design Center, Urbana, IL
rlr@alumni.colorado.edu (Roger Rose) (05/09/91)
> martin@adpplz.UUCP (Martin Golding) writes: > > The exabyte is (internally) fixed very large block. Filemarks are only > found between blocks. When you cp the file onto the exabyte, somebody > somewhere is giving you padding for the last block. When you read it > back, you get the padding ABSOLUTELY FREE!. You would find the same > effect on a cartridge tape, but not quite as much extra data (512). Almost right. The Exabyte's do use a fixed blocksize on tape; however, they automatically map other blocksizes into these. As far as things look to the computer, the drive can function as either a fixed or a variable-block device. (Also note that on the 2.3Gb model there is are significant capacity and speed penalties for blocks that aren't an even multiple of 1024 bytes.) The underlying cause of the garbage data is most likely in the device driver's handling of the last block of data. As far as the Exabyte drive is concerned, it would be perfectly acceptable to write a short block at that (or any other) point. Handling a short block when reading the file is trivial for variable-block mode; however, a bit difficult to do *portably* for fixed-block mode. (The SCSI spec leaves a bit much unstated on fixed-block transfers, so it would probably be necessary to special-case the device driver for various vendors' tape devices.) -- Roger Rose {rlr@boulder.colorado.edu}