brad@kontron.UUCP (Brad Yearwood) (08/11/87)
Write-once-read-mostly optical disks, while promising very large storage
capacities economically and in a small space, do not fit well with
the types of file allocation and directory structures used on rewritable
magnetic disks. Does anybody know of published papers or standards efforts
covering filesystem/archive formats for optical disks?
Brad Yearwood
Kontron Electronics {voder, pyramid}!kontron!brad
Mountain View, CA
(415) 965-7020 (800) 227-8834brian@casemo.UUCP (Brian Cuthie ) (08/11/87)
In article <1775@kontron.UUCP>, brad@kontron.UUCP (Brad Yearwood) writes: > Write-once-read-mostly optical disks, while promising very large storage > capacities economically and in a small space, do not fit well with > the types of file allocation and directory structures used on rewritable > magnetic disks. Does anybody know of published papers or standards efforts > covering filesystem/archive formats for optical disks? > > Brad Yearwood > Kontron Electronics {voder, pyramid}!kontron!brad > Mountain View, CA > (415) 965-7020 (800) 227-8834 There are two major problems when using optical WORM (Write Once Read Many) drives. First, the block size is usually 1024 bytes instead of the usual 512. Second, when a block is written, it's ECC is also written. Since the media is Write Once, it is impossible to simulate a 512 byte/block file system using the usual read block, update the half you want, write block. Note, that this is impossible even if the half in question has never been written; it's ECC has. The only reasonably transparent sollution I have seen is to have the controller map the directory part of the filesystem to a magnetic drive and the data portion to the Optical drive. Thus part of the optical drive has been mapped (or overlayed) with magnetic media. Since this is done by the controller all the operating system must do is write data in even numbers of blocks and limit directory blocks to the portion of the WORM drive that has been mapped to magnetic media. When the data area (the optical drive) is full, the magnetic disk is copied to the area on the optical it had previously overlayed. This freezes the filesystem onto the optical platter. Note that the magnetic drive only needs to be as large as the directory area of the WORM filesystem. This scheme works well for operating systems like VMS where directory structures may be forced to live within a specific region of the disk. VMS also alows the user to specify the cluster size (the number of blocks transfered) to be even. Once a file system has been created and frozen, it may be mounted under VMS as READONLY without the use of a companion magnetic drive. Thus platters may be created and then sent to sites containing no ability to write optical media. Several companies have implimented such schemes, some more succesful than others. US Design (Lanham Maryland, 301 577 - 2880) has a good implimentation. One final note. WORM drives tend to be EXTREMELY slow. They can have seek times on the order of seconds ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Brian Cuthie CASE Communications Corp. Columbia, Md 21046 (301) 290 - 7443 ARPA: brian@umbc3.umd.edu UUCP: ...!seismo!mimsy!aplcen!casemo!brian
zemon@felix.UUCP (Art Zemon) (08/11/87)
We have a spec for the format of the optical disks used in the OSAR library which we build. I know that under some circumstances it is available to other companies but don't know the conditions of such divulgence. If you have serious interest, contact our Marketing department. (Sorry for that advice but we do have some people over there who are better than the average market-droid.) The phone number is (714)966-2344 or write to FileNet Corp., 3530 Hyland Avenue, Costa Mesa, CA 92626, USA. If you just have casual, personal interest, I doubt very much that they will release the spec to you. Please don't write to me. I don't know the format and cannot tell you any more than I have written above. -- -- Art Zemon FileNet Corporation Costa Mesa, California ...!hplabs!felix!zemon
dad@cipric.UUCP (08/13/87)
I guess I was under the impression (and still am) that WORM drives would be good for doing things like: dd if=/dev/rrf0g of=/dev/rworm99 bs=32k And then mounting /dev/worm99 read-only... Of course this presumes that rf0g is a usable good file-system that you would like to preserve on optical media.
rbl@nitrex.UUCP (08/13/87)
In article <212@casemo.UUCP> brian@casemo.UUCP (Brian Cuthie ) writes: >In article <1775@kontron.UUCP>, brad@kontron.UUCP (Brad Yearwood) writes: >> Write-once-read-mostly optical disks, while promising very large storage >> capacities economically and in a small space, do not fit well with >> the types of file allocation and directory structures used on rewritable >> magnetic disks. Does anybody know of published papers or standards efforts >> covering filesystem/archive formats for optical disks? >> > > >There are two major problems when using optical WORM (Write Once Read Many) >drives. First, the block size is usually 1024 bytes instead of the usual 512. >Second, when a block is written, it's ECC is also written. Since the media >is Write Once, it is impossible to simulate a 512 byte/block file system >using the usual read block, update the half you want, write block. Note, >that this is impossible even if the half in question has never been written; >it's ECC has. > >The only reasonably transparent sollution I have seen is to have the controller >map the directory part of the filesystem to a magnetic drive and the data >portion to the Optical drive. Thus part of the optical drive has been >mapped (or overlayed) with magnetic media. Since this is done by the controller > ... [deleted for brevity] I have notes of a conversation with (either) Don Wilde or Mike Konshak of Optotech, Inc. at NCC in July 1985. One of the two developed an efficient scheme for bitmap updating on optical media so that files could be updated efficiently. Optotech is in Colorado Springs, 303-570-7500. Rob Lake
raj@ho95e.UUCP (08/14/87)
In article <4988@felix.UUCP>, zemon@felix.UUCP writes: > > know the conditions of such divulgence. If you have serious > interest, contact our Marketing department. (Sorry for that > advice but we do have some people over there who are better > than the average market-droid.) The phone number is ........ > -- Art Zemon > FileNet Corporation I have dealt with the marketing people in FileNet and can testify that they are one of the best. Rajeev AT&T Bell Labs
jvs@micas.UUCP (Jo stockley) (08/17/87)
in article <212@casemo.UUCP>, brian@casemo.UUCP (Brian Cuthie ) says: > Summary: WORM disk structures > Xref: micas comp.periphs:175 comp.unix.wizards:1846 > Xpath: icdoc ivax > > There are two major problems when using optical WORM (Write Once Read Many) > drives. First, the block size is usually 1024 bytes instead of the usual 512. > Second, when a block is written, it's ECC is also written. Since the media > is Write Once, it is impossible to simulate a 512 byte/block file system > using the usual read block, update the half you want, write block. Note, > that this is impossible even if the half in question has never been written; > it's ECC has. > > The only reasonably transparent sollution I have seen is to have the > controller > map the directory part of the filesystem to a magnetic drive and the data > portion to the Optical drive. Thus part of the optical drive has been > mapped (or overlayed) with magnetic media. Since this is done by > the controller > all the operating system must do is write data in even numbers of blocks and > limit directory blocks to the portion of the WORM drive that has been mapped > to magnetic media. When the data area (the optical drive) is full, > the magnetic > disk is copied to the area on the optical it had previously overlayed. This > freezes the filesystem onto the optical platter. Note that the magnetic > drive only needs to be as large as the directory area of the WORM filesystem. We have recently evaluated a WORM subsystem consisting of a OSI WORM drive (I think there's some link with CDC the OSI drive) and a software package called DOROFILE which is produced by a French company called DOROTECH. To quote from the release 2.2 manual: " ... DOROFILE has been written with three major objectives: - To stay in complete accordancewith all UNIX concepts concerning files and directories. - To offer the same user and programmer's interfaces as UNIX does. - To use the optical disc to the most of it's possibilities. ...." They go on to say that " Each disk is totally independant of other storage devices (unlike some WORM file systems where part of the disk indexing is stored in a file on a seperate magnetic disk). " They have also implemented version numbers for files on the optical disk which is of great use to us. The commands are like UNIX only with an 'o' in front, e.g. ols, ocd, ocp etc. The software supports several WORM drives including CDC-OSI, Optimem and GIGADISC. Well it works and we are going to use it in our imaging system on the Sun as we have not seen or heard of anything better. I don't know if it works on anything other than the Sun systems. The only drawback we have seen is that for some reason all the error messages come out in French, followed by a cryptic UNIX message. I trust they will change this at some point. A distributer in the UK is: Program Products Ltd. Tileman House, 131 Upper Richmond Road, London SW15 2TP Telephone: (+44) 01 785 3055 And a contact name is: John Slater. Make of this what you will. Jo. -- ------------- Jo. Stockley. (jvs@micas.uucp or ...!mcvax!ukc!micas!jvs) Nodecrest Computer Systems Ltd Byfleet, UK. Phone: +44 09323 40555
jh@materna.uucp (Juergen Hackenbroich) (08/17/87)
We have developed our own Filesystem for optical disks. It currently runs with LMS-LaserDrives 1200 and correspondant jukebox versions 1250. It keeps the directory during work on the magnetic disk and puts it at the end to the optical disk, so you can ship it or anything else. This process can be done at any time, not only when the cartridge is full. There is a set of calls which are provided to use the OFS1200. Despite the normal calls for working on files (o_open, o_close, read, o_write, etc.) you have calls for mounting and dismounting of cartridges. These calls take the directory from the optical disk and cache it on the magnetic disk and vice versa. For further information please contact me via email or normal mail. Dr. Materna GmbH Juergen Hackenbroich Vosskuhle 38 4600 Dortmund 1 West-Germany Email: ...!seismo!unido!materna!jh
michael@orcisi.UUCP (08/17/87)
> > ... [deleted for brevity] > > I have notes of a conversation with (either) Don Wilde or Mike Konshak of > Optotech, Inc. at NCC in July 1985. One of the two developed an efficient > scheme for bitmap updating on optical media so that files could be updated > efficiently. > > Optotech is in Colorado Springs, 303-570-7500. Optotech uses a special construct called a "post field" in their recording format that can direct a READ command to read an alternate block of data rather than the immediately following the current block (the latter being the usual case for optical media). This scheme helps reduce the "burn rate" at which optical media is consumed as far as file updates go. There are special commands for writing (once) the post field of a block and for reading it. One of the better models for optical media is "magtape with random read access". Michael Herman Optical Recording Corporation 141 John Street, Toronto, Ontario, Canada M5V 2E4 UUCP: { cbosgd!utcs ihnp4!utzoo seismo!mnetor }!syntron!orcisi!michael ALSO: mwherman@watcgl.waterloo.edu
michael@orcisi.UUCP (08/17/87)
> I guess I was under the impression (and still am) that WORM drives would > be good for doing things like: > dd if=/dev/rrf0g of=/dev/rworm99 bs=32k > And then mounting /dev/worm99 read-only... > Of course this presumes that rf0g is a usable good file-system that you > would like to preserve on optical media. %s/read-only/user read-only/ ? Don't read-only Unix file systems still update last access times? Michael Herman Optical Recording Corporation 141 John Street, Toronto, Ontario, Canada M5V 2E4 UUCP: { cbosgd!utcs ihnp4!utzoo seismo!mnetor }!syntron!orcisi!michael ALSO: mwherman@watcgl.waterloo.edu
guy@gorodish.UUCP (08/18/87)
> Don't read-only Unix file systems still update last access times?
No.
Guy Harris
{ihnp4, decvax, seismo, decwrl, ...}!sun!guy
guy@sun.combrian@casemo.UUCP (Brian Cuthie ) (08/18/87)
In article <75@cipric.mn.org>, dad@ciprico.mn.org (Dan A. Dickey) writes: > I guess I was under the impression (and still am) that WORM drives would > be good for doing things like: > dd if=/dev/rrf0g of=/dev/rworm99 bs=32k > And then mounting /dev/worm99 read-only... > Of course this presumes that rf0g is a usable good file-system that you > would like to preserve on optical media. You're right! you can do this. But, do you have 1 Gbyte in magnetic disk space to first create the filesystem you want to dd ?? The problem with WORM drives is that to use the scheme you suggested you must have a magnetic disk the size of the WORM drive. This presents a rather large obstacle for most people. You can of course have the controler or driver break the disk into several smaller logical volumes and then use dd to copy a complete filesyetem. Cheers, -Brian ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Brian Cuthie CASE Communications Columbia, Md. 21046 (301) 290 - 7443 UUCP: ...seismo!mimsy!aplcen!casemo!brian
john@xanth.UUCP (John Owens) (08/18/87)
In article <615@orcisi.UUCP>, michael@orcisi.UUCP (Michael Herman) writes: > > And then mounting /dev/worm99 read-only... > Don't read-only Unix file systems still update last access times? Certainly not! The whole point is to be able to mount disks with the WRITE PROTECT set, etc. People have even made filesystems on magtapes and mounted them read-only, but I doubt if they ever did it for anything besides seeing that it could be done. -- John Owens Old Dominion University - Norfolk, Virginia, USA john@ODU.EDU old arpa: john%odu.edu@RELAY.CS.NET +1 804 440 4529 old uucp: {decuac,harvard,hoptoad,mcnc}!xanth!john
ron@topaz.rutgers.edu.UUCP (08/18/87)
No, There it is not supposed to write on the disk at all (hence the name). There was a bug for a while in V6 (and hence early PWB kernels as well) where if you had a sparse file the kernel bmap routine tried to allocate a block from the free list (and hence update the superblock) when you tried to read one of the empty blocks. A slight fix to the bmap routine to differentiate between reads and writes fixed the problem. As a matter of fact, only slight changes to the UNIX kernel (seperating pipedev from rootdev) were required to allow UNIX to boot on a write-protected root disk. (4.2 actually would run for a few hours with the paging disk write protected as well, everything was fine until memory filled up). -Ron
henry@utzoo.UUCP (Henry Spencer) (08/19/87)
> Don't read-only Unix file systems still update last access times?
Nope, not for a long time now. Even V7 got that one right; anyone who has
reintroduced that bug since should be shot.
--
Support sustained spaceflight: fight | Henry Spencer @ U of Toronto Zoology
the soi-disant "Planetary Society"! | {allegra,ihnp4,decvax,utai}!utzoo!henryforster@plx.UUCP (Jim Forster) (08/20/87)
in article <615@orcisi.UUCP>, michael@orcisi.UUCP (Michael Herman) says: > >> I guess I was under the impression (and still am) that WORM drives would >> be good for doing things like: >> dd if=/dev/rrf0g of=/dev/rworm99 bs=32k >> And then mounting /dev/worm99 read-only... > > Don't read-only Unix file systems still update last access times? and, in article <218@casemo.UUCP>, brian@casemo.UUCP (Brian Cuthie) writes: >You're right! you can do this. But, do you have 1 Gbyte in magnetic disk space >to first create the filesystem you want to dd ?? Plexus has support for 14" Optimem Optical drives, and in fact, we use them just like you suggested. We have a utility so that the optical disk can be carved into named partitions and sized, etc., and then associated with a particular minor device. We then copy a file system to that device, using dd, and then mount that hunk read-only. Works great. We have 1Gb optical drives on-line, each with various numbers of file systems mounted. Its real handy to have enough storage to keep old sources and usenet archives online. I didn't do any of the actual work on this project, but contributed a little to the design, so I thought I'd explain what we did. We also use them in some other ways: piping the output of cpio into the program that creates an O.D. partition is a handy way to backup something. Later you just cpio it out. One other way we use Optical Disks is with a Data Base program that knows about the optical disk, and which does space management knowing that it will never be able to re-write anything. The Optimem drives (and a Hitachi we looked at) don't have the sector-postamble-redirect feature of the Optotech units, which sounds handy, so we developed a scheme that works pretty well. With these drives, if you ask to read 10 sectors, but only 3 have been written, the controller will tell you this, and this turns out to be handy. We don't use a bitmap, but do have directory-like structures that would like to be updated. To do so, we allocate 10 sectors, say, to store one sector, and initially write the information to the first sector. To update this sector, we re-write it into the next position in the block of sectors. To read the information, we attempt to read 10, but only look at the last sector that was written, and thus get the most recent information. This burns a sector each time it's updated, but at least the location of the information on the disk hasn't changed. There's a little issue of what to do when you fill up the 10 sector block, so you move the block, and then have to update the pointer to that block, so you use the same scheme on that block, realizing that you'll only have to update it 1/10 as often as the child block. Jim Forster Plexus Computers
mike@turing.UNM.EDU.unm.edu (Michael I. Bushnell) (08/21/87)
In article <615@orcisi.UUCP> michael@orcisi.UUCP (Michael Herman) writes: >Don't read-only Unix file systems still update last access times? > > Not quite. The whole point of ROFS is so that you can mount a diskpack with the WRITE PROTECT on. That means the hardware prohibits write operations. Even updating last access times. Mount(8) makes reference to this in its man page, explaining that the reason for read only mounts is to prevent access time updating. Michael I. Bushnell a/k/a Bach II mike@turing.UNM.EDU --- I just had a NOSE JOB!! -- Zippy the Pinhead