jms@antares.UUCP (joe smith) (10/03/88)
In article <4912@cbmvax.UUCP> steveb@cbmvax.UUCP (Steve Beats) writes: >Now I have a question. How can we transparently handle disk media that >has a capacity >2.5 Gigabytes. A lot of software is going to break just >because they've gone over 31 bits of offset (a longword becomes negative >then). An even larger problem exists when you go over 4.2 Gigabytes since >that will overflow a longword. ... )It isn't possible to just limit partitions to less than )2.5 Gigs since the filing system has to make it's requests in terms of )physical blocks, not blocks relative to the start of the partition. Anyone )got any ideas they'd care to > Steve OK, I'll bite. Why can't the filing system use logical blocks? Regardless of the current data structures that the harddisk.device uses, why can't it add a cylinder (or block) offset after converting the request byte offset into block # and offset with block? -- +----------------------------------------------------------------------------+ | TYMNET:JMS@F29 CA:"POPJ P," UUCP:{ames|pyramid}oliveb!tymix!antares!jms | | INTERNET: closed for renovations PHONE:Joe Smith @ (408)922-6220 | +----------------------------------------------------------------------------+
steveb@cbmvax.UUCP (Steve Beats) (10/04/88)
In article <176@antares.UUCP> jms@antares.UUCP (joe smith) writes: > >OK, I'll bite. Why can't the filing system use logical blocks? Regardless >of the current data structures that the harddisk.device uses, why can't it >add a cylinder (or block) offset after converting the request byte offset >into block # and offset with block? > Internally, the filing system DOES use logical blocks. Right now they're hard coded at 512 bytes which allows the FS to access about 4 Terabytes of media. Under the 1.4 filing system, true variable block size will be implemented (I wonder how I knew that :-) so this limit can be raised appreciably by going to 1K or 2K block sizes. The problem arises when the filing system has to talk to an exec device. The IO_OFFSET field of an IORequest is 32 bits of BYTE offset, not BLOCK offset. This limits the range to 2^32 or about 4.5 Gigabytes. The obvious answer is to write a device driver that takes a block number in the IO_OFFSET field but this would be incompatible. I don't see any backwards compatible way for the FS to know whether it has to request block offsets or byte offsets. Well, actually I do see a way. A new device command class could be added that expects offsets in blocks as opposed to bytes. The filing system could check if this command was accepted at initialisation time and adjust accordingly. Question is: what numbers do the new commands take and is this an acceptable solution ? Steve
page@swan.ulowell.edu (Bob Page) (10/04/88)
steveb@cbmvax.UUCP (Steve Beats) wrote: >Under the 1.4 filing system, true variable block size will be implemented Hooray! Next we'll have cylinder groups, file links and an AREXX port :-) >The IO_OFFSET field of an IORequest is 32 bits of BYTE offset, not >BLOCK offset. This limits the range to 2^32 or about 4.5 Gigabytes. >The obvious answer is to write a device driver that takes a block number >in the IO_OFFSET field but this would be incompatible. Here's the current IOStdReq block: struct IOStdReq { struct Message io_Message; struct Device *io_Device; /* device node pointer */ struct Unit *io_Unit; /* unit (driver private)*/ UWORD io_Command; /* device command */ UBYTE io_Flags; BYTE io_Error; /* error or warning num */ ULONG io_Actual; /* actual number of bytes transferred */ ULONG io_Length; /* requested number bytes transferred*/ APTR io_Data; /* points to data area */ ULONG io_Offset; /* offset for block structured devices */ }; Why not have the application PASS IN a magic cookie to io_Error, or io_Flags (or somewhere obtuse like io_Message.mn_ReplyPort->mp_Flags). :-) The device then knows "OK, the value in io_Offset is in BLOCKS, not BYTES." If the magic cookie isn't there, we just treat it as a byte offset, like always. >I don't see any backwards compatible way for the FS to know whether >it has to request block offsets or byte offsets. It doesn't have to know. Dump theory and go with what's practical. Have the file system use the old method to access any data under 4.5GB. If you have to go over 4.5GB, only the devices that know about BLOCK offsets will be able to do it, and since you haven't defined how to do that, nobody has devices that can, and nobody will until you define how to do it. By the time 1.4 is in developer's hands, they'll all have known about the BLOCK offset for a long time. Nobody's going to get burned. >A new device command class could be added that expects offsets in blocks as >opposed to bytes. The filing system could check if this command was >accepted at initialisation time and adjust accordingly. Question is: >what numbers do the new commands take and is this an acceptable solution ? I think this is too much work. You're trying to be compatible with something that doesn't exist, and may actually be less compatible with existing stuff, since you might be sending currently private commands to the driver. ..Bob -- Bob Page, U of Lowell CS Dept. page@swan.ulowell.edu ulowell!page
stevex@ziebmef.uucp (Steve Tibbett) (10/06/88)
Well, if we've got 2.5 gigs of storage to play around with, or 4.x gigs, why not up the block size? A 4096 byte block size would let you have 8 times the amount of storage the current 512 byte block size would before overflowing... ...Steve
thad@cup.portal.com (10/10/88)
Re: Steve Tibbett's question about larger block/sector sizes ... it's always a trade-off. A 1-byte or a (block-size)-byte file would both consume the same-size data block; same for a one-byte overflow. Block/sector sizes around 512 to 1024 bytes have generally proven to be reasonable.
page@swan.ulowell.edu (Bob Page) (10/12/88)
stevex@ziebmef.UUCP (Steve Tibbett) wrote: >A 4096 byte block size would let you have 8 times the amount of storage >the current 512 byte block size would before overflowing... That's OK for DOS, bot not for Exec. Exec wants IOStdReq's in terms of byte offsets, and there's only 32 bits in the offset. So your proposal wouldn't work without some modification, to do something like tell Exec that the offset is not in bytes, or put the high-order 32 bits in another place. If you just decreed that the offset was in terms of 'blocks' (which is driver/unit/partition dependent), you'd break all existing I/O requests. ..Bob -- Bob Page, U of Lowell CS Dept. page@swan.ulowell.edu ulowell!page "I can't tell the difference between ABC News and Hill Street Blues" -Bono
peter@sugar.uu.net (Peter da Silva) (10/13/88)
In article <9579@swan.ulowell.edu>, page@swan.ulowell.edu (Bob Page) writes: > stevex@ziebmef.UUCP (Steve Tibbett) wrote: > >A 4096 byte block size would let you have 8 times the amount of storage > >the current 512 byte block size would before overflowing... > That's OK for DOS, bot not for Exec. Exec wants IOStdReq's in terms > of byte offsets, and there's only 32 bits in the offset. Is this a 32-bit limit on *device* size or just on *file* size? I think that you could live with a 32-bit limit on files with a 48-bit limit on devices/partitions... But, yeh. It's totally appropriate to start working on ways around this. Best would probably be a second set of calls with bigger offsets. -- Peter da Silva `-_-' peter@sugar.uu.net Have you hugged U your wolf today?
dillon@CORY.BERKELEY.EDU (Matt Dillon) (10/13/88)
:>A 4096 byte block size would let you have 8 times the amount of storage
:>the current 512 byte block size would before overflowing...
:
Bob Page, U of Lowell CS Dept. page@swan.ulowell.edu ulowell!page Writes:
:bits in another place. If you just decreed that the offset was in
:terms of 'blocks' (which is driver/unit/partition dependent), you'd
:break all existing I/O requests.
:
:...Bob
There is another problem. If we add a new command or flag
bit to the trackdisk device saying 'offset is in blocks' instead of bytes,
if anybody ever *does* put in a disk that big you couldn't use the old
method anyway and all existing programs that go directly to the trackdisk
will break.
The best thing to do is to have CMD_READ/WRITE fail with the
new drivers and add new CMD_? calls to the trackdisk for read/write ops
that takes blocks instead of bytes. Or BETTER, if the device is too
large have CMD_READ/WRITE fail, else allow them to work. The main point
is to NOT allow application programs access via the old calls if the disk is
too big.... better to have them fail, no?
I like that second idea ... after all, what will fail on the super
large disks? The system software won't because it will have been changed
to use the new calls exclusively. Games might but it isn't a problem if you
run them off other storage. About the only thing that WILL fail are
third-party/shareware/freeware packages that access that particular
trackdisk.device ... not many, and not a major problem because only the
super-large storage device is affected.
-Matt