[comp.sys.atari.st] 16 meg partition limit

swklassen@tiger.waterloo.edu (Steven W. Klassen) (02/03/90)

In article <1325@crash.cts.com> canada@crash.cts.com (Diane Barlow Close) writes:
>I used this program until I formatted my hard drive to have partitions
>bigger than 16 megabytes (allowable under TOS 1.4).  I had no problems with
>FSCK *until* it tried checking partitions over 16 megs, then it gave false 
>readings (ie. bad sectors where there were none).  Others in the local
>atari club (San Diego) have reported the same thing.  Be careful...

Here is a question about the 16 meg partition limit.  I purchased an
Atari Megafile 60 a number of months ago.  From the part number I 
believe it used a Seagate drive.  The manual that came with it
mentioned the 16 meg limit, however, the utilities which came 
with it allowed me to make larger partitions.  I decided to experiment
with this.  I made a partition a fair bit larger than 16 meg (I believe
it was about 24 meg but I don't recall for sure) then tested it by
copying a program onto it multiple times (giving it a unique name each
time) and then reading the files (actually some selected copies of it
from throughout the hard drive) and everything worked fine.  I have
kept my large partition and have not had a problem with it yet (although
I also haven't filled more than about 11 meg on that partition, except
of course during the test I mentioned earlier).

Does anyone have comments on this?  Did some of the later versions
of TOS 1.0 or 1.1 or whatever it is in my 1040, allow for larger
partitions?  Did some of the later versions of the driver program
allow for larger partitions, patching TOS appropriately?  Did I
just get lucky during my test and is my >16 meg partition going
to be unreliable for long term use?  I am being sure to back
everything up on occasion (usually every 4 months unless I add
something significant to the drive in which case I back it up
immediately following the installation) just in case.  (Actually
I would do this anyway.)



Steven W. Klassen                       +-----------------------------+
Computer Science Major                  | Support the poor...buy fur! |
University of Waterloo                  +-----------------------------+

brazil@pawl.rpi.edu (Timothy E. Onders) (02/03/90)

In article <20465@watdragon.waterloo.edu> swklassen@tiger.waterloo.edu (Steven W. Klassen) writes:
>
>mentioned the 16 meg limit, however, the utilities which came 
>with it allowed me to make larger partitions.  I decided to experiment

Those would be HDX 3.0. HDX 3.0 allows partitions up to 1 GBy (that's
1024 MB in case you didn't know.), It also allows more partitions per
physical drive.

>partitions?  Did some of the later versions of the driver program
>allow for larger partitions, patching TOS appropriately?  Did I

See above.

>to be unreliable for long term use?  I am being sure to back
>everything up on occasion (usually every 4 months unless I add
>something significant to the drive in which case I back it up
>immediately following the installation) just in case.  (Actually
>I would do this anyway.)

The only problem, however, is that many disk utility programs, such
as DL II, and TuneUp(i think), don't know how to handle the larger
partitions. Make sure you find out if a utilities package can handle
the big partitions before you fork() out $$$$ for it. You might not
get your money back.

					-Tim Onders
					brazil@pawl.rpi.edu
					70135.563@CompuServe.COM

apratt@atari.UUCP (Allan Pratt) (02/06/90)

brazil@pawl.rpi.edu (Timothy E. Onders) writes:
>In article <20465@watdragon.waterloo.edu> swklassen@tiger.waterloo.edu (Steven W. Klassen) writes:
>>mentioned the 16 meg limit, however, the utilities which came 
>>with it allowed me to make larger partitions.

>Those would be HDX 3.0. HDX 3.0 allows partitions up to 1 GBy (that's
>1024 MB in case you didn't know.), It also allows more partitions per
>physical drive.

Right. Actually it's 3.01; the driver that came with 3.0 had a bug,
though the formatter (HDX) is the same.

>>unreliable for long term use?

Certainly not.

>The only problem, however, is that many disk utility programs ...
>don't know how to handle the larger partitions.

Also right.  Here's the bottom line: when you have a partition larger
than 16MB, the driver creates the partition with 1KB sectors (or more!)
rather than 512-byte sectors, which is the old value.  Cluster size is
still two sectors, whatever size that is.  The disk itself is formatted
with 512-byte sectors, and Rwabs does blocking and deblocking, turning
requests for "one logical sector" into as many physical sectors as
necessary.

The reason for the large-sector implementation is that all GEMDOSes can
handle any (power-of-two) sector size, but not all can handle a cluster
size other than two.

The drawback is that some (most) utilities assume that one sector is
always 512 bytes.  They could have used the Getbpb() call and used the
sector size returned from that, but the original TOS documentation all
but guaranteed 512 bytes forever.  These programs will crash, or worse,
will fail quietly because they read "one sector" into a buffer which is
only 512 bytes long; the Rwabs transfers 1K, clobbering whatever came
after the buffer in memory.  Also, their notion of how big a sector is
(e.g. number of directory entry slots in a directory sector) will be
wrong.

As an additional complication, if you use the "raw mode" (also called
"physical device") bit in the first arg to Rwabs, you get physical
sectors, which are still 512 bytes.  That number is not obtainable as a
variable anywhere, so it'll probably stay at 512.

The hardest-hit class of programs is the "add cache sectors" or  other
disk-caching kind, because it  needs to know how big a buffer to
allocate.  The answer to that question is impossible to compute, so
it's available from the hard-disk driver.  (It's impossible to compute
because it involves a user-preference value which can be patched into
the driver: with removable media you could pop the cartridge and put in
one with larger logical sectors, so the user tells the driver a
minimum for the "maximum sector-size" value to use.)

CACHExxx.PRG from Atari, of course, interrogates the driver and uses
that number correctly.

I'm not publishing the way to interrogate the driver here because I
would not get it right.  The straight scoop is avaiable to developers
in the HDX 3.0 Release Notes document.

The moral is, utilities should always use Getbpb to find out how big
sectors are, how many sectors per cluster, etc., and base all your
other calculations and "constants" on the answers.  If you're adding
GEMDOS buffers or other permanent objects, get the release notes and do
it right.

============================================
Opinions expressed above do not necessarily	-- Allan Pratt, Atari Corp.
reflect those of Atari Corp. or anyone else.	  ...ames!atari!apratt

greg@sj.ate.slb.com (Greg Wageman) (02/07/90)

Opinions expressed are the responsibility of the author.

In article <20465@watdragon.waterloo.edu> swklassen@tiger.waterloo.edu (Steven W. Klassen) writes:
>
>Here is a question about the 16 meg partition limit.  I purchased an
>Atari Megafile 60 a number of months ago.  From the part number I 
>believe it used a Seagate drive.  The manual that came with it
>mentioned the 16 meg limit, however, the utilities which came 
>with it allowed me to make larger partitions.  I decided to experiment
>with this.  I made a partition a fair bit larger than 16 meg (I believe
>it was about 24 meg but I don't recall for sure) then tested it by
>copying a program onto it multiple times (giving it a unique name each
>time) and then reading the files (actually some selected copies of it
>from throughout the hard drive) and everything worked fine.  I have
>kept my large partition and have not had a problem with it yet (although
>I also haven't filled more than about 11 meg on that partition, except
>of course during the test I mentioned earlier).
>
>Does anyone have comments on this?  Did some of the later versions
>of TOS 1.0 or 1.1 or whatever it is in my 1040, allow for larger
>partitions?  Did some of the later versions of the driver program
>allow for larger partitions, patching TOS appropriately?

One reason there was a 16M limit on hard disk partitions is because
the Rwabs() function which does the physical I/O is specified as
taking a signed integer (16 bit) value for the "recno" parameter.
This means that the sector number can only range from 0 to 32767, for
a total of 32768 sectors.  Since Atari-compatible disks have 512 bytes
per sector, that means you are limited to 32768 * 512 = 16 777 216
bytes (16 Megabytes) per partition.

What Atari has changed, I believe, is to treat the "recno" parameter
as an UNSIGNED int, with a range of 0 to 65535, and consequently a
32Mb limit.  However, any software which calls Rwabs() to directly
access the hard disk (such as disk checkers might do) must understand
this convention, or else they will get confused by the (apparently)
negative sector numbers (i.e. any in the second 16 Meg with the
high-order bit set).

Personally, I am absolutely paranoid about hard disk integrity, and
will limit myself to 16 Meg or smaller partitions.  Besides, once you
get about 4 Meg of stuff in a partition, the old Dfree()/FAT search
bug rears up, and file creation takes nearly forever.  It is, however,
a good motivation to get Rainbow TOS.

Copyright 1990 Greg Wageman	DOMAIN: greg@sj.ate.slb.com
Schlumberger Technologies	UUCP:   {uunet,decwrl,amdahl}!sjsca4!greg
San Jose, CA 95110-1397		BIX: gwage  CIS: 74016,352  GEnie: G.WAGEMAN
 Permission is granted for reproduction provided this notice is maintained.

bluneski@pogo.WV.TEK.COM (Bob Luneski) (02/07/90)

apratt@atari.UUCP (Allan Pratt) writes in response:

>Also right.  Here's the bottom line: when you have a partition larger
>than 16MB, the driver creates the partition with 1KB sectors (or more!)
>rather than 512-byte sectors, which is the old value.  Cluster size is
>still two sectors, whatever size that is.  The disk itself is formatted
>with 512-byte sectors, and Rwabs does blocking and deblocking, turning
>requests for "one logical sector" into as many physical sectors as
>necessary.

<stuff deleted>

>The moral is, utilities should always use Getbpb to find out how big
>sectors are, how many sectors per cluster, etc., and base all your
>other calculations and "constants" on the answers.  If you're adding
>GEMDOS buffers or other permanent objects, get the release notes and do
>it right.

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<  WARNING  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Under TOS 1.0,1.2, and 1.4 repeated use of Getbpb or Dfree will bring your 
system crashing to a halt! The number of calls until failure depends on your 
system memory utilization but averages in the 35-65 call range.  This 
little bug(feature) was discovered at great loss of time, sleep, and hair
(with a number of forehead bruises also).  The real moral of the story is 
never let someone else do what you can do yourself.  Read in the boot sector 
and pull the sector size off there!  

I strongly agree with Alan that it is the developers responsibility to query 
the system and never to take anything for granted.

****************************************************************************
*  Bob Luneski                                                             *
*  Diamond Back Support Hotline:  bluneski@pogo.WV.TEK.COM                 *
*                                                                          *
*  The opinions expressed herein are my own and in no way reflect the      *
*  opinions of Tektronix, Inc.                                             *
****************************************************************************

covertr@force.UUCP (Richard E. Covert) (02/08/90)

In article <8503@pogo.WV.TEK.COM>, bluneski@pogo.WV.TEK.COM (Bob Luneski) writes:
> 
> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<  WARNING  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 
> Under TOS 1.0,1.2, and 1.4 repeated use of Getbpb or Dfree will bring your 
> system crashing to a halt! The number of calls until failure depends on your 
> system memory utilization but averages in the 35-65 call range.  This 
> little bug(feature) was discovered at great loss of time, sleep, and hair
> (with a number of forehead bruises also).  The real moral of the story is 
> never let someone else do what you can do yourself.  Read in the boot sector 
> and pull the sector size off there!  
> 
> I strongly agree with Alan that it is the developers responsibility to query 
> the system and never to take anything for granted.
> 
> ****************************************************************************
> *  Bob Luneski                                                             *

Bob, I am writing Yet Another Disk Backup Program (YADBP tm of ME!!) and just how
can I get the disk free space of a mountable device (like a floppy, or a sysquest
cartidge, or even an optical floppy!!) if I can not rely on the repeated use of
dfree??

This is a serious bug for me because I had planned to just call dfree() and be done
with it. Now I need some other routine. I guess I can take your hint about reading
the boot sector and getting the sector size from there. But are there risks in doing
that?? Will that crash the system also?? Or does someone have a non-crashing 
version of dfree() that I can use?

Does any one have C code (or ASM that is callable by MWC) that will work w/o
crashing the system??

HELP!!!


HELP!!!


HELP!!!


-- 
 Richard E. Covert, Lead Engineer of Software Tools Group
 AG Communications Systems, Phoenix AZ   (602) - 581-4652
 TCP/IP: covertr@gtephx
 UUCP: {ncar!noao!asuvax | uunet!zardoz!hrc | att}!gtephx!covertr

bluneski@pogo.WV.TEK.COM (Bob Luneski) (02/09/90)

In article <4885cc32.14a1f@force.UUCP>, covertr@force.UUCP (Richard E. Covert)
writes:

>Bob, I am writing Yet Another Disk Backup Program (YADBP tm of ME!!) and just 
>how can I get the disk free space of a mountable device (like a floppy, or a 
>sysquest cartidge, or even an optical floppy!!) if I can not rely on the 
>repeated use of dfree??

Why YADBP? Just buy Diamond Back(grin)

>This is a serious bug for me because I had planned to just call dfree() and be 
>done with it. Now I need some other routine. I guess I can take your hint about 
>reading the boot sector and getting the sector size from there. But are there 
>risks in doing that?? Will that crash the system also?? Or does someone have a 
>non-crashing version of dfree() that I can use?

You don't want to just call Dfree because 1) It will cause system failure with
repeated use and 2) it is VERRRRRRRY SLOOOOOOW for even small sized media.
You can do the same job yourself reliably and much faster. All the information 
you need to determine free space on the disk is contained in the disk boot 
sector and the FAT table, you just have to know how to piece it together. When 
making your own disk free calculations it is important to remember that the ST
file system is based on clusters not on bytes.  If you do it right, there is
no danger in doing it yourself and significantly less danger than letting
TOS do it. 

****************************************************************************
*  Bob Luneski                                                             *
*  Diamond Back Support Hotline:  bluneski@pogo.WV.TEK.COM                 *
*                                 Genie: B.LUNESKI1                        * 
*                                                                          *
*  The opinions expressed herein are my own and in no way reflect the      *
*  opinions of Tektronix, Inc.                                             *
****************************************************************************

brazil@pawl.rpi.edu (Timothy E. Onders) (02/13/90)

In article <1990Feb6.223637.15845@sj.ate.slb.com> greg@sj.ate.slb.com (Greg Wageman) writes:
>
>What Atari has changed, I believe, is to treat the "recno" parameter
>as an UNSIGNED int, with a range of 0 to 65535, and consequently a
>32Mb limit.  However, any software which calls Rwabs() to directly

Actually, the logical sector size was also increased to 1024 bytes, along
with some other modifications.  Maximum partition size is now 1024 Megs,
or 1 gig.  I'm presently running off of a 40 meg partition, and I wouldn't
go back for anything.

>access the hard disk (such as disk checkers might do) must understand
>this convention, or else they will get confused by the (apparently)
>negative sector numbers (i.e. any in the second 16 Meg with the
>high-order bit set).

The big problem now is that the program may not be able to deal with the
1024 byte logical sectors, since most just assume 512 byte sectors.  The
physical sectors, however, are still 512 bytes.

>get about 4 Meg of stuff in a partition, the old Dfree()/FAT search
>bug rears up, and file creation takes nearly forever.  It is, however,
>a good motivation to get Rainbow TOS.
>
Never seen this problem, even though I've still got 1.2. Hmmm.

				-Tim Onders
				brazil@pawl.rpi.edu

apratt@atari.UUCP (Allan Pratt) (02/14/90)

brazil@pawl.rpi.edu (Timothy E. Onders) writes:

>In article <1990Feb6.223637.15845@sj.ate.slb.com> greg@sj.ate.slb.com (Greg Wageman) writes:
>>
>>What Atari has changed, I believe, is to treat the "recno" parameter
>>as an UNSIGNED int, with a range of 0 to 65535, and consequently a
>>32Mb limit.

Wrong.

>Actually, the logical sector size was also increased [...]

Right.  The logical sector size is now "large enough so the partition
can be represented by 32767 sectors," which is to say 512 bytes
for <16MB, 1024 bytes for <32MB, etc. all the way up to 8K/sector
or more.

This means BIG TROUBLE for any program which does Rwabs() calls and
expects to get 512 bytes per sector: you should use Getbpb() to FIND
OUT how big sectors are, rather than assuming any one value.  It's also
BIG TROUBLE for programs which add buffers to GEMDOS via the "bufl"
system variables: it's hard (not impossible) to tell how large each
buffer must be.  Get the AHDI 3.00 Release Notes from Atari (available
to developers) for the grubby details.

>>the old Dfree()/FAT search bug rears up

There is no such bug; I think you mean "really horribly slow implementation."

>>and file creation takes nearly forever.

That part is true.  So take your own advice and get TOS 1.4!

============================================
Opinions expressed above do not necessarily	-- Allan Pratt, Atari Corp.
reflect those of Atari Corp. or anyone else.	  ...ames!atari!apratt

csbrod@medusa.informatik.uni-erlangen.de (Claus Brod ) (02/15/90)

greg@sj.ate.slb.com (Greg Wageman) writes:

>What Atari has changed, I believe, is to treat the "recno" parameter
>as an UNSIGNED int, with a range of 0 to 65535, and consequently a
>32Mb limit.  However, any software which calls Rwabs() to directly
>access the hard disk (such as disk checkers might do) must understand
>this convention, or else they will get confused by the (apparently)
>negative sector numbers (i.e. any in the second 16 Meg with the
>high-order bit set).

Actually, ATARI did extend the range of sector numbers, but not only
to 2^16 but to 2^32. This is, however, not the reason why partitions
larger than 16 MB became possible. The new ATARI driver supports
larger logical sector sizes. Before, the logical sector size has
been 512 bytes througout all kind of media. Now, the new hard disk#
driver blocks 512 byte sectors to larger logical sectors. A partition
with 24 MB, for instance, has a logical sector size of 1024 bytes.
This trick works with ALL TOS versions, so you don't have to install
TOS 1.4 to use them. (Nevertheless, TOS 1.4 is a must for every
hard disk user because it speeds up everything quite a lot.)

Claus Brod